[ENH] Jupyter Backend - CouchDB

General
1

Hey Everyone,

I’m looking into starting an enhancement to re-do the ContentsManager backend in CouchDB (https://couchdb.apache.org)

I think I have an idea how I could handle, TreeView, File Load / Save and Checkpoints but it’s a quite a bit of work.

Does anyone have resources on ContentsManager integrations I could learn from?

-J

2

The Jupyter ContentsManager is responsible for translating file interactions into operations on a particular storage medium.

Quantopian’s pgcontents offers a “PostgreSQL backed ContentsManager”, and Jupyter S3 provides a “Jupyter Notebook Contents Manager for AWS S3”. Alternatively, hdfscontents implements a “contents manager for Jupyter that stores files in Hadoop File System (HDFS)” and s3contents"an S3 and GCS backed ContentsManager".

The HybridContentManager is a fork of Quantopian’s original pgcontents content manager that removes the Postgres dependencies and allows you to mix and match different content managers for different directories, allowing you to “easily move files between different content managers (i.e local files to s3 backed manager)”. Path validation is enabled to keep naming schemes consistent and/or prevent illegal characters.

There’s also a jupyter-pyfilesystem ContentsManager that uses the PyFilesystem for storing files. (PyFilesystem “simplifies behaviour compared to io and os modules” and provides yet another level of indirection, allowing you to write to remote locations, such as remote FTP directories, for example, using the same API as you would to write to the local filesystem.)

via

1 Like
3

Also worth noting


had the same idea as me of wrapping PyFilesystem for Jupyter. Given that’s written by a commercial organisation you should probably use that in preference to my https://github.com/manics/jupyter-pyfilesystem