This document provides a summary of findings about extending Jupyter classic and Jupyter Lab.
Extending the File Manager of Jupyter and Jupyter Lab
This document summarizes research of Jupyter and Jupyter Lab integration into ERP5 - in particular building a plugin/extension to allow notebooks to be stored in ERP5.
The Task
The desired outcome is to store Jupyter notebooks in ERP5 instead of the server's file system. In particular this means that the Jupyter (Lab) server needs to be able to
- list: get a list of all notebooks (stored in ERP5) from ERP5. It then displays this list in the filemanager component of Jupyter (Lab)
- get: get meta data and cells content of a notebook from the ERP5 storage
- save: save modifications made within Jupyter (Lab) to the stored version in ERP5
- create/delete/rename: typical filemanager operations on notebooks stored in ERP5
Additional (non-core) functionallity may include: upload notebooks, manage checkpoints, etc.
Essentially, a simple file manager has to implemented such that it interfaces with ERP5 via hateoas. This modification/extension should be applicable for both Jupyter classic and Jupyter Lab as both will be relevant for Nexedi.
Extending Jupyter and Jupyter Lab
To achieve this task Jupyter (Lab) needs to be extended to either:
- replace the default file manager with a new version that interfaces with ERP5 instead of the local filesystem
- extend the default file manager to show notebooks stored in ERP5 in addition to the local filesystem
The second approach has the advantage that local storage is also possible. This might be especially relevant for Jupyter Lab, as it is a multi-function tool and not exclusively used for Jupyter Notebooks.
Extending Classic Jupyter
For the classic Jupyter there exist two possibilities.
- A frontend nbextension written in Javascript (undocumented). Reference implementation: jupyter-drive
- An ipython extension implementing the contentsmananger interface (reference). Reference Implementation: PGContents
Note, that these two are extending the built-in file manager of the Jupyter classic and not the notebook functionality itself.
Extending Jupyter Lab
Jupyter Lab is composed of plugins which are written in TypeScript and based on phosphorjs (reference). In case of the notebook plugin an interface layer is written in TypeScript to communicate with the jupyter server (as far as we understand) (reference).
The default file manager (or file browser) of Jupyter Lab is such a plugin (see here).
There are two or three relevant ways to extend Jupyter Lab (documentation is somewhat confusing, see here vs here):
- Jupyter Lab extensions: Extensions to plugins, written in TypeScript (reference)
- Ipython widget extensions: These seem specific for the notebook plugin and are most likely similar to the ipython extension of the classic Jupyter Notebook (reference). They do not seem to be implemented yet (see note at the bottom of reference).
The Problem
Based on this information it seems impossible to make a single extension for both Jupyter and Jupyter Lab. The file managers of Jupyter and Jupyter Lab are completely different. One is built into classic Jupyter and one is a phosphor/typescript widget (Jupyter Lab), which can not be extended with the same tools.
Another point of evidence for this is, that there are two completely seperate implementations of google drive file manager for classic Jupyter and Jupyter Lab:
In other words: Jupyter Lab has decoupled file management from Jupyter Notebook and extensions to the file manager are different from extensions to the Notebook plugin which no longer needs to provide file management itself.
Thus writing a single extension for both classical Jupyter and Jupyter Lab to enable storage of notebooks in ERP5 seems impossible to me.
Moving forward
I still need to find out if ipython extensions to the Jupyter plugin still work in Jupyter Lab. My guess would be: yes. Still, this does not solve the problem. In my opinion two different extensions need to be written to support both Jupyter (classic) and Jupyter Lab.