I’ve been working on a Jupyter notebook where as I work I’m building a large nested list. I want to have that list be persistent, so that if I close the browser (or even restart the computer) and then launch notebook again from a command prompt, then not only will the notebook code be visible, but the variable for this list can be re-bound to its value at the end of the previous session by typing a line in the notebook. Effectively, that it “follows the .ipynb file around”, whether or not it’s actually inside the .ipynb itself.
I tried using the %store “magic” , i.e. ‘%store {listname}’ yesterday when I finished working on the notebook and tried doing ‘%store -r {listname}’ today when I re-launched, however it didn’t work. Both times when using %store, I got a warning that this is now an optional feature, and that it needs pickleshare installed. Firstly I thought that warnings should be non-critical and that operations that only give them (and not errors) should have worked, and in fact it did say “Stored ‘{listname}’ list” after storing yesterday. And trying to restore it doesn’t give an error (only the warning) but the variable was still “None” afterward. Luckily I printed the list shortly before I quit yesterday so I could copy-paste and reassign to the list name (aside from some complex non-printable objects–see below), so I only lost one element that will take a few minutes to re-create, but I need to avoid this in the future.
Secondly, I’ve now installed pickleshare and I can import it without errors, but I still get that warning if I use ‘%store’. I’m wondering how to fix this so that it will work. It seems like this is something that should work.
One thing that MAY be influencing it is that within the hierarchy of nested lists, the bottom-most elements are dictionaries, and while most values in these dicts are simple strings or ints, one of them is a complex object. I did anticipate that these objects themselves might not be saved using %store, so I wrote a function that can be called before closing the notebook that loops through the dicts and writes these objects to files, using the package that created them, and with some of the other values in the filename. Then, a companion function does the reverse, loops over the dicts, uses the other values to build the path, loads the files back into objects, and then sets dict[‘{key-of-complex-object}’] = {rebuilt-object}.
This is not a hassle in itself, in fact it’s a more useful representation of these objects for long-term use than some opaque Python object. But the list-of-dicts itself takes very little space, is only useful inside Python, and I should not need to create a special custom function just to write it to a file. But I’m wondering if the “broken links” to these objects are a possible source of the problem saving and restoring the list between sessions. It’s possible the error has nothing to do with that, but I wanted to mention it just in case.