Does anyone have ideas on how to set up a system, where interactive notebook sessions would be run through a queueing system at the individual cell level?
The scenario is, that some typical commands/cells use plenty of memory for a short while. So when many users are using the system at the same time, there is a high probability for running out of memory, if everything is just executed instantly in an interactive session. By having the execution of a cell wait for resources, this would be avoided.
It will not work to schedule the whole notebook through a queue system, as most of the time, the notebook is idle. It is only when running the cells that the start of the execution should wait for resources to become available (if necessary).
Any and all ideas on how to implement this is welcome and highly appreciated!