It looks like the plotly issue is a limitation of dash? I see in Dash in Jupyter Environments | Dash for Python Documentation | Plotly the following:
JupyterLab version 3 is required to run a Dash app in jupyterlab mode.To use
jupyterlabmode, you’ll also need to build the@plotly/dash-jupyterlabextension. When you launch a new JupyterLab notebook after installing Dash, you’ll be prompted to build@plotly/dash-jupyterlab.
It also looks like plotly/dash has issues with JupyterLab 4 (Does plotly work with Jupyterlab 4? - #12 by schausson31 - 📊 Plotly Python - Plotly Community Forum), which is maybe why you are having issues once you click ‘build’. It tries to build for JupyterLab 3, even though you are using JupyterLab 4. Unfortunately I don’t think there’s anything we (as in Jupyter folks) can currently do about this.