If users manually add this file to correct location then the server extension starts working properly else they get 404 on the API calls. Any pointers?
Sorry, I am out of my depth on this one, and I would not want to suggest and incorrect solution that could cause problems later on. If no one chimes in soon-ish I would suggest asking on jupyterlab/jupyterlab - Gitter as it seems as a fairly advanced question.
The most widely-supported practice seems to put jupyter-packaging in pyproject.toml#/build-system/requires, as in the cookiecutter. Definitely don’t make it a runtime dependency.
A perusal of the official data_files docs can help clarify some things, and you might find you don’t even need jupyter-packaging.