# JupyterAI Local LLM Integration

**URL:** <https://discourse.jupyter.org/t/jupyterai-local-llm-integration/28407>\
**Category:** JupyterHub\
**Tags:** community, jupyterlab, jupyterhub, how-to, help-wanted\
**Created:** [September 16, 2024, 8:36am UTC](https://discourse.jupyter.org/t/jupyterai-local-llm-integration/28407 "2024-09-16T08:36:55Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![kabirdureja](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/kabirdureja/32/14909_2.png) [@kabirdureja](https://discourse.jupyter.org/u/kabirdureja)\
**Post date:** [September 16, 2024, 8:36am UTC](https://discourse.jupyter.org/t/jupyterai-local-llm-integration/28407/1 "2024-09-16T08:36:55Z")

</div>

Hi all,  
I have a locally hosted Llama 3.1 70b model running on our server at the address: 10.1xx.1xx.50:8084/generate. Since there is no API key required for this setup, I use the following code to get responses from the model:

response = requests.request(method=“POST”, url=“10.1xx.1xx.50:8084/generate”, headers=headers, data=payload)

How can I integrate this endpoint into Jupyter-AI?  
Any help would be really appreciated!

---

<div class="post-metadata">

**Author:** ![krassowski](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/krassowski/32/1685_2.png) [@krassowski](https://discourse.jupyter.org/u/krassowski)\
**Post date:** [September 16, 2024, 10:18am UTC](https://discourse.jupyter.org/t/jupyterai-local-llm-integration/28407/2 "2024-09-16T10:18:50Z")

</div>

Please do not cross post across every single platform/medium possible (I saw this on 5 different channels), it only spreads thin the already sparse resources of volunteers trying to help out. Answered in [jupyter notebook - JupyterAI Local LLM Integration - Stack Overflow](https://stackoverflow.com/a/78989646/6646912)