# How to load AWS Profiles to Jupyter PySpark or Spark sessions

**URL:** <https://discourse.jupyter.org/t/how-to-load-aws-profiles-to-jupyter-pyspark-or-spark-sessions/8808>\
**Category:** Notebook\
**Created:** [April 17, 2021, 11:52am UTC](https://discourse.jupyter.org/t/how-to-load-aws-profiles-to-jupyter-pyspark-or-spark-sessions/8808 "2021-04-17T11:52:41Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![vara-bonthu](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/vara-bonthu/32/4187_2.png) [@vara-bonthu](https://discourse.jupyter.org/u/vara-bonthu)\
**Post date:** [April 17, 2021, 11:52am UTC](https://discourse.jupyter.org/t/how-to-load-aws-profiles-to-jupyter-pyspark-or-spark-sessions/8808/1 "2021-04-17T11:52:42Z")

</div>

I have Jupyter Notebook running in local docker container and its started with the following shell script inside the container. I can access the Jupyter Notebook session (PySpark or Spark or Python3) However i would like to make use of my AWS profile (credentials) to my Jupyter Notebook session.

How can i achieve this? I normally use `export AWS_PROFILE =<profile name>`. This will provide the credentials for that session so that i can run Spark jobs from the Terminal but i would like to do the same with the Jupyter Notebook sessions as well.

Is there a workaround to set the profiles? or any other approach

```
#!/bin/bash
nohup /home/livy/bin/livy-server &
/usr/local/bin/jupyter notebook --allow-root --NotebookApp.token='' --NotebookApp.password='' --no-browser --ip=0.0.0.0
```

---

<div class="post-metadata">

**Author:** ![manics](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/manics/32/85_2.png) [@manics](https://discourse.jupyter.org/u/manics)\
**Post date:** [April 17, 2021, 7:22pm UTC](https://discourse.jupyter.org/t/how-to-load-aws-profiles-to-jupyter-pyspark-or-spark-sessions/8808/2 "2021-04-17T19:22:59Z")

</div>

You could try setting environment variables for all your AWS credentials on the Docker command line, something like:

```auto
docker run -e AWS_ACCESS_KEY_ID=<...> -e AWS_SECRET_ACCESS_KEY=<...> -e AWS_SESSION_TOKEN=<...> ...

```

or you could try mounting your `~/.aws` directory into the container:

```auto
docker run -e AWS_PROFILE=<> -v ~/.aws:/home/jovyan/.aws:ro ...

```

You’ll may need to modify your Docker user or the permissions on your `~/.aws` files to ensure they’re readable inside the Docker container.