Open Data Week 2024

Step 1: Anaconda and Jupyter Notebook download instructions

Python is the programming language we use to prepare data for and create our Open Datasets on our Socrata platform, and what you'll use in this demonstration. Anaconda is a 'distribution' of Python. Downloading Anaconda lets you use Python, as well as other useful programs for programming in Python.

1. Download this repository by clicking the "Code" button, and then the "Download Zip" button beneath that.
   

2. Extract the open-data-week-2024 folder to somewhere in your computer by clicking or double-clicking it, and hitting the "Extract All" button.
   

3. Go to this link and download Anaconda.

4. If you go to the Downloads folder on your computer, you should see the Anaconda installer there. It should look something like this:
   

Double click on that, and then you’ll be prompted through a number of steps to install Anaconda on your computer. Go through all of the steps until the installation is complete.

5. Once you finish Step 2, Anaconda will be downloaded on your computer. The icon for the app will look like this:
   

Open the application.

6. Once you open Anaconda-Navigator, the interface should look similar to this:
   

7. Click on “Launch” under the VS Code square. VS Code is a code-editor that we'll use to edit and run our Python scripts.
   

It should pop up a new window like this:

8. Click on Open Folder. Then navigate to the open-data-week-2024 file in the folder, click on the folder named , and click "select folder."
   

This will set this folder as your workspace. You should be able to see what files are available in this folder on the left-hand side of your screen.

9. VSCode has extensions that make programming using Python easier. You can install extensions by clicking on the "Extensions" button on the left hand side.
   

You should search for and install the following extensions:

• Python
• Jupyter

Step 2: Alternative for experienced Python users: Installing Jupyter with pip/pip3

As an existing Python user, you may wish to install Jupyter using Python’s package manager, pip, instead of Anaconda.

The launch methods on Windows and Mac are different.

Windows launch Jupyter Notebook server steps

1. Navigate to the working folder you want to write or run Jupyter Notebook file. And on the address bar on the top, enter cmd and hit Enter.
   
   
2. Then type pip install jupyter in the command prompt (in case you don't know where to run the pip instruction last step).
   
   
3. After the successful installation, type jupyter notebook and hit Enter.
   
   

Mac OS launch Jupyter Notebook server steps

1. Open terminal in the working file and type pip3 install jupyter.
   
   
2. Instead, you should type python3 -m notebook to lauch the Jupyter Notebook server successfully.
   
   

When successful, this will print some information about the notebook server in your terminal, including the URL of the web application (by default, http://localhost:8888):

[I 2024-03-06 18:37:47.801 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2024-03-06 18:37:47.806 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2024-03-06 18:37:47.811 ServerApp] jupyterlab | extension was successfully linked.
[I 2024-03-06 18:37:47.814 ServerApp] notebook | extension was successfully linked.
[I 2024-03-06 18:37:48.416 ServerApp] notebook_shim | extension was successfully linked.
[I 2024-03-06 18:37:48.590 ServerApp] notebook_shim | extension was successfully loaded.
[I 2024-03-06 18:37:48.592 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2024-03-06 18:37:48.593 ServerApp] jupyter_server_terminals | extension was successfully loaded.

When the notebook opens in your default web browser, you will see the Notebook Dashboard, which will show a list of the notebooks, files, and subdirectories in the directory where the notebook server was started. Most of the time, you will wish to start a notebook server in the highest level directory containing notebooks. Often this will be your home directory.

Step 3: Install packages in Jupyter Notebook steps

1. Create an ipynb file.
   
   
   
2. Rename the file.
   
   
   
   
3. In the cell, use !pip list to check all packages installed. Exclamation Mark required!
   
   
   
4. For example, install pandas package. Use !pip install pandas.
   
   
   
5. Check installation by using !pip show pandas.
   
   
   

Step 4: Proxy setup steps

Config proxy

If you are on a corporate network, you might need to define and use proxies to access the API. Your IT department should be able to provide proxies for you. Feel free to ask them.

FYI, local_config.py file locates in the local repo open-data-week-2024 that you just clone from GitHub.

Now, in your local_config.py file, customize your proxy for variables https and http. The format should look like domain:port.

https = "insertYourProxyHere"
http = "insertYourProxyHere"

Step 5: NYS Data API setup steps

Create an account

1. Go to DATA.NY.GOV to signup if you don't have an account.
   
   
   
2. After verifying your email address, you can sign in and will be redirected to the dashboard.
   
   
   

Config with your info

1. On the top-left corner, click the pencil icon.
   
   
   
2. Navigate to the sidebar on the left, click "Developer Settings".
   
   
   
3. Create New API Key.
   
   
   
   
   
   
   
4. Copy API Key ID and API Key Secret to local_config.py file. And replace it with your own API Key ID and API Key Secret.

apikey_id = "yourAPIKeyID"
apikey_secret = "yourAPIKeySecret"

5. Create New App Token.
   
   
   
   
   
   
   
6. Copy App Token to local_config.py file and replace it with your own App Token.

apptoken = "yourAppToken"

7. Unreveal App Secret Token by clicking Show Secret Token button. Copy it and replace the value of apptoken_secrete variable.

apptoken_secret = "yourAppSecretToken"