This was a project done in Azure Data Factory.
I began by extracting data from New York City Open Data: https://opendata.cityofnewyork.us/
From there, I created a Blob Container within an existing storage account. Then I initialized Azure Data Factory to do a series of T-SQL transformations on CSV files. I ultimately wanted to load data into a parquet file. The dataflow looks like this:
The final, loaded result of the ETL process resulted in the creation of a parquet file hosted within a generated blob in the container:
I then went back into my container to look for any issues to trouble shoot. There were no issues to resolve, so I monitored activity in my container in a private dashboard:
