2110531 Data Science and Data Engineering Tools @Chula 2023

Syllabus:

Syllabus

Code:

Week01: Intro to Numpy, Pandas

1. Numpy:

2. Pandas:

3. Pandas with Youtube stat data:

4. (Advanced) Pandas with Youtube stat data:

Assignment (Pandas with Youtube stat data):

Week02: Data Preparation

1. EDA:

2. Impute Missing Value:

3. Split Train/Test:

4. Outliers with Log:

5. Outliers with Log (Titanic DataSet):

Assignment:

Week03-04: Traditional ML

1. Decision Trees:

2. Linear Regression:

3. Logistic Regression:

4. Neural Network:

5. K Nearest Neighbors:

6. SVM:

7. Save and Load Model:

8. K-Means:

9. Market-Basket Analysis:

Week05-06: Intro to Deep Learning

1. Image classification (basic): CIFAR10:

2. Image classification (advanced): Animal:

3. Object detection: VOCDetection:

4. Semantic segmentation: CamSeq2007:

5. Time series Forecasting: Stock Price:

Week08: Advanced ML (Transformer, GenerativeAI, Model Monitoring, Text Classification)

1. Image classification with Hugging Face:

2. Model Monitoring with MLflow:

3-1. Model Monitoring with Tensorboard:

3-2. Model Monitoring with Weight and Biases:

4-1. Diffusion on Text-to-Image:

4-2. Diffusion on Image-to-Image:

5. OpenAI ChatGPT (Simulated Toyota data):

6-1. Text Classification (TF-IDF):

6-2. Text Classification (BERT):

Week07: Big Data Architecture and Data Storage

1. Simple Example

2. Redis Assignment

Week08: Data Extraction

1. Basic Web Scraping

2. Wikipedia Scaping

3. REST API Extraction

4. Selenium

5. Excel Extraction

6. PDF Extraction

Week09: Data Ingestion

The Kafka sample codes cannot be openned in Colab as it requires connection to local kafka server. See more details in week09 code section.

Week10: Spark

1. Basic Spark

2. Spark SQL

3. Spark ML

4. Spark Assignment

Week11: Ops Stars

The airflow and fastapi sample codes cannot be openned in Colab as it requires local airflow and fastapi installation. See more details in week11 Ops Stars.