Skip to content

ieee-sun/interpolation

Repository files navigation

Interpolation for Missing Data

An overview & demo of various Interpolation methods for missing data.

Effective for Dynamic-detection(LIDAR/CFAR), Deep Image Sensor, and Time-series records with NaN entries.

Motion-Detection Data

Imputational trace with classical interpolation (Linear/Polynomial/Spline/Seasonal) and Machine Learning Methods.

Click Download to View, HTML

  • [Basic] - Linear Interpolation
  • [Advanced] - Curved Interpolation:
    • Polynomial Interpolation
    • Spline Interpolation
  • Dealing with Seasonal Data:
    • Seasonal Decomposition of Time Series
    • Filling Missing Values Using Seasonal Decomposition
  • Machine Learning for Imputation:
    • Using K-Nearest Neighbors to Impute Missing Values
  • Notes for Data Engineers
  • Demo & Comparison
Interpolated Demo Charts

[Basic] - Linear Interpolation

Linear interpolation is a simple and effective method for estimating missing values when the data is relatively smooth, and exhibits linear trends between data points.

[Advanced] - Curved Interpolation:

  • Polynomial Interpolation

    Polynomial interpolation fits a polynomial of specified order, or complex curves through the known data points, making it suitable for datasets with non-linear trends.
  • Spline Interpolation

    Spline interpolation fits piecewise polynomials between data points, ensuring smooth transitions. It effectively captures complex patterns without the risk of overfitting.

Dealing with Seasonal Data:

  • Seasonal Decomposition of Time Series

  • Filling Missing Values Using Seasonal Decomposition

    Seasonal decomposition is highly effective for time series data with clear seasonal patterns. It separates the data into trend, seasonal, and residual components, allowing for targeted imputation. Filling missing values using the trend component can be effective in seasonal data.

Machine Learning for Imputation:

  • Using K-Nearest Neighbors to Impute Missing Values

    KNN imputation is a versatile method that estimates missing values based on the similarity to other data points. It can capture local patterns in the data.

Notes for Data Engineers

Click Download to View, HTML

Demo & Comparison

  • Download to View:

Interpolation for Missing Data - Demo (HTML, 3.3MB) Comparison of 6 interpolation models with dataset and illustrations.

Python solution - Linear Interpolation (.py)

Python solution - Polynomial Interpolation (.py)

Prepared & Published by:
Sun CHUNG, SMIEEE M.Sc. HKU - colab w/ MIT-IDSS
KNN Graphical Imputation Automobile Trajectory Prediction MIMO FMCW Deep Image Radar Detection