AQI Predictor

A time-series model will happily report an excellent score while quietly reading its own answer. Most of the work here went into making that impossible: backward-only causal rolling-median imputation, limited forward-fill windows and shift(1) change-rate baselines, so no feature at time t can observe the target it predicts. The training and inference paths were split into separate feature groups for the same reason - an observed-only historical store and a forecast-weather store - so forecast data can never contaminate training.

1. 01Engineered a 45-feature representation combining EDA-driven lags, cyclical sine/cosine encodings for hour, day-of-week and month, AQI change-rate and momentum features, temperature–humidity and wind–pressure interactions, and one-hot seasonal indicators.
2. 02Selected the lag structure empirically from ACF/PACF, cross-correlation and mutual-information analysis, applying a full [1, 3, 6, 12, 24]-hour set to PM2.5 and carbon monoxide while restricting temperature and pressure to their 12-hour peak-predictive lag.
3. 03Trained and benchmarked five models - Random Forest, XGBoost, LightGBM, ElasticNet and a four-layer Keras network - under TimeSeriesSplit cross-validation with a chronologically held-out test set, promoting the best by a deterministic rule of lowest validation RMSE, then MAE, then R², with a degenerate-fit guard excluding perfect scores.
4. 04Reached a best validation RMSE of 3.24 and R² of 0.98 with LightGBM, exporting a full train/validation/test comparison across all five models on every run.
5. 05Implemented autoregressive multi-step forecasting maintaining 26-hour rolling buffers, feeding each predicted hour back into the next through an inverted EPA breakpoint table that converts predicted AQI to the PM2.5 concentration driving downstream lags.
6. 06Designed a three-tier storage abstraction - MongoDB Atlas primary, Hopsworks secondary, bundled local artifacts as last resort - where reads take the first backend returning data and writes fan out to all of them, switchable at runtime.
7. 07Built a versioned model registry on MongoDB GridFS storing serialised artifacts, scalers, feature manifests and metrics, making models portable without a shared filesystem.
8. 08Delivered an 11-endpoint FastAPI service covering forecasts, current AQI, health guidance, alerts, per-model and cross-model metrics, feature importance and SHAP explainability, with runtime model selection and an in-process cache.
9. 09Automated the lifecycle with two GitHub Actions workflows: an hourly feature pipeline ingesting the last 26 observed hours to compute 24-hour lags correctly while inserting only new timestamps, and a daily pipeline that retrains, re-selects, re-registers and regenerates the forecast cache.
10. 10Backfilled two years - roughly 17,500 hourly records - in 90-day batches with three-attempt exponential backoff, time-keyed deduplication and chronological ordering.
11. 11Built a six-page Streamlit dashboard with Plotly visualisations and plain-language SHAP summaries that translate feature contributions into readable explanations.
12. 12Wrote 29 Pytest tests over the storage abstraction, covering backend-priority resolution, read fallback, write fan-out and model registry load and publish paths.

• Time-series feature engineering
• Data leakage prevention
• Lag selection via ACF/PACF and mutual information
• Multi-model training and automated selection
• Autoregressive multi-step forecasting
• MLOps pipeline automation
• Feature store and model registry design
• Multi-backend storage abstraction with failover
• Model explainability with SHAP
• REST API design
• Dashboard development and data visualisation
• CI/CD scheduling and deployment