MLWatch - Model Monitoring
Kafka Streams + Statistical Monitoring
Developed MLOps monitoring platform using Spring Boot 3 WebFlux and Kafka Streams for real-time prediction event processing. Implements statistical drift detection algorithms (KL divergence, Population Stability Index, Wasserstein distance) monitoring model behavior. System ingests prediction events from Kafka topics, computes rolling window metrics, detects anomalies via statistical process control, sends alerts when drift thresholds exceeded. Built React dashboard with WebSocket real-time updates showing model health, feature distributions, performance trends over time.
The Challenge
ML models degrade silently in production as data distributions shift. Traditional monitoring tracks infrastructure only, not model-specific metrics like prediction confidence, feature drift, or output distribution changes. Need real-time detection before model degradation impacts results. Statistical drift algorithms (KL divergence, PSI) require streaming architecture.
The Solution
Built reactive monitoring platform using Spring Boot 3 WebFlux + Kafka Streams for real-time event processing. Implemented drift detection algorithms (KL divergence, Population Stability Index, Wasserstein distance). WebSocket dashboard shows real-time model health. InfluxDB time-series storage enables fast historical queries. Alert rules trigger on threshold violations.
Technology Stack
Architecture
- Spring Boot 3 WebFlux reactive streams processing
- Kafka Streams topology for event aggregation
- InfluxDB time-series storage with retention policies
- WebSocket Server-Sent Events for real-time dashboard
- Statistical drift detection: KL divergence, PSI, Wasserstein
- Rolling window aggregations: 1min, 5min, 1hour, 1day
- Alert rule engine with configurable thresholds
- Python microservice for statistical computations
- Redis caching for frequently accessed metrics
- Docker Compose multi-container deployment
Key Features
Results & Impact
- Drift detection algorithms implemented: KL divergence, PSI, Wasserstein distance
- Real-time WebSocket dashboard with React 18 showing model health trends
- Kafka Streams processing pipeline handling 1K+ events/sec in testing
- InfluxDB time-series storage with <100ms query latency for metric retrieval
- Rolling window metrics computed: 1min, 5min, 1hour, 1day aggregations
- Statistical process control charts detecting anomalies automatically
- Tested monitoring 5 ML models with simulated prediction data
- Alert engine with configurable thresholds for drift, accuracy, latency