Algorithms For Pre-Processing And Noise Filtering In Streaming Telemetry Data Of Iot Devices

Abstract

This article examines algorithms for preprocessing and filtering noise in streaming telemetry data from IoT devices. We classify noise sources (measurement noise, ADC quantization, and impulse noise during packet loss and retransmission) and formulate requirements for online processing: limited memory, on-the-fly processing without re-passing the data, and predictable computational complexity per sample. We consider moving averages, exponential smoothing, median filters, Hampel filters, recursive Kalman filters, and wavelet noise suppression, as well as hybrid pipelines that combine outlier removal with subsequent smoothing. Using a synthetic telemetry stream simulating temperature sensor readings with Gaussian noise and impulse spikes, we compare the methods in terms of root mean square error (RMSE), processing time per sample, and response latency. The hybrid Hampel filter + exponential smoothing pipeline is shown to reduce RMSE by 84.1% while maintaining O(1) computational complexity per sample, making it suitable for deployment on low-power edge devices.