A Novel Filter for Tracking Trends in Noisy Real-Time Data
ISEF · 2014 Earth and Environmental Sciences
Overview
Noise in data can mask underlying trends. These trends can consist of ramps, steps, spikes, sinusoids, etc., and may provide important information about the process being monitored. In order to observe these trends, the data can be filtered to reduce the noise. However, filtering tends to remove useful information as well; it rounds corners, introduces phase lag (delay), and fails to fully capture legitimate rapid changes such as spikes in the data. This work presents a novel computer algorithm that can smooth noisy data to expose the underlying trend, but can react quickly to fully capture sudden changes in the trend. The algorithm is based on an exponential smoothing filter with a modification that enables the coefficients to change when the measurement is sufficiently far from the previous smoothed value. This serves two purposes. First, it provides excellent tracking of large rapid changes in the signal. Second, for certain applications where the time of the rapid change in the signal is important, the coefficient itself can be used as an indicator of this change. Three simple tuning parameters make the filter very flexible and the adjustment intuitive. The algorithm was tested on a variety of noisy datasets containing steps and spikes. It worked very well, and in particular, when there were features of the data that needed to be extracted (e.g., P and S wave leading edges in seismic data, and QRS complex in ECG data), the filter coefficient was very effective detector.
Competition history
- ISEF 2014
Resources
Related projects
CWSF · 2026
Signal Through the Noise: Improving Radon Readings with a Kalman Filter
ISEF · 2026
Physics-Anchored Short-Term Earthquake Hazard Characterization Using Large-Scale Seismic Data Waveform-Derived Sequence Information Improves Leakage-Safe Short-Term Regional Hazard Indicators Beyond Catalog-Only Baselines
ISEF · 2025
Hybrid Incremental Learning: A Novel Approach to Time Series Forecasting
ISEF · 2025
Using Artificial Neural Networks and Kolmogorov-Zurbenko Filtering to Predict Future Sunspot Numbers and Temperatures
Closest projects by meaning, across every fair and year in the corpus.
Source: Regeneron International Science and Engineering Fair