Real-Time Flight Data Analysis with Apache Storm
🔗View on GitHubA distributed real-time stream processing system to analyze and filter live flight data based on altitude, vertical rate, and proximity to airports using Apache Storm.
Overview
Developed a real-time flight monitoring system using Apache Storm to process live ADS-B flight streams. Implemented topology components including Spouts for data ingestion, and Bolts such as HubIdentifier and AirlineSorter to filter, group, and analyze flights within a 20-mile radius of airports. The system identifies aircraft below 3000 ft with significant vertical movement, providing near real-time situational awareness for air traffic analytics.
System Architecture
Storm Topology Architecture: ADS-B Flight Data Spout ingests real-time flight streams. Data flows through parallel Bolts: DecoderBolt (deserializes flight data), AltitudeFilterBolt (filters aircraft below 3000ft), VerticalRateBolt (detects vertical movement >500fpm), and ProximityBolt (checks 20-mile radius from airports using geospatial indices). Results aggregated by HubIdentifier and AirlineSorter Bolts. Topology deployed on Storm cluster with configurable parallelism. Redis/Kafka stores intermediate results. Bolt grouping strategies ensure data locality and fault tolerance.
Setup & Implementation
Clone the repository and configure Apache Storm locally or on a distributed cluster. Use Maven to build the project and deploy the topology using 'storm jar'. Provide a flight data stream (e.g., flights.txt or live ADS-B feed) to the Spout. Filter parameters such as altitude threshold and proximity radius can be adjusted in the configuration file.