Stock Market Data Pipeline
Python • Apache Airflow • AWS S3 • PostgreSQL • Pandas
- Built a fully automated ETL pipeline to extract, transform, and load stock market data into an analytics-ready warehouse, implementing a data lake pattern in AWS S3 for raw and processed datasets.
- Orchestrated daily pipeline execution with Apache Airflow, implementing task sequencing and failure handling.
- Integrated PostgreSQL as a data warehouse for downstream SQL analytics.