Academic and Personal Projects

• Real-time Data Streaming Pipeline: Orchestrating with Apache Airflow and Containerized Technologies: This project demonstrates the creation of a real-time data streaming pipeline, employing Apache Airflow for orchestration, Apache Kafka and Kafka Connect for data ingestion, Zookeeper for synchronization, Apache Spark for data processing, and Cassandra and PostgreSQL for storage, all encapsulated within Docker containers.
• Robust Data Pipelines using Databricks DBT: End-to-end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our cloud provider. This project illustrate the process of data ingestion to the lakehouse, data integration with ADF and data transformation with Databricks, and DBT.
• Reddit Data Pipeline Engineering with AWS: This project integrates various technologies to streamline data extraction from Reddit using its API, orchestrating ETL processes with Apache Airflow and Celery, storing efficiently in Amazon S3, cataloging with AWS Glue, querying and transforming data using Amazon Athena, and implementing best practices for loading data into Amazon Redshift for analytics
• Home Made Food Delivery App: Developed a full functioning Android App that can be used as a food order-delivery system, as a part of project assigned by the Software Engineering Lab Course.
• Graph Algorithms to Maze Graphs: Applying Graph Algorithms to solve Maze problems. The website that hosts this is being built on Ruby on Rails. We will be using OpenGL for the graphics.
• SRRIP Cache Eviction Algorithm: Implemented Static Rereference Interval Prediction algorithm as a part of Cache Simulator Project as a part the Computer Architecture/ Systems Programming course.