About this programme
Programme overview
The Data Engineering is a comprehensive 22-week, part-time program designed to take learners from the fundamentals of data engineering to building production-grade data pipelines. The journey begins with Python, SQL, Linux, Git, and development environment setup, and progresses through advanced SQL, data modeling, data warehousing, ETL/ELT, dbt, PySpark, Apache Spark, and workflow orchestration with Airflow. Learners gain hands-on experience working with real-world data engineering concepts and tools while developing reliable, scalable, and maintainable data pipelines.
The program then moves into modern data engineering technologies, including Kafka and streaming pipelines, cloud data platforms such as Snowflake, BigQuery, and Databricks, Delta Lake, data quality and observability, and LLM-assisted data engineering. Throughout the program, learners complete practical assignments and build portfolio-worthy projects covering batch processing, streaming, cloud data platforms, data quality, and AI-assisted workflows. The curriculum emphasizes practical implementation, documentation, performance optimization, testing, reliability, and real-world engineering practices.
The final stage focuses on transforming technical skills into career opportunities. Learners complete end-to-end capstone projects, create a professional portfolio, prepare an ATS-optimized Data Engineer resume, practice SQL, Python, pipeline and system design interviews, and complete recorded mock interviews. The program concludes with a structured job-search and outreach process covering targeted applications, referrals, networking, application tracking, follow-ups, and offer preparation. By the end of the track, learners will have developed practical Data Engineering skills, completed demonstrable projects, built their professional career materials, and prepared themselves for Data Engineer opportunities.
Curriculum
What you'll learn
The Data Engineering Lifecycle
Lesson 1: The Data Engineering LifecycleLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 0 — Orientation & Environment SetupWeek: 1Category: CoreLesson D...
Linux & Shell Fundamentals
Lesson 2: Linux & Shell FundamentalsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 0 — Orientation & Environment SetupWeek: 1Category: CoreLesson D...
Git Workflows
Lesson 3: Git WorkflowsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 0 — Orientation & Environment SetupWeek: 1Category: CoreLesson DescriptionThis le...
Lesson 4: Python Environment Management
Lesson 4: Python Environment ManagementLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 0 — Orientation & Environment SetupWeek: 1Category: CoreLesson De...
Introduction to Distributed Systems Concepts
Lesson 5: Introduction to Distributed Systems ConceptsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 0 — Orientation & Environment SetupWeek: 1Category...
Local Development Environment & 22-Week Learning Plan
Advanced SQL — Window Functions & CTEs
Lesson 1: Advanced SQL — Window Functions & CTEsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 1 — Python & SQL for Data EngineeringWeeks: 2–3Categ...
Lesson 2: Pandas & PySpark Basics
Lesson 2: Pandas & PySpark BasicsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 1 — Python & SQL for Data EngineeringWeeks: 2–3Category: CoreLesson...
Python Packaging & Testing
Lesson 3: Python Packaging & TestingLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 1 — Python & SQL for Data EngineeringWeeks: 2–3Category: CoreLes...
API Ingestion & Pagination
Lesson 4: API Ingestion & PaginationLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 1 — Python & SQL for Data EngineeringWeeks: 2–3Category: CoreLes...
Idempotent Script Design
Lesson 5: Idempotent Script DesignLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 1 — Python & SQL for Data EngineeringWeeks: 2–3Category: CoreLesson De...
Parquet, JSON & CSV Formats
Lesson 6: Parquet, JSON & CSV FormatsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 1 — Python & SQL for Data EngineeringWeeks: 2–3Category: CoreLe...
Python + SQL API Ingestion Pipeline
Star & Snowflake Schemas
Lesson 1: Star & Snowflake SchemasLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 2 — Data Modeling & WarehousingWeeks: 4–5Category: CoreLesson Desc...
Dimensional Modeling (Facts & Dimensions)
Lesson 2: Dimensional Modeling (Facts & Dimensions)Learning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 2 — Data Modeling & WarehousingWeeks: 4–5Category...
Slowly Changing Dimensions
Lesson 3: Slowly Changing DimensionsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 2 — Data Modeling & WarehousingWeeks: 4–5Category: CoreLesson Descri...
Normalization vs. Denormalization Trade-offs
Lesson 4: Normalization vs. Denormalization Trade-offsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 2 — Data Modeling & WarehousingWeeks: 4–5Category:...
dbt Models & Tests
Lesson 5: dbt Models & TestsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 2 — Data Modeling & WarehousingWeeks: 4–5Category: CoreLesson Descriptio...
dbt Star Schema & Data Warehouse Modeling
Batch vs. Streaming ETL
Lesson 1: Batch vs. Streaming ETLLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 3 — ETL/ELT FundamentalsWeeks: 6–7Category: CoreLesson DescriptionThis less...
Extract–Load–Transform Patterns
Lesson 2: Extract–Load–Transform PatternsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 3 — ETL/ELT FundamentalsWeeks: 6–7Category: CoreLesson DescriptionT...
Incremental Loads & CDC Basics
Lesson 3: Incremental Loads & CDC BasicsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 3 — ETL/ELT FundamentalsWeeks: 6–7Category: CoreLesson Descripti...
Schema Enforcement & Validation
Lesson 4: Schema Enforcement & ValidationLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 3 — ETL/ELT FundamentalsWeeks: 6–7Category: CoreLesson Descript...
Idempotency & Backfills
Lesson 5: Idempotency & BackfillsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 3 — ETL/ELT FundamentalsWeeks: 6–7Category: CoreLesson DescriptionThis...
ELT Pipeline with Incremental Models & Backfill
PySpark DataFrame API
Lesson 1: PySpark DataFrame APILearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 4 — Batch Processing with SparkWeeks: 8–9Category: CoreLesson DescriptionThis...
Partitioning & Shuffling
Lesson 2: Partitioning & ShufflingLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 4 — Batch Processing with SparkWeeks: 8–9Category: CoreLesson Descript...
Joins at Scale
Lesson 3: Joins at ScaleLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 4 — Batch Processing with SparkWeeks: 8–9Category: CoreLesson DescriptionThis lesson...
Performance Tuning
Lesson 4: Performance TuningLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 4 — Batch Processing with SparkWeeks: 8–9Category: CoreLesson DescriptionThis le...
Spark SQL
Lesson 5: Spark SQLLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 4 — Batch Processing with SparkWeeks: 8–9Category: CoreLesson DescriptionThis lesson intr...
PySpark Batch Processing & Performance Tuning
DAG Design
Lesson 1: DAG DesignLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 5 — Workflow Orchestration with AirflowWeek: 10Category: CoreLesson DescriptionThis less...
Task Dependencies & Sensors
Lesson 2: Task Dependencies & SensorsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 5 — Workflow Orchestration with AirflowWeek: 10Category: CoreLesson...
Retries & SLAs
Lesson 3: Retries & SLAsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 5 — Workflow Orchestration with AirflowWeek: 10Category: CoreLesson DescriptionT...
Scheduling & Backfilling
Lesson 4: Scheduling & BackfillingLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 5 — Workflow Orchestration with AirflowWeek: 10Category: CoreLesson De...
Connections & Secrets Management
Lesson 5: Connections & Secrets ManagementLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 5 — Workflow Orchestration with AirflowWeek: 10Category: CoreL...
Airflow Pipeline Orchestration & Failure Alerting
Kafka Producers & Consumers
Lesson 1: Kafka Producers & ConsumersLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 6 — Streaming Data with KafkaWeeks: 11–12Category: AILesson Descrip...
Topics, Partitions & Consumer Groups
Lesson 2: Topics, Partitions & Consumer GroupsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 6 — Streaming Data with KafkaWeeks: 11–12Category: AILesso...
Schema Registry & Avro/Protobuf
Lesson 3: Schema Registry & Avro/ProtobufLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 6 — Streaming Data with KafkaWeeks: 11–12Category: AILesson Des...
Spark Structured Streaming
Lesson 4: Spark Structured StreamingLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 6 — Streaming Data with KafkaWeeks: 11–12Category: AILesson DescriptionT...
Exactly-Once Semantics
Lesson 5: Exactly-Once SemanticsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 6 — Streaming Data with KafkaWeeks: 11–12Category: AILesson DescriptionThis...
Windowed Aggregations
Lesson 6: Windowed AggregationsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 6 — Streaming Data with KafkaWeeks: 11–12Category: AILesson DescriptionThis l...
Kafka Streaming Pipeline & Windowed Aggregations
Cloud Warehouse Architecture
Lesson 1: Cloud Warehouse ArchitectureLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 7 — Cloud Data Platforms (Snowflake/BigQuery/Databricks)Weeks: 13–14Ca...
Snowflake or BigQuery SQL & Cost Controls
Lesson 2: Snowflake or BigQuery SQL & Cost ControlsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 7 — Cloud Data Platforms (Snowflake/BigQuery/Databric...
Databricks Lakehouse & Delta Lake
Lesson 3: Databricks Lakehouse & Delta LakeLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 7 — Cloud Data Platforms (Snowflake/BigQuery/Databricks)Weeks...
Data Sharing & Access Control
Lesson 4: Data Sharing & Access ControlLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 7 — Cloud Data Platforms (Snowflake/BigQuery/Databricks)Weeks: 13...
Query Optimization & Clustering
Lesson 5: Query Optimization & ClusteringLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 7 — Cloud Data Platforms (Snowflake/BigQuery/Databricks)Weeks:...
Cloud Warehouse Migration & Cost Optimization
Great Expectations Suites
Lesson 1: Great Expectations SuitesLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 8 — Data Quality & ObservabilityWeek: 15Category: AILesson Descriptio...
Data Contracts
Lesson 2: Data ContractsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 8 — Data Quality & ObservabilityWeek: 15Category: AILesson DescriptionThis lesso...
AI-Assisted Anomaly Detection
Lesson 3: AI-Assisted Anomaly DetectionLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 8 — Data Quality & ObservabilityWeek: 15Category: AILesson Descri...
Freshness & Volume Monitoring
Lesson 4: Freshness & Volume MonitoringLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 8 — Data Quality & ObservabilityWeek: 15Category: AILesson De...
Lineage Tracking
Lesson 5: Lineage TrackingLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 8 — Data Quality & ObservabilityWeek: 15Category: AILesson DescriptionThis les...
Data Quality, Validation & AI-Assisted Anomaly Detection
AI-Assisted Schema Inference from Raw Files
Lesson 1: AI-Assisted Schema Inference from Raw FilesLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 9 — LLM-Assisted Data EngineeringWeek: 16Category: AILe...
LLM Code Generation for Transformations
Lesson 2: LLM Code Generation for TransformationsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 9 — LLM-Assisted Data EngineeringWeek: 16Category: AILesson...
Auto-Generated Pipeline Documentation
Lesson 3: Auto-Generated Pipeline DocumentationLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 9 — LLM-Assisted Data EngineeringWeek: 16Category: AILesson D...
Prompting for SQL Generation & Review
Lesson 4: Prompting for SQL Generation & ReviewLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 9 — LLM-Assisted Data EngineeringWeek: 16Category: AILess...
Guardrails for AI-Generated Code
Lesson 5: Guardrails for AI-Generated CodeLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 9 — LLM-Assisted Data EngineeringWeek: 16Category: AILesson Descri...
LLM-Assisted Schema, Transformation & Documentation
End-to-End Pipeline Design
Lesson 1: End-to-End Pipeline DesignLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 10 — Capstone ProjectsWeeks: 17–18Category: CareerLesson DescriptionThis...
Combining Batch, Streaming & Warehouse Work
Lesson 2: Combining Batch, Streaming & Warehouse WorkLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 10 — Capstone ProjectsWeeks: 17–18Category: CareerL...
Documentation & Reproducibility
Lesson 3: Documentation & ReproducibilityLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 10 — Capstone ProjectsWeeks: 17–18Category: CareerLesson Descri...
Capstone Data Engineering Pipelines
Site Structure
Lesson 1: Site StructureLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 11 — Portfolio CreationWeek: 19Category: CareerLesson DescriptionThis lesson focuses...
Architecture-Diagram Case Studies
Lesson 2: Architecture-Diagram Case StudiesLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 11 — Portfolio CreationWeek: 19Category: CareerLesson Description...
GitHub Hygiene
Lesson 3: GitHub HygieneLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 11 — Portfolio CreationWeek: 19Category: CareerLesson DescriptionThis lesson focuses...
Phase 11 Assignment: Professional Data Engineering Portfolio
ATS Formatting
Lesson 1: ATS FormattingLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 12 — Resume BuildingWeek: 20Category: CareerLesson DescriptionThis lesson focuses on...
Quantified Bullets
Lesson 2: Quantified BulletsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 12 — Resume BuildingWeek: 20Category: CareerLesson DescriptionThis lesson focuse...
LinkedIn Optimization
Lesson 3: LinkedIn OptimizationLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 12 — Resume BuildingWeek: 20Category: CareerLesson DescriptionThis lesson foc...
Phase 12 Assignment: ATS-Checked Data Engineer Resume
SQL & Python Coding
Lesson 1: SQL & Python CodingLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 13 — Interview PreparationWeek: 21Category: CareerLesson DescriptionThis le...
Pipeline & System Design
Lesson 2: Pipeline & System DesignLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 13 — Interview PreparationWeek: 21Category: CareerLesson DescriptionTh...
Behavioral / STAR
Lesson 3: Behavioral / STARLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 13 — Interview PreparationWeek: 21Category: CareerLesson DescriptionThis lesson p...
Mock Interviews
Lesson 4: Mock InterviewsLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 13 — Interview PreparationWeek: 21Category: CareerLesson DescriptionThis lesson pro...
Interview Preparation & Mock Interviews
Target Company List
Lesson 1: Target Company ListLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 14 — Job Search & OutreachWeek: 22Category: CareerLesson DescriptionThis le...
Referrals & Cold Outreach
Lesson 2: Referrals & Cold OutreachLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 14 — Job Search & OutreachWeek: 22Category: CareerLesson Descript...
Application Tracking
Lesson 3: Application TrackingLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 14 — Job Search & OutreachWeek: 22Category: CareerLesson DescriptionThis l...
Offer Negotiation
Lesson 4: Offer NegotiationLearning Format: Self-Paced CurriculumRecommended Learning Time: 12–15 Hours per WeekPhase: 14 — Job Search & OutreachWeek: 22Category: CareerLesson DescriptionThis less...
Job Search & Outreach
Free Counselling — 30 Minutes
The Data Engineering
No sales pitch. Just clear next steps for your goal.
Your details stay private. Privacy policy.
Have a friend's code? Apply it for a discount.
By submitting you agree to be contacted by Digital Aicademy.
You're in good hands.
Your counsellor will reach out on WhatsApp within 60 minutes during business hours.
Message us on WhatsApp