Course Overview:

This course is designed to build your foundational skills in data engineering on Microsoft Fabric, focusing on the Lakehouse concept. This course will explore the powerful capabilities of Apache Spark for distributed data processing and the essential techniques for efficient data management, versioning, and reliability by working with Delta Lake tables. This course will also explore data ingestion and orchestration using Dataflows Gen2 and Data Factory pipelines. This course includes a combination of lectures and hands-on exercises that will prepare you to work with lakehouses in Microsoft Fabric.

Course Objectives:

● Describe end-to-end analytics in Microsoft Fabric
● Describe core features and capabilities of lakehouses in Microsoft Fabric
● Create a lakehouse
● Ingest data into files and tables in a lakehouse
● Query lakehouse tables with SQL
● Configure Spark in a Microsoft Fabric workspace
● Identify suitable scenarios for Spark notebooks and Spark jobs
● Use Spark dataframes to analyze and transform data
● Use Spark SQL to query data in tables and views
● Visualize data in a Spark notebook
● Understand Delta Lake and delta tables in Microsoft Fabric
● Create and manage delta tables using Spark
● Use Spark to query and transform data in delta tables
● Use delta tables with Spark structured streaming
● Describe Dataflow capabilities in Microsoft Fabric
● Create Dataflow solutions to ingest and transform data
● Include a Dataflow in a pipeline

Who Should Attend?

The primary audience for this course is data professionals who are familiar with data modeling, extraction, and analytics. It is designed for professionals who are interested in gaining knowledge about Lakehouse architecture, the Microsoft Fabric platform, and how to enable end-to-end analytics using these technologies. These include:

• Data Analyst
• Data Engineer
• Data Scientist

Course Prerequisites

There are no prerequisites for this course.

Course Content:

Module 1: Introduction to end-to-end analytics using Microsoft Fabric

● Introduction
● Explore end-to-end analytics with Microsoft Fabric10
● Data teams and Microsoft Fabric
● Enable and use Microsoft Fabric
● Knowledge Check
● Summary

Module 2: Get started with lakehouses in Microsoft Fabric

● Introduction
● Explore the Microsoft Fabric lakehouse
● Work with Microsoft Fabric lakehouses
● Explore and transform data in a lakehouse
● Exercise – Create and ingest data with a Microsoft Fabric lakehouse
● Knowledge Check
● Summary

Module 3: Use Apache Spark in Microsoft Fabric

● Introduction
● Prepare to use Apache Spark
● Run Spark code
● Work with data in a Spark dataframe
● Work with data using Spark SQL
● Visualize data in a Spark Notebook
● Exercise – Analyze data with Apache Spark
● Knowledge Check
● Summary

Module 4: Work with Delta Lake tables in Microsoft Fabric

● Introduction
● Understand Delta Lake
● Create delta tables
● Work with data tables in Spark
● Use delta tables with streaming data
● Exercise – Use delta tables in Apache Spark
● Knowledge Check
● Summary

Module 5: Ingest Data with Dataflows Gen2 in Microsoft Fabric

● Introduction
● Understand Dataflows Gen2 in Microsoft Fabric
● Explore Dataflows Gen2 in Microsoft Fabric
● Integrate Dataflows Gen2 and Pipelines in Microsoft Fabric
● Exercise – Create and use a Dataflow Gen2 in Microsoft Fabric
● Knowledge Check
● Summary