05696601 Introduction to Big Data Ecosystem 🏖️🌏
Academic Year 2025 | Semester 2
BigGrade | 4rum
Course Description
This course wraps around three main characteristics of Big Data, also known as 3Vs (Volume, Variety, Velocity). Students will learn challenges posed by each characteristic along with techniques to tackle each of them. Those techniques include database partitioning/replication, distributed processing, schemaless data models, object storage, data warehouse, data lake, lakehouse, and stream processing. Modern data platforms (e.g., Spark, Ray, MongoDB, Neo4j, MinIO, dbt, DuckLake, Databricks, Apache Kafka) will be leveraged to demonstrate those techniques.
Course Information
  • Time: Monday 9:00 - 12:00
  • Location: Sc08 Room 714
  • Office Hours: Monday 13:00 - 16:00
Course Evaluation
  • Homework Assignment 20%
  • Lab Assignment 20%
  • Project 40%
  • Final Exam 20%
Course Staff

Yuttapichai Kerdcharoen (Guide)
Instructor
Malapchai Chaisihat (Nueng)
Head Teaching Assistant
Schedule
Date Topic
Nov 24 Lecture #1 - Introduction to Big Data
Online A1 Out 		Slides Video
Dec 1 Lecture #2 - Evolution of Data Technologies
Online 		Slides Video
Dec 8 Lecture #3 - Distributed Databases (Sharding and Replication)
Online A1 Due 		Slides Video
Dec 15 Lab #1 - Building Data Pipelines with dbt Slides
Dec 22 Lecture #4 - Distributed Processing ⚡️ Slides Video
Dec 29
New Year Week 		New Year Week (No Class)
Jan 5 Lab #2 - MongoDB 101 / Neo4j 101 Slides
Jan 12 Commencement Day (No Class)
Jan 19 Midterm Week (No Class)
Jan 26 Lab #3 - Setting up a Spark Cluster and PySpark
A2 Out
Feb 2 Lecture #5 - Data Warehousing ⚡️
Project Team Registration 		Slides Video
Feb 9 Lab #4 - Building Data Lakes with DuckLake and MinIO
Project Proposal 		Slides
Feb 16 Lecture #6 - Stream and Incremental Processing ⚡️
A2 Due 		Slides
Feb 23 Lab #5 - Apache Kafka
Project Checkpoint 1 Due
Mar 2 Lightning Talk (Remaining) ⚡️
Mar 9 Guest Lecture by Kulsawasd Jitkajornwanich, Ph.D. (Assistant Professor @ Texas Tech University)
Project Checkpoint 2 Due
Mar 25 Final Exam
Mar 31 (TBA) Project Presentation
Assignment