URL study guide
https://tue.osiris-student.nl/onderwijscatalogus/extern/cursus?cursuscode=2ID70&collegejaar=2025&taal=enDescription
As we enter the era of Big Data and Data Science, the database and data engineering market is going through unprecedented shifts and challenges, demanding new approaches for processing heterogeneous data at massive scale. As a result, a new breed of database systems which aims to handle smart data-driven applications is emerging. This course prepares students to meet the new challenges of contemporary data engineering in which traditional assumptions break, where new data models, programming paradigms, and platforms are required.The topics of the course include:
• Advanced relational databases: transactions, indices, OLAP, query execution optimizations
• Big Data platforms: Spark and Hadoop/MapReduce
• Stream processing: Spark Streaming
Objectives
Learning outcomes:At the end of the course, students will
• be able to summarize and explain the main characteristics of different major data management models in contemporary data-intensive systems;
• be able to effectively use state-of-the-art relational databases for both online analytical processing and online transaction processing;
• be able to describe, compare, and use effectively the two mainstream Big Data ecosystems (Hadoop/MapReduce and Spark); know their limitations
• understand the challenges associated with stream processing, and be able to use standard contemporary solutions such as Spark streaming, to address memory and performance constraints;
• acquire the ability to decide, based on a problem description, which contemporary or non-standard data management platform (a big data ecosystem, a relational database, or even a solution adapted for data streams) is more appropriate for solving a problem.
• acquire problem-solving skills related to big data; discuss the suitability of different solutions for different contexts, combine different strategies, and propose alternatives; identify difficult problems, and know how to design solutions.