Special Topic

Description

Title: Data472 - Data Engineering

The course offers an introduction to the most common concepts and work practices in Data Engineering. Moreover, it tackles advanced and professional topics in Data Wrangling, beyond what covered in more introductory courses such as Data201/422 Data Wrangling. The aim of the course is to offer the students provable understanding and coding experience of designing, implementing, and mantaining complex data ingestion, curation, and publication pipelines. Data472 takes a vertical data science approach, providing the student the opportunity to manage a data science project from start to end.
Topics included in the course are:
• Handling of streaming data sources and sinks
• AI/Deep Learning model input and output data
• Large Volume Audio/Video data
• Data Wrangling task automation
• RESTful APIs for data publication
• ETL/ELT
• Network Data
Technologies used in the course span include both open source (Julia/R/Python, Java/Scala, Postgres, Apache Kafka, Apache Flink, ...) and proprietary (Snowflake, ...).
The course assessment is project-centric and will encourage the student to work on both an individual and a group projects. These projects will be handled through github, and are designed so that they can become part of the public portfolio of the students.

Prerequisites:
Knowledge of a scientific programming language, and familiarity with introductory data wrangling topics (ideally, what covered by Data201/422) are strongly suggested.