COSC440-20S2 (C) Semester Two 2020

Deep Learning

15 points

Details:
Start Date: Monday, 13 July 2020
End Date: Sunday, 8 November 2020
Withdrawal Dates
Last Day to withdraw from this course:
  • Without financial penalty (full fee refund): Friday, 24 July 2020
  • Without academic penalty (including no fee refund): Friday, 25 September 2020

Description

This course introduces students to theoretical foundational concepts of deep neural networks. The focus of this course is on both fundamental theoretical and applied methods in deep neural networks. A range of topics from convolutional and recurrent type networks to neural-network generative models and attention mechanisms will be introduced.

The description text above is for a previous occurrence. For 2020 this course introduces students to the core concepts of deep neural networks. The course focuses on the computational process of problem formulation, model selection and design, implementation, analysis, and refinement for deep neural networks. We analyze a range  of advanced neural network designs with transformative results in computer vision, natural language, anomaly detection, molecular design, and deep fakes. Students build competency in the theory and practice of creating deep neural network applications and will research, propose, and implement their own deep learning network for a given application domain (2020 S2 domain: astrophysics).

Learning Outcomes

  • Understand the fundamental algorithms of deep learning systems such as backpropagation
  • Show competency in programming deep learning models
  • Know which model architectures to use for processing different types of data (images, sequences, and graphs)
  • Train neural networks through supervised learning, unsupervised learning, and reinforcement learning
  • Analyze advanced network designs in major problem domains such as generative models for creating synthesized (fake) text, images, video, and audio
  • Refine and engineer models for domain constraints such as performance in edge systems
  • Research, propose, and implement a project involving state of the art deep learning networks for a given application domain (2020 S2 domain: astrophysics)

Pre-requisites

(1) COSC480 or
equivalent and at least 30 points at 200-300 level MATH/EMTH/STAT; (2) Subject to approval of the Head of Department of the Computer Science and Software Engineering.

Timetable 2020

Students must attend one activity from each section.

Lecture A
Activity Day Time Location Weeks
01 Wednesday 16:00 - 18:00 Jack Erskine 315 13 Jul - 23 Aug
7 Sep - 18 Oct
Lab A
Activity Day Time Location Weeks
01 Friday 17:00 - 18:00 Jack Erskine 131 Lab 1 13 Jul - 23 Aug
7 Sep - 18 Oct
02 Friday 09:00 - 10:00 Jack Erskine 131 Lab 1 13 Jul - 23 Aug
7 Sep - 18 Oct

Course Coordinator

James Atlas

Assessment

Assessment Due Date Percentage  Description
Test 10%
Assignments 20% 4 Assignments
Research Project 30%
Exam 40%

Notes

The prerequisites listed on this page are for a previous occurrence. For 2020, students can be approved into this course who have passed at least:

(1) COSC262 and (2) 30 points of 300-level COSC/SENG/DATA

Additional Course Outline Information

Syllabus

Machine learning concepts
Types of learning (supervised, self-supervised, reinforcement)
Types of problems (classification, regression)
Loss functions, gradient descent and optimization
Automatic differentiation, forward and backward mode
Diagnosing problems: under and over-fitting, regularization, initialization
Deep learning concepts
Multi-dimensional arrays and memory models, views and vectorized operations
Neural networks: perceptron, layers, types of operations (linear, convolutional, pooling, sampling, nonlinearity), visualization
Sequential and recurrent networks
Transfer learning, synthesis, ensemble networks
Deep learning problems, models, and research
Computer Graphics and Vision
object detection, segmentation, image retrieval, face reidentification
feature pyramid networks, similarity learning, discriminatory networks, adversarial networks, generative networks
Natural language
parsing, window prediction, generation, translation
encoders/decoders, latent space, autoencoders
transformers, long short-term memory, attention
Audio and Video Synthesis
Text-to-speech, music generation, deep fakes, semantic models
Time series forecasting, autoregression, dilated convolution, few-shot learning
Search using deep reinforcement learning
Molecular design, game playing
neural network architecture design, compression, quantization
Agents, Markov decision processes, Monte Carlo, policy gradient methods
Anomaly detection
Intrusion detection, fraud detection, scientific discovery
One-class neural networks, zero shot learning
Irregular networks
Recommender systems, molecular structure and property prediction
Graph convolutional networks, point cloud processing, spatial-temporal networks

Indicative Fees

Domestic fee $1,022.00

International fee $4,800.00

* Fees include New Zealand GST and do not include any programme level discount or additional course related expenses.

For further information see Computer Science and Software Engineering.

All COSC440 Occurrences

  • COSC440-20S1 (C) Semester One 2020 - Not Offered
  • COSC440-20S2 (C) Semester Two 2020