Hands-on Introduction to Deep Learning with PyTorch

Course Description

This course is intended as a quick introduction of fundamental concepts of deep learning, covering neural network basics, training methods, as well as a few examples of specific applications such as convolutional neural networks for computer vision and the transformer model for natural language processing.

More specifically, the following topics will be covered:

Fundamentals of neural networks.
Training deep learning models: The stochastic gradient decent, optimizers, loss functions, regularization, etc.
Convolutional Neural Networks (CNNs) for computer vision: Basics of CNNs. Image classification and generation.
Natural Language Processing (NLP) with transformers: Basics of NLP, the transformer model, attention mechanism, etc.

The lessons will blend theory with hands-on practice, using PyTorch for practical exercises. We will run these sessions on the Piz Daint supercomputer at CSCS.

Target Audience

This course is intended for researchers who are starting to use deep learning in their work and would like a fast introduction to the field. The goal of the course is to provide the basic concepts and some pointers so the participants can continue their deep learning journey by themselves.

Although PyTorch will be used for the practical sessions, the focus of the course is on the deep learning concepts that can be applied with other frameworks (such as TensorFlor or JAX). Participants are required to be familiar with Python (including basic OOP) and some library for numerical computation, linear algebra, and array programming such as NumPy.

Schedule

The course will start on Wednesday, February 28, 2024, at 10:00 and end on Friday, March 1, at 16:30 Central European Time (CST).

Wednesday, February 28
10:00 - 12:00 Introduction to Neural Networks
12:00 - 13:00 Lunch
13:00 - 16:45 Convolutional Neural Networks

Thursday, February 29
09:00 - 12:00 Convolutional to Neural Networks (continuation)
12:00 - 13:00 Lunch
13:00 - 16:45 Training Neural Networks

Friday, March 1
09:00 - 12:00 Natural Language Processing with transformers (continuation)
12:00 - 13:00 Lunch
13:00 - 14:30 Natural Language Processing with transformers (continuation)
15:00 - 15:30 Tour of the CSCS machine room
15:30 - 16:30 Natural Language Processing with transformers (continuation)

There will be a coffee break in the morning (day 2 and 3) and one in the afternoon around 10:30 and 14:30, respectively.

Instructors

Dr. Rocco Meli (Research Software Engineer, CSCS)
Dr. Rafael Sarmiento (HPC Application Engineer, CSCS)

Participation Fee and Registration

All participants must register for the course. The registration fee includes lunches and coffee breaks.

Course Fee: CHF 240.00

Deadline for registration: Wednesday, February 14, 2024

Kindly note that the course can take place only if there are sufficient confirmed registrations received by the deadline. The minimum number of participants is eight. Should the minimum number of participants not be reached by the deadline, registered attendees will be informed about the course cancellation. No communication will be sent to confirm the event. Registration for the course will automatically close when we reach the maximum number of participants (30).

Directions

Kindly note that no parking space is available at the Swiss National Supercomputing Centre. There is a Park & Ride Resega parking lot, within five minutes walk from CSCS.

You are encouraged to travel by public transportation. The closest bus stop to CSCS is "Lugano, Stadio." From Lugano railway station, you should take bus number 4 or 6.

Suggestions regarding travels and accommodation are available here.

Inquiries

Please contact Rafael Sarmiento (rafael.sarmiento@cscs.ch) for questions related to the course content, and info@cscs.ch for questions related to the event logistics.