Meet MPCDF

Distributed Training of Neural Networks

  • Date: Jun 1, 2023
  • Time: 03:30 PM - 04:30 PM (Local Time Germany)
  • Speaker: Members of the MPCDF
  • Location: online via Zoom
  • Host: MPCDF
  • Contact: training@mpcdf.mpg.de
The series "Meet MPCDF" offers the opportunity for the users to informally interact with MPCDF staff, in order to discuss relevant kinds of technical topics.

Neural networks have emerged as powerful tools for solving complex problems across various domains.
However, training large neural networks on massive datasets is a time-consuming process. An effective approach to overcome this challenge is to distribute the training process across multiple nodes or GPUs. In this presentation, we discuss distributed training techniques like data parallelism and model parallelism, and we address challenges such as communication overhead and synchronisation. We also highlight popular frameworks like Horovod, PyTorch and PyTorch Lightning and their role in facilitating distributed training. Real-world applications that benefit from distributed training are showcased, demonstrating the impact on performance and accuracy.

Meet MPCDF: A seminar of this series is given on the first Thursday of the month, from 15:30 to 16:30, and features a technical talk (ca. 20-30 minutes) about an AI, HPC or data-related topic given by a staff member of the MPCDF. In addition, this meeting offers the opportunity for the users to informally interact with MPCDF staff, in order to discuss relevant kinds of technical topics. Optionally, questions or requests for specific topics to be covered in more depth can be raised in advance via email to training@mpcdf.mpg.de.

Target audience are intermediate to advanced users of MPCDF services as well as computational scientists and software developers of the MPG. Users seeking a basic introduction to MPCDF services are referred to our semi-annual online workshop "Introduction to MPCDF services" (see above) which offers a consistent introduction for new users of the MPCDF.

A registration is not necessary, just use the link on the left which directly connects you to the Zoom room of the event. The explicit connection details are

Meeting ID: 936 3550 3145

Passcode: 622791

Go to Editor View