Meet MPCDF: An introduction to the MPCDF LLM Inference Service

MPCDF provides the LLM Inference Service at llm.mpcdf.mpg.de, a convenient web UI for spinning up LLM inference endpoints on MPCDF-operated HPC systems. The service handles submitting and managing Slurm jobs and, once the model is running, provides a routed inference endpoint accessible from your local machine. This allows researchers to run open-source language models within MPCDF infrastructure, without exposing sensitive data to external providers. In this talk, we will introduce the service and its use cases, walk through spinning up and querying an inference endpoint, and discuss when it is the right tool - and when it is not.

Meet MPCDF: A seminar of this series is given on the first Thursday of the month, from 15:30 to 16:30, and features a technical talk (ca. 20-30 minutes) about an AI, HPC or data-related topic given by a staff member of the MPCDF. In addition, this meeting offers the opportunity for the users to informally interact with MPCDF staff, in order to discuss relevant kinds of technical topics. Optionally, questions or requests for specific topics to be covered in more depth can be raised in advance via email to training@mpcdf.mpg.de.

Target audience are intermediate to advanced users of MPCDF services as well as computational scientists and software developers of the MPG. Users seeking a basic introduction to MPCDF services are referred to our semi-annual online workshop "Introduction to MPCDF services" (see above) which offers a consistent introduction for new users of the MPCDF.

A registration is not necessary, just use the link on the left which directly connects you to the Zoom room of the event. The explicit connection details are

Meeting ID: 936 3550 3145

Passcode: 622791