GPU MODE IRL Hackathon - PyTorch Conference Europe side event

Join us to wrap up the PyTorch Conference with a GPU MODE x PyTorch IRL hackathon. You can expect an intimate yet advanced day-long hackathon alongside researchers and engineers working at the bleeding edge of ML systems. Here's what you can expect: Two tracks on distributed training and inference optimization Access to a B300 cluster from Verda and H200 instances from Sesterce Cloud credits as prizes, including 48-hour access to a GB300 NVL72 rack Talks from PyTorch (Helion), vLLM, Prime Intellect, and more Food and refreshments More details on the tracks, teams, and prizes below ⬇️ Schedule 9:30 - Doors open 10:00 - Kick off 11:00 - Tasks start 19:00 - Tasks end 20:00 - Closing ceremony 00:00 - Doors close Tracks 1st track: LLM training on B300 cluster by Verda Task: Pre-train an LLM from scratch in a limited time on a B300 cluster. You will participate as a team, trying to make optimal use of 360 PFLOP/s (BF16). Will this require asynchronous optimizer steps, large-batch size training, models much larger than chinchilla, or something else entirely? Compute: Your team will get a B300 node for development and a set time on a B300 cluster. Teams: Come with a pre-formed team or get matched to a team on-site The expected team sizes are 2-4 Prizes: 🥇 1st place: 48 hours on GB300 NVL72 + 2,500 EUR in cloud credits 🥈 2nd place: 2,000 EUR in cloud credits 🥉 3rd place: 1,500 EUR in cloud credits 2nd track: LLM inference on H200 GPUs by Sesterce Task: Compete as a team on a leaderboard for the fastest inference for a fixed model. Compute: Your team will get access to 4x H200s for the duration of the hackathon. Teams: Similarly, you can come with a pre-formed team or get matched on-site. Prizes: 🥇 1st place: 5,000 EUR in cloud credits 🥈 2nd place: 2,000 EUR in cloud credits 🥉 3rd place: 1,500 EUR in cloud credits Lightning talks Speakers, topics, and times TBA soon! Sponsors PyTorch Prime Intellect Verda Sesterce SemiAnalysis