Join us to wrap up the PyTorch Conference with a GPU MODE x PyTorch IRL hackathon.
You can expect an intimate yet advanced day-long hackathon alongside researchers and engineers working at the bleeding edge of ML systems.
Here's what you can expect:
Two tracks on distributed training and inference optimization
Access to a B300 cluster from Verda and H200 instances from Sesterce
Cloud credits as prizes, including 48-hour access to a GB300 NVL72 rack
Talks from PyTorch (Helion), vLLM, Prime Intellect, and more
Food and refreshments
More details on the tracks, teams, and prizes below ⬇️
Schedule
9:30 - Doors open
10:00 - Kick off
11:00 - Tasks start
19:00 - Tasks end
20:00 - Closing ceremony
00:00 - Doors close
Tracks
1st track: LLM training on B300 cluster by Verda
Task: Pre-train an LLM from scratch in a limited time on a B300 cluster. You will participate as a team, trying to make optimal use of 360 PFLOP/s (BF16). Will this require asynchronous optimizer steps, large-batch size training, models much larger than chinchilla, or something else entirely?
Compute: Your team will get a B300 node for development and a set time on a B300 cluster.
Teams:
Come with a pre-formed team or get matched to a team on-site
The expected team sizes are 2-4
Prizes:
🥇 1st place: 48 hours on GB300 NVL72 + 2,500 EUR in cloud credits
🥈 2nd place: 2,000 EUR in cloud credits
🥉 3rd place: 1,500 EUR in cloud credits
2nd track: LLM inference on H200 GPUs by Sesterce
Task: Compete as a team on a leaderboard for the fastest inference for a fixed model.
Compute: Your team will get access to 4x H200s for the duration of the hackathon.
Teams: Similarly, you can come with a pre-formed team or get matched on-site.
Prizes:
🥇 1st place: 5,000 EUR in cloud credits
🥈 2nd place: 2,000 EUR in cloud credits
🥉 3rd place: 1,500 EUR in cloud credits
Lightning talks
Speakers, topics, and times TBA soon!
Sponsors
PyTorch
Prime Intellect
Verda
Sesterce
SemiAnalysis
