Internship - MScAC 2027

Research Projects

• Audio data extraction and annotation. Our speech models learn from very large amounts of audio in 100+ languages. You will build pipelines that collect, segment, transcribe and annotate audio with as little manual engineering as possible, using unsupervised and semi-supervised methods, and you will close the loop so that annotation and model training improve each other (see NVIDIA Granary, arXiv:2505.13404). This suits you if you enjoy scripting, experiments and large datasets, and have a working knowledge of statistics and machine learning.
• Audio evaluation and benchmarking. Good speech is more than correct pronunciation. It has to fit the context, emotion and style, sound natural in a conversation, and match the voice it clones. You will design benchmarks and automatic metrics for expressive, multilingual and conversational speech, for voice cloning, and for speech recognition under the conditions real voice agents face (see our EmergentTTS-Eval, arXiv:2505.23009, and Back to Basics: Revisiting ASR in the Age of Voice Agents, arXiv:2603.25727), and use them both to evaluate our models and as signals for training them. This suits you if you like experimentation, have a good ear, ideally speak more than one language, and are comfortable automating evaluation in Python.
• Video generation and computer vision. Higgs Avatar turns a single image into a streaming, talking video. You will work on the data pipeline behind it (collection, multi-stage filtering and synthesis) and on the models: audio-driven video generation with accurate lip sync and natural expressions, distilled diffusion models that run in real time, and computer vision that lets our models see and react to the person they are talking to (see Wan-S2V, arXiv:2508.18621, and LTX-Video, arXiv:2501.00103). This suits you if you have a background in image or video generation (diffusion models, VAEs, GANs, NeRFs) or in computer vision, and are comfortable with large-scale data and model optimization.
• Efficient training. You will make training on our 900-GPU cluster faster and more robust: numerical formats (FP8, FP4, BF16), fused CUDA or Triton kernels for architectures that go beyond standard Transformers, overlapping communication with computation, faster checkpointing and recovery from failures, newer optimizers such as Muon, and stable reinforcement-learning post-training when the training and rollout systems differ (see our Let the Batch Decide, arXiv:2605.12380). This suits you if you enjoy systems work and optimization, like writing CUDA or Triton, and want experience training across hundreds of GPUs. Strong coding skills are required.
• Serving and real-time inference. Higgs Realtime responds in under a second, and Higgs Avatar streams live video into calls. You will make our streaming speech and video models faster and cheaper to serve, on our own GPUs and in the cloud, using SGLang or vLLM: 8-bit and 4-bit quantization, speculative decoding, kernel and memory-layout optimization, KV-cache management across many concurrent users, and scaling capacity with demand (see our Alignment Drift in Single-Model Speculative Decoding for ASR, arXiv:2608.12703, and TARQ, arXiv:2605.27808). This suits you if you enjoy systems work, have some experience with GPU hardware and CUDA or Triton, and want to work with large numbers of GPUs. Strong coding skills are required.
• Agents: conversational design, theory of mind and reinforcement learning. Our voice agents handle customer support, sales and training calls for businesses. They must follow business rules exactly while sounding natural, keep track of context, and adapt to the person on the other end. Projects include declarative dialog frameworks that separate workflow state from language generation and keep agents on track through multi-step workflows, even after the user interrupts (see our IHBench, arXiv:2606.19595); theory of mind, so that agents model the user's state of mind and act on needs the user has implied but not said (see our ProactBench, arXiv:2605.09228); and reinforcement learning that improves agents from real call traces (see Flow-GRPO, arXiv:2510.05592). This suits you if you are a rigorous logical and systems thinker, know the basics of LLMs and reinforcement learning, and are curious about human behavior. Experience with dialog systems or human-computer interaction is a plus, and so is a love of storytelling or role-playing games.
• Full-duplex conversation, for audio and audio-video. People talk over each other, nod, say "mm-hm" and react while they listen. Turn-based voice AI does none of this. You will work on full-duplex models that listen and speak at the same time, handling interruptions, backchannels and turn-taking, following instructions about when to speak, and keeping up with several people in one conversation: first for speech, then for audio-video, where the avatar also has to look like it is listening and react in real time (see our Instruct-FD, arXiv:2607.20460, and MSI-Bench, arXiv:2609.24812). This suits you if you have a background in speech, sequence modeling or multimodal learning, and want to work on an open research problem with a direct path into our products.

Minimum Qualifications

• Enrolled in the MScAC program at the University of Toronto. This is mandatory, and applications from anyone else are automatically declined
• Strong programming skills in Python, plus C++, CUDA or Triton for the systems projects
• Solid foundations in machine learning and deep learning, with hands-on experience in PyTorch or JAX
• Careful experimental practice: you design experiments, measure what matters, and report results honestly
• Available for the full 8-month internship and able to work from our Toronto office on a hybrid schedule

Preferred Qualifications

• Research, coursework or project experience in one of our project areas: speech and audio, video generation, computer vision, LLM agents, reinforcement learning, or ML systems
• Papers, open-source contributions or GitHub projects you can point to
• Experience with large datasets or multi-GPU training
• Fluency in a language besides English. Our models speak 100+ of them

Internship Details

• 8-month applied research internship, starting in May 2027. An earlier start is possible
• Based in our Toronto office on a hybrid schedule
• Compensation: CAD 70,000–100,000 for the 8-month term
• In your application, tell us which one to three projects interest you most