Want to know if this job is worth applying to?
Boston, Massachusetts, United States
Onsite
Job Description:
DataRobot delivers AI that maximizes impact and minimizes business risk. Our platform and applications integrate into core business processes so teams can develop, deliver, and govern AI at scale. DataRobot empowers practitioners to deliver predictive and generative AI, and enables leaders to secure their AI assets. Organizations worldwide rely on DataRobot for AI that makes sense for their business — today and in the future.
As a Principal Software Engineer, you’ll be responsible for technical leadership and vision. You’ll lead by example—rolling up your sleeves as a technical contributor to solve complex problems, shaping architecture, and mentoring engineers to do their best work and advance their careers. You’ll work across our control plane systems, influence cross-team roadmaps, and bring pragmatic engineering practices into how we build, test, and operate infrastructure software.
This is not a “stay in your swim lane” role. You’ll question assumptions, challenge complexity, and help drive a high-performance culture. You’ll be trusted to bring clarity where there’s ambiguity, and momentum where there’s inertia.
This role includes participation in an on-call rotation—we believe in shared ownership of our platform and aim to build systems that are resilient, observable, and require minimal intervention.
What You’ll Do:
You will help design, develop, and optimize the inference engine that powers DataRobot's agentic infrastructure API., ensuring large language model (LLM) serving systems are fast, scalable, and efficient. Your work will touch the full GenAI inference stack — from kernels and runtimes to orchestration and memory management.
Contribute to the design and implementation of the inference engine, and collaborate on model-serving stack optimized for large-scale LLMs inference
Collaborate with partners such as NVIDIA to bring new model architectures or features (sparsity, activation compression, mixture-of-experts) into the engine
Optimize for latency, throughput, memory efficiency, and hardware utilization across GPUs, and accelerators
Build and maintain instrumentation, profiling, and tracing tooling to uncover bottlenecks and guide optimizations
Develop and enhance scalable routing, batching, scheduling, memory management, and dynamic loading mechanisms for inference workloads
Integrate with federated, distributed inference infrastructure – orchestrate across nodes, balance load, handle communication overhead
Collaborate cross-functionally: with platform engineers, cloud infrastructure, and security/compliance teams
Document and share learnings, contributing to internal best practices and open-source efforts when possible
What We’re Looking For:
10+ years of engineering experience, with at least 5+ in infrastructure, platform, or backend systems roles.
Deep expertise in Kubernetes internals and operations, including networking, scheduling, scaling, and controller patterns.
Proven ability to design and build systems from scratch, making pragmatic tradeoffs along the way.
Strong proficiency in modern programming languages such as Python or Go. Experience building production-quality, reliable, and observable systems that are used across engineering organizations.
A growth-oriented mindset—driven to teach, learn, and improve systems as well as people.
Experience operating across multiple cloud providers (AWS, GCP, Azure) and/or hybrid environments.
Strong experience with Helm, container orchestration patterns, and CI/CD automation.
Comfortable working with IaC (Terraform, Pulumi) and GitOps workflows.
Ability to influence without authority and align diverse stakeholders around technical decisions
Nice to Have:
Familiarity with Cilium, Kyverno, KEDA, Gateway API, OPA, or similar technologies.
Experience building and running multi-tenant SaaS platforms.
Exposure to on-prem delivery models or regulated environments.
Experience with performance tuning for large-scale data or compute workloads.
Past success driving infrastructure transformation or decomposing legacy systems.
Experience working with GPU infrastructure for training and inference.
The talent and dedication of our employees are at the core of DataRobot’s journey to be an iconic company. We strive to attract and retain the best talent by providing competitive pay and benefits with our employees’ well-being at the core. Here’s what your benefits package may include depending on your location and local legal requirements: Medical, Dental & Vision Insurance, Flexible Time Off Program, Paid Holidays, Paid Parental Leave, Global Employee Assistance Program (EAP) and more!
DataRobot Operating Principles:
All DataRobot hires are required to complete a background check prior to starting employment, which includes identity verification, criminal history check, employment verification and education verification. Additionally, all DataRobot employees must be available to attend in-person company trainings and meetings.
Research shows that many women only apply to jobs when they meet 100% of the qualifications while many men apply to jobs when they meet 60%. At DataRobot we encourage ALL candidates, especially women, people of color, LGBTQ+ identifying people, differently abled, and other people from marginalized groups to apply to our jobs, even if you do not check every box. We’d love to have a conversation with you and see if you might be a great fit.
DataRobot is proud to be an Equal Employment Opportunity and Affirmative Action employer. We do not discriminate based upon race, religion, color, national origin, gender (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. DataRobot is committed to working with and providing reasonable accommodations to applicants with physical and mental disabilities. Please see the United States Department of Labor’s EEO poster and EEO poster supplement for additional information.
All applicant data submitted is handled in accordance with our Applicant Privacy Policy.
