Want to know if this job is worth applying to?
India
Remote only
This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Specialist in Machine Learning Engineering based in India.
As a Specialist in Machine Learning Engineering, you will help build, operate, and scale the infrastructure powering advanced AI services and applications. You will work at the intersection of Site Reliability Engineering, distributed systems, and machine learning infrastructure, ensuring that production AI workloads remain reliable, secure, scalable, and cost-efficient. Your responsibilities will span large-scale LLM inference, model serving, vector databases, and retrieval-augmented generation (RAG) platforms. You will also drive automation, improve system observability, and optimize GPU resources to enhance performance and availability. Working closely with AI researchers, software engineers, and platform teams, you will contribute to the development of resilient AI infrastructure in a dynamic, hybrid multi-cloud environment.
You will take ownership of the reliability, performance, and operational excellence of critical AI infrastructure, helping ensure that large-scale machine learning services operate efficiently in production. Your work will combine hands-on engineering, automation, incident management, and cross-functional collaboration.
Operate, scale, and optimize multi-tenant LLM inference, model-serving, VectorDB/RAG, and Model Context Protocol (MCP) platforms across global cloud and GPU clusters.
Establish and monitor Service Level Objectives (SLOs), Service Level Indicators (SLIs), automated health checks, and observability dashboards using tools such as Prometheus, Grafana, Splunk, and OpenTelemetry.
Develop automation for deployment rollouts, autoscaling, canary releases, self-healing systems, and disaster recovery to reduce operational overhead.
Optimize GPU utilization and inference performance by improving throughput, latency, model scheduling, batching, caching, and resource allocation using Kubernetes, vLLM, and Ray.
Lead complex production incident responses, conduct root-cause investigations using methodologies such as 5 Whys, and implement long-term solutions to prevent recurring issues.
Collaborate with AI researchers, product developers, and platform engineering teams to conduct production readiness reviews and ensure reliable service delivery.
Participate in a sustainable on-call rotation and continuously improve operational practices, system resilience, and service availability.
Requirements:
The ideal candidate combines strong software engineering expertise with practical experience operating distributed, cloud-native systems in production environments. You should have a solid understanding of SRE principles, observability, and infrastructure automation, alongside an interest in the challenges of deploying and scaling modern AI workloads.
Bachelor's degree in Computer Science, Engineering, or a related field, with at least 5 years of relevant experience, or a Master's degree with at least 3 years of experience.
At least 3 years of software development experience using Python, Go, or Java, particularly in building automation tools, infrastructure solutions, or production backend services.
Strong experience operating distributed and containerized production systems using Kubernetes, microservices architectures, and CI/CD pipelines.
Proven background in Site Reliability Engineering, production operations, distributed systems troubleshooting, and observability.
Practical experience with monitoring and observability tools such as Prometheus, Grafana, Splunk, or OpenTelemetry.
Working knowledge of at least one major cloud platform: AWS, Microsoft Azure, or Google Cloud Platform (GCP).
Hands-on experience with LLM inference and serving frameworks such as vLLM, TensorRT-LLM, Ray Serve, or KServe is highly desirable.
Familiarity with GPU orchestration, capacity planning, model parallelism, and performance profiling.
Experience managing vector databases, search engines, or distributed caching solutions such as Milvus, Pinecone, or Redis.
Understanding of generative AI architectures, including RAG pipelines, model routing, Model Context Protocol (MCP), and agent runtimes.
Experience designing highly available multi-region systems, automated remediation solutions, and chaos engineering practices is an advantage.
Strong problem-solving, analytical, communication, and collaboration skills, with the ability to work effectively across multidisciplinary engineering teams.
Benefits:
Fully remote working arrangement.
Full-time employment opportunity.
Opportunity to work on large-scale AI infrastructure and advanced generative AI technologies.
Exposure to distributed systems, GPU computing, and modern machine learning platforms.
Collaboration with experienced professionals across AI research, software development, and platform engineering.
Opportunities for continuous technical learning, innovation, and professional development.
The opportunity to contribute to mission-critical AI services with global reach and impact.
