About the Role
PubMatic is seeking a Data Analytics-focused Principal Software Engineer with experience in building and optimizing AI agents, including strong skills in Hadoop, Spark, Scala, Kafka, Spark Streaming, and cloud-based solutions, with proficiency in programming languages such as Scala and Python. You will be responsible for developing advanced
AI agents that enhance our data analytics capabilities, enabling our platform to handle complex information retrieval, contextual understanding, and adaptive interactions, ultimately improving our data-driven insights and analytical workflows.
What You'll Do:
- Build, design, and implement our highly scalable, fault-tolerant big data platform to process terabytes of data and provide customers with in-depth analytics.
- Develop backend services using Java, REST APIs, JDBC, and AWS.
- Build and maintain Big Data pipelines using technologies like Spark, Hadoop, Kafka, and Snowflake.
- Architect and implement real-time data processing workflows and automation frameworks.
- Lead multiple projects to develop features for data processing and reporting platforms, and collaborate with product managers and cross-functional teams.
- Collaborate with functional teams to build products to deliver end-to-end products and features, and fix bugs for better performance.
- Design and develop GenAI-powered agents for analytics, operations, and data enrichment use cases using frameworks like LangChain, LlamaIndex, or custom orchestration systems.
- Integrate LLMs (e.g., OpenAI, Claude, Mistral) into existing services for query understanding, summarization, and decision support.
- Manage end-to-end GenAI workflows, including prompt engineering, fine-tuning, vector embeddings, and retrieval-augmented generation (RAG).
- Working closely with cross-functional teams on improving the availability and scalability of large data platforms and the functionality of PubMatic software.
- Participate in Agile/Scrum processes such as sprint planning, sprint retrospective, backlog grooming, user story management, and work item prioritization.
- Frequently discuss with product managers about the software features to include in the PubMatic Data Analytics platform.
- Support customer issues over email or JIRA (bug tracking system), provide updates, and patches to customers to fix the issues.
- Perform code and design reviews for code implemented by peers or as per the code review process.
We'd Love for You to Have:
- 6+ years of coding experience in Java and backend development.
- Solid computer science fundamentals, including data structure and algorithm design, and creation of architectural specifications.
- Experience in developing the implementation of professional software engineering best practices for the full software development life cycle, including coding standards and code reviews.
- Hands-on experience with Big Data tools and systems like Scala Spark, Kafka, Hadoop, and Snowflake.
- Proven experience in building GenAI applications, including:
o LLM integration (OpenAI, Anthropic, Cohere, etc.)
o LangChain, or similar agent orchestration libraries
o Prompt engineering, embedding, and retrieval-based generation (RAG) - Experience in developing and deploying scalable, production-grade AI or data systems.
- Ability to lead end-to-end feature development and debug distributed systems.
- Experience in developing and delivering large-scale big data pipelines, real-time systems & data warehouses would be preferred.
- Demonstrated ability to achieve stretch goals in a very innovative and fast-paced environment.
- Demonstrated ability to learn new technologies quickly and independently.
- Excellent verbal and written communication skills, especially in technical communications.
- Strong interpersonal skills and a desire to work collaboratively.
Qualification:
Should have a bachelor’s degree in engineering or an equivalent degree from a well-known institute/university.
Additional Information: