Business Area:
EngineeringSeniority Level:
DirectorJob Description:
At Cloudera, we empower people to transform complex data into clear and actionable insights. With as much data under management as the hyperscalers, we're the preferred data partner for the top companies in almost every industry. Powered by the relentless innovation of the open source community, Cloudera advances digital transformation for the world’s largest enterprises.
Principal Engineer - Semantic Search
Cloudera is looking for a Principal Engineer to join the Search engineering team. This team is responsible for data management, search and discovery of unstructured data for AI Applications. The goal is also to make semantic search capability accessible to every organization. The current search engine helps drive the development of our next-generation Generative AI Toolsto troubleshoot and auto resolve Cloudera Cloud and on-prem product related technical issues.
We are looking for a principal engineer to architect a backend data and modeling platform based on unstructured data embedding to support the sharing, management, and access of state-of-art pre-trained model libraries and datasets. To be successful in this position you will need to have a deep interest in AI solutions and world-class engineering skills. In this role, you will have the opportunity for providing technical guidance for the engineering team while improving capabilities in the platform. You will also get a chance to interact with our customers to co-innovate on solutions. You will also have the opportunity for developing reusable components code that can be used by other members in the team. You can learn and influence clean, contemporary coding practices and be part of developing a great product.
As a Principal Data Engineer, you will:
Define the architecture of the system that can cater to the Semantic Search capabilities as a part of CDP.
You will explore multiple ways in which the vectorized data can be persisted optimally.
Stay up-to-date with the latest advancements in the Semantic Search algorithms and make decisions of integrating new algorithms into the platform.
To be abreast of Generative AI techniques and implementation of new algorithms and architectures as appropriate.
Collaborate with cross-functional teams to identify and prioritize the different business use cases by leveraging the internally available AI models or developing new models to support them.
Develop and implement best practices for Semantic Search engine capabilities, training, and tuning.
Communicate technical concepts and results to both technical and non-technical stakeholders.
We’re excited about you if you have:
BS/BE/BTech, MS/ME/MTech or Ph.D. in Computer Science or related field
15+ years relevant industry experience designing and developing enterprise software with a strong understanding of scaling and performance
Hands-on programmer with strong data structures and algorithms skills. Java experience required.
Excellent debugging skills across various components in a n-tiered stack.
Ability to convert a Marketing PRD/Feature description into a technical design document.
Mentor junior members of the team and delegate project tasks.
Strong oral and written communication skills.
Ability to work effectively on cross-functional projects.
Experience with one of Apache HBase, Phoenix and one of Lucene, Solr , or Open Search open source projects or an RDBMS
Experience with the Hadoop ecosystem
Experience with public cloud infrastructures such as Microsoft Azure, Amazon Web Services and Google Cloud Platform
What you can expect from us:
Generous PTO Policy
Support work life balance with Unplugged Days
Flexible WFH Policy
Mental & Physical Wellness programs
Phone and Internet Reimbursement program
Access to Continued Career Development
Comprehensive Benefits and Competitive Packages
Employee Resource Groups
EEO/VEVRAA
#LI-NK1