Want to know if this job is worth applying to?
województwo mazowieckie, Polska
Onsite
GT was founded in 2019 by a former Apple, Nest, and Google executive. GT’s mission is to connect the world’s best talent with product careers offered by high-growth companies in the UK, USA, Canada, Germany, and the Netherlands.
On behalf of the client, GT is looking for a Senior ML Engineer / Applied ML Engineer who is interested in solving complex matching and search problems at scale, applying ML and algorithms to hundreds of millions of real-world data records.
Our client is a leading global management consultancy known for tackling some of the world’s most complex business challenges. With a focus on strategy, transformation, and performance improvement, the firm partners with major organizations across industries to drive lasting impact.
Recognized consistently as a top workplace, it combines deep industry expertise with a collaborative, innovative culture. Its centralized European hub plays a key role in supporting operations across the EMEA region, ensuring excellence and efficiency at scale.
We are looking for a Senior ML Engineer / Applied ML Engineer to build and evolve the client’s internal entity resolution system — a core part of the data platform that uses machine learning, LLMs, and search and matching algorithms to turn complex, noisy data into trusted, unified entities.
The system operates at significant scale, processing hundreds of millions of records, and the role will focus on developing and improving the ML models, matching logic, algorithms, and service capabilities behind it.
This is a hands-on engineering role requiring strong Python and SQL, practical ML engineering experience, experience with large-scale data pipelines, and strong algorithmic and problem-solving skills.
This is not a traditional Data Engineering role. The main focus is on building intelligent, production-ready ML systems and solving complex matching and algorithmic problems, with Data Engineering technologies such as Spark/PySpark used to support scalability and productionization.
Entity Matching & Distributed Algorithms
Design and improve entity matching, clustering, and deduplication algorithms at scale
Implement distributed matching approaches such as:
blocking strategies
multi-pass matching pipelines
nearest-neighbor and similarity-based methods
Apply and combine rule-based, statistical, and ML-assisted techniques (including embeddings where relevant)
Optimize candidate generation and scoring to balance accuracy, recall, performance, and cost
Translate algorithmic ideas into scalable implementations using Spark and SQL transformations
Continuously experiment with different approaches and iterate based on performance metrics and results
Large-Scale Analytic Engineering
Design, build, and operate a large-scale analytic system processing hundreds of millions to billions of records.
Implement and optimize Apache Spark pipelines for entity matching, deduplication, and clustering.
Build and maintain complex workflows using Airflow and DBT.
Ensure pipelines are fault-tolerant, observable, and cost-efficient in distributed environments.
SQL & Data Modelling
Develop and maintain analytical SQL models using Snowflake or a similar cloud data warehouse.
Optimize large joins, aggregations, and window functions over very large datasets.
Design data models that support both matching pipelines and downstream consumers.
Data Quality, Validation & Iteration
Build validation logic and metrics to measure match rate, precision, recall, and accuracy.
Support continuous improvements to the matching engine through iterative releases.
Debug and resolve data quality issues across heterogeneous and imperfect data sources.
Core Requirements:
5+ years of software development using Python, ideally in a team lead capacity.
Solid understanding of distributed systems and algorithms (partitioning, shuffles, joins, scalability trade-offs).
Experience building & working with complex data pipelines or data systems
Strong SQL skills, ideally with Snowflake or similar analytical databases.
Strong hands-on experience with Apache Spark – Databricks (PySpark or Scala) in production.
Experience with AI/ML-assisted systems (embeddings, inference, re-ranking).
Matching, Search & Similarity (Required or Willingness to Learn)
Experience with, or strong interest in, fuzzy and semantic matching techniques, such as:
Levenshtein / edit distance
Token-based similarity
BM25 or other lexical ranking methods
Vector embeddings and cosine similarity
Approximate nearest-neighbor or vector search concepts
Strong willingness to learn and apply advanced semantic matching techniques if not already experienced.
Nice-to-Have:
Experience with data orchestration tools such as Airflow (or equivalents).
Experience building entity resolution, deduplication, or record linkage systems.
Familiarity with search or retrieval systems (e.g., Elasticsearch, OpenSearch, vector databases).
Experience operating data pipelines at very large scale (100M+ records).
Background in data quality frameworks, validation automation, or QA at scale.
Experience with Kubernetes or Containerized Functions (e.g., Azure Container Apps, AWS Fargate)
Experience with Serverless Functions
Azure or other Cloud Experience
GitHub Actions
What Success Looks Like
Matching pipelines scale reliably across hundreds of millions to billions of records.
Measurable improvements in matching accuracy, recall, and runtime efficiency.
Faster iteration on matching algorithms without compromising system stability.
A robust, high-quality entity foundation that enables advanced analytics and downstream applications.
Understanding of algorithmic and resource constraints to balance system cost and performance
Interview with GT Recruiter
Cultural Fit Interview
Technical Interview
Final Interview (optional)
Offer
Join a fast-growing, high-impact team in a top-tier company
Contribute to an ambitious effort to create the highest quality, most comprehensive business directory in the world.
Be part of a dinamic-style group within the company that’s redefining how they deliver consulting through productization and data innovation.
Work with cutting-edge data tools, including AI/ML enrichment, semantic matching, and modern cloud-based infrastructure.