Want to know if this job is worth applying to?
Reston, Virginia, United States
Hybrid
Babel Street is the trusted technology partner for the world’s most advanced identity intelligence and risk operations. We deliver advanced AI and data analytics solutions providing unmatched, analysis-ready data regardless of language, proactive risk identification, 360-degree insights, high-speed automation, and seamless integration into existing systems. Babel Street empowers government and commercial organizations to transform high-stakes identity and risk operations into a strategic advantage. The actionable insights we deliver safeguard lives and protect critical assets around the world. Babel Street is headquartered in Reston, Virginia, with regional offices in Boston, MA and Cleveland, OH, and international offices in Australia, Canada, Israel, Japan, and the U.K. For more information, visit www.babelstreet.com.
ROLE SUMMARY:
The primary purpose of this Software Engineer III position is to architect, develop, and maintain advanced automated data extraction systems (spiders) that harvest critical business intelligence from complex web environments. This role ensures the continuous, reliable flow of high-quality data into our internal databases by engineering solutions to overcome sophisticated technical barriers and anti-bot security measures. Candidates near our Somerville, MA or Reston, VA offices are preferred but remote work it potential for the ideal candidate.
ROLE FOCUS;
As a Software Engineer III specializing in data extraction, you will be responsible for the end-to-end lifecycle of web-based data collection. This includes designing scalable crawling architectures, reverse-engineering web applications to identify data points, and implementing evasion techniques to bypass IP rate-limiting and bot detection. You will also manage the storage and integrity of this data using advanced SQL and relational database management.
We are looking for a Senior Python Engineer with a "hacker" mindset to join our team as a Software Engineer III. This role is dedicated to large-scale web scraping and data harvesting. If you have deep experience with Scrapy or Playwright, know how to defeat Cloudflare or DataDome, and can write high-performance SQL to manage millions of records, we want to hear from you. This is a specialized role for an engineer who enjoys reverse-engineering the web to unlock data.
The primary purpose of this Software Engineer III position is to architect, develop, and maintain advanced automated data extraction systems (spiders) that harvest critical business intelligence from complex web environments. This role ensures the continuous, reliable flow of high-quality data into our internal databases by engineering solutions to overcome sophisticated technical barriers and anti-bot security measures.
Responsibilities:
Requirements:
Education & Experience:
Benefits at Babel Street (just to name a few...)
Babel Street is an equal opportunity/affirmative action employer. All qualified applicants will receive consideration for employment without regard to sex, gender identity, sexual orientation, race, color, religion, national origin, disability, protected Veteran status, age, or any other characteristic protected by law. Further, Babel Street will not discriminate against applicants for inquiring about, discussing or disclosing their pay or, in certain circumstances, the pay of their co‐worker, Pay Transparency Nondiscrimination. In addition, Babel Street's policy is to provide reasonable accommodation to qualified employees who have protected disabilities to the extent required by applicable laws, regulations and ordinances where a particular employee works. Upon request, we will provide you with more information about such accommodations.