We are seeking a seasoned scraping development lead to drive the technical vision of our high-load web scraping platform.
the successful candidate will lead our development team, overseeing architectural decisions and shaping the future of our cutting-edge scraping infrastructure.
key responsibilities:
* data-plane: high-parallel crawler, session management, anti-bot bypass, captcha solver
* control-plane: api, authorization, billing, quotas, monitoring
* infrastructure: microservices, docker/k8s, ci/cd, observability
leadership & team management:
* team building: participate in hiring backend, browser, devops, and qa engineers
* establish development and release processes
* code quality control and technical decision oversight
anti-bot research & bypass:
* browser and network-level fingerprinting (ja3, tls, cdp)
* protection bypass: cloudflare, akamai, qrator, servicepipe, variti, and others
devops / sre:
* ci/cd setup, helm charts, scaling configuration
* monitoring organization (prometheus, grafana, loki) and sla processes
requirements:
* 5+ years of engineering experience in high-load backend/data platforms
* extensive scraping experience (mandatory)
* excellent understanding of http, tls, http/2, proxies, browser stack (cdp, webrtc)
* experience with async python and go (ideally both), redis, queues, message brokers
* advanced level devops tools: kubernetes, helm, prometheus, ci/cd
* experience analyzing and bypassing bot protection systems — strong plus
* fluent english (all documentation and part of the team are english-speaking)
* decision-making skills, mentoring abilities, and accountability for results