Data Integration Engineer

We are looking for a talented software engineer with a strong background in data to join our team. In this role, you will work embedded with our core development team, acting as the bridge between complex data acquisition and our software applications.

Key Responsibilities:

• Software Engineering Best Practices: Ensure data codebases follow strict software development standards.
• Internal API Development: Work closely with the software engineering team to design and build robust internal APIs that serve structured data and search capabilities directly to our backend and frontend applications.
• Data Acquisition & Scraping: Design, scale, and maintain robust web scraping pipelines to collect vast amounts of data.
• Data Analytics: Ability to understand which data or types of data is needed to support certain business processes.
• Data Pipelines & ETL Architecture: Build, manage, and evolve reliable data pipelines. You will work with our custom internal orchestration tools, handling parsing, deduplication, and stratification of unstructured text data into clear automotive taxonomies.
• Database & Search Engineering: Design scalable database schemas and engineer structured data specifically to power a high-performance search engine.

Required Skills & Qualifications:

• Experience: 3+ years of hands-on experience as a Backend Engineer, Data Engineer, or in a similar data-heavy software role. Startup experience is a major plus.
• Core Languages & Frameworks:
• Strong proficiency in Python and its backend/data ecosystem.
• Experience building and integrating with robust RESTful APIs (using FastAPI, Flask, or similar).
• Web Scraping: Proven, extensive experience with web scraping libraries, frameworks, and browser automation (Scrapy, BeautifulSoup, Playwright, Selenium).
• Databases & Search:
• Solid understanding of relational databases and SQL (PostgreSQL).
• Hands-on experience structuring data for and querying search-oriented NoSQL databases (Elasticsearch).
• Event-Driven Architecture: Knowledge of or experience with message brokers and event streaming (Ex: Kafka, RabbitMQ, AWS SQS).
• Observability: Familiarity with monitoring and observability tools (Prometheus, Grafana, Datadog).

Application Question(s):

• Do you have hands-on experience building and integrating robust RESTful APIs using frameworks like FastAPI, Flask, or Django?
• Which web scraping and browser automation tools do you have proven experience using in a production environment?
• Briefly describe the most complex web scraping challenge you have solved?
• Which message brokers or event streaming platforms have you worked with previously?
• Are you familiar with setting up or using observability and monitoring tools (such as Prometheus, Grafana, or Datadog) to troubleshoot production issues?
• Please provide a brief overview of a recent backend project where you integrated Python, web scraping, and a database like PostgreSQL or Elasticsearch. What was your specific role?

Experience:

• Backend, Data Engineer or similar data-heavy software: 3 years (Required)
• Python: 3 years (Required)

Work Location: Remote