Data Engineer

• Build and maintain data pipelines from various data sources.
• Develop and maintain web crawling/scraping processes to collect data.
• Perform ETL (Extract, Transform, Load) processes to prepare data for further use.
• Clean, transform, and validate data to ensure good data quality.
• Develop scripts to process documents and generate metadata from PDF files.
• Monitor and troubleshoot data crawling and processing processes.
• Automate data processing workflows using tools such as Airflow, n8n, or similar tools.
• Collaborate with the team to support data needs for applications, analytics, and AI-related projects.

Qualification:

• Bachelor’s degree in computer science, mathematics, statistics, or related fields.
• Minimum 1 Years experience as Data engineer, Data Processing, Web Scarping, or a related field.
• Proficient in SQL and at least one programming language, preferably Python.
• Familiar with web crawling/scraping and data cleaning. • Familiar with ETL/data pipeline processes.
• Familiarity with Airflow, n8n, or similar tools.
• Familiar with relational databases such as PostgreSQL or MySQL.
• Experience with PDF/document processing is a plus.
• Strong analytical and problem-solving skills.
• Able to work independently and within a team.
• Willing to continuously learn and grow.