Junior Data Engineer

Key Responsibilities

• Assist in developing and maintaining data pipelines using Databricks.
• Work with Python, PySpark, and SQL for data processing and transformation.
• Assist in developing and maintaining ETL/ELT workflows.
• Work with Delta Lake and Databricks Lakehouse concepts.
• Perform data extraction, transformation, cleansing, and validation.
• Write SQL queries for data transformation, analysis, and data validation.
• Process structured and semi-structured data such as CSV, JSON, and Parquet.
• Assist in monitoring and troubleshooting data pipeline issues.
• Support data quality checks and validation processes.
• Participate in testing and documenting data pipelines.
• Collaborate with senior data engineers and technical teams.
• Follow best practices for coding, version control, testing, and deployment.
• Continuously learn and adapt to new data engineering technologies and tools.

Required Skills

• Bachelors degree in Computer Science, Information Technology, Data Science, Engineering, or a related field.
• Strong fundamentals in Python.
• Good knowledge of SQL.
• Basic to intermediate knowledge of PySpark / Apache Spark.
• Hands-on knowledge or academic/project experience with Databricks.
• Basic understanding of ETL/ELT concepts and data pipelines.
• Understanding of databases and basic data warehousing concepts.
• Good analytical and problem-solving skills.
• Strong willingness to learn and work in a data engineering environment.

Preferred Skills

• Knowledge of Delta Lake and Databricks Lakehouse architecture.
• Basic knowledge of Azure, AWS, or GCP.
• Familiarity with Git/GitHub.
• Exposure to Databricks Workflows/Jobs.
• Basic knowledge of data orchestration tools such as Azure Data Factory or Airflow.
• Experience working with APIs and JSON data.
• Academic or personal projects involving Databricks, PySpark, and SQL.