Data Engineering Intern

Data Engineering Intern

About the Role

We are looking for a Data Engineering Intern to join our engineering team and work on building data pipelines that collect, process, structure, and store information from various websites.

In this role, you will work on real-world data engineering problems where information needs to be collected from websites with different structures and formats, transformed into structured data, validated, and stored in a PostgreSQL database.

You will also work with OpenAI models to process unstructured web content and convert it into structured, usable data.

Key Responsibilities

• Build and maintain data collection and processing pipelines for websites.

• Analyze different website structures to identify and extract relevant information.

• Develop data extraction workflows for collecting information from web pages.

• Process raw HTML and web content into structured data.

• Use OpenAI models/LLMs to extract and structure information from unstructured web content.

• Design and improve prompts for accurate and consistent data extraction.

• Transform, clean, normalize, and validate extracted data.

• Store processed data in PostgreSQL databases.

• Write and optimize SQL queries for data insertion, retrieval, and validation.

• Develop APIs and backend services required for data processing and pipeline operations.

• Handle different data formats and website structures.

• Identify and resolve data quality, extraction, and pipeline issues.

• Implement appropriate error handling, logging, and retry mechanisms.

• Monitor and improve the reliability and efficiency of data pipelines.

• Collaborate with the engineering team to improve data collection and processing workflows.

Technology Stack

Data Engineering

• Web Data Extraction

• Data Processing & Transformation

• Data Cleaning & Validation

• Data Pipelines

• Structured Data / JSON

AI / LLM

• OpenAI APIs

• Large Language Models

• Prompt Engineering

• AI-based Data Extraction

• Structured JSON Generation

Database

• PostgreSQL

• SQL

• Relational Database Concepts

Backend

• Node.js

• LoopBack

• REST APIs

Development Tools

• Git

• GitHub

• API Testing / Debugging

Requirements

• Basic understanding of JavaScript / TypeScript.

• Basic understanding of SQL and relational databases.

• Familiarity with PostgreSQL is a plus.

• Understanding of HTML and web page structure.

• Basic understanding of APIs and JSON.

• Good problem-solving and debugging skills.

• Ability to analyze different data structures and identify patterns.

• Strong willingness to learn and work with new technologies.

• Interest in data engineering, automation, web data, and AI/LLMs.

Good to Have

• Experience with Node.js.

• Experience with web scraping or web crawling.

• Experience with PostgreSQL or any relational database.

• Familiarity with OpenAI APIs or other LLM APIs.

• Knowledge of JSON Schema or structured outputs.

• Experience with data cleaning and transformation.

• Understanding of asynchronous programming.

• Experience with Git/GitHub.

What You Will Learn

As a Data Engineering Intern, you will get hands-on experience with:

• Designing real-world data pipelines

• Web-based data collection and extraction

• Processing large amounts of unstructured data

• Data cleaning, transformation, and validation

• Building structured datasets from web content

• Using LLMs for automated data extraction

• Prompt engineering for data processing

• PostgreSQL database management

• SQL and data modeling

• Backend APIs supporting data pipelines

• Debugging and optimizing data workflows

Ideal Candidate

We are looking for someone who enjoys working with data, problem-solving, automation, and technology.

You don't need extensive prior experience. We value strong fundamentals, logical thinking, curiosity, and the ability to learn and understand new data structures and technologies.

Internship Details

Position: Data Engineering Intern

Experience: Fresher / 01 Year

Employment Type: Internship

Location: Nehru Place

Duration: [3/6 Months]