Senior Big Data Engineer

Job Title: Senior Big Data Engineer

Location: Charlotte, NC (Hybrid 3days a week)

Duration: 12+ Months

Job Description:

We are looking for a Senior Data Engineer

Resource should have minimum 10 years of hands-on experience in building production data systems.

Resource is expected to design, build, and maintain scalable data platforms while leading technical decisions.

Key technical skills - required:

Machine learning data pipelines

Generative AI and LLM data infrastructure

Programming

Python (advanced)

SQL (expert level)

Scala or Java (often preferred for big data)

Data Engineering skills required

ETL/ELT pipeline development

Data modelling (star schema, dimensional modelling)

Data warehousing (Snowflake, Big-Query, Redshift, Synapse)

Data lakes and lake-house architectures

Big Data Technologies skills required

Apache Spark (essential)

Hadoop ecosystem

Kafka (streaming)

Apache Airflow or similar workflow orchestration tools

Cloud Platforms skills required

AWS (S3, Glue, EMR, Lambda, Redshift)

Azure (ADF, Synapse, Databricks)

Google Cloud (Big-Query, Dataflow, Data-proc)

Databases skills required

PostgreSQL, MySQL

NoSQL databases (MongoDB, Cassandra, DynamoDB)

DevOps & Engineering Practices

Git

CI/CD pipelines

Docker

Kubernetes

Infrastructure as Code (Terraform is a plus)

Data Quality & Governance

Data validation and testing

Metadata management

Security and access control

Monitoring and observability

Leadership Skills

System design and architecture

Mentoring junior engineers

Stakeholder communication

Project estimation and technical planning

Nice-to-Have Skills

Databricks

DBT (Data Build Tool)

Delta Lake or Apache Iceberg