Senior Data Engineer

Senior Data Engineer

Full Time Professional
Remote
Salary Range: $120,000.00 To $140,000.00 Annually
Job Title: Senior Data Engineer
Clearance Requirement: Active DoD Secret Clearance required
Location: Remote / Pentagon

Position Overview

The Senior Data Engineer is a hybrid role combining Data Analyst, Data Engineer, and Data Scientist responsibilities. This position operates with a high level of autonomy, owning projects end-to-end across functional areas. The ideal candidate is proactive, solution-oriented, and capable of identifying opportunities, reducing complexity, and delivering efficient, high-quality data solutions.

This role also provides guidance to team members, helps remove roadblocks, and contributes to the continuous improvement of processes, systems, and deliverables.

Key Responsibilities

Data Analysis & Insights

• Gather, clean, and analyze complex datasets using advanced analytical techniques
• Apply statistical methods to identify trends, patterns, and anomalies
• Conduct exploratory data analysis (EDA) to support decision-making
• Translate data findings into actionable insights for technical and non-technical stakeholders

Data Engineering & Pipeline Development

• Design, develop, and maintain ETL/ELT pipelines using Databricks and Delta Lake
• Implement Medallion Architecture (Bronze, Silver, Gold) to ensure data quality and scalability
• Manage data ingestion processes with a focus on traceability and governance (e.g., Collibra)
• Build and maintain data pipelines within DoD environments such as Advana/Jupiter
• Troubleshoot, debug, and optimize data pipelines, identifying root causes and implementing solutions

Data Visualization & Reporting

• Develop dashboards, reports, and visualizations using tools such as Tableau, Power BI, Qlik, or Foundry
• Create interactive dashboards and reports to support operational and strategic decisions
• Ensure accuracy, quality, and usability of all data visualizations and deliverables

Integration & System Support

• Collaborate with data engineers, analysts, scientists, and stakeholders to deliver integrated data solutions
• Support real-time data streaming using Databricks Structured Streaming
• Optimize data storage solutions leveraging Delta Lake capabilities (e.g., schema evolution, time travel)

Governance & Compliance

• Maintain data lineage and governance using tools such as Collibra
• Ensure compliance with data security, privacy, and DoD requirements
• Establish and enforce data quality standards across multiple data sources

Collaboration & Communication

• Work closely with cross-functional teams to gather and refine requirements
• Communicate findings, recommendations, and metrics clearly and professionally
• Track assigned tasks and provide regular updates to leadership
• Mentor junior team members and support team development initiatives

Required Qualifications

• Active DoD Secret Clearance
• Bachelor’s degree in Data Science, Computer Science, Statistics, Engineering, or a related field
• 5+ years of experience in data engineering and building data pipelines and architectures
• 5+ years of experience in data analysis, data mining, or business intelligence
• 3+ years of experience developing data solutions using Python, SQL, and/or PySpark in big data environments (e.g., Databricks, Snowflake)
• Strong experience with data visualization tools (Tableau, Power BI, Qlik, Foundry) and advanced Excel (macros, pivot tables, VBA)
• Experience with cloud platforms such as AWS, Azure, or Google Cloud
• Familiarity with big data frameworks (Hadoop, Spark, Kafka)
• Strong understanding of ETL processes, data warehousing, and BI best practices
• Proven ability to analyze large datasets and communicate insights effectively
• Strong problem-solving skills and attention to detail
• U.S. work authorization required

Preferred Qualifications

• Master’s degree in Data Engineering, Computer Science, or related field
• Experience with DoD data platforms such as Advana and Jupiter
• Familiarity with statistical methods (e.g., hypothesis testing, regression analysis)
• Experience with SAP, Foundry, or advanced analytics platforms
• Experience with NoSQL databases (e.g., Cassandra, MongoDB)
• Knowledge of containerization tools (Docker, Kubernetes)
• Experience with version control systems (Git) and Agile methodologies (Scrum, Kanban)
• Prior experience in a client-facing government environment

Equal Employment Opportunity Statement

ShadowObjects, LLC is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, sexual orientation, and gender identity), national origin, age, disability, genetic information, protected veteran status, or any other characteristic protected by applicable federal, state, or local law.

Pay: $120,000.00 - $140,000.00 per year

Benefits:

• 401(k)
• 401(k) matching
• Dental insurance
• Health insurance
• Life insurance
• Paid time off
• Professional development assistance
• Retirement plan
• Vision insurance

Work Location: Remote