Data Lake Engineer

Description Prescient Edge is seeking a Data Lake Engineer to support a Federal Government client.

Please note that the availability of this position is contingent upon contract award. Benefits: At Prescient Edge , we believe that acting with integrity and serving our employees is the key to everyone's success. To that end, we provide employees with a best-in-class benefits package that includes:

• A competitive salary with performance bonus opportunities.
• Comprehensive healthcare benefits, including medical, vision, dental, and orthodontia coverage.
• A substantial retirement plan with no vesting schedule.
• Career development opportunities, including on-the-job training, tuition reimbursement, and networking.
• A positive work environment where employees are respected, supported, and engaged.

Description:

• Design, implement, and maintain scalable Data Lake architectures to support structured and unstructured data ingestion, ensuring efficient data access and retrieval.
• Configure and manage the integration interface between the Data Lake and the managed knowledge graph platform (Stardog), including SPARQL endpoint access, metadata federation, and catalog alignment.
• Follow access control policies and usage scope.
• Confirm compliance with access policies on a quarterly basis and document the results in the Data Governance & Compliance Report.
• Optimize ETL pipelines for high-volume data transformation, ensuring compliance with DoD IL-4/IL-5 security standards
• Implement storage tiering strategies and access controls, ensuring data is properly classified, retained, and accessed per DoD governance requirements.
• Submit the Data Lake Performance & Optimization Report, detailing ingestion efficiency, access control improvements, and storage utilization metrics.

Requirements Experience:

• Possess the knowledge and capability to support Data Lake platform administration and enterprise data architecture for DoD data-driven projects.

• Skilled in Data Lake Platform administration, including workspace management, cluster optimization, and Unity Catalog integration for secure data governance.

• Proficient in ETL pipeline development, Delta Lake architecture, and Data Lakehouse optimization for DoD analytics and mission critical data workflows.

• Experienced in SysEngOps, DevSecOps, version control systems (Git), and CI/CD pipelines to streamline Data Lakedevelopment and deployment.

• Knowledgeable in identity and access management (IAM), role-based access control (RBAC), and cloud security best practices across AWS, Azure, and GCP.

• Hands-on expertise in Python, SQL/NoSQL, Apache Spark, Databricks SQL, Terraform, and cloud native data services for large-scale data processing and analytics.

• Has strong background in:

• Databricks platform administration, including workspace configuration, Unity Catalog, cluster performance tuning, and cloud integration.

• Data pipeline development (ETL/ELT), Delta Lake optimization, and AI/ML workflow integration.

• SysEngOps, DevSecOps, version control systems (Git), and CI/CD pipelines for automated deployment and governance.

• Cloud security, role-based access control (RBAC), and compliance with DoD data policies across AWS, Azure, and GCP.

Education:

• A Master’s degree or higher (e.g. Ph.D.) in Computer Science, Information Technology, Systems Engineering, Data Science, Business Administration, Engineering Management, or a closely related field; or a minimum of 11+ years of experience managing complex technical projects in enterprise data architecture, Databricks administration, and cloud-based data platforms.

Security Clearance:

• Active TS/SCI clearance.

Location:

• Doral, Florida. (USSOUTHCOM Headquarters)