big data etl engineer

Duties/Responsibilities • Design and implement data pipelines and transformations using big data technologies such as Spark, Hadoop, and related ecosystems • Develop and maintain complex SQL queries for data extraction and reporting, and optimize query performance and scalability • Design and develop software components that integrate with our identity graphs, client data, and keying processes • Participate in capacity monitoring and planning • Develop and maintain technical documentation • Ensure data integrity and quality in development process, including conducting data research and analysis to identify trends and patterns • Work with cross-functional teams to ensure software systems meet business requirements and are scalable for future growth, including developing predictive models and enhancing data quality • Position will require that the individual understands all regulations and laws applicable to their assigned roles and responsibilities. Additionally, the individual will be responsible for the development, implementation, and regular maintenance of policies and procedures that govern the work of assigned roles and responsibilities, including compliance with the security requirements of ePHI.