Data Engineer - Cosmos DB

About the Role

We are looking for a Senior Data Engineer with 8 - 13 years of experience who has strong hands-on experience with Azure Cosmos DB and PostgreSQL.

The person in this role will work on designing and building scalable data solutions, optimizing database performance, developing data pipelines, and supporting data integration across different systems. Strong experience with Azure and good problem-solving skills are important for this role.

What You'll Be Doing

• Design and develop data solutions using Azure Cosmos DB and PostgreSQL.
• Work on Cosmos DB data modeling, partitioning, indexing, query optimization, and performance tuning.
• Manage PostgreSQL databases, including schema design, complex SQL queries, indexing, and performance optimization.
• Build and maintain ETL/ELT pipelines for data ingestion and transformation.
• Work on data migration and integration between Cosmos DB, PostgreSQL, and other data platforms.
• Monitor database performance and troubleshoot issues related to scalability, availability, and data quality.
• Work closely with application developers, architects, DevOps, and business teams to understand requirements and deliver data solutions.
• Follow best practices around data security, reliability, and governance.
• Participate in technical discussions, code reviews, and solution design.
• Help mentor junior engineers and share knowledge with the team.

What We're Looking For

• 8 - 13 years of overall experience in Data Engineering, Database Engineering, or a similar role.
• Strong hands-on experience with Azure Cosmos DB.
• Good understanding of Cosmos DB partition keys, indexing, RU consumption, Change Feed, consistency levels, and performance optimization.
• Strong experience with PostgreSQL, including advanced SQL, database design, indexing, query tuning, and troubleshooting.
• Experience working with Azure data services, preferably Azure Data Factory, Azure Data Lake, Synapse, or similar services.
• Good programming/scripting experience in Python.
• Experience developing ETL/ELT pipelines and working with large volumes of data.
• Good understanding of data modeling and distributed database concepts.
• Experience with Git and CI/CD tools such as Azure DevOps or GitHub Actions.
• Good communication skills and the ability to work independently as well as with cross-functional teams.

Good to Have

• Experience with Databricks / Spark.
• Knowledge of Kafka or Azure Event Hubs.
• Experience with data migration projects.
• Knowledge of NoSQL databases and distributed systems.
• Experience working on high-volume, low-latency applications.
• Azure certification related to Data Engineering.