StreamSets Developer

Key Responsibilities

• Design, develop, and test streaming and batch data pipelines using StreamSets Data Collector.
• Build data integration solutions across Snowflake, Apache Kafka, and cloud data platforms.
• Monitor pipeline executions, analyze SLAs, troubleshoot failures, and resolve data issues.
• Optimize pipeline performance and improve data quality.
• Implement data governance and validation processes.
• Automate pipeline scheduling and orchestration.
• Develop resilient job templates and failover mechanisms.
• Collaborate with Data Engineers, Architects, and Business teams to deliver scalable data solutions.

Required Skills

• 10+ years of IT experience.
• Minimum 3 years of hands-on StreamSets Development.
• Strong experience with StreamSets Data Collector (SDC).
• Experience building streaming and batch data pipelines.
• Knowledge of Snowflake.
• Experience with Apache Kafka.
• Experience working with cloud data warehouses.
• Strong troubleshooting and pipeline monitoring skills.
• Experience with data quality and data governance.
• Good communication and analytical skills.

Preferred Skills

• Experience with cloud platforms (AWS, Azure, or GCP).
• Knowledge of ETL/Data Engineering concepts.
• Experience in automation and orchestration of data pipelines.
• Agile/Scrum project experience.