Senior Software Engineer (Data Pipelines)

• As a Senior Engineer, you will play a crucial role in the development, optimization, and automation of our data pipelines.
• You will lead complex data projects, architect technical initiatives, and ensure the scalability and performance of our data solutions.
• Your role will involve strategic planning, collaboration with senior management, and mentoring other engineers.
• You will be responsible for pushing the delivery speed and quality of our data ELT, leading the implementation of innovative tooling and architecture patterns, and fostering a culture of data engineering excellence
• Acting as a technical leader and Subject Matter Expert (SME) in data pipeline automation and workflow orchestration
• Designing, implementing, and maintaining complex, reliable, data solutions with a focus on automation using Airflow, dbt, and Google Cloud data products
• Managing a large portfolio of dbt models, leveraging macros and DRY patterns
• Advocating for test-driven development and assisting QA in developing a robust and reliable process for continuous integration and delivery
• Monitoring, troubleshooting, and optimizing the performance of data pipelines and workflows
• Architecting solutions and reusable patterns that scale with business needs
• Providing implementation, configuration, and deployment documentation
• Proactively addressing issues and problems, generating and implementing innovative solutions
• Participating in all agile ceremonies, including daily standups and regular sprint planning
• Mentoring other engineers and fostering a culture of technical excellence
• Staying up-to-date with the latest industry trends and technologies to drive continuous improvement and innovation in data engineering practices
• Ensuring data security, governance, and compliance with relevant standards and privacy restrictions

Benefits

• Half-day Fridays every Friday
• 3 different medical plans to suit your needs, including a plan under $10 per month, as well as dental, vision, disability, and life insurance
• 401(k) with company matching
• Free, confidential Employee Assistance Program
• Free weekly virtual workout classes
• 100% paid parental leave policy for birth, adoption, and foster placement
• Medical and dependent care FSAs
• LinkedIn learning and access to internal training and development
• Unlimited PTO
• Organizational transparency - our c-suite and senior leadership are easily accessible to all employees, they share business updates, and recognize top employees through a monthly all-hands and a weekly CEO message
• Virtual experiences, from Slack cooking challenges to trivia tournaments and employee awards
• Wprotege mentorship program- Expertise in building and automating ETL/ELT pipelines using Airflow and DAG-based workflow management software. Airbyte for data ingestion is a plus, but not required
• Familiarity with version control systems and CI/CD tools like GitHub Actions
• Ability to influence and guide cross-functional teams and projects
• Exceptional problem-solving skills and technical leadership
• Mastery of Python or comparable scripting language, API integrations, and software architecture
• Good understanding of business intelligence tools like Looker or comparable alternatives
• Strong command of agile methodologies, continuous integration, and test-driven development
• Deep experience with databases like PostgreSQL and BigQuery, including query optimization for performance and cost
• Proficiency with advanced data formats (Parquet, Avro, Hive, JSONL) and data integration techniques
• Experience with monitoring and logging tools (e.g., Prometheus, Grafana) is a plus
• Deep experience managing dbt models using macros and dbt tests
• Experience with Google Cloud Platform, Kubernetes, and managing infrastructure as code using Terraform
• Minimum 7+ years of experience in software development, with extensive experience in Python, SQL, and data pipelines