Sr. Software Engineer Database

bebo Technologies is a leading complete software solution provider. bebo stands for 'be extension be offshore'. We are a business partner of QASource, inc. USA[www.QASource.com]. We offer outstanding services in the areas of software development, sustenance engineering, quality assurance and product support. bebo is dedicated to provide high-caliber offshore software services and solutions.

Our goal is to 'Deliver in time-every time'.

For more details visit our website: www.bebotechnologies.com

Let's have a 360 tour of our bebo premises by clicking on below link:

https://www.youtube.com/watch?v=S1Bgm07dPmMKey

Job Responsibilities:

• Design and implement scalable data platforms leveraging Data Lake, Lakehouse, Data Mesh, and modern data architecture patterns
• Build and optimize data pipelines for batch and real-time processing using Databricks, Apache Spark, dbt, and cloud-native AWS services
• Develop robust data ingestion frameworks for structured, semi-structured, and unstructured data from APIs, files, databases, and streaming sources
• Design and develop scalable Python-based microservices to enable secure and efficient data sharing across systems and applications
• Build RESTful APIs and event-driven services for exposing curated datasets from Data Lake and Lakehouse platforms
• Work extensively with Python, PySpark, and SQL for data transformation, processing, and data engineering workflows
• Implement streaming data pipelines using Kafka/Kinesis and integrate them with downstream analytics and data platforms
• Design and manage large-scale datasets using formats such as Parquet, JSON, CSV, and IoT/sensor data
• Optimize data storage, partitioning, and query performance for high-volume analytical workloads
• Design and implement data orchestration workflows using Apache Airflow and/or Databricks Workflows
• Implement infrastructure and data platform components using Terraform and Infrastructure as Code (IaC) practices
• Containerize data applications and services using Docker where applicable
• Collaborate with cross-functional teams, including Data Architects, Data Analysts, and BI teams, to operationalize Data Lake and Lakehouse solutions
• Contribute to data modeling in Lakehouse environments, including Medallion Architecture and dimensional modeling
• Implement data quality, validation, monitoring, and observability practices across data pipelines
• Use data quality and observability tools to identify data issues, monitor pipeline health, and improve data reliability
• Ensure data quality, reliability, security, and observability across data pipelines
• Implement serverless data processing and pipeline solutions where applicable

Job Requirements:

• 6–8 years of experience in Data Engineering or related roles
• Strong understanding of modern data architectures such as Data Lake, Lakehouse, Data Mesh, and Data Products
• Strong proficiency in Python, PySpark, and SQL
• Hands-on experience with Databricks, Delta Lake, and/or Snowflake
• Experience with ETL/ELT frameworks and data orchestration tools
• Hands-on experience with Apache Airflow and/or Databricks Workflows
• Practical experience with streaming technologies such as Kafka and/or AWS Kinesis
• Strong understanding of batch processing frameworks and technologies such as Apache Spark, AWS Glue, and dbt
• Proficiency in handling structured, semi-structured, and unstructured data
• Experience with modern data modeling techniques, including Star Schema, Snowflake Schema, and 3NF
• Familiarity with vector databases and data architectures supporting AI/ML use cases
• Hands-on experience with AWS services including S3, Glue, Glue Data Catalog, Athena, and Redshift
• Experience with Infrastructure as Code (IaC) tools such as Terraform
• Experience with containerization technologies such as Docker
• Experience with Git, CI/CD, and DevOps practices
• Experience implementing data quality, monitoring, and observability solutions using relevant tools and frameworks
• Good understanding of cloud security concepts, including IAM, encryption, access control, and data governance
• Strong understanding of Data Lakehouse concepts, architecture, and implementation patterns
• Familiarity with serverless architectures and Python-based serverless data processing solutions