Databricks Admin

Job highlights

• Cloud experience . Below are the general responsibilities. Hadoop / Databricks Operations is part of the Hadoop Support team

Databricks platform (not as a data engineer, but as a Databricks platform administrator)
Cloud experience

Below are the general responsibilities.

• Hadoop/Databricks Operations is part of the Hadoop Support team. This team is responsible for managing the Hadoop/Databricks plant across the system.
• Working within an Agile delivery (Scrum/Kanban/Scrumban)/ DevOps methodology. Support the application development teams and help them in debugging and fixing an issue.
• Configure, deploy and maintain Databricks clusters and workspaces.
• Use Terraform to provision and deploy Databricks clusters, workspaces, and associated AWS resources.
• Ensure consistency and repeatability by treating infrastructure as code
• Monitor cluster performance, resource utilization, and troubleshoot issues to ensure optimal performance.
• Implement and manage access controls and security policies to protect sensitive data.
• Manage platform feature and software releases and updates.
• Provide technical support to Databricks users, including troubleshooting and issue resolution.
• Conduct training sessions to educate users on platform best practices and capabilities.
• Optimize cluster configurations for performance and cost-effectiveness.
• Monitor and manage auto-scaling to ensure resources are allocated efficiently.
• Implement and enforce security policies, RBAC, access controls, and encryption mechanisms.
• Develop and maintain backup and disaster recovery strategies to ensure data integrity and availability.
• Collaborate with data engineers to integrate Databricks with other data sources, data warehouses, and data lakes.
• Monitor and manage platform costs, including optimizing resource utilization to control expenses.
• Collaborate with cross-functional teams, including data scientists, data engineers, and business analysts to understand their requirements and provide technical solutions