What are the responsibilities and job description for the Databricks Admin/Unity Catalog position at capgemini?
Responsible for provisioning Databricks and Unity Catalog services while taking care of organizational policies and governance requirements as well as streamlining and automating data-related processes.
Required skills and Experience (Hands-on):
Account and Workspace Administration
Configuration of workspaces across environments
Management and optimization of compute resources
Strong knowledge of security and governance: user and group management, provisioning, identify federation
Role based access control, cluster policies and workspace object permissions
Unity Catalog administration of metastores, catalogs, schemas, external locations and Delta sharing)
Knowledge of security compliance procedures
Strong engineering knowledge to guide data engineering teams on ways to build pipelines and optimize existing ones
Usage monitoring and troubleshooting bottlenecks with spark jobs, ETL pipelines and ML workloads
Scripting experience in Python and SQL
Data sharing knowledge and implementing guardrails
Preferred:
Understanding of Agentic Architecture
Familiarity with data requirements of common ML/AI use cases
Responsibilities
Implement data provisioning patterns based on business requirements, following follow predefined processes, policies, standards, and metadata management rules
Create and manage distributed workspaces in Databricks, set up workspace policies, provision Databricks clusters and manage data infrastructure sizing and capacity
Create Python notebooks, implement data masking processes, create UDFs (SQL/Python), troubleshoot data pipelines
Ensure data security and compliance with regulations using Databricks and Privacera's features
Navigate multi-step enterprise approval process across architecture, security, and governance teams
Design and implement data architecture leveraging technologies such as Databricks, Unity Catalog, Privacera, and Collibra
Develop, optimize, and manage data pipelines for ETL processes using Databricks, with a focus on data integrity and quality
Design and maintain data models and schemas, incorporating Unity Catalog and Collibra data governance practices
Operationalize Machine Learning models in Batch and Real Time Data Pipelines, leveraging relevant governance setups
Collaborate with cross-functional teams including data scientists, engineers, and analysts to translate business requirements into scalable solutions
The pay range that the employer in good faith reasonably expects to pay for this position is $46.23/hour - $72.23/hour. Our benefits include medical, dental, vision and retirement benefits. Applications will be accepted on an ongoing basis.
Tundra Technical Solutions is among North America’s leading providers of Staffing and Consulting Services. Our success and our clients’ success are built on a foundation of service excellence. We are an equal opportunity employer, and we do not discriminate on the basis of race, religion, color, national origin, sex, sexual orientation, age, veteran status, disability, genetic information, or other applicable legally protected characteristic. Qualified applicants with arrest or conviction records will be considered for employment in accordance with applicable law, including the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act. Unincorporated LA County workers: we reasonably believe that criminal history may have a direct, adverse and negative relationship with the following job duties, potentially resulting in the withdrawal of a conditional offer of employment: client provided property, including hardware (both of which may include data) entrusted to you from theft, loss or damage; return all portable client computer hardware in your possession (including the data contained therein) upon completion of the assignment, and; maintain the confidentiality of client proprietary, confidential, or non-public information. In addition, job duties require access to secure and protected client information technology systems and related data security obligations.
Salary : $46 - $72