What are the responsibilities and job description for the Gen AI with Databricks certification position at Hash Technologies?
We are seeking a highly skilled AI Engineer with proven experience in developing, deploying, and optimizing AI models using the Databricks AI platform. The ideal candidate will have a strong background in machine learning, distributed data processing, and will be a champion of Databricks platform.
Responsibilities
Generative AI: Experience of LLM-based solutions (LlamaIndex, LangChain, RAG pipelines, or similar frameworks). Ability to integrate GenAI and Agentic AI into business workflows.
Design and implement end-to-end Generative AI solutions on Databricks, leveraging Unity Catalog, MLflow, Delta Lake, and Vector Search
Design, build, and deploy large-scale AI/ML models using the Databricks environment.
Implement data validation, lineage, and monitoring using Delta Live Tables and Unity Catalog.
Leverage Databricks’ data engineering workflows for feature engineering, model training, and evaluation.
Optimize training pipelines for efficiency, scalability, and accuracy.
Integrate AI models into production systems using APIs and microservices.
Build reusable ML pipelines using Databricks Repos, MLflow, and Feature Store.
Implement robust testing, monitoring, and retraining protocols for deployed models.
Ensure adherence to compliance, security, and performance standards.
Stay updated on advancements in AI frameworks, distributed computing, and Databricks platform updates.
Required Skills and Qualifications
Bachelor’s or Master’s degree in Computer Science, Data Science, or related field.
Certified Databricks Certified Generative AI Engineer
Proven experience developing AI solutions on Databricks.
Strong knowledge of Python, PySpark, MLflow, Spark and Databricks Notebooks.
Strong understanding and knowlege of Databricks platform features such as Unity Catalog, DLT, MosaicAI, Data Assets Bundles, etc.
Experience with Transformer-based models, generative AI, and Databricks pipelines.
Proficiency in integrating AI models with cloud-native architectures (AWS, Azure, or GCP).
Solid understanding of MLOps practices, Data Assets bundles (CI/CD), and containerization (Docker, Kubernetes) on Databricks Platform.
Familiarity with vector databases, embeddings, and retrieval-augmented generation (RAG).
Strong problem-solving, analytical thinking, and communication skills