What are the responsibilities and job description for the Data & AI Engineer position at Hierarch Soft Technologies, Inc.?
Job Title: Data & AI Engineer (Scala, Databricks & Java Microservices)
Location: Hybrid/Onsite (SFO,CA)
Job Summary
Key Responsibilities
Design and develop Scala-based Spark pipelines on Databricks.
Build scalable Java Spring Boot microservices for enterprise applications.
Develop high-performance ETL/ELT data pipelines using Scala, Python, Spark, and Databricks.
Implement AI-powered solutions using LLMs, RAG, LangChain, Azure OpenAI, or similar technologies.
Develop event-driven architectures using Kafka and Spark Streaming.
Expertise in MarTech platforms with hands‑on experience in Adobe Campaign or similar orchestration tools.
Optimize Spark jobs and Databricks workloads for large-scale data processing.
Build and deploy cloud-native applications on AWS, Azure, or Google Cloud Platform.
Collaborate with Data Scientists, AI Engineers, and Product teams to deliver enterprise AI solutions.
Mentor engineers and drive technical design decisions.
Required Skills
8 years of software engineering experience.
Strong hands-on Scala development.
Strong Python programming.
Production experience with Databricks and Apache Spark.
Strong Java Spring Boot Microservices development.
Experience building AI/GenAI applications.
Experience with Kafka and distributed systems.
Experience working on any major cloud platform (AWS, Azure, or Google Cloud Platform).
Strong understanding of distributed data processing and cloud-native architectures.