What are the responsibilities and job description for the Senior Databricks Data Engineer position at Conquest Consulting?
Only Local Austin, TX candidates who can attend a Face-to-Face interview and work in the required hybrid schedule (2 days onsite every Monday and Thursday) will be considered.
Senior Databricks Data Engineer
Client: State of Texas
Job Title: Senior Databricks Data Engineer
Experience: 12 Years (Mandatory)
Location: Austin, Texas
Work Schedule: Hybrid (3 Days Remote / 2 Days Onsite – Mondays & Thursdays)
Interview Mode: Face-to-Face Interview in Austin, TX (Mandatory)
Employment Type: Contract (C2C/W2)
📍 Candidate Requirements
- LOCAL AUSTIN, TX CANDIDATES ONLY
- Must reside within commuting distance of Austin, Texas.
- Must be available to attend a Face-to-Face Interview in Austin.
- Must be able to work onsite every Monday and Thursday.
- Candidates with State Government/Public Sector experience are highly preferred.
Required Technical Skills
Databricks, Azure Databricks, Apache Spark, PySpark, Scala, SQL, Python, ETL, ELT, Delta Lake, Lakehouse Architecture, Medallion Architecture (Bronze, Silver, Gold), Lakeflow Declarative Pipelines (DLT), Lakeflow Jobs, Databricks SQL, Databricks Apps, Data Warehousing, Dimensional Data Modeling, Star Schema, Snowflake Schema, Data Governance, Data Quality, Data Security, CI/CD, Azure DevOps, Git, GitHub, Airflow, Performance Tuning, Data Pipeline Development, Enterprise Analytics, Dashboard Development.
Key Responsibilities
- Design, develop, and optimize scalable data engineering solutions using Databricks.
- Build enterprise ETL/ELT pipelines using Apache Spark, PySpark, or Scala.
- Develop and maintain Lakeflow Declarative Pipelines (DLT) for production workloads.
- Design and implement Medallion Architecture (Bronze, Silver, Gold).
- Develop scalable Delta Lake and Lakehouse solutions.
- Build Databricks SQL Dashboards and Databricks Apps for business analytics.
- Design dimensional data models using Star and Snowflake schemas.
- Implement data governance, data quality, security, and validation frameworks.
- Optimize data pipelines for performance, scalability, and cost efficiency.
- Schedule and monitor workloads using Lakeflow Jobs, Databricks Workflows, or Airflow.
- Implement Git-based CI/CD pipelines for enterprise data engineering.
- Collaborate with business stakeholders to deliver high-quality analytical solutions.
- Present technical solutions and business insights to leadership and cross-functional teams.
Required Experience
- 12 years of overall IT experience.
- Strong hands-on experience with Databricks and Apache Spark.
- Expertise in PySpark, SQL, and Python (Scala is a plus).
- Experience building enterprise ETL/ELT pipelines.
- Strong experience with Delta Lake and Lakehouse Architecture.
- Experience implementing Medallion Architecture (Bronze, Silver, Gold).
- Experience with Lakeflow Declarative Pipelines (formerly Delta Live Tables).
- Experience with Lakeflow Jobs, Databricks Workflows, or Airflow.
- Strong experience in Data Warehousing and Dimensional Data Modeling.
- Experience developing Databricks SQL Dashboards and Databricks Apps.
- Strong understanding of Data Governance, Data Quality, and Security.
- Experience with Git, Azure DevOps, and CI/CD pipelines.
- Excellent verbal and written communication skills.
- Public Sector or State Government project experience is highly preferred.
Preferred Qualifications
- Databricks Certified Data Engineer Associate or Professional Certification.
- Experience with Azure Cloud Data Engineering ecosystem.
- Experience working on enterprise-scale analytics and reporting solutions.
- Strong knowledge of performance optimization and workload tuning.
📩 Interested Candidates, Please Share:
- Updated Resume
- Full Name
- Current Location
- Work Authorization
- Total IT Experience
- Databricks Experience
- PySpark / Spark Experience
- SQL & Python Experience
- Delta Lake & Medallion Architecture Experience
- DLT / Lakeflow Experience
- State Government Experience (If Any)
- Databricks Certification Details
- LinkedIn Profile
- Availability for Face-to-Face Interview
- Earliest Joining Date
- Contact Number