What are the responsibilities and job description for the Data Engineer position at Elite Workforce Inc?
Position: Data Engineer Azure Databricks & Synapse Lakehouse
Location: Indianapolis, IN 46204
Client: State of Indiana
Duration: 12 Months
Job Description: Seeking a data engineer to design, build, and maintain secure, scalable data solutions using Azure Databricks and Azure Synapse. You'll develop batch and streaming pipelines, optimize data warehouse workloads, implement data governance, and collaborate with architects, scientists, analysts, and project managers to translate business needs into robust technical designs. Focus areas include data integration, data modeling, and documenting solutions with strong emphasis on security and compliance (PHI/PII).
Key Responsibilities:
- Design, build, and maintain batch and streaming data pipelines in Azure Databricks (PySpark, Spark SQL, Delta Lake) using a medallion lakehouse architecture.
- Develop and optimize dimensional models and SQL workloads in Azure Synapse Analytics (dedicated/serverless) for reporting and dashboards.
- Orchestrate pipelines with Databricks Workflows, Synapse, and Data Factory; implement scheduling, monitoring, alerting, and failure recovery.
- Apply data governance and security controls (Unity Catalog, RBAC, data masking) to protect sensitive health data per HIPAA and policy.
- Collaborate with data architects, scientists, analysts, and PMs; document solutions and provide technical support.
Required/Desired Skills:
- Master's degree in data-related field or Bachelor's 4 years of relevant experience (or 4 years in a similar data engineering role)
- Databricks: PySpark, Spark SQL, R; medallion lakehouse, Auto Loader, Structured Streaming
- Unity Catalog, workspace catalogs, permissions, lineage, external locations
- ADLS Gen2, Parquet/Delta formats; OPENROWSET, external tables, views
- Star/schema design, SCD Types 1 & 2; COPY INTO, CETAS, Synapse/ADF pipelines
- Git/GitHub, CI/CD, Azure DevOps; Databricks Asset Bundles
- Monitoring with Azure Monitor/Log Analytics; troubleshooting failed jobs
- Data governance, PHI/PII protection, HIPAA compliance
- Strong communication skills
Nice-to-have:
- Databricks Certified Data Engineer
- Azure Data Engineer Associate (DP-203)
- Power BI integration with Synapse (Import vs. DirectQuery)
To Apply: Please send your resume to with the exact subject line: Resume Submission
Note: This posting emphasizes hands-on data engineering with a focus on secure, scalable, and governed data solutions using Azure Databricks and Synapse.