What are the responsibilities and job description for the Databricks Architect-Migration position at Tixy Services LLC?
Job Title: Databricks Architect Migration
Chicago, IL (Hybrid)
Long Term Contract
Job Description:
We are seeking an experienced Databricks Architect with a strong background in data migration and cloud modernization. The ideal candidate will design, develop, and implement scalable data solutions on the Databricks Lakehouse platform while leading migration initiatives from legacy data platforms to Databricks.
Key Responsibilities:
- Design and implement Databricks-based data architectures and ETL/ELT pipelines.
- Lead end-to-end migration projects from legacy data warehouses or big data platforms to Databricks.
- Develop data pipelines using PySpark, SQL, Delta Lake, and Databricks Workflows.
- Optimize data processing, performance, scalability, and cost.
- Collaborate with business stakeholders, architects, and engineering teams to deliver high-quality solutions.
- Ensure data governance, security, and best practices using Unity Catalog and CI/CD processes.
Required Skills:
- 8 years of experience in Data Engineering/Data Architecture.
- 3 years of hands-on experience with Databricks.
- Strong expertise in PySpark, Spark, SQL, Python, and Delta Lake.
- Experience migrating platforms such as Hadoop, Snowflake, Oracle, SQL Server, Synapse, or Teradata to Databricks.
- Experience with Azure Databricks (preferred), AWS, or Google Cloud Platform.
- Knowledge of ETL tools, Git, Azure DevOps/Jenkins, and cloud data architecture.