What are the responsibilities and job description for the Databricks Migration Engineer position at ConsultNet Technology Services and Solutions?
Databricks Migration Engineer
Job Summary
The Databricks Migration Engineer moves workloads off legacy platforms and lands them on the lakehouse without breaking the business that depends on them. This role handles assessment, code conversion, data movement, validation, and cutover, and is measured on whether the new pipeline produces the same answer as the old one. It suits engineers who are methodical, comfortable with legacy code, and willing to read someone else's stored procedures for a living.
Key Responsibilities
Inventory and assess legacy workloads across Hadoop, Teradata, Netezza, Informatica, SSIS, Oracle, or Snowflake estates
Build the migration wave plan, sequencing workloads by dependency, risk, and business criticality
Convert legacy ETL, stored procedures, and SQL dialects into Spark and Databricks SQL
Design and execute historical data movement, including reconciliation of row counts, checksums, and business metrics
Build automated parity testing so old and new outputs can be compared continuously during parallel run
Refactor rather than lift and shift where the legacy pattern will not perform on the lakehouse
Manage cutover, rollback planning, and decommissioning of retired systems
Document the migrated state so the receiving team can support it
Required Skills
5 years in data engineering with at least one completed large scale platform migration
Strong Spark and PySpark skills plus advanced SQL across more than one dialect
Experience reading and translating legacy ETL tooling or procedural SQL
Data reconciliation and validation experience at scale
Understanding of Delta Lake and how legacy modeling choices should change on the lakehouse
Disciplined documentation and change control habits
Preferred Skills
Experience with migration accelerators such as BladeBridge, LeapLogic, or SQLMorph
Hadoop ecosystem depth including Hive, HDFS, Sqoop, Oozie, and Impala
Unity Catalog migration from Hive metastore
Experience running parallel run periods with business stakeholders
Welcome to ConsultNet, a premier national provider of technology talent and solutions. Our expertise spans across project services, contract-to-hire, direct search, and managed services onshore, nearshore, and hybrid. For over 25 years, we have connected thousands of consultants with meaningful roles through a personal, communication-driven approach, partnering with a diverse client base to build high-performing teams and create lasting impact. Our comprehensive service offerings cover a wide range of technology and engineering positions across key markets nationwide. Learn more at www.consultnet.com .
We champion equality and inclusivity, proudly supporting an Equal Opportunity Employer policy. We welcome applicants regardless of Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other status protected by law.