What are the responsibilities and job description for the Lead Data Engineer - W2 Contract position at ConnectedX, Inc.?
Need Consultants Local to Dallas, TX
Job Title: Lead Data Engineer
Location: Plano, TX Onsite
Duration: 12 Months W2 Contract
Job Summary
Experienced Lead Data Engineering with a strong background in Big Data technologies, Scala.
Key Responsibilities
- Lead end-to-end delivery of data engineering projects, ensuring timely and quality outcomes.
- Design, develop, and maintain scalable data pipelines using Databricks, Apache Spark (Scala), and AWS.
- Manage and coordinate with cross-functional teams including data analysts, data scientists, and business stakeholders.
- Drive data integration initiatives, including Workday HCM data extraction, transformation, and integration with downstream systems.
- Oversee ETL development and migration, especially from tools like Informatica PowerCenter to modern data platforms.
- Ensure data quality, validation, governance, and security across pipelines and storage systems.
- Optimize data processing workflows for performance, scalability, and cost efficiency.
- Translate business requirements into technical design and implementation plans.
- Provide technical leadership and mentorship to engineering teams.
- Manage project timelines, risks, dependencies, and stakeholder communication.
Required Skills & Qualifications
- 12 years of IT experience with at least 5 years in Big Data/Data Engineering.
- Strong expertise in Scala programming and Apache Spark (RDD, DataFrames, Spark SQL).
- Hands-on experience with Databricks and AWS ecosystem.
- Experience with Workday HCM integrations, including data extraction and transformation.
- Proficiency in SQL and data modeling concepts.
- Experience in ETL tools, especially Informatica PowerCenter, and migration strategies.
- Strong understanding of data pipeline architecture and distributed systems.
- Proven experience in project management and team leadership.
- Excellent communication and stakeholder management skills.
Preferred Qualifications
- Experience in Agile/Scrum methodologies.
- Exposure to data governance and compliance frameworks.
Certification in AWS / Databricks / Project Management (PMP, Scrum Master) is a plus