Data Engineering Manager

Job Description

Senior Manager Data Engineering (Irvine California - 3 days in the office).

As a Senior Manager Data Engineering, you will lead and drive the development of scalable, high-performance data solutions leveraging Big Data technologies. You will be responsible for designing, implementing, and optimizing data pipelines, ensuring data quality, and enabling advanced analytics and AI-driven insights. Your expertise in Big Data platforms and cloud-based data architectures will be instrumental in shaping the future of data engineering at scale.

Your Impact

• Lead the design, development, and implementation of large-scale data solutions using Big Data technologies such as Hadoop, Spark, Kafka, and Snowflake.
• Architect and optimize data pipelines, ensuring high performance, scalability, and reliability.
• Collaborate with cross-functional teams to define data engineering best practices and drive data-driven decision-making.
• Implement real-time and batch data processing frameworks to support business intelligence, analytics, and machine learning use cases.
• Ensure data security, governance, and compliance with industry standards and regulations.
• Drive the adoption of cloud-based data platforms (AWS, GCP, Azure) and serverless data architectures.
• Mentor and lead a team of data engineers, fostering a culture of innovation, collaboration, and continuous learning.
• Partner with business stakeholders to understand data requirements and deliver actionable insights.
• Automate data workflows, monitoring, and performance tuning to enhance efficiency.
• Stay ahead of emerging trends in Big Data, AI, and data engineering to drive innovation and competitive advantage.

Skills & Experience

• 15 years of experience in data engineering, with a strong focus on Big Data technologies.
• Proven expertise in Hadoop, Spark, Kafka, Flink, and other distributed computing frameworks.
• Hands-on experience with cloud-based data platforms such as AWS and AWS services
• Strong programming skills in Python, Scala, or Java for data processing and analytics.
• Experience in building and optimizing ETL/ELT pipelines using tools like Apache NiFi, Airflow, or Talend.
• Deep understanding of data modeling, data lakes, and data warehousing concepts.
• Knowledge of SQL, NoSQL databases, and data governance frameworks.
• Strong problem-solving and analytical skills with a focus on performance optimization and scalability.
• Experience in leading and mentoring data engineering teams in an Agile environment.
• Excellent communication and stakeholder management skills, with the ability to translate complex technical concepts into business insights.

Qualifications

• Exceptional data engineering skills with distributed computing background and proven experience in delivering large scale data platforms
• Good grasp of analytics, measurement, reporting, and business intelligence including modeling, insights generation, and data science
• Hands-on technologist with deep expertise in big data ecosystem for data integration, data storage, compute framework, analytics, and advanced visualization (i.e., ETL Tools, Streaming Tools, No-SQL data bases, Databricks, Spark, Airflow, ELT tools like DBT, Reporting Tools, AI/ML Platforms)
• Hands-on experience with Amazon Web Services
• Exposure to AWS EMR, Glue, Athena, S3, SQS/SNS or equivalent technologies in other cloud platforms
• Experience in building & applying best practices w.r.t. performance, security, and cost-efficiency for data lake
• Ability to lead teams that rapidly learn the client’s current digital ecosystem and produce a future data landscape vision and strategy considering the transformation agenda and business goals
• Have a point of view and understanding of build vs. buy, performance considerations, hosting, commercial models, business intelligence, reporting, and analytics
• Excellent client communication and facilitating skills, ability to influence others and gain consensus, and team collaboration
• Combination of proficient consulting, business, strategy, technical and people skills
• A Bachelor s degree in Engineering, Computer Science, or related field

Set Yourself Apart With

• Certifications in Big Data technologies
• Experience in real-time data processing, event-driven architectures, and streaming analytics.
• Knowledge of AI/ML frameworks and their integration with Big Data platforms.
• Expertise in data security, compliance, and governance best practices.
• Background in financial services, investment management or asset management