What are the responsibilities and job description for the Lead Data Engineer / Data Architect position at ExecutivePlacements.com?
Key Responsibilities
- Lead the data architecture and engineering strategy, ensuring alignment with business and technology roadmaps.
- Design and implement data models, pipelines, and data integration frameworks across multiple platforms.
- Partner with stakeholders to translate business requirements into scalable data solutions.
- Performance Optimization: Optimize data pipelines for performance, scalability, and reliability, including query tuning and resource management within Snowflake.
- Drive adoption of best practices in data engineering design patterns and modern cloud architectures.
- Data Quality Assurance: Implement and monitor data validation procedures to ensure data accuracy and consistency across systems.
- Collaboration and Communication: Work closely with project managers, data architects, and business analysts to align project milestones and deliverables with business goals.
- Mentor and guide data engineering teams (onsite & offshore) to deliver high-quality outcomes.
- Ensure compliance with data governance, security, and privacy standards.
- Documentation: Create and maintain detailed documentation of data pipelines, data flow diagrams, and transformation logic.
- Issue Resolution: Troubleshoot and resolve issues related to data pipelines, including job failures and performance bottlenecks.
- Bachelor's degree in Computer Science, Information Technology, or a related field.
- 8 years of experience in data engineering with a strong focus on Data Architecture, Data Modelling (Conceptual, Logical, Physical), ELT processes and data pipeline development.
- Hands-on experience with Snowflake cloud data platform, including data sharing, secure views, and performance optimization.
- Proficiency in SQL and familiarity with data integration and ETL/ELT tools.
- AWS Technologies (Glue, EMR)
- Python for data engineering workflows
- Strong understanding of data engineering design patterns
- Strong problem-solving skills and the ability to work independently to meet deadlines.
- Excellent communication skills for effectively interacting with technical and non-technical stakeholders.
