What are the responsibilities and job description for the Lead Data Engineer (Snowflake, AWS, StreamSets) position at Apexon?
Lead Data Engg
Location : Frisco, Texas
Responsibilities & Duties
- Lead the design, development, and scaling of data platforms and pipelines using Snowflake on AWS
- Build and optimize data ingestion frameworks using StreamSets and AWS S3 for structured, semi-structured, and unstructured data
- Develop and manage robust ETL/ELT pipelines leveraging AWS services (S3, Glue, Lambda, DMS) and Python-based data processing
- Design and enhance database schemas (MySQL/PostgreSQL) with a focus on performance, scalability, and maintainability
Required Qualifications & Skills
- Bachelor’s degree in Computer Science, Engineering, or related field
- 8–12 years of experience in data engineering, ETL development, or data platform engineering
- Strong hands-on experience with Snowflake (data modeling, performance tuning, virtual warehouses)
- Deep expertise in AWS ecosystem, especially AWS S3, Glue, Lambda, DMS, and IAM
- Proven experience with StreamSets for data ingestion and pipeline orchestration
- Strong proficiency in Python for data engineering, automation, and pipeline development
- Advanced experience in SQL, schema design, and performance optimization (MySQL/PostgreSQL)
- Solid understanding of data lakes, data warehouses, and ETL/ELT best practices
- Experience with Apache tools such as Airflow, Kafka, or Superset
- Familiarity with Linux systems, networking basics, and system performance tuning
- Strong problem-solving skills with experience in incident management and root cause analysis
- Excellent communication and collaboration skills