What are the responsibilities and job description for the Principal Data Engineer position at IDR, Inc.?
IDR is seeking a Principal Data Engineer to join one of our top clients for an opportunity in Alpharetta, Georgia. This role is ideal for a highly experienced data professional with a strong background in designing and scaling complex data pipelines and APIs in cloud environments. The company operates within the data technology and analytics industry, focusing on innovative solutions for analytics and AI/ML use cases.
Position Overview for the Principal Data Engineer:
- Drive the strategic design and evolution of highly optimized, secure, and scalable ETL/ELT data pipelines across diverse, complex data sources
- Architect and operationalize data marts and domain-oriented data models that enable reliable, performant, and democratized access for analytics and AI/ML
- Lead the design and rollout of API-based, event-driven data services that power mission-critical products and real-time decisioning
- Establish enterprise-grade observability frameworks, including automated testing, lineage tracking, and monitoring systems for proactive detection and rapid remediation of data issues
- Provide technical leadership, setting architectural direction and guiding multiple project teams to successful delivery
Requirements for the Principal Data Engineer:
- 12 years of hands-on experience in software development, data structures, and algorithms
- 5 years of Snowflake experience
- 5 years of Spark experience
- 4 years of experience in technical leadership, setting architectural direction, and guiding multiple project teams to successful delivery
- Deep expertise in cloud platforms and services such as AWS (Redshift, Kinesis, EMR) or GCP (BigQuery, Dataflow, Pub/Sub)
- Mastery of programming languages including Python, Node, Scala, or Go
- Proficiency in SQL and NoSQL databases including PostgreSQL, MySQL, MongoDB, Cassandra
What's in it for you?
- Competitive compensation package
- Full Benefits; Medical, Vision, Dental, and more!
- Opportunity to get in with an industry leading organization.