What are the responsibilities and job description for the Data Infrastructure Engineer position at STEM Search Group?
What you'll do
- Keep production data pipelines running across a cloud data warehouse and a real-time streaming platform, catching problems before they turn into outages
- Tune warehouse performance, manage access roles, and keep compute costs in check as usage grows
- Configure and maintain streaming infrastructure (topics, partitions, consumer groups) so data moves in near real time without gaps or duplicates
- Build automated checks that catch bad data, failed loads, or delivery delays before anyone downstream notices
- Write scripts and small tools in Python and SQL to cut down manual, repetitive pipeline work
- Support CI/CD for data infrastructure changes so updates ship safely and roll back cleanly if something breaks
- Work with data engineering and analytics teams to fix root causes, not just symptoms, when pipelines fail
What we're looking for
- Hands-on production experience running and tuning a cloud data warehouse (Snowflake or similar)
- Real, working experience with a streaming platform (Kafka or similar): producers, consumers, schema handling, and troubleshooting
- Comfortable in SQL and at least one scripting language, Python preferred
- Experience with a major cloud provider (AWS, Azure, or GCP)
- Familiarity with orchestration tools like Airflow or dbt
- Solid grasp of data modeling and the full data lifecycle, from ingestion to delivery
- Able to explain a pipeline issue clearly to both engineers and non-technical stakeholders