What are the responsibilities and job description for the Sr. Data Engineer position at Jobs via Dice?
Dice is the leading career destination for tech experts at every stage of their careers. Our client, Prudent Technologies and Consulting, is seeking the following. Apply via Dice today!
Mandatory skills: Data Engineer, Python, Spark, Azure Databricks, Snowflake, complex Sql query, AWS, Airflow
Must have skills:
Mandatory skills: Data Engineer, Python, Spark, Azure Databricks, Snowflake, complex Sql query, AWS, Airflow
Must have skills:
- Data Engineering skills using -
- Complex or Advanced SQL queries
- Python
- Spark
- Snowflake
- Databricks
- Airflow or Prefect
- Experience with at least one cloud platform (AWS / Azure / Google Cloud Platform)
- Familiarity with tools like Datorama / Improvado / FiveTran for integrating, harmonizing, and visualizing data across platforms.
- Data Modeling (e.g., Dimensional Modeling)
- Experience working with Kafka
- Familiarity with CI/CD tools (e.g., Jenkins, GitHub Actions, etc.) & Docker containers
- Exposure to monitoring tools like Datadog
- Partner with technical and non-technical colleagues to understand data and reporting requirements
- Work with engineering teams to collect required data from internal and external systems
- Design table structures and create data pipelines to build performant data solutions that are reliable and scalable in a fast growing data ecosystem
- Develop automated data quality checks
- Develop and maintain ETL routines using ETL and orchestration tools such as Airflow
- Perform ad hoc analysis as necessary.
- Perform SQL and ETL tuning as necessary.