What are the responsibilities and job description for the Sr Data Engineer - (Hybrid - NYC, NY / Seattle, WA / Santa Monica, CA / San Francisco, CA) position at XFORIA Inc?
Job Details
Sr. Data Engineer
Fulltime / Direct Hire
Location: Hybrid - NYC, NY / Seattle, WA / Santa Monica, CA / San Francisco, CA
Full time – (9 – 15 Years exp)
Mandatory skills: Data Engineer, Python, Spark, Azure Databricks, Snowflake, complex Sql query, AWS, Airflow
Must have skills:
• Data Engineering skills using -
• Complex or Advanced SQL queries
• Python
• Spark
• Snowflake
• Databricks
• Airflow or Prefect
• Experience with at least one cloud platform (AWS / Azure / Google Cloud Platform)
Nice to have skills:
• Familiarity with tools like Datorama / Improvado / FiveTran for integrating, harmonizing, and visualizing data across platforms.
• Data Modeling (e.g., Dimensional Modeling)
• Experience working with Kafka
• Familiarity with CI/CD tools (e.g., Jenkins, GitHub Actions, etc.) & Docker containers
• Exposure to monitoring tools like Datadog
Responsibilities:
• Partner with technical and non-technical colleagues to understand data and reporting requirements
• Work with engineering teams to collect required data from internal and external systems
• Design table structures and create data pipelines to build performant data solutions that are reliable and scalable in a fast growing data ecosystem
• Develop automated data quality checks
• Develop and maintain ETL routines using ETL and orchestration tools such as Airflow
• Perform ad hoc analysis as necessary.
• Perform SQL and ETL tuning as necessary.