Data Engineer

Data Platform Engineer

📍 Location: Chicago, IL, US

🏢 Industry: Financial Services

💼 Work Setting: Hybrid

Are you passionate about building scalable cloud data platforms, designing modern data pipelines, and delivering analytics solutions that power meaningful customer experiences and business outcomes?

We are seeking a Senior Data Engineer to design, build, and optimize cloud-first data platforms, pipelines, and data products that support large-scale analytics, machine learning, and business intelligence initiatives. This role combines data engineering, cloud technologies, distributed data processing, and technical leadership to deliver reliable, secure, and scalable data solutions.

The ideal candidate brings expertise in Python, Spark, SQL, Databricks, Snowflake, cloud platforms, distributed systems, and data architecture while collaborating closely with Product, Engineering, Data Science, and Analytics teams.

Key Responsibilities

Data Engineering & Platform Development

• Design, develop, test, deploy, and support enterprise-scale data solutions.
• Build scalable cloud-first data platforms and applications.
• Develop reusable data assets, frameworks, and engineering standards.
• Ensure data platforms support analytics, reporting, and machine learning initiatives.
• Drive innovation through modern data engineering practices.

Focus Areas

• Data Engineering
• Data Platform Development
• Cloud Data Solutions
• Distributed Systems
• Lakehouse Architecture

Data Pipeline Design & Development

• Design and implement scalable batch and streaming data pipelines.
• Develop resilient data ingestion, transformation, and processing frameworks.
• Optimize pipeline performance for growing business and data demands.
• Ensure reliability, maintainability, and operational efficiency.
• Support end-to-end data lifecycle management.

Responsibilities

• ETL/ELT Development
• Data Integration
• Pipeline Automation
• Real-Time Processing
• Data Transformation

Cloud Data Architecture

• Build and deploy data solutions in cloud environments.
• Develop architecture patterns supporting high-volume analytical workloads.
• Enable scalable and secure cloud-based data processing.
• Support modernization initiatives through cloud-native technologies.
• Improve platform performance and operational effectiveness.

Cloud Platforms

• AWS
• Microsoft Azure
• Google Cloud Platform (GCP)

Analytics & Machine Learning Enablement

• Partner with Data Scientists and Analysts to support analytics and machine learning workloads.
• Deliver trusted, high-quality datasets for business users.
• Support large-scale analytical and predictive modeling initiatives.
• Improve accessibility and usability of enterprise data assets.
• Enable data-driven decision-making across the organization.

Data Modeling & Database Design

• Design and maintain data models supporting analytical and operational use cases.
• Develop solutions using both relational and non-relational databases.
• Ensure efficient storage, retrieval, and processing of data.
• Improve data consistency and scalability across systems.

Technologies

• SQL Databases
• NoSQL Databases
• Data Warehouses
• Lakehouse Architectures

Distributed Data Processing

• Develop solutions using distributed data technologies.
• Build scalable compute workloads supporting complex analytics.
• Optimize large-scale processing jobs for performance and reliability.
• Support both structured and semi-structured data processing.

Technologies

• Apache Spark
• Databricks
• EMR
• Glue

Technical Leadership & Mentorship

• Influence engineering best practices and design standards.
• Mentor engineers and share technical expertise.
• Promote reusable patterns and scalable engineering solutions.
• Participate in internal technology communities.
• Drive continuous learning and innovation.

Data Security & Governance

• Implement data security controls and governance standards.
• Protect sensitive information through encryption and access controls.
• Ensure compliance with data privacy and security requirements.
• Support data quality and stewardship initiatives.
• Maintain trust and integrity of enterprise data assets.

Areas of Focus

• Data Security
• Data Privacy
• Governance
• Access Management
• Compliance

Cross-Functional Collaboration

Partner closely with:

• Product Managers
• Software Engineers
• Data Scientists
• Data Analysts
• Business Stakeholders

Responsibilities

• Translate business needs into technical solutions.
• Communicate technical concepts effectively.
• Align data initiatives with organizational objectives.
• Drive adoption of data products and platforms.

Required Qualifications

Education

• Bachelor's Degree in:
• Computer Science
• Engineering
• Statistics
• Mathematics
• Analytics
• Economics
• Operations Research
• Related Quantitative Field

Experience

• 4 years of application development experience.
• 2 years working with distributed data systems.
• 2 years of SQL experience.
• 2 years of experience with Python, Java, or Scala.
• 2 years of experience designing and developing data pipelines.
• 1 year of experience designing data models and end-to-end data solutions.
• Experience with relational and non-relational databases.

Preferred Qualifications

Experience

• 7 years of software or data engineering experience.
• 4 years designing and operating cloud-based data workloads.
• 4 years working with Spark, Databricks, EMR, or similar distributed platforms.
• 4 years building real-time and streaming data pipelines.
• Experience developing reusable enterprise data products.
• Experience working in Agile environments.

Technical Skills

Programming

• Python
• SQL
• Scala
• Java

Data Engineering

• Data Pipelines
• ETL/ELT
• Data Transformation
• Data Integration
• Streaming Data

Cloud Technologies

• AWS
• Azure
• GCP
• Cloud-Native Data Platforms

Data Processing

• Apache Spark
• Databricks
• EMR
• Glue

Databases

• Relational Databases
• NoSQL Databases
• MongoDB
• Cassandra
• DynamoDB

Data Warehousing

• Snowflake
• Amazon Redshift
• Lakehouse Architecture

Orchestration & Observability

• Airflow
• Dagster
• Splunk
• Monte Carlo

Professional Competencies

• Analytical Thinking
• Problem Solving
• Technical Leadership
• Collaboration
• Communication Skills
• Innovation
• Adaptability
• Continuous Learning

Core Competencies

• Data Engineering
• Python
• SQL
• Spark
• Databricks
• Snowflake
• Cloud Computing
• AWS
• Azure
• GCP
• Data Pipelines
• Data Architecture
• Data Modeling
• Data Warehousing
• Streaming Data
• NoSQL
• Lakehouse Architecture
• Machine Learning Data Platforms
• Data Security
• Agile Development