What are the responsibilities and job description for the Senior Data Engineer position at ITTConnect?
ITTConnect is seeking a Data Platform Engineer for a direct-hire full time position with a client that is a large financial institution.
Position is hybrid in Miami.
Our client is at a strategic moment in our data platform transformation, migrating from an on premise environment to a modern cloud native stack based on AWS, Databricks, and PySpark.
They are looking for a Platform Engineer to support the implementation of the new cloud data infrastructure, implementing technical standards, development pipelines, automation with CI/CD and Terraform, and deliver with that data governance, security, and data quality embedded from day one.
This person will help to build the foundation of the new data ecosystem, acting as a bridge between Data, Cloud, Security, and Engineering teams.
Key Responsibilities:
• Implementation of the cloud data infrastructure, with a focus on AWS Databricks.
• Implement the DataOps strategy for a cloud native data platform.
• Build and maintain CI/CD pipelines for data and infrastructure, using Terraform and automation best practices.
• Replicate implementation standards for:
- PySpark based data pipelines
- Repository structure and organization
- Deployment, versioning, and rollback strategies
• Review data governance policies, including cataloging, lineage, ownership, and data layers.
• Ensure data security controls, including Access management, encryption and environment segregation
• Apply data quality practices, such as validations, SLAs, monitoring, and observability.
• Work closely with engineering teams to standardize development workflows.
• Support the migration of data from the on premise environment to the cloud, ensuring reliability and consistency.
• Act as a technical support for architectural decisions and the evolution of the data stack.
Qualifications:
- 8 years of experience in IT.
- MUST HAVE experience with AWS (S3, IAM, VPC, monitoring and security services), Databricks and PySpark.
- Highly desirable previous experience working for Financial Services / Banks.
- Solid experience in DataOps, Data Engineering, or Platform Engineering.
- Hands on experience building cloud based data platforms.
- Experience with Infrastructure as Code (Terraform).
- Experience with data pipelines and orchestration concepts.
- Proven experience with CI/CD for data platforms, data governance, data quality, data security
- Ability to define technical standards and lead end to end implementations.
Nice to have:
- Experience migrating platforms from on premise to cloud.
- Knowledge of Lakehouse architectures.
- Experience in regulated environments (financial services, insurance, healthcare).
- Experience with data observability tools.
- AWS, Databricks, or Data Engineering certifications.
- Bachelor’s degree in Computer Science or a related field