What are the responsibilities and job description for the Platform Engineer position at ITTConnect?
ITTConnect is seeking a DataOps Engineer expert in AWS and Databricks for a direct-hire full time position with a client that is a large financial institution.
Position is hybrid
Key Responsibilities:
- Lead the design and implementation of the cloud data infrastructure, with a focus on AWS Databricks.
- Define and implement the DataOps strategy for a cloud native data platform.
- Design, implement, and maintain CI/CD pipelines for data and infrastructure, using Terraform and automation best practices.
- Define implementation standards for PySpark based data pipelines, Repository structure and organization, Deployment, versioning, and rollback strategies
- Implement data governance policies, including cataloging, lineage, ownership, and data layers.
- Ensure data security controls, including Access management, encryption and environment segregation
- Define and operationalize data quality practices, such as validations, SLAs, monitoring, and observability.
- Work closely with engineering teams to standardize development workflows.
- Support the migration of data from the on premise environment to the cloud, ensuring reliability and consistency.
- Act as a technical authority for architectural decisions and the evolution of the data stack.
Requirements
- 10 years of experience in IT.
- MUST HAVE experience with AWS (S3, IAM, VPC, monitoring and security services), Databricks and PySpark.
- Highly desirable previous experience working for Financial Services / Banks.
- Solid experience in DataOps, Data Engineering, or Platform Engineering.
- Hands on experience building cloud based data platforms.
- Experience with Infrastructure as Code (Terraform).
- Experience with data pipelines and orchestration concepts.
- Proven experience with CI/CD for data platforms, data governance, data quality, data security
- Ability to define technical standards and lead end to end implementations.
Nice to have:
- Fluence in Portuguese or Spanish
- Experience migrating platforms from on premise to cloud.
- Knowledge of Lakehouse architectures.
- Experience in regulated environments (financial services, insurance, healthcare).
- Experience with data observability tools.
- AWS, Databricks, or Data Engineering certifications.
- Bachelor’s degree in Computer Science or a related field