What are the responsibilities and job description for the Senior Cloud Data Engineer position at National Computer Systems?
Benefits:
Remote Position
Position Overview: We are seeking a highly skilled and experienced Sr Cloud Data Engineer to join our team for a Cloud Data Modernization project.
Key Responsibilities:
- Competitive salary
- Paid time off
- Training & development
Remote Position
Position Overview: We are seeking a highly skilled and experienced Sr Cloud Data Engineer to join our team for a Cloud Data Modernization project.
Key Responsibilities:
- Lead the migration of the ETLs from on-premises SQLServer based data warehouse to Azure Cloud and Snowflake.
- Design, develop, and implement data platform solutions using Databricks, Azure Data Factory (ADF), Self-hosted Integration Runtime (SHIR), Logic Apps, Azure Data Lake Storage Gen2 (ADLS Gen2), Blob Storage, and Snowflake.
- Review and analyze existing on-premises ETL processes developed in SSIS and T-SQL.
- Implement DevOps practices and CI/CD pipelines using GitActions.
- Collaborate with cross-functional teams to ensure seamless integration and data flow.
- Optimize and troubleshoot data pipelines and workflows.
- Ensure data security and compliance with industry standards.
- Minimum of 6 years of experience as a Cloud Data Engineer.
- Hands-on experience with Databricks, Azure Cloud data tools (ADF, SHIR, Logic Apps, ADLS Gen2, Blob Storage) and Snowflake.
- Strong experience in ETL development using on-premises databases and ETL technologies
- Experience with Python or other scripting languages for data processing.
- Proficiency in DevOps and CI/CD practices using GitActions.
- Experience with Agile methodologies.
- Excellent problem-solving skills and ability to work independently.
- Strong communication and collaboration skills.
- Strong analytical skills and attention to detail.
- Ability to adapt to new technologies and learn quickly.
- Experience with the application of AI/ML tools and models to data processing and ETL workloads
- Design and implement AI/ML-enabled data pipelines to improve data quality, anomaly detection, classification, forecasting, and operational insights.
- Leverage Databricks Machine Learning, MLflow, and cloud-native AI services to support machine learning workflows.
- Integrate Generative AI capabilities, Large Language Models (LLMs), and Azure OpenAI services into data engineering processes.
- Develop automated solutions for metadata management, data cataloging, code generation, data validation, and documentation using AI technologies.
- Build scalable feature engineering pipelines to support model training and inference workloads.
- Collaborate with Data Scientists and AI Engineers to operationalize machine learning models within enterprise data platforms.
- Implement MLOps practices for model versioning, deployment, monitoring, governance, and lifecycle management.
- Experience with data modeling and database design.
- Knowledge of data governance and data quality best practices.
- Experience with development in Databricks for data engineering and analytics workloads.
- Familiarity with other cloud platforms (e.g., AWS, Google Cloud).
- Certification in Azure or Snowflake.
Salary : $50 - $52