Senior Cloud Data & AI Platform Engineer

Senior Cloud Data & AI Platform Engineer

Location: Birmingham AL Onsite

Duration: 6 mo to start

Pay Rate: $97-$102 per hour DOE (W2 and benefit options)

Role Overview

The Senior Cloud Data & AI Platform Engineer is a hands-on technical leader responsible for designing, building, and supporting advanced data, AI, and application solutions within an Azure/Databricks-based Lakehouse environment.

This role combines modern software engineering with data and AI platform development, focusing on scalable, secure, and auditable systems. The engineer will develop AI-enabled data pipelines, orchestration frameworks, and enterprise-grade applications while providing high-level production support in a large-scale distributed environment.

The position partners closely with data engineering, analytics, and application teams to deliver production-ready solutions aligned with enterprise architecture and regulatory standards.

Key Responsibilities

Data, AI & Application Engineering

• Design, develop, and implement scalable software solutions using modern object-oriented and web-based technologies (e.g., Python, .NET, Java).
• Build AI-enabled data pipelines and orchestration frameworks for analytics, automation, and decision support.
• Develop reusable, testable components for data validation, metadata enrichment, and AI-assisted workflows.
• Design, create, test, and modify applications and programs to meet business and operational needs.

Azure Databricks & Lakehouse Platform

• Deploy and operate solutions within Azure Databricks, leveraging:
• Delta Lake and Medallion Architecture (Bronze/Silver/Gold)
• Databricks Workflows and Jobs
• Unity Catalog for governance and access control
• Ensure solutions align with enterprise Lakehouse architecture and platform standards.

System Integration & Development

• Design and implement secure APIs and integration patterns across internal and external systems.
• Develop technical specifications and recommend tools, technologies, and solutions.
• Support development of new systems and enhancements to existing applications.

Production Support & Troubleshooting

• Provide high-level application support, including troubleshooting complex issues in distributed environments.
• Perform root cause analysis, problem determination, and resolution.
• Support software installation, configuration, updates, and ongoing system maintenance.
• Participate in 24x7 support as required.

Performance, Cost & Reliability

• Monitor and optimize Spark workloads, applications, and orchestration processes for performance and scalability.
• Ensure efficient resource utilization and adherence to cloud cost management practices.
• Build resilient systems capable of handling high-volume and variable workloads.

Technical Leadership & Standards

• Contribute to system architecture, design reviews, and engineering best practices.
• Ensure code quality, modularity, and maintainability across all solutions.
• Develop and maintain documentation, including technical specifications, runbooks, and architectural artifacts.
• Identify opportunities for process improvement and innovation.

Technical Qualifications

Core Technical Skills

• 10 years of experience in software engineering, cloud engineering, or data platform development.
• Strong experience with object-oriented and web-based technologies, including .NET and Java.
• Advanced proficiency in Python, including object-oriented and event-driven programming.
• Strong data engineering experience with PySpark, Delta Lake, and data lake architectures.
• Experience developing applications in large-scale distributed systems.

Azure & AI Technologies

• Hands-on experience with:
• Azure Databricks
• Azure Functions and/or Logic Apps
• Azure container-based services
• Experience implementing AI/LLM-enabled workflows and structured orchestration patterns (e.g., agent-based or task-driven frameworks).

Platform & DevOps

• Experience with CI/CD pipelines using GitHub Actions or Azure DevOps.
• Familiarity with infrastructure-as-code and cloud environment management.
• Strong understanding of secure development practices, especially in regulated environments.

Preferred Experience

• Experience in energy, utilities, nuclear, or other highly regulated industries.
• Familiarity with handling sensitive operational, telemetry, or regulatory data.

Performance Expectations

• Reliability & Auditability: Deliver solutions that are traceable, testable, and compliant with regulatory standards.
• Scalability & Resilience: Design systems that operate effectively under varying workloads.
• Operational Excellence: Provide strong production support and maintain system stability.
• Documentation & Standards:
• Adhere to PEP 8 and enterprise coding standards
• Maintain clear documentation, including code comments, READMEs, and architectural designs