What are the responsibilities and job description for the Automation Engineer- Ansible/Puppet/Chef position at Konane Solutions?
Automation Engineer
We are seeking someone with experience with automation and orchestration tools such as Ansible, Puppet, and Chef. Experience in UNIX, Linux, and/or Windows Operating System (OS), including file management, scripting, editing, and security. Experience with a variety of programming languages, software, or tools to develop regular or ad-hoc queries from large datasets. Familiar with network communication protocols, and other related protocols. Ability to implement solutions integrating Rest application program interface (API) based web services.
Responsibilities
- Evaluate and recommend data ingestion and pipeline tools (open source and vendor solutions) to support scalable data workflows.
- Design and implement data pipelines to ingest, process, and transform structured and unstructured data from diverse sources.
- Develop and manage data orchestration workflows using open-source or COTS tools to automate and monitor complex data pipelines.
- Design, develop, and maintain APIs or leverage existing APIs to enable seamless data integration across COTS, custom-built applications, and data platforms.
- Work with both on-premises infrastructure (Oracle/Microsoft databases, MongoDB) and cloud platforms (AWS, Azure, Google Cloud Platform, or others) to support current and future data architectures.
- Integrate data pipelines with various applications, ensuring seamless data flow and interoperability.
- Collaborate with database administrators, application owners, solution architects, and other stakeholders to understand data requirements and deliver robust solutions.
- Implement and manage data virtualization solutions, with a preference for experience in Denodo.
- Ensure data quality, security, and compliance throughout the data lifecycle.
- Document processes, architectures, and best practices for data engineering workflows
Qualifications
- BS in relevant discipline plus minimum 3 years or more years of directly related experience that demonstrates the knowledge, skills, and ability to perform the duties of the job.
- Must be able to obtain and maintain a DOE Security Clearance
- Work is performed onsite and therefore candidate must reside in the Albuquerque area.
Required Skills:
- Knowledge in automation technologies such as Ansible, Puppet, or Chef.
- Strong programming skills in Python and SQL.
- Experience designing, developing, and consuming RESTful APIs and other API technologies to facilitate data integration and interoperability.
- Experience with ETL/ELT tools and frameworks (e.g., dbt, PySpark).
- Proven experience (3 years) in data engineering, data pipeline development, and tool evaluation.
- Excellent problem-solving skills and ability to work collaboratively in a cross-functional team.
Desired Skills:
- Knowledge of data virtualization concepts and experience with Denodo preferred.
- Hands-on experience with data orchestration tools such as Apache Airflow, Prefect, Luigi, or similar platforms for workflow automation and scheduling.
- Familiarity with big data technologies (Hadoop, Spark, Kafka) is a plus.
- Hands-on experience with cloud platforms (AWS, Azure, Google Cloud Platform) and hybrid data architectures.
- Experience integrating data pipelines with COTS and custom applications.
- Strong understanding of data modeling, metadata management, and data governance.