Overview

The Machine Learning Engineer is focused on supporting data scientists by building structures to house, expose, and execute models. Often this is in support of enhancing human decision pipelines, so that humans can focus on the more creative and difficult tasks and models can drive automation on the "easy 80%". This is often a complex problem given the variability of systems at play in the credit union over the years. The data science team supports a mixture of batch and live models. The Machine Learning Engineer is involved in the entire life cycle of the model from inception tomonitoring, and if needed retirement. They work closely with Data Scientists to deploy models into various systems and Data Engineers to create the feature groups needed to support ongoing and future modeling needs, and do a lot of data engineering themselves. Their core focus and ownership within that pipeline is the tooling used to deliver models to production and serve out to the greater enterprise in general. This includes building APIs and other types of interfaces. The end state should be that Data Scientists can self serve as much as possible through the tooling and pipelinescreated by the Machine Learning Engineer.

We do not model for the sake of modeling or to pad resumes. Everything is focused on working with a business owner to get the best (balancing complexity and accuracy) model into production and executing, even if that is a decision tree and not a neural network. We do not expect you to have all of the skills or exposure to all the tools listed below but be willing to work towards mastering them, and even help evolve their implementation (we do not know everything either). We put a heavy emphasis on being about to deploy models into production which includes writing tests, setting upmonitoring, and performing code reviews.

Responsibilities

Communicate. It all starts and ends with communicating with business stakeholders. The Machine Learning Engineer works as an internal consultant, consulting with business functions across different financial functions. Communicating, and being able to learn the language of that business unit, is critical to success of any Machine Learning Engineer. So financial experience is preferred, but not a must, as we have awesome product owners that are willing to help along the way. Often the Machine Learning Engineer will be communicating directly with senior management.

Support Model Development. The data science team has numerous model domains from fraud to credit risk to recommendations. Supporting models in conjunction with the business owners is a core responsibility. We use lots of the popular model frameworks such as Logistic Regression, Random Forest...however, we also aid business owners in honing their case logic to be more accurate. Given the variety of technology, and capability, it often takes some creativity and problem solving to implement a model appropriately.

Deploy Models. A model not in production is not making business impact (some exceptions apply). Production comes in two forms at AFCU, batch and live. The Machine Learning Engineer will work with the data scientists to deploy their model into the target system using a variety of technologies using our Continuous Integration and Continuous Delivery (CI/CD) pipelines. This will include building out testing, creating or updating model documentation, and creating or fine-tuning on-going monitoring systems.

Monitor Models. A data science project is not complete without a robust monitoring solution in place. Given the stakes of a model making decisions without human intervention, a heavy emphasis is put on monitoring the models. This ranges everywhere from data metrics such as accuracy and recall to run-time and call volume. The Machine Learning Engineer will be required to set this up as a part of the model development pipeline within our monitoring tool.

Educate. One of the core functions the Machine Learning Engineer will be educating citizen data scientists and business owners on data science best practices. A lot of which is tearing down the marketing fluff that ML/AI/Data Science...is magic and can solve all your problems and instead that it is just math and programming, and can be understood.

Document. Being a financial institution, we have a significant amount of documentation required for each model to comply with auditors. While auto doc tools get us 70% of the way, the last 30% requires a human touch. This last 30% brings in the context of the business, model application, and outcome. The Machine Learning Engineer is responsible for helping with this last mile.

Innovate. The data science team is structured in a centralized manner within the credit union, which gives us the ability to see across several business functions. Part of the team's role is to look at projects from the different business functions and implement cross functional synergies between different projects.

Qualifications

• Required Master's of Science in a STEM field or a Bachelor's of Science degree in a STEM field with equivalent experience of 3 years.
• 3 years of programming experience, preferably with Python, SQL, and PySpark.
• 3 years building analytical systems.
• 3 years working with product owners to get buy in for model use.
• 3 years building enterprise interfaces for clients to consume from such as REST, SOAP.
• 3 years working with distributed and container systems.
• Demonstrated experience working in a data warehouse/feature store environment.
• Demonstrated experience in creating and maintaining data models.
• Demonstrated experience with various data management technology; NoSQL; relational database; message broker; multidimensional database; and design architecture.
• A strong understanding of the model life cycle.
• A strong understanding of how to monitor models.
• Demonstrated experience working within an SDLC environment including tools such as git, wikis, task management, and build agents.
• Financial institution experience, or active participation in the US credit system as a consumer, is desired.