What are the responsibilities and job description for the Associate Data Engineer position at Brooksource?
Jr. Data Engineer
6M CTH
Hybrid- Irving, TX
Description:
This position will be responsible for leveraging cutting edge technology to solve business problems at our client by participating in all phases of the development process from inception through transition, advocating the agile process and test-driven development, using Big Data and Data Warehousing tools to analyze, model, design, and construct.
Responsibilities:
- Work closely with source data application teams and product owners to design, implement and support analytics solutions that provide insights to make better decisions
- Implement data migration and data engineering solutions using Azure products and services: (Azure Data Lake Storage, Azure Data Factory, Azure Functions, Event Hub, Azure Stream Analytics, Azure Databricks, etc.) and traditional data warehouse tools
- Perform multiple aspects involved in the development lifecycle – design, cloud engineering (Infrastructure, network, security, and administration), ingestion, preparation, data modeling, testing, CICD pipelines, performance tuning, deployments, consumption, BI, alerting, prod support
- Provide technical support and collaborate within a team environment, as well as work independently
- Be a part of a DevOps team that completely owns and supports their product
- Implement batch and streaming data pipelines using cloud technologies
- Develops coding standards, best practices and privacy and security guidelines
Minimum Qualifications:
- Bachelor's degree in Computer Science, Computer Engineering, Technology, Information Systems (CIS/MIS), Engineering or related technical discipline, or equivalent experience/training
- 2 years software solution development using agile, DevOps, operating in a product model that includes designing, developing, and implementing large-scale applications or data engineering solutions
- 2 years data analytics experience using SQL
- 1 year of cloud development and data lake experience (prefer Microsoft Azure) including Azure EventHub, Azure Data Factory, Azure Databricks, Azure DevOps, Azure Blob Storage, Azure Data Lake, Azure Power Apps and Power BI.
- Combination of Development, Administration & Support experience in several of the following tools/platforms required:
- Scripting: Python, Spark, Unix, SQL
- Data Platforms: Teradata, Cassandra, MongoDB, Oracle, SQL Server, ADLS, Snowflake, Azure Data Explorer. Administration skills a plus
- Azure Cloud Technologies: Azure Data Factory, Azure Databricks, Azure Blob Storage, Azure Data Lake, Azure Power Apps and Azure Functions
- CI/CD: GitHub, Jenkins, Azure DevOps, Terraform
- BI Analytics Tool Stack - Cognos, Tableau, Power BI, Alteryx, Denodo, and Grafana
- Data Warehousing: DataStage, Informatica
- Data Governance and Privacy: Informatica Axon and EDC, BigID
Preferred Qualifications:
- 2 years software solution development using agile, dev ops, product model that includes designing, developing, and implementing large-scale applications or data engineering solutions.
- 2 years data analytics experience using SQL
- 1 year full-stack development experience, preferably in Azure
- 1 year of cloud development and data lake experience (prefer Microsoft Azure) including Azure EventHub, Azure Data Factory, Azure Functions, ADX, ASA, Azure Databricks, Azure DevOps, Azure Blob Storage, Azure Data Lake, Azure Power Apps and Power BI.
- Airline Industry Experience
- Experience with the Azure Technology stack for data management, data ingestion, capture, processing, curation and creating consumption layers.
- Azure Development Track Certification
- Spark Certification