What are the responsibilities and job description for the Senior Data Engineer position at ASCII Group LLC?
Job Details
The following requirement is open with our client.
Title : Senior Data Engineer
Location : Newark, NJ(Onsite)
Duration : 6 Months
Relevant Experience (in Yrs.): 8 yrs.
Role Description:
Relevant Experience 8 Years
Roles & Responsibilities
• Data Management and Storage:
o Design and implement data storage systems using Azure services like Azure SQL Database, Azure Data Lake Storage, and Azure Synapse.
o Ensure scalability, performance, and cost-effectiveness.
• Data Integration and ETL (Extract, Transform, Load):
o Develop and implement data integration processes using Azure Data Factory.
o Extract data from various sources, transform it, and load it into data warehouses or data lakes.
• Big Data and Analytics:
o Utilize big data technologies such as Apache Spark.
o Create data processing workflows and pipelines to support data analytics and machine learning applications.
• Build and maintain new and existing applications in preparation for a large-scale architectural migration within an Agile function.
• Monitor and optimize data pipelines and database performance to ensure data processing efficiency.
• Build interfaces for supporting evolving and new applications and accommodating new data sources and types of data.
• Document data engineering processes, data models, and pipelines to ensure transparency and maintainability.
Data Lake Proficiency in:
o Software development and scripting languages
o ETL tools (e.g., SSIS, Azure Data Factory, Power BI Dataflow).
o Database management (MSSQL, Azure SQL Database, Azure Data Lake Storage, and Azure Synapse).
• Knowledge of data modeling and data warehousing concepts.
• Excellent problem-solving and troubleshooting abilities.
• Attention to detail and commitment to data accuracy.
• Experience with cloud-based data migration.
• Strong analytical and problem-solving skills, with ability to conduct root cause analysis on system, process or production problems and ability to provide viable solutions.
• Experience working in an Agile environment with Scrum Master/Product owner and ability to deliver.
• 3 years of programming experience in Python/Pyspark.
• Knowledge of Jira, Confluence, SAFe development methodology & DevOps
Must Have Skills:
Azure Data Engineer, Microsoft Fabric
SQL Database, Azure Data Lake Storage, Azure Synapse.
ETL, ADF, Apache, Power BI, DevOps