What are the responsibilities and job description for the Data Engineer position at Madagoni LLC?
Job Summary
We are seeking a dynamic and innovative Data Engineer to join our team and drive the development of scalable, efficient data solutions. In this role, you will be responsible for designing, building, and maintaining robust data pipelines and architectures that support advanced analytics, machine learning models, and business intelligence initiatives. Your expertise will enable us to harness big data technologies and cloud platforms to deliver actionable insights that propel our organization forward. This is an exciting opportunity for a motivated professional eager to work with cutting-edge tools in a fast-paced, collaborative environment.
Responsibilities
- Design and implement scalable data pipelines using ETL (Extract, Transform, Load) processes to integrate data from diverse sources such as AWS, Azure Data Lake, Oracle, and Microsoft SQL Server.
- Develop and optimize data models and database schemas for data warehouses and big data platforms like Hadoop, Apache Hive, Spark, and Looker.
- Collaborate with cross-functional teams to gather requirements and translate them into technical specifications for data solutions.
- Build automated workflows using Shell Scripting, Bash (Unix shell), Talend, Informatica, and other ETL tools to ensure reliable data processing.
- Manage cloud-based data storage solutions on AWS and Azure environments while ensuring security and compliance standards are met.
- Support model training and analysis efforts by providing clean, well-structured datasets for analytics teams.
- Create RESTful APIs for seamless data access across applications while maintaining high performance standards.
- Conduct database design activities focusing on efficiency, scalability, and data integrity to support enterprise-wide analytics initiatives.
- Use Python, VBA, SQL, and Shell Scripting to troubleshoot issues, automate tasks, and improve overall system performance.
- Participate in Agile development cycles to continuously improve data infrastructure and support evolving business needs.
Requirements
- Proven experience working with big data technologies such as Hadoop, Spark, Apache Hive, and related ecosystems.
- Strong proficiency in SQL programming with experience in Microsoft SQL Server, Oracle databases, or similar systems.
- Hands-on experience with cloud platforms including AWS (Amazon Web Services) and Azure Data Lake services.
- Solid understanding of ETL processes using Talend, Informatica, or equivalent tools; familiarity with RESTful API development is a plus.
- Proficiency in programming languages such as Java and Python for data processing tasks; VBA scripting skills are advantageous.
- Knowledge of database design principles for data warehouses supporting analytics functions.
- Experience working with Looker or similar BI tools for reporting and visualization purposes.
- Familiarity with big data frameworks like Hadoop ecosystem components (HDFS), Apache Hive, Spark for large-scale data processing.
- Ability to implement shell scripting using Bash or Unix shell environments to automate workflows efficiently.
- Strong analysis skills combined with the ability to interpret complex datasets into actionable insights.
- Experience working within Agile methodologies to deliver iterative improvements rapidly while maintaining high quality standards. Join us in shaping the future of data-driven decision-making! We value energetic professionals who thrive on innovation and are eager to leverage the latest technologies in a collaborative setting. If you’re passionate about transforming raw data into strategic assets through cutting-edge tools like AWS, Spark, Hadoop, and Looker—this is your chance to make an impact!
Pay: $30.00 - $35.00 per hour
Work Location: In person
Salary : $30 - $35