What are the responsibilities and job description for the Big Data Developer position at MM International, LLC?
Job Description
Job Title: Big Data Engineer
Location: Richmond, VA / McLean, VA (Onsite)
We are seeking a Big Data Engineer with strong hands-on experience in Python and AWS Infrastructure to support the design, development, and maintenance of scalable data platforms and cloud-based data solutions. The ideal candidate will have prior experience working in enterprise environments and building robust big data pipelines using AWS-native services and infrastructure best practices.
Responsibilities
- Design, build, and maintain scalable big data pipelines and cloud-based data platforms on AWS.
- Develop and optimize data processing solutions using Python for large and complex datasets.
- Work with AWS infrastructure components such as S3, EMR, Glue, Redshift, EC2, IAM, and related cloud services.
- Support data lake, ETL, and distributed data processing workflows in a production environment.
- Monitor data pipeline performance, troubleshoot issues, and ensure system reliability, scalability, and security.
- Collaborate with cross-functional teams including data engineers, architects, developers, and business stakeholders to deliver data solutions.
Required Skills
- Strong experience with Python development for data engineering and large-scale data processing.
- Hands-on experience with AWS Infrastructure and AWS cloud services.
- Experience building and supporting big data or data warehouse/data lake solutions.
- Good understanding of ETL, distributed systems, and cloud-based data architecture.
- Experience with AWS services such as S3, EMR, Glue, Redshift, EC2, IAM, Lambda, or CloudWatch is preferred.
- Strong problem-solving, debugging, and performance tuning skills.
Must Have
- Python
- AWS Infrastructure
- Experience working with enterprise-level environments, preferably from strong end-client backgrounds.
Preferred Qualifications
- Experience with Spark, PySpark, Airflow, SQL, or data pipeline orchestration tools.
- Experience with Infrastructure as Code tools such as Terraform or CloudFormation.
- Prior experience in financial services or other large-scale enterprise environments is a plus.
Equal Opportunity Statement
We are committed to diversity and inclusivity.