What are the responsibilities and job description for the Big Data Engineer position at Princeton IT Services, Inc?
Job Title: Big Data Engineer
Location: Mt. Laurel, NJ
Type: Full-Time Contract
Job Summary
We are seeking a highly skilled Big Data Engineer to join our team in Mt. Laurel, NJ. The ideal candidate will have hands-on experience with Big Data frameworks (Spark, Flink, Databricks, Hadoop ecosystem), streaming technologies (Kafka, Kinesis, Flink Streaming), and Cloud platforms (AWS preferred). The role involves designing and optimizing data pipelines, enabling large-scale data ingestion, transformation, and analytics to support enterprise applications and reporting.
Key Responsibilities
Location: Mt. Laurel, NJ
Type: Full-Time Contract
Job Summary
We are seeking a highly skilled Big Data Engineer to join our team in Mt. Laurel, NJ. The ideal candidate will have hands-on experience with Big Data frameworks (Spark, Flink, Databricks, Hadoop ecosystem), streaming technologies (Kafka, Kinesis, Flink Streaming), and Cloud platforms (AWS preferred). The role involves designing and optimizing data pipelines, enabling large-scale data ingestion, transformation, and analytics to support enterprise applications and reporting.
Key Responsibilities
- Design, develop, and optimize scalable data pipelines for batch and real-time processing.
- Work with streaming frameworks (Flink, Spark Streaming, Kafka) to handle large event-driven data flows.
- Build and maintain ETL/ELT processes using tools like Apache Airflow, AWS Glue, and EMR.
- Implement data lake and data warehouse solutions on AWS (S3, Redshift, DynamoDB, Athena, EMR, Lambda, Step Functions, Kinesis, MSK).
- Collaborate with data scientists, analysts, and application developers to deliver curated data sets for analytics and reporting.
- Ensure data quality, governance, and security standards across systems.
- Work with Java, Python, and Scala for data transformation, API integration, and microservices.
- Participate in Agile ceremonies and contribute to sprint planning, demos, and retrospectives.
- Troubleshoot and optimize performance of big data pipelines in production environments.
- Bachelor’s degree in Computer Science, Engineering, or related field.
- 10 years of experience as a Data Engineer or Big Data Engineer.
- Strong experience with Big Data tools: Spark (PySpark, Spark SQL, Spark Streaming), Flink, Hadoop ecosystem (Hive, HDFS, HBase, Sqoop).
- Hands-on expertise in cloud platforms (preferably AWS – EMR, Glue, S3, Kinesis, Lambda, Step Functions, DynamoDB).
- Strong programming skills in Java, Python, or Scala.
- Experience with Kafka (MSK or equivalent) for real-time data streaming.
- Proficiency with SQL and NoSQL databases (Redshift, Oracle, MongoDB, Cassandra, DynamoDB).
- Knowledge of CI/CD tools (Jenkins, Git, Bitbucket, Terraform, Docker, Kubernetes).
- Strong problem-solving, analytical, and communication skills.