What are the responsibilities and job description for the Senior Data Engineer AI Engineer position at QUANTUM TECHNOLOGIES LLC?
Senior Data Engineer AI Engineer
Location: Mt Laurel, NJ, USA
Duration: 12 Months Extension
Bill Rate: $95/hr on C2C
Job Type: C2C/1099 Contract
Client: To Be Discussed Later
Work Authorization: US-Citizen, H-1B, OPT-EAD, GC-EAD
Job Description:
- Senior Data Engineer / AI Engineer to design, develop, and optimize scalable data processing solutions and AI-driven applications.
- Strong expertise in PySpark, Python, Databricks, SQL, Agentic AI frameworks, and relational databases including Oracle and SQL Server
- Design, develop, and maintain large-scale data pipelines using PySpark and Databricks.
- Build efficient ETL/ELT solutions to ingest, transform, and process structured and semi-structured data.
- Optimize Spark jobs for performance, scalability, and cost efficiency.
- Develop reusable data engineering frameworks and best practices.
- Create and maintain data models, data marts, and analytical datasets.
- Implement data quality checks, monitoring, and governance controls.
- Design and develop Agentic AI solutions using LLM-based frameworks.
- Build autonomous AI agents capable of reasoning, planning, and task execution.
- Production streaming experience Pub/Sub and Dataflow, or Kafka / Flink / Kinesis including deduplication, ordering, and replay.
- Strong Python, advanced SQL, and Apache Beam.
- Deep hands-on BigQuery: partitioning, clustering, incremental merge patterns, cost-aware design.
- Dimensional modeling built in a real warehouse, including conformed dimensions.
- Dataform or dbt at production scale with tests or assertions and dependency management.
- Terraform, Git workflow, and CI/CD for data pipelines.
- Experience integrating a major SaaS platform as a data source.
- Track record leading a small team or owning a workstream, with judgment on which decisions are theirs and which belong to the architect.
- Production streaming experience Pub/Sub and Dataflow, or Kafka / Flink / Kinesis including deduplication, ordering, and replay.
- Strong Python, advanced SQL, and Apache Beam.
- Deep hands-on BigQuery: partitioning, clustering, incremental merge patterns, cost-aware design.
- Dimensional modeling built in a real warehouse, including conformed dimensions.
- Dataform or dbt at production scale with tests or assertions and dependency management.
- Terraform, Git workflow, and CI/CD for data pipelines.
- Experience integrating a major SaaS platform as a data source.
- Track record leading a small team or owning a workstream, with judgment on which decisions are theirs and which belong to the architect.
Required Qualifications:
- Expertise in Scala (preferred) or Java, with proficiency in Python.
- Strong experience with Apache Spark for distributed data processing.
- Proficiency in building and maintaining ETL pipelines at scale.
- Experience with AWS services: S3, Glue, Athena, EMR
- Strong understanding of relational databases and SQL.
- Knowledge of data architecture, data warehousing, partitioning strategies, and columnar storage formats (e.g., Parquet).
- Experience implementing data quality checks and validation frameworks.
- Experience with workflow orchestration tools (Airflow preferred).
- Proficiency with Linux-based operating systems and shell scripting.
- Experience with Git-based version control and collaborative development workflows.
- Demonstrated ability and desire to continually expand skill set and learn from and teach others.
Salary : $80 - $100