Principal Data Engineer

Principal Data Engineer

Logistics:

• Full-time
• Hybrid in Westborough, MA or Woburn, MA 3 days/week
• This role does offer H1B sponsorship if you already hold a valid H1B visa.

Top things the team is looking for:

• Building Pipelines in a Databricks environment.

• Data Architect/Modeler experience is most important, Data Engineering is second. Need an expert Data Modeler to help their end users.

• Need someone who can help them use and choose the right AI to build autonomous agents for Data Engineering workflows

• Excellent communication skills – this person needs to showcase what they are building and doing to the business

• 7 years of experience

• Someone local who can be onsite 3 days/week in Woburn or Westborough. The team does a lot of whiteboarding together and in-person is what’s best.

What they are doing:

• Striving for accurate data with excellent performance and accuracy.

• Loading data into schemas/tables and improving them and dashboards through better efficiency/speed for the business to consume

• Improving the data pipelines delivered to the business

• Automate their processes with AI – what are the right tools, agents etc.

• Build a databricks platform from greenfield/scratch

• Then move their Legacy Oracle platform onto the databricks platform. Need strong Migration experience!

Responsibilities

• Design and enhance enterprise-grade data platforms, including ingestion, transformation, storage, orchestration, and data serving layers for both batch and streaming use cases
• Build and maintain scalable data pipelines, reusable frameworks, and enterprise data models to support analytics, artificial intelligence, and operational reporting
• Define and manage semantic data layers while implementing governance controls such as data quality validation, lineage tracking, metadata management, and secure access
• Establish engineering standards for development, testing, version control, documentation, and continuous integration and delivery practices
• Optimize data solutions for cost efficiency, scalability, and performance using modern engineering and operational practices
• Lead technical design reviews, incident response activities, and root cause analysis to improve platform stability and reliability
• Collaborate with data science teams to deploy and operationalize machine learning models for batch and real-time use cases
• Partner with cross-functional teams including analytics, security, and architecture to deliver compliant, high-quality data solutions
• Evaluate new technologies and guide architectural decisions, including build-versus-buy considerations
• Mentor engineering teams through technical guidance, code reviews, and knowledge sharing to raise overall engineering standards
• Promote consistency and reuse across distributed teams by sharing best practices and standardized components

Required Experience and Skills

• Bachelor’s degree in computer science, statistics, applied mathematics, or a related quantitative field
• At least 8 years of experience in data engineering or data platform development, including significant experience in senior or principal-level roles
• Strong proficiency in SQL, Python, and large-scale data processing frameworks such as Apache Spark or PySpark
• Hands-on experience with major cloud platforms such as AWS, Azure, or Google Cloud, along with modern data platforms such as Databricks or Snowflake
• Experience with streaming technologies such as Apache Kafka or similar tools and orchestration frameworks such as Apache Airflow
• Strong background in data modeling, including dimensional, data vault, and domain-oriented approaches
• Experience implementing data governance frameworks, including quality controls, lineage tracking, metadata management, and access controls
• Knowledge of software engineering practices including CI/CD, infrastructure as code, and automated testing
• Proven ability to lead complex technical initiatives, influence stakeholders, and guide engineering teams
• Strong communication skills with the ability to present technical concepts clearly and effectively
• Ability to manage multiple priorities and work effectively in fast-paced environments