What are the responsibilities and job description for the DATABRICKS DATA ENGINEER position at National Data Solutions?
DATABRICKS DATA ENGINEER — CONTRACT (1099)
National Data Solutions | Remote — US Based Only | 1-3 Months
ENGAGEMENT OVERVIEW
NDS is supporting a data warehouse and RCM modernization program for a large national healthcare client. The immediate need is a contract Databricks Data Engineer with strong healthcare claims data experience to build the pipeline and ingestion foundation and validate claims data integrity during the program's initial phase. Defined scope, short duration, potential for extension. Remote but must be US based — no exceptions.
WHAT YOU WILL DO
Build and configure nightly automated data extraction pipelines from multiple source systems including practice management, clearinghouse, and EHR platforms. Normalize and canonicalize claims data across heterogeneous sources into a unified enterprise data model. Validate claims data integrity, completeness, and accuracy — including charge, payment, adjustment, denial, and ERA/EOB data — against a 24-month historical baseline. Identify and document data quality gaps, feed failures, and reconciliation discrepancies. Implement data lineage, refresh cadence, and governance controls within the Databricks environment. Support Power BI connectivity from the Databricks semantic layer. Coordinate directly with NDS domain leads and client stakeholders on data dictionary alignment, KPI logic, and validation findings. Participate in UAT and resolve data quality issues surfaced during dashboard validation.
WHAT YOU NEED
3 years of hands-on Databricks engineering experience including Delta Lake, Unity Catalog, and Databricks Workflows. Direct experience with healthcare claims data — charge entry, claim submission, remittance, denial management, or AR — in a data engineering or analytics context. Demonstrated ability to validate and reconcile complex multi-source claims datasets. Proficiency in PySpark and SQL. Experience connecting Databricks to Power BI via DirectQuery or semantic layer. Working knowledge of HIPAA data handling requirements and PHI masking in non-production environments. Strong communication and client-facing skills — you will interact regularly with NDS leads and client stakeholders and are expected to present findings, ask the right questions, and manage expectations professionally. Ability to work independently against a defined scope and surface blockers clearly and promptly.
NICE TO HAVE
Familiarity with clearinghouse data structures, ERA/EOB formats, or CARC/RARC code sets. Prior work in RCM, home health, or managed care analytics environments.
ENGAGEMENT DETAILS
Structure: 1099 independent contractor
Location: Remote — must be located and authorized to work in the United States
Duration: 1-3 months with potential extension
Rate: Provide hourly rate expectation when applying