What are the responsibilities and job description for the Senior Data Engineer Palantir Foundry (W2 Only) position at 3BEES TECHNOLOGIES INC?
Job Summary:
We are seeking a Senior Data Engineer to design, build, and maintain enterprise-scale data platforms using modern cloud-native technologies. The ideal candidate will have strong experience with Palantir Foundry, Databricks, and/or Snowflake, along with expertise in scalable data pipeline development and data warehousing.
Key Responsibilities:
- Design and implement end-to-end data pipelines using Palantir Foundry, Databricks, and Snowflake
- Develop scalable ETL/ELT workflows using Python, PySpark, and SQL
- Build and maintain data lake and data warehouse architectures on Azure (Data Factory, Synapse, ADLS)
- Implement data governance, data quality, and metadata management frameworks
- Develop and optimize Databricks notebooks, Delta Lake tables, and Unity Catalog implementations
- Design and manage Palantir Foundry ontologies and transformation pipelines
- Collaborate with data scientists, analysts, and business stakeholders to deliver data products
- Optimize performance in Snowflake (clustering, partitioning, materialized views)
- Implement real-time data pipelines using Spark Streaming or similar technologies
- Develop CI/CD pipelines using Git, Jenkins, or similar tools
Required Qualifications:
- 5 years of hands-on data engineering experience
- Strong proficiency in Python and PySpark
- Strong SQL skills, including complex query optimization
- Hands-on experience with at least two of the following: Palantir Foundry, Databricks, Snowflake
- Experience with Azure data services (Data Factory, Synapse, ADLS)
- Knowledge of data modeling (dimensional modeling, star schema, snowflake schema)
- Experience with orchestration tools (Airflow, dbt, Luigi, or similar)