What are the responsibilities and job description for the Data Engineer position at Pi-Square Technologies?
Data Engineer
Location: Minnesota (MN)
Job Description:We are seeking an experienced Data Engineer with strong expertise in Python, PySpark, SQL, and Databricks to design, develop, and optimize scalable data pipelines and ETL processes. The ideal candidate should have hands-on experience working with large-scale datasets, data warehousing, and cloud-based data platforms.
Required Skills:
Preferred Skills:
Location: Minnesota (MN)
Job Description:We are seeking an experienced Data Engineer with strong expertise in Python, PySpark, SQL, and Databricks to design, develop, and optimize scalable data pipelines and ETL processes. The ideal candidate should have hands-on experience working with large-scale datasets, data warehousing, and cloud-based data platforms.
Required Skills:
- Strong experience in Python and PySpark
- Expertise in SQL and database performance tuning
- Hands-on experience with Databricks
- Experience building and maintaining ETL/ELT pipelines
- Knowledge of Data Modeling, Data Warehousing, and Big Data concepts
- Experience with cloud platforms such as Azure or AWS
- Strong problem-solving and analytical skills
Preferred Skills:
- Experience with Delta Lake, Spark Optimization, and workflow orchestration tools
- Knowledge of CI/CD and DevOps practices for data engineering