Scope of Position

The Renewable Data Lake (RDL) is focused on building renewable asset optimization solutions around the world and transforms the data into meaningful metrics for our customers (internal stakeholders). 

Our team ingests massive volumes of data and generates a series of KPIs that help us and our customers understand how our sites are performing, where and why they are under-performing, what the most common problems are, etc.

The number of sites and the volume of data is growing fast. We'll need your help building smart, reliable, and performant solutions to make the most of the data. 

Our current stack includes Redshift, RDS, Glue, Spark, Lambdas, S3, the AWS developer tools, and other AWS technologies.

If you have good ideas for alternative solutions, we want to hear about it! 

Responsibilities

• Work in a fast-paced, agile project environment with cross-functional and highly skilled teams in cadence within a product team, while remaining connected to a core engineering team on solution-based, outcome-oriented as well as highly complex projects 

• Build hands-on experience with the latest technologies and delivery methodologies
• Translate complex business use-cases into detailed technical implementations 
• Define and craft our data models and database structures 
• Create, improve, and maintain automated, highly reliable data pipelines 

• Build data integrations within our data platform and between partners
• Define, design, and build data testing and quality frameworks 
• Other duties as assigned 

Qualifications

Qualifications - Skills/Knowledge/Abilities 

• Strong experience with database design, specifically geared towards Big Data 
• Familiar with serverless approach, micro-service-oriented architecture, event-driven architecture 
• Strong experience working in a AWS cloud technology environment: Lambda functions, Step functions, SQS, DynamoDB, Spark with Glue, along with EMR, EMR serverless, Spark streaming 
• Experience building and maintaining critical, reliable ETL pipelines 
• Experience writing high-quality, production-ready code, preferably in Python
• Comfortable with Git and Unix command lines 
• DevOps experience: strive for scalability and automation via CI/CD and Infrastructure as Code. A plus if knowledge of CDK 

• Proactive communicator who can translate between technical and non-technical stakeholders 
• Self-starter and autonomous, strong bias for action in high ambiguity environment 
• Explore and experiment with the limits of cloud technologies 
• Help mentor and educate your teammates (and vice versa) to expand everyone's skillsets 
• Displays personal and corporate integrity

Qualifications - Education/Experience 

• 8 years of experience working in software engineering 
• 5 years working with Big Data in a cloud environment, AWS strongly preferred 
• Bachelor’s degree in Computer Science or other related technical field or relevant experience 
• Experience developing multi-platform software utilizing various programming environments and tools
• Knowledge of agile methodologies
• Knowledge of QA processes and test-driven development

Qualifications - Skills/Knowledge/Abilities 

• Strong experience with database design, specifically geared towards Big Data 
• Familiar with serverless approach, micro-service-oriented architecture, event-driven architecture 
• Strong experience working in a AWS cloud technology environment: Lambda functions, Step functions, SQS, DynamoDB, Spark with Glue, along with EMR, EMR serverless, Spark streaming 
• Experience building and maintaining critical, reliable ETL pipelines 
• Experience writing high-quality, production-ready code, preferably in Python
• Comfortable with Git and Unix command lines 
• DevOps experience: strive for scalability and automation via CI/CD and Infrastructure as Code. A plus if knowledge of CDK 

• Proactive communicator who can translate between technical and non-technical stakeholders 
• Self-starter and autonomous, strong bias for action in high ambiguity environment 
• Explore and experiment with the limits of cloud technologies 
• Help mentor and educate your teammates (and vice versa) to expand everyone's skillsets 
• Displays personal and corporate integrity