Senior Software Engineer

The Big Data team is a central player in the Poshmark organization. Our mission is to build a world-class big data platform to bring value out of data for us and for our customers. Our goal is to democratize data, support exploding business, provide reporting and analytics self-service tools, and fuel existing and new business critical initiatives.

This position will be responsible for designing, developing & maintaining growth data pipeline and integrating with advertising platforms like Facebook, Google etc to drive insights for business growth. The role requires to partner with Data Scientists, Analysts, Marketing and Growth campaign managers. In addition, this role involves making an impact by driving continuous improvements in moving, aggregating, profiling, sampling, testing and analyzing terabytes of data.

Responsibilities

• Build highly scalable, available, fault-tolerant data processing systems using AWS technologies - Hive, Kafka, Spark, and other big data technologies. These systems should handle batch and real-time data processing over 10s of terabytes of data ingested every day and petabytes of data warehouse.
• Build domain expertise and ownership -- become technically knowledgeable in the area of Attribution, Marketing, Integrating with Ad platforms.
• Participate in architecture discussions, influence product roadmap, and take ownership and responsibility over new projects
• Maintain and support existing platforms and evolve to newer technology stacks and architectures

6-Month Accomplishments 

• Get familiar with the poshmark ecosystem i.e data Infrastructure and data tools - Redshift, Spark, Druid etc. 
• Help optimize current tools and ETL used for various business needs 
• Identify gaps in current infrastructure and propose solutions 
• Regularly provide meaningful design feedback and code review to peers

12 Month Accomplishments

• Design technical solutions for large complexity problems with minimal supervision
• You've rolled out a high impact project to production
• Collaborate with the engineering team to improve data collection and data quality. 
• Collaborate with machine learning engineers, data scientists and analytics on different projects. 

Desired Skills

• 4 years of experience in building highly scalable solutions in data storage, real-time analysis and reporting for multi-terabyte data sets
• Expertise architecting and building large-scale data processing systems using Big Data technologies like Hadoop, Hbase, Spark, Storm, Kafka, Druid, Flink etc.
• Experience with SQL and NoSQL databases Amazon Redshift/Spectrum, AWS S3
• Solid software development skills in Java/Scala/Ruby/Python
• Solid understanding of Jenkins / Github
• Integration experience with advertising and attribution partners like FB, Google, AppsFlyer, Snapchat etc. 
• Be self-driven, take complete ownership of initiatives and make pragmatic technical decisions
• Collaborate with cross-functional teams of developers, QA and operations to execute deliverables