Job Description:

• Create new ingest batch/streaming pipelines from the ground up or maintain existing pipelines. Recommend tools/tech stack which will improve the effectiveness of existing pipelines or reduce the over all operating cost of the existing pipelines Responsibilities
• Possess an in-depth understanding of the data structures and governance Deep understanding of modern cloud computing platforms and concepts (AWS preferred)
• Work with modern schema-less big data storage solutions .

Responsibilities:

• Work closely with DevOps to foster a culture of collaboration
• Work closely with machine learning and data science teams to create scale and efficiency
• Demonstrate critical thinking for potential roadblocks; comprehend a bigger picture of the business and effectively communicate these issues to stakeholders
• Work closely with internal stakeholders to implement solutions that adhere to solution designs and schema
• Work in an agile software development team create new batch/streaming ingest data pipelines from the ground up, and/or expand upon those created by other team members.
• Act as a lead, identifying and assigning new work for other engineers.
• Test and create automated unit/integration tests for your code, ensuring every function, service, and object is compatible with your team s work and with the many systems within the Client Group system portfolio.
• Act as a development leader, help to formulate best practices and setting up toolsets and procedures for the team, leveraging internally available tools and communicating with other internal development & product teams.
• Stay up-to-date on new development and platform technologies, make recommendations for the right tool for the job, and take the lead in setting up those tools and training other developers on them.
• Create documentation for developers as well as business users to help them understand your products.
• Maintain cloud-based platforms and environments of supported applications, troubleshooting and patching functional issues and data issues in lower or production environments when necessary.

Experience:

• 8 years of related work experience

Skills:

• Embrace a builder mentality to push the business forward
• 8 years experience in a software developer role
• 4 years of experience as a data engineer
• Proficient with AWS, Python and Pyspark
• Experience with EMR, S3, SNS, Kafka, Databricks, Athena, Kinesis, Lambda and Airflow
• Experience with distributed data technologies (e.g. Spark)
• Proficient with Linux environment
• Passion for media and news

Education:

• S. degree in Computer Science, Information Technology, or equivalent experience

About US Tech Solutions:

US Tech Solutions is a global staff augmentation firm providing a wide range of talent on-demand and total workforce solutions. To know more about US Tech Solutions, please visit .

US Tech Solutions is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.