Data Engineer (Web Scraping technologies)

Title: Data Engineer (Web Scraping technologies)

Duration: FTE/Perm

Location: Onsite 4 days a week in NYC

Salary: 125-190k plus bonus

Responsibilities:

• Utilize AI Models, Code, Libraries or applications to enable a scalable Web Scraping capability
• Web Scraping Request Management including intake, assessment, accessing sites to scrape, utilizing tools to scrape, storage of scrape, validation and entitlement to users
• Fielding Questions from users about the scrapes and websites
• Coordinating with Compliance on approvals and TOU reviews
• Some Experience building Data pipelines in AWS platform utilizing existing tools like Cron, Glue, Eventbridge, Python based ETL, AWS Redshift
• Normalizing/standardizing vendor data, firm data for firm consumption
• Implement data quality checks to ensure reliability and accuracy of scraped data
• Coordinate with Internal teams on delivery, access, requests, support
• Promote Data Engineering best practices

Required Skills and Qualifications:

• Bachelor’s degree in computer science, Engineering, Mathematics or related field
• 2-5 experience in a similar role
• Prior buy side experience is strongly preferred (Multi-Strat/Hedge Funds)
• Capital markets experience is necessary with good working knowledge of reference data across asset classes and experience with trading systems
• AWS cloud experience with commons services (S3, lambda, cron, Event Bridge etc.)
• Experience with web-scraping frameworks (Scrapy, BeautifulSoup, Selenium, Playwright etc.)
• Strong hands-on skills with NoSQL and SQL databases, programming in Python, data pipeline orchestration tools and analytics tools
• Familiarity with time series data and common market data sources (Bloomberg, Refinitiv etc.)
• Familiarity with modern Dev Ops practices and infrastructure-as-code tools (e.g. Terraform, CloudFormation)
• Strong communication skills to work with stakeholders across technology, investment, and operations teams.