Demo

Software Developer

Haystack
Orlando, FL Full Time
POSTED ON 4/28/2026
AVAILABLE BEFORE 5/27/2026
We are working with a company that is at the forefront of AI innovation, providing human feedback to enhance the capabilities of advanced AI agents. They collaborate with leading AI organizations to train Large Language Models (LLMs) to design, coordinate, and optimize complex, real-world architectural workflows.

The Role

  • Help train generative AI models by contributing your software expertise.
  • Develop objective and verifiable criteria (rubrics) to evaluate system performance.
  • Review system logs and "trajectories" to refactor code and improve execution paths.
  • Design systems to ensure outputs meet strict functional requirements and reach a "Golden Path" of perfect reliability.
  • Test systems for vulnerabilities, including improper data exposure, unauthorized access escalations, and edge-case failures.

What You'll Need

  • 2 years of experience in backend engineering, AI automation, or complex systems integration.
  • Proven ability to build and maintain production-grade software with modular separation.
  • Strong command of at least two major programming languages (e.g., Python, JavaScript, Go, or Java).
  • Experience working with SQL databases.
  • Practical experience building for live, non-mocked environments and handling multi-turn system interactions.
  • Outstanding attention to detail and the ability to provide clear, high-density technical feedback on complex system behaviors.

What's On Offer

  • Opportunity to shape the future of autonomous agents.
  • Flexible, fully remote work with flexible hours.
  • Competitive hourly rates for project work, with additional incentives.
  • Contribute to cutting-edge projects with leading AI organizations.

Apply via Haystack today!

Salary : $50

If your compensation planning software is too rigid to deploy winning incentive strategies, it’s time to find an adaptable solution. Compensation Planning
Enhance your organization's compensation strategy with salary data sets that HR and team managers can use to pay your staff right. Surveys & Data Sets

What is the career path for a Software Developer?

Sign up to receive alerts about other jobs on the Software Developer career path by checking the boxes next to the positions that interest you.
Income Estimation: 
$97,257 - $120,701
Income Estimation: 
$123,167 - $152,295
Income Estimation: 
$91,370 - $117,201
Income Estimation: 
$115,390 - $147,559
Income Estimation: 
$106,780 - $140,358
Income Estimation: 
$104,963 - $131,876
Employees: Get a Salary Increase
View Core, Job Family, and Industry Job Skills and Competency Data for more than 15,000 Job Titles Skills Library

Job openings at Haystack

  • Haystack Berkeley, CA
  • We're working with a global aerospace company that develops and manufactures commercial jetliners, defense, space and security systems, and global services... more
  • 4 Days Ago

  • Haystack Berkeley, CA
  • We're working with a global aerospace company that designs, manufactures, and sells airplanes, rotorcraft, rockets, satellites, telecommunications equipmen... more
  • 4 Days Ago

  • Haystack Saint Louis, MN
  • We're working with a global aerospace leader that develops and manufactures commercial jetliners, defense, space and security systems. This is an exciting ... more
  • 5 Days Ago

  • Haystack Tucson, AZ
  • We're working with a global aerospace and defense technology company that develops and manufactures advanced systems for air, land, sea, space, and cyber d... more
  • 5 Days Ago


Not the job you're looking for? Here are some other Software Developer jobs in the Orlando, FL area that may be a better fit.

  • Software Resources, Inc. Orlando, FL
  • Software Resources has an immediate, long term contract job opportunity for a Senior Power BI Developer with a major corporation in Orlando, FL. 4 days per... more
  • 11 Days Ago

  • StraCon Services Group, LLC. Orlando, FL
  • Job ID# 1704 StraCon Services Group, LLC is seeking Full-Time, Software Developer, Journeyman with to join our Naval Enterprise Research Data Science (NERD... more
  • 4 Days Ago

AI Assistant is available now!

Feel free to start your new journey!