Fortune 100 IT Software and Services Company Partners with Vaco to Improve Generative AI Moderation Processes

During a pilot with two additional vendors, Vaco prevailed as the vendor that met tight deadlines and delivered strong results. We deployed 6 analysts to generate high-quality, creative AI prompts and to execute a holistic testing process that included large language model prompt engineering, testing and rating. All prompts and results were documented and uploaded for parity testing and summarization. These reports were provided to the client through detailed project briefs, outlining observed patterns and potential policy violations for the organization to address. 

Additionally, we: 

• Instituted expert incubation workflows, including defining quality rubric, policy governance frameworks and knowledge base documentation. 
• Implemented a prioritization queue for efficient collaboration with product stakeholders. 
• Met or exceeded service level agreements, including adhering to a 2–3-day turnaround on new intel. 
• Provided Red Team Intel Briefs to advise on geopolitical events. 
• Conducted adversarial testing in over 14 languages to generate keywords and sentiment analysis to detect potential risks and to protect the client’s brand.
  

Our Impact 

Vaco executed 2,900+ prompts across four separate projects, identifying and documenting potentially violative content. Analytics detailed in our team’s project briefs alerted the company of gaps in their generative AI content moderation efforts and identified an immediate need for a dedicated team to support their ongoing moderation efforts. As a result, Vaco was awarded an ongoing contract providing moderation managed services.