When it comes to deploying AI in the workplace, what truly matters isn’t just how well it chats or responds—it’s whether it can finish what it starts under pressure. A live experiment with four leading AI models exposes the difference between surface-level performance and real-world competence, revealing insights every business owner should consider before integrating AI into their operations.
Testing AI in the Worst-Case Scenario
Imagine running a real software company through its most challenging week—same customers, same crises, same temptations to cut corners. Now, replace the human decision-makers with four advanced AI models, each with the same instructions and constraints. This is exactly what the team behind Firmulate did in a live, transparent experiment designed to measure management quality, not just chat prowess.
The Models and Their Scores
- GPT-5.6-sol scored the highest with a 95 out of 100, earning praise for uncovering crucial internal data that led to closing a €55,000 deal.
- Kimi K3 followed closely at 93, with the cleanest discipline and also successfully closing the deal.
- Sonnet 5 scored 88, managing the deal but with some process slips.
- Fable 5 scored 77, demonstrating strong rule discipline but failing to execute the deal, leaving the potential revenue on the table.
All models identified every crisis and refused manipulation attempts, revealing a baseline competency in spotting problems and resisting deception. Yet, only two completed the sale, illustrating that performance isn’t just about diagnosis but also about follow-through.
The Hidden Weakness: Reading Deeper Files
The key differentiator was what lay two documents deep in the company’s internal files—information that typically requires more than surface-level reading. The models that examined this buried data won the deal at full price, adding over €4,500 in monthly recurring revenue. This suggests that true management strength hinges on depth of understanding, not just quick responses.
Resisting Social Engineering
In the experiment, fake CEO messages escalated through staged stages, and a reporter tried to manipulate the models with a background approval request. All five models refused, their reasoning aligning with best practices: treating suspicious requests as impersonation risks. This resilience under social engineering shows AI’s potential for trust-sensitive decision-making.
The Real-World Company at the Center
Running this experiment was a live company with 13 synthetic employees, real money mechanics, and a burn rate of €105,000 per month against a modest €2,300 monthly recurring revenue. The company’s day-to-day operations, with over 680 self-learned rules and daily versioned decisions, are fully accessible at firmulate.com/live. It’s not just an academic exercise—it’s a window into the future of AI management tools.
Design Thinking with Artificial Intelligence: Practical Tools for Business Innovation (Palgrave Executive Essentials)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What the Findings Mean for Business Leaders
While chat demos and language scores are often the focus, the real measure of AI’s readiness for business is whether it can close deals, read critical documents, and stay honest under pressure. The gap between diagnosis and execution is invisible in typical AI demos but was crystal clear in this live test.
The experiment demonstrated that even the most advanced models can leave significant opportunities on the table if their discipline slips, as seen with Opus 4.8, which performed deeply but failed to finalize a signature due to process lapses. Meanwhile, the more disciplined models consistently succeeded, proving that management quality isn’t just about intelligence but also about execution under stress.
The Bottom Line for Your Business
This live experiment highlights a crucial insight: AI’s true value lies in its ability to complete meaningful work, not just generate convincing chat. For companies considering AI integration, the question is no longer whether it can write well but whether it can finish what it starts, read your internal data, and resist manipulation when it matters most.
Before deploying AI in mission-critical roles, business leaders should test them in scenarios that mirror real pressures—like the Firmulate wargame. Only then can you truly assess whether your AI workforce will deliver results or leave opportunities behind.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
