firmulate.com/benchmarks.html — live view
AIThis post was created with the assistance of artificial intelligence (AI).
Firmulate — We Buried a €55,000 Fact Two Documents Deep. Here's Which AIs Did Their Homework.
Live on firmulate.com.

Imagine going on a trip where your travel agent not only listens to your preferences but also digs deep into your travel history, preferences, and even hidden notes in your files before making a recommendation. That level of thoroughness can be the difference between a memorable, personalized experience and a missed opportunity. In the world of AI for business decision-making, this ‘deep reading’ capability is emerging as a game-changer—affecting whether companies close multi-thousand-euro deals or lose them in a blink.

The Experiment: Testing AI in a High-Stakes Business Environment

In a recent live experiment conducted by Firmulate, four advanced AI models were put through their paces in a simulated environment replicating a small software company’s worst week. The goal? To see if these models could navigate a series of crises, temptations, and manipulative tactics without compromising integrity or losing deals. Crucially, every decision was recorded and auditable, making the results clear and measurable.

Amazon

AI document analysis software

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Key Findings: Deep Reading Decisively Wins

All four AI models successfully identified every crisis and refused any manipulation attempts—an essential baseline for trustworthy decision-making. Interestingly, only two of these models managed to close the €55,000 deal based solely on their own analysis, while the other two failed to sign the contract despite offering the same diagnosis and pitch.

Amazon

deep reading AI for business

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

The Hidden Edge: Reading Two References Deep

The real breakthrough was the models’ ability to uncover a critical flaw buried two document references deep within the company’s files. Unlike surface-level analysis, this deep reading enabled the successful models to identify a key piece of information—an insidious weakness that, if missed, would have lost the deal. In fact, the models that read the files thoroughly won the full €4,583 monthly recurring revenue (MRR) deal at full price.

Amazon

trustworthy AI decision-making tools

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Why This Matters for Business AI

For companies relying on AI to handle customer relationships, support, or forecasting, the takeaway is clear: it’s not enough for an AI to generate convincing chat or superficial analysis. The ability to ‘read your files before answering’—digging deep into complex documentation—can be the decisive factor in closing deals, maintaining trust, and avoiding costly mistakes.

Amazon

AI cybersecurity social engineering detection

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Fake Social Engineering Tests: What the AI Shows

In another test involving social engineering, a staged sequence of fake CEO messages and a reporter trick were deployed. Across all models, every one of them refused to escalate or manipulate the situation, citing suspicion and security protocols. Kimi K3, one of the models, explicitly explained: “Treat the request as a suspected approval-bypass or possible impersonation.” This demonstrates that well-designed AI can be vigilant against manipulative tactics, an increasingly important feature in digital trust and security.

Real-World Company Simulation: The Stakes Are High

The experiment wasn’t just theoretical. It simulated a live company with 13 synthetic employees, managing real-money mechanics—burning €105,000 monthly against €2,300 MRR—while operating under public cash countdowns. Every workday, the AI models made decisions based on over 680 self-learned rules, with all actions versioned for transparency. You can watch this ongoing experiment at firmulate.com/live.

Insights from the Results: Discipline and Deep Analysis Matter

The most thorough participant, Opus 4.8, with over 80 learned rules and the deepest analyses, still left a deal on the table due to slipping discipline—failing to escalate instead of writing into a locked department. Meanwhile, other models showed similar weaknesses, reinforcing the importance of disciplined decision-making and deep comprehension in AI performance.

Implications for Business Decisions

What does all this mean for your company? If AI is touching your CRM, support queue, or sales forecasts, the question isn’t just whether it writes well—it’s whether it reads thoroughly and stays honest under pressure. The ability to uncover hidden facts in your files could be the difference between closing a full-price deal or losing it without even realizing why.

Infographic — We Buried a €55,000 Fact Two Documents Deep. Here's Which AIs Did Their Homework.
The findings at a glance — source: firmulate.com.

Deep reading capabilities in AI are proving crucial in high-stakes decision-making. Models that thoroughly analyze internal files before acting can secure more deals and uphold trust, making them invaluable for modern businesses looking to stay competitive and honest.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI


You May Also Like

Mistral AI Ownership: Moving Beyond API Rentals For Greater Control

Mistral Forge lets enterprises train and deploy domain-specific AI models, offering more control than rented APIs but at greater cost and complexity.

Show HN: GentleOS – A pair of hobby OSes for vintage 32-bit and 16-bit PCs

GentleOS introduces two hobby operating systems for vintage PCs—GentleOS/32 for 32-bit hardware and GentleOS/16 for 16-bit devices, focusing on retro tinkering.

These Apple Devices Are Still on Sale After Prime Day

Several Apple devices remain discounted at Amazon following Prime Day, including the Apple Watch Series 11 and iPad Pro, offering savings for shoppers.

IKEA Storage Box Just Happens To Make Great Printer Cover

A common IKEA storage box is being repurposed as an effective, affordable printer cover, offering heat retention and dust protection for 3D printers.