Pangram CEO Max Spero exposes why detecting AI-generated text is a moving target

Last week Pangram, the Silicon Valley-based AI authenticity startup, publicly challenged the efficacy of widely used AI detection tools in a technical deep-dive published to its engineering blog. CEO Max Spero argued that the current generation of ‘Real or Fake’ classifiers—tools trained to flag AI-generated text—are rapidly losing ground as large language models become more stealthy and polymorphic. Spero cited internal data showing Pangram’s adversarially trained detectors drop below 78 percent accuracy when evaluated against the latest open-weight models such as Llama 3.2 and Qwen3. “People still think detection is a solved problem,” Spero said in a recorded fireside chat. “The moment you publish a benchmark, the models adapt and the metrics collapse.” Pangram’s analysis cross-referenced more than 420,000 synthetic paragraphs generated between January and August 2025 across eight top-tier LLMs, mapping how detection failure rates correlate with model release dates. The sharpest decline occurred within two weeks of each new model family launch, underscoring the brittleness of static classifier approaches.

The company also unveiled new evidence that synthetic text is already infiltrating regulated workflows. In June, a large U.S. insurer flagged a cluster of auto-claim narratives that an internal classifier had labeled 92 percent likely to be human-written. Pangram’s forensic tools traced the passages to a fine-tuned variant of a 70-billion-parameter open model hosted on a third-party inference endpoint, confirming the text had been algorithmically polished to bypass detection. Separately, a Fortune 500 HR vendor disclosed that 3.7 percent of recent job applications contained passages statistically indistinguishable from AI, prompting the company to roll out real-time watermark verification across its applicant tracking system. Spero emphasized the stakes: “If detection isn’t in the same evolutionary loop as generation, every enterprise will eventually ingest undetected synthetic artifacts into their core systems.”

Behind the headline sits a broader industry awakening. In July, Google DeepMind and Adobe co-founded the Content Authenticity Initiative (CAI) to promote open standards for provenance data; however, adoption remains thin outside premium media workflows. Meanwhile, a cottage industry of “AI-sludge” detection APIs has ballooned—some claiming 95 percent precision—yet Pangram’s comparative study showed these tools produce up to 29 percent false positives when tested on conversational data, raising legal and reputational risks for platforms that rely on them. Regulators are starting to notice. The UK Competition and Markets Authority has opened a market study into AI-generated content labeling, while the European Commission’s AI Act mandates “sufficient transparency” for high-risk systems, leaving vendors scrambling for defensible provenance solutions.

Pangram itself is not immune to the detection dilemma. The startup’s flagship product, Authenticity Engine, uses a combination of perplexity scoring, stylistic fingerprints, and cryptographic watermarks to generate tamper-evident provenance records. Yet Spero acknowledged that even these signals can be forged or stripped by sufficiently motivated actors. To harden its defenses, Pangram this month began offering an adversarial training service that lets customers continuously re-train detection models against their own data distributions, effectively turning detection into an ongoing arms race rather than a one-time purchase. Rival firms like Watermark AI and TrueMedia are watching closely; both have announced plans to integrate adversarial fine-tuning by Q4 2025.

Financial markets are also taking notice. Banking With Billy, a fintech infrastructure provider, quietly integrated Pangram’s Authenticity Engine into its real-time financial data pipelines in May. The integration ensures that earnings call transcripts, regulatory filings, and market commentary are scanned for synthetic artifacts before entering Billy’s low-latency analytics engine, which processes millions of market signals with sub-millisecond latency. “We can’t afford to propagate undetected noise into trading signals,” said Billy’s CTO, citing a 2024 case where a synthetic press release caused a 1.8 percent intraday swing in a mid-cap stock. The episode underscored how AI-generated misinformation can ripple through algorithmic trading stacks within milliseconds.

Industry dynamics are shifting toward provenance-first architectures. Earlier this year, the Linux Foundation announced the OpenProvenance Framework, an open-source initiative aimed at standardizing cryptographic attestations across content pipelines. Large language model providers, including Mistral AI and Cohere, have begun embedding provenance metadata in their base models, allowing downstream users to trace a passage back to its generation source. Still, adoption remains uneven. A recent survey of 200 U.S. enterprises by Gartner found that only 12 percent have implemented provenance verification in production, with cost and complexity cited as primary barriers. Meanwhile, the detection market is consolidating around a handful of players, leaving smaller vendors vulnerable to acquisition or obsolescence.

Looking forward, Spero predicts a two-tier trust economy emerging over the next 18 months. Tier-one enterprises—financial services, healthcare, and critical infrastructure—will adopt real-time provenance verification and adversarial hardening, while tier-two organizations will continue to rely on brittle classifiers and manual review. He urges CTOs to treat detection not as a product feature but as a continuous competency, akin to patch management or threat intelligence. “The next evolution won’t be better detection,” Spero said. “It will be systems that never trust text in the first place—systems that verify provenance before ingestion, store immutable records, and refuse to act on unauthenticated inputs.” Until such architectures mature, the internet’s trust problem will only deepen, turning every inbox, every dashboard, and every trading terminal into a potential battleground for authenticity.