Who wrote Anthropic’s prospectus? AI detection tool gives unexpected results

Abstract:

According to Business Insider, artificial intelligence company Anthropic submitted a

335 page

Pangram’s detection results show that 92% of this document of approximately 259,000 words was determined to be written by humans, and some sentences and paragraphs were marked with “high” confidence as containing traces of AI use.

However, this result is only the judgment of the detection tool and cannot prove which content was written by humans or AI, nor can it confirm whether Anthropic used its own AI assistant Claude.

Business Insider reporters initially tried to inspect the entire document themselves, but encountered difficulties due to the size of the file. Subsequently, Pangram co-founder and CEO Max Spero helped it complete full-text detection.

Spero said that his interpretation of this result is that the document may have been lightly edited or used AI at the sentence level, but there does not appear to be a large section of the content that was completely generated by AI.

The sentences marked by Pangram as possible uses of AI include a description of industry development, to the effect that AI is developing at an extremely fast pace, and the intervals between new version model iterations are often measured in months rather than years.

Business Insider also pointed out that AI writing detection tools are not 100% accurate. The way people use AI to edit, proofread or revise wording can affect detection results.

Previously, a Business Insider reporter entered his LinkedIn posts and published articles into Pangram. The tool marked one of the sentences as AI-written. After reviewing the writing process, she found that she originally wrote this sentence and later accepted ChatGPT's rewriting suggestions.

Anthropic did not respond to Business Insider's request for comment on the test results. The media stated that Anthropic’s own AI watermarking system has not yet been opened to ordinary users and therefore was not used for this detection.

Business Insider writer Katie Notopoulos believes that this result raises an interesting question. As an AI company, why doesn’t Anthropic use more of its own products to write prospectuses? But she also pointed out that the prospectus needs to be reviewed by the U.S. Securities and Exchange Commission, and every sentence in the document may receive attention, and it is reasonable for humans to participate in writing it.