OpenAI has developed a tool that could potentially catch students cheating by asking ChatGPT to write assignments -- but the Wall Street Journal reports the company is debating whether to actually release such a tool. In a statement, an OpenAI spokesperson confirmed the company was working on the text watermarking method described in the Daily News report, but said it was taking a "thoughtful approach" due to "the complexities involved and the impact it could have on the broader ecosystem beyond OpenAI."

"The text watermarking method we are developing is technically promising, but while we are investigating alternatives, we are weighing the important risks, including the ease of circumvention by malicious actors and the potential for disproportionate impact on groups such as non-English speakers," the spokesperson said.

This would be a different approach than most previous efforts to detect AI-generated text, which have been largely ineffective. OpenAI even shut down its previous artificial intelligence text detector last year due to "low accuracy."

In terms of text watermarking, OpenAI will only focus on detecting text in ChatGPT, rather than other companies’ models. To this end, OpenAI will make minor changes to the way ChatGPT selects text, creating an invisible watermark in the text that will then be detected by a separate tool.

After the report was published, OpenAI also updated a blog post in May to introduce its research around detecting content generated by artificial intelligence. The update states that text watermarks have proven to be "highly accurate and even effective against localized tampering, such as paraphrasing," but have proven "less robust to globalized tampering, such as using a translation system, rephrasing with another generative model, or requiring the model to insert a special character between each word and then delete that character."

As a result, OpenAI writes, this method is "easily circumventable by abusers." OpenAI's update also echoes the spokesperson's point regarding non-English speakers, arguing that text watermarking could "tarnish the use of AI as a useful writing tool for non-native English speakers."

learn more:

https://openai.com/index/understanding-the-source-of-what-we-see-and-hear-online/