Sina Technology News Beijing time on the evening of September 8th, it was reported that Company X (Twitter) recently updated its terms of service. Without permission, any third party is not allowed to capture data on the X platform to train artificial intelligence (AI) models.
This provision will come into effect on September 29. According to the new terms, no data scraping in any form is allowed on the X platform for any purpose without prior written permission. Previously, Company X allowed outsiders to crawl platform data through the robots.txt file.
The robots.txt file provides instructions to robot crawlers on which parts of your website they can access. But in the past few months, Company X modified the robots.txt file to remove all instructions for crawler robots except Google. In 2015, Company X reached an agreement with Google to allow Google to display tweets in search results.
Elon Musk, the boss of X Company, has always opposed third parties collecting data on the X platform to train artificial intelligence models. In April this year, he even threatened to sue Microsoft, claiming that Microsoft illegally used X's data to train its artificial intelligence model.
In July, Company X filed a lawsuit against four entities, accusing them of engaging in data scraping activities that severely strained X's servers and worsened user experience. X said at the time: "Scraping interferes with the legitimate operation of websites and mobile apps because it makes millions of requests, places a heavy load on the server, and harms the experience of real users."
While banning third-party crawling, X also adjusted its privacy policy earlier this month to allow X to use information posted by users to train its artificial intelligence model. Musk said that X will only use public information to train its artificial intelligence model and will not use any private content.