technology
Techradar
2026-11-08
Google, Microsoft, and OpenAI Support New Website Scraping Opt-Out Standard
Major tech firms have agreed to adopt a new robots.txt standard allowing website owners to opt-out of having their content used to train AI models.