Google’s Illyes: Robots.txt Covers Crawling, Not AI Training via @sejournal, @MattGSouthern

T2Covered by 1 source · first reported by AdExchanger / Digiday / MarTech / Search Engine Journal at 22 Oct, 10:58 UTC

Google's new robots.txt video says the standard covers crawling, not how content gets used, and recommends percent-encoding non-English robots.txt paths. The post Google’s Illyes: Robots.txt Covers Crawling, Not AI Training appeared first on Search Engine Journal.

policy

Sources

• T2Google’s Illyes: Robots.txt Covers Crawling, Not AI Training via @sejournal, @MattGSouthernAdExchanger / Digiday / MarTech / Search Engine Journal22 Oct, 10:58