Menu Close

Encyclopedia

Robots.txt

Robots.txt

A robots.txt file is a simple text file used by websites to communicate with web crawlers and other automated agents that visit the site. It…