Robots.txt data files are crucial for the correct operating and administration of websites, and for optimizing their performance in internet search engine benefits. These data files provide a variety of important reasons, generating them an integral element of web enhancement and search engine optimisation (Web optimization). Key word study is t