Despite the fact that the robots.txt file was invented to tell search engines like yahoo what pages never to crawl, the robots.txt file can be utilized to point search engines like google and yahoo on the XML sitemap. That is supported by Google, Bing, Yahoo and Request. As While using https://rankingseotoolz.com/robots-txt-generator