# http://www.robotstxt.org/orig.html # https://napoveda.seznam.cz/en/full-text-search/crawling-control/#request_rate_directive # https://neilpatel.com/blog/robots-txt/ #User-agent: * #Disallow: / #User-agent: Google #Disallow: # TLF below added 6/26/2021 User-agent: * Disallow: /ndpls/askFILE.cgi Disallow: /ndpls/form.html Disallow: /bin/news_show.cgi