About Anaconda Help Download Anaconda

Provides functions to download and parse 'robots.txt' files. Ultimately the package makes it easy to check if bots (spiders, crawler, scrapers, ...) are allowed to access specific resources on a domain.

copied from cf-post-staging / r-robotstxt
Type Size Name Uploaded Downloads Labels
conda 197.5 kB | noarch/r-robotstxt-0.7.15-r45hc72bb7e_2.conda  4 days and 11 hours ago 38 main
conda 198.5 kB | noarch/r-robotstxt-0.7.15-r44hc72bb7e_2.conda  4 days and 11 hours ago 36 main
conda 198.1 kB | noarch/r-robotstxt-0.7.15-r44hc72bb7e_1.conda  11 months and 26 days ago 872 main
conda 195.7 kB | noarch/r-robotstxt-0.7.15-r43hc72bb7e_1.conda  11 months and 26 days ago 918 main
conda 195.6 kB | noarch/r-robotstxt-0.7.15-r43hc72bb7e_0.conda  1 year and 26 days ago 962 main

© 2025 Anaconda, Inc. All Rights Reserved. (v4.2.2) Legal | Privacy Policy