# See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file # Sitemap: https://www.cdp.net/sitemap.xml.gz Sitemap: https://cdn.cdp.net/cdp-production/sitemap.xml.gz User-agent: AhrefsBot Disallow: / User-agent: * Disallow: /checkout Disallow: /cart Disallow: /orders Disallow: /user Disallow: /account Disallow: /api Disallow: /password Disallow: /users Disallow: /dashboards