# Don't allow web crawlers to index Craft CMS User-agent: * Allow: / Disallow: /craft/ Disallow: /staff/ Disallow: /my-property/ Disallow: /public/staff/ Disallow: /public/my-property/ Disallow: /CraftV3/web/assets Disallow: /assets