User-agent: * Disallow: /admin Disallow: /en/admin # Don't crawl query URLs Disallow: /*?