User-agent: * Allow: / # Disallow all admin, application, and API routes Disallow: /admin/ Disallow: /careers/apply/ Disallow: /api/