User-agent: * Allow: / # Public profile pages (/@handle, /u/handle, /user/handle — served by apps/web, not the landing) are # LINKABLE but NOT INDEXABLE: profile SEO is worth close to nothing, while publishing users' full # libraries into a search index is a one-way door on their expectations (project/architecture/sharing.md §3.6). # # That is enforced by `` on the pages themselves, and there is # deliberately NO `Disallow` here to match. Two reasons, both of which a Disallow would get wrong: # 1. A disallowed URL can still be indexed URL-only — the crawler never fetches the page, so it # never sees the noindex. Allowing the crawl is what actually keeps profiles out of the index. # 2. Unfurl bots (X, Facebook, Slack, iMessage) consult robots.txt too. Blocking them would kill the # link previews that are the entire point of sharing a profile. # Title and episode pages are indexable on purpose and are covered by the Allow above. Sitemap: https://bingers.app/sitemap-index.xml