# robots.txt for https://americanlight.design # # Policy: AI crawlers are ALLOWED. Read the pages, answer questions about # motorized screens and awnings, and cite us with a link. What is closed # off is machine noise only — the POST-only lead endpoints, the # post-submission page, and query-string duplicates of canonical URLs. # # Machine-readable surfaces, all of them cheaper to read than the HTML: # https://americanlight.design/llms.txt site summary for language models # https://americanlight.design/llms-full.txt the full corpus in one fetch # https://americanlight.design/agents.md how to use this site programmatically # https://americanlight.design/graph.jsonld the whole schema.org @graph # https://americanlight.design/.well-known/agent-policy.json what agents may do here # https://americanlight.design/.well-known/security.txt how to report a problem # # Content pages have a Markdown twin at their path + index.md # (https://americanlight.design/motorized-screens/index.md; /motorized-screens.md and # /motorized-screens/.md redirect there), or send Accept: text/markdown. # Identical content, a fraction of the tokens. https://americanlight.design/api/index.json # names each page's twin under alternates (the giveaway pages have none). # # Crawl-delay appears in exactly one group below. Bing documents support # for it; Google states it ignores the directive, and no other operator # named here documents honoring it, so asking them would be noise. This # origin rate-limits per IP instead, which actually binds. # # Every named agent below gets the same answer. The tokens are listed # individually anyway, because a named token is a statement — it says we # know this agent exists and decided to admit it, rather than leaving it # to fall through to the wildcard by accident. # GPTBot OpenAI training # OAI-SearchBot OpenAI search-index # ChatGPT-User OpenAI user-triggered # anthropic-ai Anthropic training # PerplexityBot Perplexity search-index # Perplexity-User Perplexity user-triggered # Googlebot Google search-index # Google-Extended Google training # Google-CloudVertexBot Google user-triggered # Applebot Apple search-index # Applebot-Extended Apple training # Amazonbot Amazon search-index # meta-externalagent Meta training # meta-externalfetcher Meta user-triggered # facebookexternalhit Meta link-preview # Bytespider ByteDance training # cohere-ai Cohere training # Timpibot Timpi search-index # omgili Webz.io training # Webzio-extended Webz.io training User-agent: GPTBot User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: anthropic-ai User-agent: PerplexityBot User-agent: Perplexity-User User-agent: Googlebot User-agent: Google-Extended User-agent: Google-CloudVertexBot User-agent: Applebot User-agent: Applebot-Extended User-agent: Amazonbot User-agent: meta-externalagent User-agent: meta-externalfetcher User-agent: facebookexternalhit User-agent: Bytespider User-agent: cohere-ai User-agent: Timpibot User-agent: omgili User-agent: Webzio-extended Allow: / Allow: /api/service-area Allow: /api/reviews Allow: /api/media Allow: /api/agent-log Disallow: /api/lead/ Disallow: /api/quote Disallow: /api/schedule Disallow: /thank-you/ Disallow: /*?product= Disallow: /*?utm_ Disallow: /*?limit= Disallow: /*?offset= # ClaudeBot Anthropic training User-agent: ClaudeBot Allow: / Allow: /api/service-area Allow: /api/reviews Allow: /api/media Allow: /api/agent-log Disallow: /api/lead/ Disallow: /api/quote Disallow: /api/schedule Disallow: /thank-you/ Disallow: /*?product= Disallow: /*?utm_ Disallow: /*?limit= Disallow: /*?offset= Crawl-delay: 1 # Claude-User Anthropic user-triggered User-agent: Claude-User Allow: / Allow: /api/service-area Allow: /api/reviews Allow: /api/media Allow: /api/agent-log Disallow: /api/lead/ Disallow: /api/quote Disallow: /api/schedule Disallow: /thank-you/ Disallow: /*?product= Disallow: /*?utm_ Disallow: /*?limit= Disallow: /*?offset= Crawl-delay: 1 # Claude-SearchBot Anthropic search-index User-agent: Claude-SearchBot Allow: / Allow: /api/service-area Allow: /api/reviews Allow: /api/media Allow: /api/agent-log Disallow: /api/lead/ Disallow: /api/quote Disallow: /api/schedule Disallow: /thank-you/ Disallow: /*?product= Disallow: /*?utm_ Disallow: /*?limit= Disallow: /*?offset= Crawl-delay: 1 # bingbot Microsoft search-index User-agent: bingbot Allow: / Allow: /api/service-area Allow: /api/reviews Allow: /api/media Allow: /api/agent-log Disallow: /api/lead/ Disallow: /api/quote Disallow: /api/schedule Disallow: /thank-you/ Disallow: /*?product= Disallow: /*?utm_ Disallow: /*?limit= Disallow: /*?offset= Crawl-delay: 1 # CCBot Common Crawl training User-agent: CCBot Allow: / Allow: /api/service-area Allow: /api/reviews Allow: /api/media Allow: /api/agent-log Disallow: /api/lead/ Disallow: /api/quote Disallow: /api/schedule Disallow: /thank-you/ Disallow: /*?product= Disallow: /*?utm_ Disallow: /*?limit= Disallow: /*?offset= Crawl-delay: 2 # Diffbot Diffbot training User-agent: Diffbot Allow: / Allow: /api/service-area Allow: /api/reviews Allow: /api/media Allow: /api/agent-log Disallow: /api/lead/ Disallow: /api/quote Disallow: /api/schedule Disallow: /thank-you/ Disallow: /*?product= Disallow: /*?utm_ Disallow: /*?limit= Disallow: /*?offset= Crawl-delay: 1 # ImagesiftBot ImageSift (Hive) training User-agent: ImagesiftBot Allow: / Allow: /api/service-area Allow: /api/reviews Allow: /api/media Allow: /api/agent-log Disallow: /api/lead/ Disallow: /api/quote Disallow: /api/schedule Disallow: /thank-you/ Disallow: /*?product= Disallow: /*?utm_ Disallow: /*?limit= Disallow: /*?offset= Crawl-delay: 5 # YouBot You.com search-index User-agent: YouBot Allow: / Allow: /api/service-area Allow: /api/reviews Allow: /api/media Allow: /api/agent-log Disallow: /api/lead/ Disallow: /api/quote Disallow: /api/schedule Disallow: /thank-you/ Disallow: /*?product= Disallow: /*?utm_ Disallow: /*?limit= Disallow: /*?offset= Crawl-delay: 1 # Everything else, including human browsers and agents not named above. User-agent: * Allow: / Allow: /api/service-area Allow: /api/reviews Allow: /api/media Allow: /api/agent-log Disallow: /api/lead/ Disallow: /api/quote Disallow: /api/schedule Disallow: /thank-you/ Disallow: /*?product= Disallow: /*?utm_ Disallow: /*?limit= Disallow: /*?offset= Sitemap: https://americanlight.design/sitemap.xml