Skip to content
English
Contact us
  • There are no suggestions because the search field is empty.

How to Ensure the ads.txt file is Accessible and Crawlable

An ads.txt file helps you control who is authorized to sell your ad inventory and protects your brand from fraudulent or unauthorized reselling. For it to work effectively, ad platforms must be able to crawl and read your file without issues.

Once your ads.txt file is created and placed at your domain’s root, crawlers will attempt to access it. Follow the steps below to ensure everything is set up correctly and functioning smoothly.

Key checks to ensure proper crawling:

1. Make sure the ads.txt file is available

Open domain.com/ads.txt in your browser and confirm it loads successfully. Avoid errors like: 404 (Not Found) and 500 (Server Error). These indicate the file is inaccessible and cannot be crawled.

2. Confirm that the ads.txt file can be reached from the root domain

Your ads.txt must be accessible from the root domain. 

  • Example: domain.com/ads.txt

If your site uses “www”, ensure proper redirection:

  • from domain.com/ads.txt to www.domain.com/ads.txt

Without this, crawlers may not find your file.

3. Check robots.txt permissions

Your robots.txt file should not block access to the ads.txt file.

Make sure it does NOT:

  • Restrict the ads.txt URL path
  • Block specific crawlers (user agents)

Otherwise, your ads.txt may be ignored.

4. Confirm a successful response (HTTP 200)

Your ads.txt file must return a 200 OK status, which confirms:

  • The file is correctly implemented
  • The request is successful

5. Avoid formatting issues

Even small errors can cause problems. Watch out for:

  • Invalid characters
  • Incorrect spacing or hidden whitespace

These issues can prevent crawlers from processing your file properly.

6. Ensure an ads.txt file access via HTTP and HTTPS

Crawlers check both versions:

  • http://domain.com/ads.txt
  • https://domain.com/ads.txt

You should:

  • Either make the file available on both
  • Or set up proper redirects between them

Examples:

For HTTP only (e.g., http://domain.com/ads.txt): set up a redirection so that when the crawler searches https://domain.com/ads.txt the site redirects to http://domain.com/ads.txt.

For HTTPS only (e.g., https://domain.com/ads.txt): set up a redirection so that when the crawler searches http://domain.com/ads.txt the site redirects to https://domain.com/ads.txt.

For both HTTP and HTTPs protocols: requesting http://domain.com/ads.txt and https://domain.com/ads.txt should show the ads.txt file.