← AI crawler directory

Meta

Meta-ExternalFetcher

AI answersPublished ASNVerified July 2026

What it is

Meta-ExternalFetcher fetches a URL after a person shares or requests it in a Meta product, including link-preview generation. Aiola classifies it as AI answers because access happens on demand.

In Aiola's taxonomy, AI answers means fetched to answer a user's question now.

User agent

Look for the match token inside the complete HTTP User-Agent header. Tokens can be spoofed, so use the network checks below for authentication.

Full example User-Agent
Meta-ExternalFetcher
Case-insensitive match token
Meta-ExternalFetcher

Official IP ranges

Meta's network is identified here as AS32934. ASN ownership is a useful signal, not a bot-specific allowlist; combine it with the User-Agent and request behavior.

Verify authenticity

Start with the source IP recorded by your trusted edge or server, not an untrusted forwarded header.

Reverse- and forward-DNS double lookup
host REQUEST_IP
# Confirm the hostname ends in an official suffix
host RETURNED_HOSTNAME
# The forward lookup must return REQUEST_IP

For production checks, test against every current prefix in the vendor feed. Re-fetch feeds regularly: a July 2026 snapshot is evidence of publication, not a permanent firewall list.

Control it with robots.txt

Meta-ExternalFetcher is user-triggered. Meta's controls may differ from automated crawling, so do not assume a robots.txt block is an authentication boundary.

Allow Meta-ExternalFetcher
User-agent: Meta-ExternalFetcher
Allow: /
Block Meta-ExternalFetcher
User-agent: Meta-ExternalFetcher
Disallow: /

Read Meta's official crawler documentation ↗

Track it

Aiola tracks this crawler on your site — crawls, pages, trends. See when Meta-ExternalFetcher arrives, which URLs it requests, and how activity changes over time.

Explore Aiola Analytics

FAQ

What is Meta-ExternalFetcher?

Meta-ExternalFetcher fetches a URL after a person shares or requests it in a Meta product, including link-preview generation. Aiola classifies it as AI answers because access happens on demand.

What user-agent token identifies Meta-ExternalFetcher?

Match the case-insensitive token “Meta-ExternalFetcher” in the User-Agent header. A matching header alone does not authenticate the sender.

Can I verify Meta-ExternalFetcher by IP address?

Meta traffic can be checked against AS32934, but an ASN is broad. Combine network ownership with the exact user-agent token and your own risk controls.

Can robots.txt block Meta-ExternalFetcher?

Meta-ExternalFetcher is user-triggered. Meta's controls may differ from automated crawling, so do not assume a robots.txt block is an authentication boundary. Use the specific “Meta-ExternalFetcher” group when you want a rule for this identity without changing rules for every crawler.

Related