Bytespider is ByteDance's Toutiao Search crawler. Aiola also lists TikTokSpider under ByteDance, with a separate token and fewer documented details.
What each crawler is for
Bytespider crawls pages so Toutiao Search can build and serve an index. ByteDance describes that pipeline on its Bytespider guide. The crawler follows links from page to page and keeps a snapshot of each page. Later processing extracts keywords, drops duplicates, and builds the index. Search then shows a title, a URL, and a short snippet from the page.
Aiola's notes describe TikTokSpider as fetching pages for TikTok search, previews, and discovery. The TikTokSpider record does not cite a ByteDance page for that description. Attribute this description to Aiola's record rather than ByteDance documentation.
Where ByteDance documents them
ByteDance documents Bytespider on the Toutiao webmaster guide. That page is the source for the strings, ranges, and DNS checks below. A separate crawl frequency guide explains the pressure tool for this crawler. Toutiao also publishes a robots matching guide for path and query rules.
Aiola's TikTokSpider record has no documentation URL. The three Toutiao pages cited here name Bytespider but do not document TikTokSpider. This record does not establish whether other public documentation exists.
Which user-agent strings to match
ByteDance publishes the token as Bytespider, with a capital B. The same guide publishes three full strings, for desktop, Android, and iPhone. Aiola's Bytespider page uses the desktop string as its sample. Aiola's documented crawler snippets match these tokens without regard to letter case. Use the address checks below when checking a request's origin.
- Desktop:
Mozilla/5.0 (compatible; Bytespider; https://zhanzhang.toutiao.com/) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.0.0 Safari/537.36 - Android:
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; https://zhanzhang.toutiao.com/) - iPhone:
Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_2 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko) Version/7.0 Mobile Safari/537.36 (compatible; Bytespider; https://zhanzhang.toutiao.com/)
The Android example reports version 5.0, while the iPhone example reports iOS 7.1.2. Match the Bytespider token rather than requiring a newer browser version.
Aiola's TikTokSpider sample is the bare name TikTokSpider, and its record marks the sample as unverified. Keep the original header from your logs when you write a rule.
How to check a Bytespider address
Compare the source IP on your edge or server with these ten ranges from the Toutiao page. Aiola's crawler record links to this published list and was last verified on 2026-10-23. Check the live vendor page before using a copied list.
110.249.201.0/24110.249.202.0/24111.225.148.0/24111.225.149.0/24220.243.135.0/24220.243.136.0/24220.243.188.0/24220.243.189.0/2460.8.123.0/2460.8.151.0/24
Treat any reverse name outside .bytedance.com as an impostor, per ByteDance's guide. Linux uses the host command with the logged address. Windows and IBM OS/2 use nslookup with the -qt=ptr flag and the address. On macOS, run dig -x with that same logged address. The page shows bytespider-111-225-148-148.crawl.bytedance.com as a sample reverse name.
Aiola's TikTokSpider record lists no IP ranges and was last verified on 2026-07-11. Do not apply Bytespider's published ranges to TikTokSpider based only on their shared vendor.
What to do when crawl volume spikes
ByteDance says heavy Bytespider crawling can slow your site or take it down. Use the crawl frequency tool on the Toutiao platform when that happens. Bytespider already adjusts volume from update frequency and server pressure. The tool's chart shows recent fetches for the site you select. A change you submit takes effect as early as the next day.
ByteDance recommends pressure adjustments only when crawling looks abnormal. It warns that limiting the upper bound can cause crawl problems. Select the affected hostname before submitting feedback. These instructions concern Bytespider; the cited tool documentation does not describe TikTokSpider controls.
Which robots.txt lines to publish
Publish a separate group for each token you want to limit. The block below requests that both tokens avoid all paths. Place it in /robots.txt if that is your intended policy.
User-agent: Bytespider
Disallow: /
User-agent: TikTokSpider
Disallow: /Toutiao's matching guide accepts a path rule only when it starts with a slash or an asterisk. Its examples distinguish /fish/ from /fish: the trailing slash limits matching to that directory. The guide also documents query matching, including wildcard support for parameter names and values. Read the live file after deployment to confirm the published groups.
How Aiola Analytics labels both
The AI view shows crawl totals, bots, pages, and activity over time. It includes Answers, Indexing, Training and Other buckets, AI-answer referrals, crawler 404s, and discovery-file health.
Bytespider's index kind and TikTokSpider's search kind both map to Indexing. Read the Analytics docs for the bucket map.
Crawlers do not run JavaScript, so the browser tag cannot see them. One server or edge snippet from the setup guide posts each matching hit. Only known crawler user agents are stored.
A recorded crawl is not proof that the vendor cited the page. An empty period is not proof that the vendor cannot reach the site.
Try it in Aiola
Download Aiola free for macOS 12 or later and Windows 10 or later. Hosted Analytics needs Pro or the 7-day trial, and the pricing page lists both. Open the AI view to compare Bytespider and TikTokSpider activity.
