ChatGLM-Spider, KimiBot and Timpibot appear in Aiola's crawler directory under Zhipu AI, Moonshot AI and Timpi. Their records describe different collection purposes and provide different evidence for identifying requests. A recorded user agent identifies a claimed crawler name; it does not authenticate the sender.
Who publishes a page about each one?
Zhipu's about page dates ChatGLM's launch to 2023. The ChatGLM-Spider record associates the crawler with Zhipu AI and lists bigmodel.cn. Its documentation field is empty, so that company history provides context without establishing a crawler policy.
The KimiBot record names Moonshot AI and links to a dedicated vendor policy. Unlike the other two records, it also links to a published IP feed. Those fields provide additional material for checking a request's claimed identity.
Timpi's about page reports more than 2.2 billion indexed web pages and over 1,700 decentralized nodes. Its search engine uses its own web index. The Timpibot record names Timpi and lists timpi.com as the vendor domain. The company page describes its infrastructure, while the directory supplies the crawler-specific fields.
What does each crawler collect?
Aiola's ChatGLM-Spider note describes public web collection for Zhipu AI's ChatGLM model ecosystem. The record categorizes it as training and gives its kind as crawl. It does not specify file types, individual datasets, or which model receives any particular page.
KimiBot's directory note describes content potentially used for foundation-model training. That wording leaves the eventual use of any individual page unspecified.
Timpibot's note describes crawling public pages for Timpi's decentralized search index. Its stored category is nevertheless training, and its kind is crawl. This reflects Aiola's classification rule: crawl maps to Training, while search and index map to Indexing.
The bucket therefore needs to be read alongside the crawler's purpose. Timpibot's directory note supports a search-index purpose, without stating that Timpi trains a model on your page. All three records use the training category, including KimiBot.
Which string appears in the log?
The table reproduces Aiola's stored samples and tokens. ChatGLM-Spider and Timpibot have token-only samples, with no documentation URL attached to either record. Treat these as directory identifiers rather than vendor-confirmed complete request headers.
| Crawler | User-agent sample | robots.txt token | IP feed in record |
|---|---|---|---|
| ChatGLM-Spider | ChatGLM-Spider | ChatGLM-Spider | None listed |
| KimiBot | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; KimiBot/1.0; +https://www.kimi.com/policies/kimi-crawlers | KimiBot | kimibot.json |
| Timpibot | Timpibot | Timpibot | None listed |
Kimi's published header matches the stored sample. Its linked JSON feed contains IPv4 prefixes. The absence of equivalent links on the other records only describes those records; it establishes no worldwide publication claim.
How does Kimi say to use robots.txt?
Kimi's crawler policy recommends robots.txt at the site root, separately for each subdomain. Each crawler has independent settings.
For a full block, its example uses:
User-agent: KimiBot
Disallow: /Its path example instead disallows /private/ and /paid/. Another example uses Crawl-delay: 1, setting a one-second minimum request interval.
Kimi says a training disallow signals that content should not be used for model training. It publishes changing IP ranges for verification and warns that IP blocks can prevent reading updated robots.txt instructions.
What can you say about the other two?
Neither ChatGLM-Spider nor Timpibot has a documentation URL or published IP feed in Aiola's directory data. Their records therefore cannot establish whether either crawler honors a particular robots.txt directive. A company about page does not supply that missing policy.
You can write groups using the stored tokens, but these examples carry no vendor-backed compliance promise:
User-agent: ChatGLM-Spider
Disallow: /
User-agent: Timpibot
Disallow: /Keep those groups distinct from KimiBot's documented example. When describing a rule change, distinguish the rule you published from behavior actually observed in requests. Neither record supplies an IP-based check for the two token-only samples.
How do you collect these hits in Aiola?
Aiola's Analytics setup guide separates browser tracking from optional crawler tracking. The browser tag cannot see these crawlers because they do not execute it. Crawler collection requires one server or edge implementation at the layer receiving page requests.
Open Settings, then Analytics, in the project you want to track. Enter the production hostname and select Enable Analytics if needed. Enabling Analytics creates a public Tracking ID, and the saved domain is required for collection. Enter a hostname without a scheme, path, or port.
Choose one documented implementation: Next.js middleware, a Cloudflare Pages Function, or Express middleware. The snippets check user agents before sending hits and skip the listed static asset extensions. They retain requests for robots.txt, llms.txt and sitemaps.
Crawler submissions include the Tracking ID, requested path, hostname and user agent. Country and response status are optional fields. Aiola stores recognized crawler user agents only, and the hostname must match the configured domain.
What should you check after a rule change?
Aiola's Analytics documentation describes an AI view with crawler totals, bots, pages and timeseries. It also shows crawler 404s, uncrawled popular pages and AI-answer referrals. Use the bot and page views to examine collected requests around the affected paths.
Response-status reporting depends on the integration. Next.js middleware runs before the final response status is known, so the guide adds a separate not-found handler. The Cloudflare and Express examples report status after a response is available. Check that distinction before interpreting missing crawler 404s.
A crawler request and an AI-answer referral represent separate activity in Analytics. A hit alone does not demonstrate that an assistant cited your page or sent a visitor. Compare the relevant views without treating one measurement as proof of the other.
Try it in Aiola: Analytics names 70 crawlers in the desktop app for macOS 12+ and Windows 10+. Hosted Analytics requires Pro or an active trial. The 7-day trial is for first-time Pro Monthly customers and requires a card. Download Aiola free, then follow the crawler setup steps for your project.
