# How Blue Jetty measures crawler activity Blue Jetty records requests for your published app information and sends occasional milestone emails to the app owner. Milestones are available to all publishers, regardless of plan. Manage them at https://bluejetty.ai/account/notifications. ## Setup completion and first activity These are separate milestones. Operational `onboarding: onboarded` means app data has a source, a successful update and current inventory. Publisher authorization establishes permission to represent the app. `publisherOnboarding` and the completion report also check the website's discovery setup. A setup-completion confirmation reports those checks; it does not report an assistant using the packet. The dashboard's final step, “First crawler activity detected,” requires observed packet activity after setup is complete. The first-request email has its own threshold of 25 qualifying requests, described below. Neither milestone establishes that an assistant used the context in an answer. If activity has not arrived yet, a completed setup can still be valid. ## Dashboard counts All dashboard analytics capabilities are available on Free, limited to the last 30 days. Growth adds broader history and deeper API reporting; it does not unlock a different dashboard chart. The dashboard has a visible **Index activity | Publisher context** switch. Both views share the app selector and the last 7 or 30 UTC days. Publisher context combines JSON and Markdown packet requests. Index activity separately labels HTML crawls, JSON crawls, API record retrievals and Blue Jetty API search-result appearances. Record retrievals can overlap JSON crawls and are not added to the chart. The Performance chart is available to Free publishers: daily observations for the selected 7 or 30 UTC dates, including partial today, with a total line and provider-level lines. Select a line or provider box to isolate it, select it again to restore all lines, and expand the chart for more room. The app selector switches between workspace and per-app counts. Dates without recorded requests show zero; unavailable analytics are not treated as zero. Provider boxes below the chart match the selected view. These are requests or observations, not clicks or unique people. The publisher-context chart's provider totals and last-crawled timestamps cover packet requests only. The index chart covers HTML and JSON crawler observations only; record retrievals and search appearances have separate provider breakdowns. Unclassified API requests appear in the metric totals but not the crawler chart. Detailed API reports and broader history retain their existing Growth/pilot access checks. The Free chart does not grant detailed API access or change retention. Dashboard counts use the existing activity counters, including recognized preview crawlers and the generic crawler bucket. They are not unique visitors and do not apply milestone emails' per-minute grouping or qualification rules. ## Crawler identity Crawler names come from recognized user-agent strings supplied with requests. These identities are **self-declared, not independently verified**. A request identifying as a provider's crawler is not proof that the provider made it. Search crawlers, user-triggered fetchers and training crawlers can have different purposes; their requests do not all indicate recommendation activity. ## What qualifies for milestone emails We count successful HTTP 200 responses from the public JSON and Markdown packet endpoints for the current publication revision, while onboarding is complete. HEAD requests, HTTP 304 revalidations, errors, generic or unrecognized bots, unknown-purpose agents and preview crawlers do not qualify. Repeated responses are grouped to at most one counted request per app and crawler name per minute across both formats. Counts are conservative request observations, not unique visitors. Normal Blue Jetty setup checks do not qualify; deliberate crawler impersonation cannot be distinguished using user-agent identity alone. Milestone tracking begins with new qualifying traffic after activation. Existing historical traffic is not replayed into welcome milestones. These counts can differ from dashboard totals, which cover other surfaces and aggregation rules. ## Milestones and frequency - **First requests:** after 25 counted packet requests across recognized crawlers. - **New crawler activity:** newly observed crawler names, grouped together after the first-request milestone. - **Increased activity:** at least 100 counted requests in seven complete UTC days, at least three times the preceding seven days, and a preceding baseline of at least 10 requests. At least 14 complete days of observation history are required. Each crawler has a 30-day activity-alert cooldown. - **Updated app information fetched:** a qualifying request for a new current publication revision. Frequent updates are combined, and superseded update notices are suppressed before sending. Milestones are combined into at most one email per app per seven days. Delivery retries reuse the same message identity. Operational messages, including sign-in and onboarding confirmations, are separate from this preference. ## Coverage and interpretation These observations establish that Blue Jetty served a successful packet response. They do not establish that an assistant read, understood, indexed, cited, recommended or used the information. They do not measure installs, purchases or attribution. First-party CDN cache hits and self-hosted reads may never reach Blue Jetty and therefore may be absent. A lack of observed traffic does not establish a lack of access elsewhere. Provider-authenticated retrieval metrics are not currently available; historical declared traffic is not treated as verified traffic. The milestone system retains aggregate counts and delivery state rather than raw IP addresses or full user-agent strings. Daily detail is bounded; lifetime milestone markers prevent repeat announcements. Disabling milestone emails does not disable packet availability or ordinary crawler analytics. ## Index activity and publisher context The dashboard separates **Index activity** from **Publisher context**, using the same UTC range. Index HTML/JSON crawls, API record retrievals and Blue Jetty API search appearances are separately labeled; overlapping counters are never summed. Packet JSON and Markdown are combined only in the publisher-context view. Read [index activity reporting](https://bluejetty.ai/developers/index-activity.md) for `GET /v1/apps/{appId}/index-activity`, coverage and API entitlements.