Crawl & URLs
Check crawl stats, crawl issues, URL details, inbound links, and Fetch as Bingbot with bwt.
Crawl
bwt crawl stats # daily crawled pages, index counts, errors, and HTTP status classes
bwt crawl issues # URLs Bing had trouble crawling, with the issue bitmask decoded
bwt crawl settings # hourly crawl rate and crawl boostcrawl issues adds issue_labels, decoded locally from Bing's Issues bitmask. To change the crawl
rate or boost, see Site settings.
URLs
bwt url info https://www.example.com/page # last crawl, discovery, status, size
bwt url traffic https://www.example.com/page # clicks and impressions
bwt url children https://www.example.com/blog/ --http 4xx,5xx --all
bwt url children-traffic https://www.example.com/blog/url children filters by --http, --crawl-date, --discovered, and --doc-flags. Pages start at
0, and --all continues until Bing returns an empty page, bounded by --max-pages (default 100).
Inbound links
bwt links counts --all # pages with inbound link counts
bwt links list https://www.example.com/ --all # links to one page, with anchor textFetch as Bingbot
bwt fetch request https://www.example.com/new-page # ask Bingbot to fetch it
bwt fetch list # requests and whether they finished
bwt fetch get https://www.example.com/new-page --save fetched.txtfetch get shows the status in a table; --save writes what Bingbot fetched (its crawl record and the
response) to a file, and -o json includes it too.
url info has no "indexed" field, and empty links or crawl issues results are not proof of
health. On a site without crawl data, the URL methods return UnknownError. See
Reading the data.