Home › Blog › Free server log analyser

A free server log analyser that never uploads your log

SEP 17, 2026 6 MIN READ

Your analytics only saw the visits that ran JavaScript. Your access log saw everything, including every crawler that came and everything it could not fetch. Here is how to read it without handing it to anyone.

Free server log analyser: a log window above a mountain range at dawn, showing access log lines with timestamps, 200 and 404 status codes and requested paths beside the user agent strings of Googlebot, Bingbot, AhrefsBot, ClaudeBot, GPTBot, SemrushBot, PerplexityBot and Applebot, with each crawler's route traced across the peaks below and a note that the log is read locally in the browser.

Three things claim to tell you how search engines treat your site, and two of them are guessing.

Analytics only records the visits that ran its JavaScript, which crawlers do not. Search Console shows you a sampled, aggregated, two-day-late version of what one search engine did. Your access log is the complete record of every request anyone made to your server, including all the ones that failed.

It is also the file nobody reads, because the good tool for reading it costs money and the free ones want you to upload it. We built a free server log analyser that reads the file in your browser instead. This is what to look for once you open it.

What is actually in the file

One line per request. Who asked, when, what they asked for, what your server answered, and what they claimed to be.

66.249.66.1 - - [14/Sep/2026:10:14:32 +0000] "GET /pricing HTTP/1.1" 200 8431 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

That is the combined log format, and it is what Apache and nginx write by default. The address at the front, the timestamp, the request, then 200 for the status and 8431 for the bytes sent, then the referring page and the user agent string. Multiply by a few hundred thousand and you have a month of your site's life, which is unreadable by eye and trivial for a machine.

Why free is the hard part

The best known desktop log analyser is genuinely good and the free tier stops at a thousand lines, which for most sites is under an hour of traffic. A licence is around a hundred pounds a year, which is fine if you do this professionally and hard to justify if you want to look at your logs twice.

The browser-based ones are free in a different sense: you upload the file and they process it on their servers. That is worth pausing on, because an access log is a list of the IP addresses of everyone who visited you, each one tied to the pages they read and the times they read them. In most jurisdictions that is personal information about your visitors, and sending it to a vendor is a disclosure you probably have not written down anywhere.

There is no technical reason it has to work that way. Parsing text and counting things is exactly what a browser is good at. So ours does the whole job in the page, which means no line limit, no account, and nothing to disclose because nothing is transmitted.

The five things worth looking at

1. What failed for a crawler

This is the one to read first and the one people skip. Every 404 and 500 returned to a search engine is a fetch it spent on nothing, and a page it cannot fetch is a page it cannot rank. Sort the errors by how often they were hit. A URL that has returned 404 to a crawler two hundred times in a month is still linked from somewhere, and the fix is either to restore it or to redirect it, not to hope.

2. Where the crawling actually went

Split the crawler requests four ways: real pages, static files, URLs with query parameters, and errors. A healthy site is mostly the first. If a significant share is parameter URLs, something is generating endless variants of the same page, usually filters, sorts, session identifiers or tracking parameters that were never meant to be crawlable. If a large share is images and stylesheets, the crawl is being spent on assets rather than content.

3. Which crawlers, and how the mix is changing

Search engines and AI crawlers are not the same thing and your log will show both. GPTBot, ClaudeBot, PerplexityBot, CCBot, Bytespider, Amazonbot and a growing list of others fetch pages for reasons that have nothing to do with ranking you. Some are gathering training data, some are fetching a page because a person just asked a question about it. Whether you allow them is a decision worth making deliberately, and you cannot make it sensibly without first knowing how much of your bandwidth they are using.

Worth knowing: Google-Extended and Applebot-Extended are robots.txt tokens for opting out of training, not crawlers. They will never appear in your log, and any tool that reports them is inventing traffic.

4. Crawling over time

Plot requests per day and look for the cliff. A crawl that stops abruptly is almost always a robots.txt change, an accidental noindex, a certificate that expired, or an outage nobody noticed because the humans were asleep. A crawl that spikes is usually something that is not a search engine. Either way the date tells you where to look in your deploy history.

5. What is in your sitemap and never gets fetched

Compare the URLs in your sitemap against the ones search engines actually requested. Anything in the sitemap that has never once been fetched is a page you have asked to be indexed and nobody has come for. Usually it has no internal links pointing at it, which is a five minute fix, or it is thin enough that nothing wants it, which is not.

Proving a crawler was really Googlebot

Anything can put Googlebot in its user agent, and plenty of scrapers do, precisely because people allowlist it. Your log cannot settle this on its own, and a tool that prints a green tick next to a claimed crawler identity is guessing.

The real method is DNS, in two steps. Look the address up backwards, then look the name you get back up forwards, and check you land on the address you started with:

host 66.249.66.1
# 1.66.249.66.in-addr.arpa domain name pointer crawl-66-249-66-1.googlebot.com

host crawl-66-249-66-1.googlebot.com
# crawl-66-249-66-1.googlebot.com has address 66.249.66.1

A genuine Googlebot resolves to a name ending googlebot.com or google.com, and Bingbot to search.msn.com. Google, Bing and the larger AI crawler operators also publish the address ranges their crawlers use, which you can check an address against directly. No browser can make a DNS query, so this step happens in a terminal, and our tool gives you the command with the address already filled in rather than pretending it can do it for you.

What a log can do is flag the claims that contradict themselves. One address that is both Googlebot and a person browsing in Safari is not Googlebot. One that claims to be crawlers run by three different companies is not any of them. One that asks for /wp-login.php and /.env is not a search engine at all.

What a log will not tell you

It is a record of requests, so it knows nothing about what happened after the response left your server. It cannot tell you whether a page rendered, what a visitor did next, whether anyone read anything, or what position you hold for a query. It also stops at your edge: if a CDN served the request from cache, your origin log never saw it, which is why a site behind Cloudflare or Fastly should read the edge logs rather than the origin's.

Within those limits it is the only source that is neither sampled nor inferred. Every other tool tells you what it thinks happened. The log is what happened.

Reading yours

Download your access log, open the free server log analyser, and drop the file on the page. Apache or nginx, combined or common format, plain or gzipped, any size. It is free, it has no line limit, it needs no account, and your visitors' addresses stay on your machine, which is the only arrangement we would use ourselves.

Questions people ask about this

Is there a genuinely free log file analyser?

There are free tiers, and they are small. The best known desktop tool stops at a thousand log lines and one project unless you buy a licence, which is roughly an hour of traffic for a modest site. Most of the web-based ones are free in the sense that you pay by uploading the file. Trufend's server log analyser is free with no line limit, no project limit and no account, because it runs in your browser rather than on a server somebody has to pay for.

Is it safe to upload my access log to an online tool?

Think carefully before you do. An access log is a list of the IP addresses of everyone who visited you, timestamped and tied to the pages they read. Under the GDPR, the UK GDPR and the New Zealand Privacy Act an IP address is generally personal information, so uploading the file is a disclosure to a third party, and if you have a data processing agreement covering your visitors' data then that vendor should be in it. The safest version of the tool is the one that never receives the file. Ours reads it in the page with the browser's File API, and you can confirm that by opening your network tab while it works.

Where do I find my server's access log?

On cPanel or Plesk, look for Raw Access Logs or Web Statistics and download the file for your domain. On your own server, nginx usually writes to /var/log/nginx/access.log and Apache to /var/log/apache2/access.log or /var/log/httpd/access_log, often with yesterday's rotated into a .gz beside it. On a managed platform such as Vercel, Netlify or Cloudflare, logs are an export or a log drain rather than a file on disk, and on the cheaper plans they may not be available at all.

How big a log file can a browser handle?

Larger than you would expect. The file is streamed in chunks and only counters are kept, never the individual lines, so memory use stays flat. A hundred megabyte log of six hundred thousand requests reads in a couple of seconds. Gzipped logs are unzipped in the browser, so there is no need to expand them first.

Does crawl budget matter for a small site?

Less than people think, and not in the way the phrase suggests. A two hundred page site is not going to exhaust anything. What the same numbers do show you is whether your crawl is being spent on the pages you care about or on parameter variants, duplicate paths and files that should never have been crawled, and that is a structural problem worth fixing whatever your size.

Read your own access log, free

Drop an Apache or nginx log on the page and see which crawlers fetched what, and what failed for them. No upload, no line limit, no account.

Open the free log analyser

Related reading