A couple of e-mailers have confided in me that they have done private page-to-page tests, and, at least in their isolated cases, PW did a better job than Alexa in representing the shape of their traffic.

You can end up in situations like Gordon in a variety of ways. For one, Google massages their data in a way we don’t: we just record hits and uniques for the day and display that. If their code is on more pages, or ours is on different pages, you’ll get different results. To further muddy the waters, Google’s JavaScript doesn’t detect hits if the viewer has JavaScript disabled, while Project Wonderful still loads an image that we can track – but then, if they’re running adblock software, they may not see it even IF they’re running JavaScript. But again, PW wants to show you how many viewers your ad will get, so the smaller number that ignores people running adblock software is what we want.

I get a few emails at Project Wonderful occasionally asking about this discrepancy between Google and PW stats (sometimes one’s bigger, sometimes it’s the other) and generally it’s often the case that each of us are tracking different things on different pages. The bottom line is that, in all cases, the most reliable data you’re going to have is your own log files: they can record things like direct image access (ie: someone inlining your comic on a forum) that neither PW or Google are aware of.

But I think here we’re more concerned with estimating audience size, which is always going to be fuzzy. I think Project Wonderful does make that more open than it was in the past, but you still need to take into account the forum / archive / “hosted on multiple domains with ad boxes only on one” issues when you’re looking at them. Unfortunately PW is designed to show you what exposure an ad will get, not how much traffic a given comic receives – though in some cases they should be the same.

Does that help?

Some problems with PW data:

Sites like mine use one PW ad for multiple domains, but only as a default; therefore, our PW score is much lower than it was when we used it as our sole advertising network.

Sites like QC use one ad format for their main page and another for their archives.

Some sites put PW on their forums, greatly increasing the total number of ad views, and some do not.

To my knowledge, no one has independently tested the data’s accuracy against something like Google Analytics, even allowing for the problems cited above. I never used PW in a straightforward way with one PW ad on every single page, so I couldn’t test for a pageview-to-pageview match. Ryan North is a great guy, and I have no doubt he’s proud of his system, but if you’re going to state that the PW data is superior, you really want to be able to back that up.

That’s interesting… and perplexing.

For Sunday, for instance, Multiplex clocked 22.6k visitors and 30.6k pageviews according PW — but according to Google Analytics, those numbers were 10.4k (9.3k unique) and 45.7k respectively.

They’re — presumably — similarly inaccurate with all the other sites on the network, so it’s still useful if you’re a prospective advertiser, but compared to real-world metric, no. Not really accurate.