Thursday, April 16, 2009

Lies, Damned Lies And Statistics v3

In the distant past I posted about why Internet TV statistics are an art not a science.

Nothing has brought this more to the fore than the following two screenshots:














The tops one is the tracking of pageviews for Adsense (the lower figure in the relevant one) and the bottom one for Google Analytics for exactly the same page (the code has only been placed on one page in both cases) over exactly the same period of time. The discrepancy is +/- 300%!

OK, this is for a web page, but it shows you how inconsistent statistics can be even for an 'accountable' medium like the internet.

For Internet TV delivery the results are not dissimilar. I've run tests for Windows Media logs which show that they are widely out from actual delivery.

But if Google can get away with this, what chance for online video ? It's worse than TV 1.0!

3 comments:

Rob Powell said...

How many Adsense ads do you have on the page? If you have three, it will count triple the impressions of Google Analytics which counts the page.

I do see discrepencies between Adsense and Analytics, but they are usually in the 10-20% range.

Iolo Jones said...

So, Rob, a page impression in Google Adsense is actually an ad panel impression, not a page impression ? Do you have a reference for this ?

There are two panels on the page, btw, so under your thesis, this would imply a 50 - 60% error margin.

So, either Google needs to get accurate about its terminology or its metrics.

I think the point is still well made - web metrics are as problematic as old world marketing metrics, unfortunately.

Rob Powell said...

No reference, just personal experience with adsense. When I have one ad on the page in one channel, the data matches up pretty closely. When I do anything else, it goes haywire - I can only assume they are double and triple counting.

All things considered, Adsense's pageviews aren't ever to be trusted IMHO. Google Analytics is as good as anyone else, but you're right it's problematic and more art than science.