Google Spying on Web Surfers?

According to Paul Chvostek, Google has some sort of an arrangement with Opera that gives Google a head start on indexing web sites by monitoring user behavior.

"I created a new page at a URL that had never existed.  I visited that page with Opera.  And within 30 seconds, I was seeing traffic for the exact same page from hosts in 64.68.86.0/24 with a user-agent string that included "Mediapartners-Google/2.1" Paul mentioned in a recent post to inet-access.

He goes on to describe the tests that he ran to track down who was responsible for this traffic. Using tcpdump, he found that the traffic was emanating from 216.239.32.0/19 (a single block of 8192 IP addresses).  

"216.239.32.0/19 is also owned by Google. 'Mediapartners' my ass." he wrote.

So what's the scoop? Has Opera cut a deal with Google to give it immediate line of sight to new websites coming online? Not having to "discover" new websites through regular means would give Google a pretty decent advantage over competing search engines. But Opera doesn't have much market share. Less than 1% by some accounts. That really means squat in terms of competitive advantage. If true, it does show that Google is serious about making sure that they are using technology to its fullest advantage.

You can almost bet that Microsoft hasn't cut a deal with Google. If anything, Internet Explorer is going to sell out its users to MSNBot.  That really just leaves AOL. They have tons of eyeballs and more than a couple of ways to get this information into Google's hands. AOL 9.0, ICQ, WinAMP, AIM...Netscape.

And that's just for starters. Remember, Google powers AOL Search. Not a big separation between church and state here.

Paul needs to test his theory further and analyse some of these other internet client applications. If true, the implications are staggering. As he says in his message, "I don't know if this is known behaviour, but it scares me..."

I hope you're wrong Paul, otherwise I'm scared too.