L023: Has Google been Naughty? Yes. Should the Government Get Involved? No

Read Vivek Wadhwa's thoughts on this piece, as posted to his blog, here.
Robert Scoble's response is here.  

=-=-=-=-=-=-=-=

By Jason Calacanis

Last week the LAUNCH team wrote a piece on Google being in a big of pickle with governments -- plural -- over its search results.

Now I am giving my personal perspective on the matter.

[ EDITOR'S NOTE: LAUNCH is written by a team with a shared byline unless otherwise noted (as in this piece -- which is me, Jason, and me alone). While I did contribute some knowledge to that piece, Kirin Kalia, who worked with me at the Silicon Alley Reporter, did most of the heavy lifting. She is an analyst here at LAUNCH, and she talked to a bunch of CEOs and founders we all know, as well as Google and the FTC. ]

The fact is government regulators are up in Google's algorithm, and it's having a bad impact on our industry. Any company as powerful and successful as Google is going to raise eyebrows from time to time, but a decade of Microsoft-like investigations is not good for anyone in the ecosystem.

Government officials are sometimes good, sometimes bad and almost always clueless when it comes to the technology industry. By the time the regulators figured out what Microsoft had done wrong, the war was long over.

Microsoft's approximately $2B in fines from the European Commission equals 24% of what Microsoft paid for Skype -- or about 12 days of Microsoft's revenues in 2010.

It's up to our industry to get ahead of regulators and educate them. That's why I was so critical of Facebook last year. Facebook was taunting regulators, senators and attorneys general with its nefarious practices. The punchline is the stupid things Facebook does have little to no impact on its success. The company is talented enough to win without cheating (or hiring PR firms to ankle competitors).

The real issue is that these investigations are slowing down Google and spinning it around. Everyone in their right mind knows that Google puts its thumb on the scale of search results and that human-powered search is the best solution (albeit non-scalable at this time, as my scars prove).

Has Google behaved so badly that the government needs to come in and regulate it? No.

Has Google been naughty? Yes.

Very, very naughty in some cases.

Here are the four naughtiest and most puzzling things Google needs to self-regulate (read: stop). Now, I'm not ascribing intent here. I don't know if Google is doing these things accidentally, because it lacks empathy or because it simply wants to make a compelling product. I suspect it's a little of each.   

After outlining each problem, I offer a very simple solution for Google to once again become a friend of the startup ecosystem.

[ SIDENOTE: I realize that talking about Google’s behavior in light of Facebook’s pillaging of the industry -- which has been 10 times worse -- may seem silly, but as a Google fanboy I'd like to see the company get back to its "don’t be evil roots" and be more Google-y. ]


1. Problem: Search Updates Crushing Startups and Costing Jobs

If you’ve got so much power that a large part of the consuming, funding and developing content is impacted by your search results, you have an obligation to make changes to those results with care, communication and consideration.

Google showed none of that with its Feb. 24 Panda update.

Now, I’m biased here as the CEO of Mahalo, but what is the point of taking hundreds of sites with tens of thousands of employees -- and hundreds of thousands of freelancers -- and cutting their traffic 25% to 85% overnight?

Is this some sadistic revenge for Demand Media CEO Richard Rosenblatt saying he could out-algorithm Google?

Is this because the New York Times made Google executives look incompetent and stupid with the eyeglasses and JCPenney stories -- which occurred within three months of each other?

Since Panda, Google has screwed up its relationship with content owners (not great to begin with) and has produced worse search results. It was, simply put, Google's worst search update. The company should have rolled it back, and since it's clear it won't, it should act quickly to solve the countless examples of horrible search results that Panda has created.

I’m hopeful Google will make big improvements in 2011 because, frankly, it is going to lose market share if it doesn't. Panda has exposed many of Google weaknesses, and Microsoft should be pouncing on these errors right now.


Solution: Communicate Search Updates and Don't Whipsaw Companies

Google should create a calendar of algorithm updates and inform the ecosystem, in broad strokes, of its plans. The company should show the new search results and proposed changes on a set of Google servers called googleupdate.com.

Since Google already has a system for consumers to voice their opinions (the blocking toolbar/remove sites feature), googleupdate.com would be reserved for domain owners. So, if I’m from the confirmed domain name of ehow.com or associatedcontent.com, I could comment on rankings related to those sites. This would help avoid the SEO group sabotage against sites.

There should be a 60-day open discussion period for those changes -- a massive show of transparency and good faith without giving away trade secrets. Doing so would cost Google practically nothing, and I’m certain the free feedback from content owners would actually make for better content, better behavior and hundreds of actionable insights.


2. Problem: Comprehensive Search Results Killing Competitors (the Yelp Problem)

Only five years ago most of Google’s search results were limited to 10 blue links. Then Google started adding sports scores, maps, stock prices and movie times, all the while assuring its content providers it was not competing with them.

Savvy folks at the time pointed to Google’s nuking of the entire comparison-shopping industry by creating the arbitrary and absurd "no search results in our search results" policy. Froogle didn’t have to compete for a slot in search result because, coincidentally, it was part of Google’s main navigation bar.

Why is bad for me to get search results in a search result exactly? If I’m looking to compare the prices of a certain pair of jeans, you’re telling me I won’t appreciate finding that in Google’s results?

Then there is local. What a disaster this has been for Google. The company won’t respect Yelp’s content, similar to what it did with other people’s books in the Google Books project.

Yelp has been pushed all the way down the page, and Google Places, which scrapes (a.k.a. steals) Yelp’s content without permission, gets the top of the page [ see this example ]. In addition, Google photocopied Yelp’s layout.

You get the sense that Google wants to take every content and community provider in its ecosystem and slowly push them below the third or fourth link.

And don’t give me this BS that the Places items are not the organic search results, they are comprehensive search results. They *are* part of the search results, and they push down the organic results.   


Solution: Stay out of the Content Business, Otherwise Pay Content Providers for Their Content

Google putting its content above the folks in the ecosystem seems like the "Microsoft bundling" of IE on the Windows desktop issue.

Google has a simple way to address this: stay out of the content and community business. Google says it is not a content company, but photocopying Yelp’s design and moving Yelp down in the rankings and putting your Places service up top is disingenuous.

If you feel you must get into the content business to make better search results -- and perhaps  Google does -- spread the wealth.

Why not license Yelp’s reviews?

Yes, cut Yelp a check. Google is massively profitable, and if Google is going to scrape other folks’ data for its local project -- while pushing them down in the rankings -- why not give them each $50M a year deal for their content? That would cost Google, what, $250M in licenses?

Then Yelp, Insider Pages, CitySearch and Zagat would be even MORE in Google’s corner. Google is a juggernaut and will make billions in profits off local every year. Why not build a sustainable model that spreads the wealth?

That’s exactly what Google did with its CPC advertising network: the company made it available to partners. Those off-site ads and clicks now generate the majority of Google’s revenue!

Again, why is Google stuck in this "we don’t pay for content" model? Since the company makes more money than any startup, Google should go “splashy-cashy!” on the industry.

This could apply to Google News as well. While Google may be in its right to scrape a little content, and while it provides the ability to opt out, why not recognize that scanning headlines is half of content consumption?

Why not give news providers in Google News the ability to select the length of their abstracts and split the revenue generated on Google News pages (of course you would need to add advertising)? If those pages start making a $10 CPM and Google gave half to the publishers just for showing their headlines, or based on which headlines got clicked, Google would become a HERO to the news industry.

Can you imagine if Google not only sent you traffic but also paid each of 50 stories abstracted on their pages $0.10 CPM ($5 CPM in total)?  


3. Problem: Google Competing with Partners on Content (the Google Knol & YouTube Problem)

Years ago I wrote what a disaster Google Knol was for Google on a PR basis because it was designed to steal writers from content companies. Google came up with every reason for why paying writers and publishing their content wasn’t competitive -- wait, what? -- but it all became moot when Knol was stillborn.

However, YouTube is the opposite of Knol. It’s a brilliant success, and Google is sharing 55% of the revenue with partners. The company also announced giving advances to content companies against this revenue.

YouTube is the shining example of what Google should be doing as a company: paying partners for content used on Google sites.

That being said, YouTube is only paying 55% and that feels like 15% too low.


Solution: Google Content Sites Should Pay 70% -- Like the App Store

This is a no-brainer. Google should increase its generosity to content partners on YouTube and take an Apple 70-30 split approach. That would double the amount of activity on YouTube.

I’m not saying Apple should charge 30% -- that feels like a huge tax for doing very little work. However, selling ads in the content business costs a company 10% to 30% depending on your scale.

Of course, now that Mahalo has pivoted to educational videos I’ve got a big, black stallion in this race that is tearing up the turf. We’re probably the largest publisher of non-UGC content on YouTube. Do tell me if you know of another professional firm publishing almost 1,000 high-quality videos to YouTube each week.


4. Problem: Google’s Lack of Transparency in Search Rankings

It’s time for Google to admit that its power in the ecosystem is so great that the company owes it to the rest of the ecosystem to become more transparent in how it ranks stuff. Google should also discuss specific examples and how to make them better.

Right now getting someone at Google to talk about search rankings is like getting someone in the Obama administration to talk about how often the POTUS is smoking.


Solution: Creating an Independent Content Ratings Agency

Google’s opaqueness creates too many problems. We need a third-party service that rates the rater as a check-and-balance to Google’s dominance.

An independent, transparent, PageRate system would, although subjective, be a service for all of us in this debate.  

PageRate would simply rate pages and search results and publish a Moody’s or Klout-like score. Details of the report would be open for everyone to see and debate.

Why doesn’t this yet exist?!

If it did, you could look at Google rankings and say, "This result is missing the top three ranked pages which are over 80 in PageRate, and the top five include three with a PageRate score of under 60."

I just bought PageRate.com for $1,700, and if I can find two tech leads in the Y Combinator format, I will start this company with you and put in $250K of my own money to get it going.


Wrapping up

Here are the big points you should take away:

1. Google is a great company with great people who don’t communicate well with the rest of the industry.

2. Google is so big and powerful that it is crushing the competition and companies without really considering it.

3. Google’s leadership needs to double down on paying content companies. Facebook does nothing for content providers (but that’s another editorial), so Google should do more. LEAN IN WHEN IN DOUBT! (hat tip Doug Leone).

4. I want to start a rating agency at PageRate.com, and I’m looking for two tech leads to build it under my direction. Email jason@calacanis.com. Must be willing to work, and not sleep often, in Santa Monica.

all the best,

Jason


P.S. Yes, I will be at the D Conference.
P.P.S. No, there isn’t a sick, sick poker game occurring for the fourth year.
P.P.P.S. No, there are no seats open for said game.