In this case, the difference in user interaction is so clear and marked we could tell extremely quickly which one worked better: the difference is in the thickness of the plus box next to the stock quote. Now, coming to the conclusion that one is "better" is tricky, and there's many a possible slip on the way there. Does more interaction with the plus box mean that it is better? How about if users then miss good results because they are distracted by the more prominent plus box? Keep watching Google to see which version won! If we've done our job right, almost without your noticing, things will work just that little bit better for you. The world will seem rosier. Birds will sing. Or maybe not - but at least you will have the best-designed plus box we can come up with :)

Okay, so not all of our experiments are insane eye tests. My main point in highlighting the above experiments is that we test almost everything, even things that you would think are so small that we could not possibly care (nor could they possibly matter). In fact, small changes do matter, and we do care.

Another class of experiments have to do with changes that are not purely visual, but rather involve changes to the underlying presentation algorithms. For instance, the algorithm that is responsible for the titles and snippets of result pages now highlights stems and some synonyms of the original query term. For the query [hp printer drivers] we will also return results that include and highlight the word "driver".

This sort of "stemming," as it's called, is generally a good idea, because it helps you better identify results that match your query, but not always. Experiments of this sort help us verify (or, occasionally, overturn) our assumptions regarding changes in these algorithms.

There is a further class of experiments - the kind that are hard to miss - which introduces fairly prominent features. Even with these larger features, the goal of experimentation always remains the same: are we adding something that really helps people, or is this just another distraction? Google does not really come with a user manual (actually, there are some nicely-written help pages, but we're pretty sure most of you don't bother to read them!). So features need to stand on their own feet, without the help of a careful explanation. Part of the goal of an experiment is to understand just how a feature will be used, which might be quite different from what we initially intended.

Here's an example of an experiment that lets you comment on search results and move them around on the result page:

At this point, I can't say what we expect from this feature; we're just curious to see how it will be used.

These are a small sample of the kinds of experiments we run as we test everything from the barely visible to the glaringly obvious. So the next time you use Google and it seems a little different - well, maybe it is. Just for you!

Posted by Ben Gomes, Distinguished Engineer