In my previous post, I described the components of your web search experience and the principles behind creating a great search experience. There are complex algorithms underlying simple features such as spelling correction and the two line snippets that describe each search result. We figure out what works by running experiments - tiny tests for a small number of users which help us determine whether that feature helps or hurts.

Experimentation is a very powerful tool, and we use it very widely to test potential changes to search. At any given time, we run anywhere from 50 to 200 experiments on Google sites all over the world. I'll start by describing experimental changes so small that you can barely tell the difference after staring at the page, and end with a couple of much more visually obvious experiments that we have run. There are a lot of people dedicated to detecting everything Google changes - and occasionally, things imagined that we did not do! - and they do latch on to a lot of our more prominent experiments. But the experiments with smaller changes are almost never noticed.

For example, can you tell the difference between the two pages below?

Choice 1:

Choice 2:

I'm pretty sure I would not be able to tell the difference if I were to see each of them on their own. But apparently you can! At least in the aggregate, there is a measurable difference with a change like this. In case you can't tell after staring, the white space around the first search result has changed, which makes the first result in Picture 2 slightly more visually prominent. This visual prominence conveys the fact that according to our ranking signals, the first result is a substantially better match than the next result. On the plus side, it helps you focus on the first result. But if you were looking for one of the other results, it can disrupt your scanning of the page. An experiment helps us determine which effect is more prominent, and whether a change would help you search faster.

Another change, almost as minimal visually, is between these two results: