When you logon to Google and perform a search, it never crosses your mind what is happening behind the scenes to make that page appear on your computer. In fact it takes more processing power than your computer can provide alone to make a single search, and it all happens in a fraction of a second. Behind the scenes, the queries related to your result run in parallel with many of Google’s servers, and this is why the result is returned so quickly.
The hardware Google runs on is a mixture of off the shelf parts and customised hardware designed specifically for use in the Google data centres, but originally Google ran on just a few off the shelf computers. It quickly out-grew this hardware as the site became popular and in just 12 years they have grown from 6 servers to in excess of 500,000 spanned over many data centres, but this figure is at least 3 years old so the current figure is likely to be much higher.
One of the customised parts Google uses is the power supplies running each server, which they re-designed to be as efficient as possible. They also announced they would be making their own switches rather than using off the shelf equipment.
A standard Google Web Server
There are several types of server roles that allow the searches to run the way they do. Servers we are aware of are:
- Load Balancers, These take a request and forward it to an appropriate server capable of handling the request at the time.
- Proxy Servers, These take the request and forward a result if it is in the cache, otherwise they forward it on to be processed further.
- Web Servers, These servers execute the request and format it into HTML ready for your browser to display.
- Data Gathering Servers, These servers have a dedicated role of crawling the web, updating the index and applying algorithms to the data they retrieve.
- Index Servers , These servers are responsible for retrieving the correct documents that link with the keyword(s) you are searching for.
- Document Servers, All documents gathered from the web crawling are stored so when you perform a search, a summary can be produced based on the keywords you searched for. They also provide the cached version of the page when you request it.
- Ad Servers, These provide the advertisements you may see around the web.
- Spelling Servers, These provide spelling suggestions if you make a mistake in your search query.
Unlike the hardware, most of the software Google runs is completely bespoke. They run on a highly modified version of Linux and a lot of custom software written to run the Google searches, crawling and other services. Google even had to design their own file system that was robust enough to cope with the huge number of files and folders their tens of thousands of servers in each data centre uses. They also needed to incorporate a high level of redundancy given their huge servers numbers. If one or more servers go down, the others must be able to take over and continue with all data accessible.
Now you know the basics of what goes on behind the scenes, it makes you realise how complex Google really is and why they have become the “God” of web searching in so little time. Without Google, Search Engine Optimisation probably wouldn’t be the same.
The information written above is very basic, and there is a lot more you can learn about the way Google works by looking at various articles around the web. One of the best articles I have found is the Google Platform Wiki page. Although it is now out of date, some of the most basic functions of Google haven’t changed a whole lot, and it is well worth a look.
This blog was written over 6 months ago and Internet Marketing and is an always changing industry which means the information within this blog may be out of date. Use caution when using any methods or suggestions within it.
