What Content Can Search Engines “See” on a Web Page?

Search engine crawlers and indexing programs are basically software programs. These
programs are extraordinarily powerful. They crawl hundreds of billions of web pages, analyze
the content of all these pages, and analyze the way all these pages link to each other. Then
they organize this into a series of databases that can respond to a user search query with a
highly tuned set of results in a few tenths of a second.

This is an amazing accomplishment, but it has its limitations. Software is very mechanical, and
it can understand only portions of most web pages. The search engine crawler analyzes the
raw HTML form of a web page. If you want to see what this looks like, you can do so by using
your browser to view the source.

The Search Engines: Reflecting Consciousness and Connecting Commerce

Search has become integrated into the fabric of our Society. With more than 12 billion searches being performed each month as of January 2009 (according to com Score), approximately 400 million web searches are performed every day. This means that on average more than 4,500 searches are performed every single second of every day.

As Google owns approximately 65% of the search market share, Google’s search technology
handles more than 2,900 searches per second. In addition, users have grown to expect that the
responses to their search queries will be returned in less than one second.

Now people can obtain information in mere seconds—information that 20 years ago would
have required a trip to the library, a search through the Dewey Decimal System, and a foot
search through halls of printed volumes, a process that could easily have consumed two hours
or more. Through the new channel of search, people can also conduct much of their shopping,
banking, and social transactions online—something that has changed the way our global
population lives and interacts.

This dramatic shift in behavior represents what investors like to label a disruptive event—an
event that has changed something in a fundamental way. Search engines are at the center of
this disruptive event, and having a business’s website rank well in the search engines when
people are looking for the service, product, or resource it provides is critical to the survival of
that business. However, like most paths to success, obtaining such prime search result real estate is not a simple matter, but it is one that this book aims to deconstruct and demystify as
we examine, explain, and explore the ever-changing art of search engine optimization (SEO).