A Web page, or document, can contain various kinds of content (as opposed to display or presentation options like sound, animation or frames), some of which is not shown when you view the document in your browser
Title – an embedded description provided by the document designer; viewable in the title bar (it is also used as the description of a newly created bookmark by most browsers)
Description – a type of metatag which provides a short, summary description provided by the document designer; not viewable on the actual page; this is frequently the description of the document shown on the documents listings by the search engines that use metatags
Keywords – another type of metatag consisting of a listing of keywords that the document designer wants search engines to use to identify the document. These too, are not viewable on the actual page
Body – the actual, viewable content of the document. Search engines may index all or some of these content fields when storing a document on their databases. (Over time, engines have tended to index fewer words and fields.) Then, using proprietary algorithms that differ substantially from engine to engine, when a search query is evaluated by that engine its listing of document results is presented in order of ‘relevance.’
Because of these differences in degree of indexing and algorithms used, the same document listed on different search engines can appear at a much higher or lower ranking (order of presentation) than on other engines. A good search engine optimisation company should be able to deliver traffic from all major search engines.
Though not hard and fast, and highly variable from engine to engine, four factors tend to influence greatly the ranking of a document in a given query:
Order a keyword term appears – keyword terms that appear sooner in the document’s listing or index tend to be ranked higher.
Frequency of keyword term – keywords that appear multiple times in a document’s index tend to be ranked higher.
Occurrence of keyword in the title – keywords that appear in the document’s title, or perhaps metatag description or keyword description fields, can be given higher weight than terms only in the document body.
Rare, or less frequent, keywords – rare or unusual keywords that do not appear as frequently in the engine’s index database are often ranked more highly than common terms or keywords. Some engines, notably Excite, attempt to “infer” what you mean in a query based on its context. Thus, the meaning of heart can differ if the context of your search is cardiac disease as opposed to Valentine’s Day. The methods by which these inferences are made are statistically based on the occurrence of some words in conjunction with others. Though useful for simpler queries, such inference techniques tend to break down when the subject of the query or its modifiers do not fit expected query relationships. For commonly-searched topics, this is generally not a problem; for difficult queries, it is a disadvantage to standard full-text indexing.
Cottage industries have emerged to help Web site developers place themselves higher in the search engines’ listings (it is clearly more valuable to be within the first few listings sent to a user than be buried hundreds, or thousands, of documents lower). A constant battle is being waged between the engines and those desiring high listings from jimmying the system to “unfair” advantage.
Crude, early attempts to “spam” search engines to get higher listings included adding hidden terms like “sex” that were searched frequently but not the real subject of the document. Other techniques were to repeat certain keywords repeatedly, such as “cars cars cars cars cars” to get a higher frequency rating. Another was to cram the page with high-interest terms using the same colour as the overall Web page, thus “hiding” the added keywords. The leading search engines have caught on to these and now have automated ways to prevent the worst of these spamming techniques.
More subtle techniques, however, are hard to prevent. For example, a listing for ski resorts in France could also add hidden tags for “Caribbean” or “beach resort” knowing that wealthy Caribbean travelers may also be looking to take ski vacations. If you as the searcher asked for Caribbean vacations you may logically wonder why you’ve gotten a listing for French ski resorts. It is because of such techniques (among others) that you can sometimes get document listings from a search that seemingly have nothing to do with your query.
So, differences in how search services rank documents, how developer’s themselves choose to characterize their Web documents, and just simple errors in how computers process and index these pages can all lead to highly variable ranking results from different search services.