Greed Feedback Loops: Internet Indexing, Search Engine Optimization, And Content : Past Search
December 5, 2010
Wow, I thought the teeth gnashing over objective search results was a dead issue. Objectivity is not part of the free Web search method. Uninformed people accept results as factual, relevant, and worth an invitation to have lunch with Plato. Wrong. Objective search results are a bit of myth and have been for decades.
Some education, gentle reader. A commercial database exercises editorial control. If you ran a query for ESOP on the Dialog system for File 15, you got a list of results in which the controlled term was applied or, if you were a savvy searcher, in documents in which the string ESOP appeared in a field or an abstract/full text field. The only objectivity involved was that Dialog matched on a string. No string. No match.
Online information is rife with subjectivity.
In the commercial database world, the subjectivity comes into play when the database producer selected an article to summarize, the controlled terms to apply, how the searcher framed his or her query, and what file to use in the first place. In ABI/INFORM the content set guaranteed that you would get only articles from magazine and journals we thought were important. The terms were the domain of the editors. The searcher controlled the query. Dialog was passive.
Flash forward to free Web search.
Search is expensive and the money to pay for content processing and the other bits and pieces of the so called free system. The most used Web search services get money mostly from advertising; that is third party payers. The reason advertisers pay money is to get access to Web search users. The present Web search system is largely built to maximize the money that flows to the search service provider. Nothing about the process is objective in my opinion. Unlike Dialog, free Web search meddles with the search results anywhere it can in order to derive benefit for itself. A happy user is not the goal of the system. A happy advertiser is the main focus in my opinion.
In the good old days, there was overt meddling, but was the the users query and the database producers editorial policy. The timesharing company providing the service selected some databases for its service and excluded others. Users had no control over the timesharing vendor. Dialog and LexisNexis did what was necessary to maximize revenues and control the customer, the database producer, and the revenues.
But even in the good old days most online searchers di=d not worry much about the database producers editorial policies. Today almost no one thinks about the provenance of a content object. The Web search service wants clicks and advertisers. The advertiser wants clicks, leads, and sales. The content is not the main concern of the advertiser. Getting traffic is the main concern. And the Webmaster of an individual Web site wants traffic. The user wants information for free. The SEO industry sprang up to help anyone with money spoof the free Web indexes in order to get more traffic for a Web site which had little or no traffic in many cast. These are the ingredients of the feedback loop that has made free Web search the biased service it is. And the feedback loop that almost guarantees a lack of subjectivity.
Now read When Businesses Attack Their Customers or one of the dozens of other write ups by English majors, failed programmers, and search engine optimization experts. The notion of a Web search system fiddling the results seems to be a real light bulb moment. Give me a break. Consider these typical functions in Web indexing and posting today:
Lousy content created to get clicks from the clueless. Theres big money in crap content because of programs like Google AdWords. But those annoying pop up ads, those are just variations on the crap content scheme. Lousy content exists because search engines incentive the creators of this content. Users are unable to think critically about information, preferring to take whatever is dished up as gospel.
The Web indexes are not in the education business. Web indexes are in the traffic and advertising business, and these outfits will do whats necessary to get traffic. If the National Railway Retirement Board adds an important document, that document may want a long time before a Web search engine indexes it. Put up a post about Mel Gibsons court battle, and that document is front and center really fast. Certain content attacks clicks, and that content gets the limelight.
People who use the Web describe themselves as good researchers. Baloney. Most people look for information the way a Stone Age person made a fire: Wait for a lighting strike, steal or borrow a burning stick from a tribesman, or get two rocks and bash them together. Primitive queries cause Web search systems to deliver what the user wants without the user having to think about source, provenance, accuracy, or freshness. By delivering what users may want, Web search engines create a way to offer advertisers what appears to be a great sales advantage. I think the present approach delivers advertisers meaningless clicks, big bills, and lots of wacky metrics. Sales. Not so much.
I dont think the commercial online search systems and the commercial database producers have a future filled with exploding revenues and ever higher quality content. I think the feedback loop set up and fed by free Web search is broken. In its wake is the even more subjective and probably easier to manipulate social search method. If you dont know something, just ask a fried. That will work really well on certain topics. The uninformed are now leading the uninformed. Stupid is and as stupid does.
I use the Exalead Web index. No index is perfect, but I am more confident in Exaleads approach because the company is not into the ad game. I also use DuckDuckGo and Blekko. Neither is perfect, but I have more confidence in the relevancy of the results, but I dont know the scope of the companies indexes, not their respective editorial policies. The other Web indexes are little more than ad engines.
And SEO or search engine optimization? That discipline was created to get a Web page to the top of a results list. Never was the SEO motivation precision, recall, or relevancy. Accuracy of the content was not a primary concern. Clicks were it. As SEO experts trashed relevancy methods, the Web search engines abandoned objectivity and went for the clicks and money. I dont have a problem with this, what I have a problem with is the baloney manufactured about bias, lousy search results, and other problems. These problems, in my opinion, complement the the naive and uninformed approach to research most users of Web search systems rely upon.
A failure in some education systems virtually ensures that critical thinking is in danger of becoming extinct. In an iPad mad world with attention deficit disorder professionals running rampant, I suppose the howls of outrage may be news. For me, this is an old story and an indication of the state of Web search.
The feedback loop is up and operating. Irrelevancy will increase in the quest for ad revenue. No easy fix in sight for a problem thats been around for a decade. Now the Web search providers want to push search results to users before the users search. Gee, thats a great opportunity to deliver subjectively ordered results based on advertiser needs. The scary part is that many Web users neither know no care about provenance, precision, recall, or relevance.
Welcome to a future with lots of lousy searchers who think they are experts.
Give me a break.
If you know an information professional, sometimes called a librarian, take a moment and get some advice from a real pro about searching. Too much work? Maybe thats why so many bad decisions are evident today? Bad data, uninformed decisions, a lack of critical thinking, and flawed information skills are nutrients for big and bad mistakes.
Stephen E Arnold, November 30, 2010
Freebie
Comments
Got something to say?