Until last fall, the search function at NeimanMarcus.com was typical of e-retailers. ?It was very frustrating. Results very rarely were spot on and in many cases there were no results,? says Michael Crotty, vice president of marketing of Neiman Marcus Group Inc.?s Neiman Marcus Direct. ?It was terribly flawed.?

As many as 30% of visitors to the site started their trips with the search function and many exited when the searches turned up no results. ?It was making an impact on the conversion rates,? Crotty says. ?We were leaving money on the table.?

Neiman Marcus?s problem was the same problem that most retailers faced when building out their web sites: It was relying on the search function that came with the Oracle database and that just wasn?t good enough for Neiman Marcus?s purposes. ?It was looking for a perfect match between what the customer typed in and the phrases that were in the database,? Crotty says. ?And you just can?t expect consumers to know how things are worded in your database.?

Flourishing

Neiman Marcus decided last year it needed to beef up its search function. In doing so, it came to a conclusion that more retailers today are reaching. ?I talk to a lot of retailers and search just wasn?t a priority a year ago,? Crotty says. ?But now it is a much higher priority and everyone?s come to that conclusion at the same time.?

Indeed, if the vendors? reports about customer signings are any indication, search has climbed in the rankings of retailers? priorities. EasyAsk Inc. reports that it?s tripled its customer base in the last year, adding BassPro, The Bombay Co., Talbots Inc., Joann.com, Lenox Inc., LampsPlus, J. Jill Group Inc., and SmartBargains.com. Mercado Software Inc.?s customer signings in the past 12 months include U.K. retailer the John Lewis Partnership, Macys.com and Boise Office Solutions. Endeca Technologies Inc., a relative newcomer to the market, reported it landed high-profile online retailing sites Ritz Interactive Inc., Crate & Barrel, BarnesandNoble.com and HSN.com as well as specialty retailer ChristianBook.com and Canada?s Indigo Books & Music Inc. IPhrase Technologies Inc., which powers the Neiman Marcus search, also drives the search at Staples.com as well as Gateway.com. ?We?re seeing a lot of interest this year,? says Yaron Dycian, Mercado?s director of product marketing. ?Last year was a slow year in general, but we?ve seen a lot of interest in Q1 this year.?

Arcane terminology

In fact, search is rising fast enough on retailers? hit lists that the industry continues to attract new players. In addition to EasyAsk, Mercado, Endeca and iPhrase, other search vendors are touting their retail wins. Verity Inc., a major provider of enterprise search, for instance, provides search technology for Cabela?s Inc. and The Home Depot Inc. New Zealand-based S.L.I. Systems has landed Etronics.com and PCRush.com in the past six months for its ASP-based product and is making a push into the U.S. Meanwhile, Chicago-based Logika Corp. is also promoting its ASP-based model to retailers. ?There?s been a tremendous flourishing of vendors,? says Harley Manning, research director of Forrester Research Inc.?s customer experience practice. ?There are so many vendors, it?s hard to keep track of them all.?

Forrester estimates that more than 100 vendors deliver some kind of search product. And to complicate matters, Forrester notes in its Upgrade to the Right Search Engine report: ?Companies selling search solutions tout a dizzying array of capabilities based on arcane technology.? Among the terms that Forrester says confuse users, and which Forrester decodes, are Boolean, clustering, linguistic analysis, natural language processing and vector-based (see box). Forrester offers a simple way through the thicket: ?Buyers should focus on user needs and content attributes,? Forrester says. User needs address whether search customers need keyword results or more sophisticated results such as guided navigation. Content attributes include data type, metadata and breadth of subject matter.

With the proliferation of search providers, vendors are boosting the wattage of their offerings to outdo one another. Enhancements have come so quickly that mere accurate search results from a relational database, which some vendors were touting as their edge only 18 months ago, are no longer enough. The most common enhancement is to add navigation to search, itself almost old news by now. Others offer merchandising options so retailers can set rules as to which products to present high up in the search results. Many also have added analytic functions to help retailers understand and improve not only the search function itself, but also what search reveals about how customers use the site and how they react to products. And some have added yet other functionality, such as EasyAsk?s My Store feature that allows customers to store search criteria and access the most up-to-date inventory when they return. ?Having a solution that works only in relational databases is not to have a product at all,? Manning says.

Good feedback

Combined search and navigation functions present search results as well as a menu of options in case the product the customer wants does not show up in the search results. Usually, the options are presented by category across the top or down one of the sides, organized in such a way that customers can zero in on products. Getting customers to the right products quickly is key, retailers say. ?E-commerce is more about destination shopping and less about browsing,? says Linsly Donnelly, COO and co-founder of Idea Forest, which operates the Joann.com crafts site for Joann Stores Inc.

In Joann.com?s case, ease of search was extra important because its customers are older women?average age 47?who don?t have sophisticated connections or a lot of online experience. ?We have to get her to what she is interested in quickly because she doesn?t have the bandwidth or the experience on the Internet,? Donnelly says.

Joann.com installed search-and-browse technology from EasyAsk in January. So, for instance, a customer who searches on ?scissors? sees results of scissors, but also sees sorts by other criteria. For instance, she will see subcategories by use?scissors for kids, crocheting or scrapbooks. While Donnelly says Joann.com does not have enough history with the product yet to measure its effectiveness, early feedback is positive. ?We?ve gotten e-mails from customers saying how easy it is to find products on the site,? she says.

The next most common enhancement is a merchandising function, which uses business rules to permit merchants to display products they wish to promote based on the retailer?s own criteria. Federated Department Stores Inc.?s Macys.com has doubled conversion rates among search customers using the merchandising function, says Gary Beberman, director of technical research at Macys.com. ?There are strong indications that featured products drive a lot of reaction among customers,? he says. Macys.com installed Mercado?s Intuifind product just before the holidays last year. ?Previously, search results were based on quality of match and merchants had little ability to display what they thought was important or to downplay what they thought was less important,? Beberman says. ?Now we are able to allow merchandising and other criteria to influence the ranking of results.? Besides merchandising considerations, Macys.com can display products based on inventory level or best sellers, for two examples.

Pointing customers to other paths either by offering options or by presenting merchandise the retailer wants to promote is crucial, Manning says. ?The basic problem with search engines is that what they?re being asked to do is beyond them,? he says. ?So you give the best answer you can, then show customers the menu structure.?

He cautions, however, that while merchandising is important and becoming more widespread, retailers have to approach that function carefully. ?You have to make sure that results aren?t driven totally by business rules so you end up where you started?presenting merchandise that the customer isn?t interested in,? he says.

Typically, merchandising rules are applied after the search results are calculated, retailers and vendors say. So, for instance, a search on skirts may display high up the skirts of any length that the retailer wants to promote, but a search on mid-length skirts won?t show shorter or longer skirts that the retailer wants to promote.

The third dimension

Now that retailers are used to the search, browse and merchandising functions that search products offer, vendors are starting to promote analysis of the data that the search products gather. Although some vendors have incorporated analytics into their products for some time, they are only now beginning to promote it to retailers. ?It?s the third dimension of site search,? Manning says. ?It?s important to know such information as how many times did a query come up with no results.?

That information is important not just so a retailer knows where the database needs repair, or the merchandising mix altered, but also as a merchandising tool. ?If you sell clothes, but a lot of people are searching on footwear, you either need to add footwear or make it clear upfront that you don?t sell footwear. That?s the kind of information that analysis of search results can tell you,? Manning says.

Apparel retailer Blair Corp. sets a lot of store by the analytics portion of search products. ?It?s another lens through which we can look at the customer experience,? says Dave Edwards, operations manager for Blair.com.

Blair.com employs the EasyAsk search product. Vice president and general manager of e-commerce Jeff Parnell says Blair is looking into adding EasyAsk?s analytics enhancement. ?We?re excited about the functionality we?ve seen,? Parnell says. Blair has not made a commitment to the service yet, but Parnell says, ?The ability to drill down into a lot of different areas is appealing.? While analysis of search data is not new, automated analysis and presentation are new, he says. ?We?ve all got web logs,? Parnell says. ?The issue is how to get to the data when you want a quick snapshot of what?s happening. This is a more sophisticated approach to get at the data.?

Similarly, Mercado says it is working on refining its search stream analysis, although it will not release details about refinements until this month.

While the search products are becoming more sophisticated about allowing merchants to showcase products and understand how customers use the site, search technology vendors are starting to develop more uses for their technology that go beyond simply delivering product results to consumers. EasyAsk, for one, has developed a My Store product that allows customers to create search files, then save those for future use. So, for instance, a shopper might create a search file that would return results when seeking educational toys for a 5-year-old boy. The customer could save the search criteria in the My Store section of a retail site. When the customer accesses the file later, it will automatically search the retailer?s database for in-stock products that meet the criteria. None of EasyAsk?s clients has installed the technology yet, although product manager Steve Morse says a couple are interested.

In addition, search vendors are promoting use of their products for purposes other than site search. John Lewis Direct, a division of the John Lewis Partnership, a ?4 billion a year U.K.-based retailer, for instance, has installed Mercado?s Intuifind product in its call centers to help reps answer customers? questions and is using it on its web site to help customers find information about policies, such as returns and exchanges. John Lewis Direct says installation in the call center has reduced the average call time from seven minutes to two minutes. Installation on the web site has reduced the number of customer inquiry calls by about 5%, says Robert Smith, director of IT for John Lewis Direct. He says John Lewis expects to reduce those calls by as much as 30%.

No dominator, yet

And then there are the small guys. Probably nothing is so indicative of the state of the market for site search than the fact that no large vendors control it yet. S.L.I. in early February hired a California-based sales rep as part of its efforts to crack the U.S. Shaun Ryan, CEO, says what differentiates S.L.I.?s search technology is that it learns from customer searches and moves products higher in rankings based on click-throughs. ?It makes a measurable difference to the relevance of rankings,? he says.

S.L.I. offers its products on a hosted basis as does Logika. Logika sells a spider-based site search product that crawls retailers? data, then presents search results and categorization. ?Our spider is specifically designed to follow the dynamic content and then create a robust index of just that site,? says John Sortino, vice president of sales and marketing. Depending on contracted frequency, the spider re-crawls the site daily, weekly or monthly. As an indication of the competitiveness of the marketplace, both S.L.I. and Logika offer free tests to prospective customers.

As the proliferation of vendors indicates, retailers are understanding the importance of search and many are dong something about it. ?This is a very interesting market for search vendors,? says Manning. ?Retailers are looking to upgrade to the right search engine. They?re coming to the realization that if they?re online retailers, they are underselling if they?re not using the right search technology.?

kurt@verticalwebmedia.com

And that means what?

Don?t be daunted by search vendors? terminology, says Forrester Research. To help, Forrester offers the following definitions:

Boolean Retrieves documents based on the number of times keywords appear in the text. Using AND, OR and NOT makes queries more specific.

Clustering Dynamically creates groups (?clusters?) of documents based on similarity, usually based on statistical analysis of the contents and structure of document

Linguistic analysis Dissects words using grammatical rules and statistics. Finds roots of words, alternate tenses, equivalent terms and likely misspellings. Related terms: stemming, morphology, synonym-handling, spell-checking.

Natural language processing Uses grammatical rules to find and understand words in a particular category like product names. More advanced approaches classify words by parts of speech to understand their meaning. Related terms: named entity extraction, semantic analysis.

Ontology Formally describes the terms, concepts and interrelationships in a particular subject area.

Probabilistic Calculates the likelihood that the terms in a given document refer to the same concepts as the terms in the query.

Taxonomy Establishes the hierarchical relationships between concepts and terms in a particular subject area.

Vector-based Represents documents and queries as arrows on a graph with a very large number of dimensions and determines relevance based on their physical proximity in the graph.