Benchmark Study of Desktop Search Tools

Description

April 20, 2005
Benchmark Study of
Desktop Search Tools
There’s More to Search than Google & Yahoo!
An Evaluation of 12 Leading Desktop Search Tools
Tom Noda
Shawn Helwig
www.uwebc.org/decisiontoolsDecision Tools | Desktop Search 2
Ex Ecu Tiv E Summary
A new generation of desktop search tools is emerging that allows users to quickly find relevant documents in computers
across the enterprise the same way search engines help locate information on the Internet. Companies expect that this
technology will boost employee productivity and creativity and allow them to compete successfully in today’s knowledge-
driven economy.
Desktop search technology itself is nothing new. In fact, it has been around for years. However, some well know names
(i.e. Google and Yahoo!) have recently entered the space giving this technology a well-deserved boost in visibility. In an
effort to help understand the differences between the latest desktop search tools on the market, the UW E-Business Con-
sortium recently conducted a benchmark study of 12 popular desktop search tools. The benchmark criteria that were used
for the evaluation included usability, versatility, accuracy, efficiency, security, and enterprise readiness.
When all the results were reviewed, it was determined that most of the desktop search tools were still too immature for
significant business use due primarily to a lack of mature security and overall manageability. However, considering the
evolution of Instant Messaging ...

Sujets

• Moteur de recherche de bureau
• Consortium for Small Scale Modeling
• Enterprise JavaBeans
• Bétail
• Application linéaire
• Google Moon

Informations

• Publié par: Nozur
• Nombre de lectures: 209
• Langue: English
• Poids de l'ouvrage: 2 Mo

Extrait

April 20, 2005 Benchmark Study of Desktop Search Tools There’s More to Search than Google & Yahoo! An Evaluation of 12 Leading Desktop Search Tools Tom Noda Shawn Helwig www.uwebc.org/decisiontools

Decision Tools | Desktop Search2 ExEcuTivE Summary A new generation of desktop search tools is emerging that allows users to quickly ind relevant documents in computers across the enterprise the same way search engines help locate information on the Internet. Companies expect that this technology will boost employee productivity and creativity and allow them to compete successfully in today’s knowledge- driven economy. Desktop search technology itself is nothing new. In fact, it has been around for years. However, some well know names (i.e. Google and Yahoo!) have recently entered the space giving this technology a well-deserved boost in visibility. In an effort to help understand the differences between the latest desktop search tools on the market, the UW E-Business Con- sortium recently conducted a benchmark study of 12 popular desktop search tools. The benchmark criteria that were used for the evaluation included usability, versatility, accuracy, eficiency, security, and enterprise readiness. When all the results were reviewed, it was determined that most of the desktop search tools were still too immature for signiicant business use due primarily to a lack of mature security and overall manageability. However, considering the evolution of Instant Messaging from a pure consumer tool to a valuable enterprise application, desktop search may have similar potential. KEy FinDingS TOP 3 DESKTOP SEarchBased on our evaluation, the best overall desktop search tool is Copernic 1.5 Usability Beta with Coveo. R E e nt a e d r i p n r e is s e s Versatility Yahoo! Desktop Search 1.1 Beta was rated the second best tool in our evalu- ation. See other notes. Likasoft Archivarius 3000 came in a surprisingly close third in our evaluation. SecurityAccuracy This software is available commercially from Likasoft. Archivarius’ index efi- ciency is outstanding and was the clear winner in our tests. The user inter- Eficiency face and navigation scheme is well designed and easy to use. 1. copernic 1.5 Beta with coveo 2. yahoo! Desktop Search 1.1 Beta 3. likasoft archivarius 3000 Source: UW E-Business Consortium TaBlE OF cOnTEnTS 2Executive Summary 3Overall ratings 3Benchmark criteria 4criteria ratings 4Product reviews 11appendix a 12appendix B www.uwebc.org BEnchmarK nOTES The benchmark evaluation testing was performed in March, 2005. This research was not funded or supported by any speciic companies or institutions. The benchmark evaluations were conducted solely by the UW E-Business Consortium. © 2005. UW E-Business Consortium

Decision Tools | Desktop Search3 OvErall raTingS These are the overall benchmark evaluation ratings. Some tools are very good in speciic areas such as usability, versatil- ity or search accuracy (explained later), but to be the best desktop search tool, a balance of all criteria is critical.  Desktop Search ToolVersion Score (Min = 1.00, Max = 5.00)Better copernic Desktop Search 1.5 Beta4.11 yahoo! Desktop Search 1.1 Beta3.66 likasoft archivarius 3000 3.143.62 mSn Toolbar Suite 2.0 Beta3.45 google Desktop 1.03.26 ask Jeeves 1.0 Beta3.16 Enish Professional 6.13.10 iSyS Desktop 6.03.05 dtSearch Desktop 6.53.02 diskmETa Pro 1.0.1 Blinkx 3.0 hotBot Desktop Beta 36.2 36.2 43.2 Source: UW E-Business Consortium BEnchmarK criTEria Our benchmark evaluation was performed across six main criteria. Each criterion was quantiied and was given a rating, ranging from 1 (worst) to 5 (best). The rating is based on sub criteria, which align with the main criterion’s objective. For sub criteria and their rating details, please refer to Appendix A - Comparison Table. 1. usability2. versatility Good desktop search tools must be easy to use, have Versatility describes how wide and deep the tool al- a lower learning curve, have professional aesthetics, lows you to search. This includes factors such as sup- and require fewer steps to reach desired output.ported document types, web/e-mail integration, and  multi-language support. w 3. accuracy “Can you ind what you are looking for?” This criterion addresses accuracy of search results as well as other factors that help users ind the desired information. 5. Security Security and privacy are big concerns, especially in an enterprise environment. This criterion considers how well vendors have incorporated security mechanisms. wwu.ewcbo.gr 4. Eficiency This criterion assesses the tool’s technical eficiency including memory usage, indexing time or indexed ile sizes. The best tool should not jeopardize overall PC performance. 6. Enterprise readiness While most tools are designed for the consumer/home PC environment, some are ready to be used in an enterprise. This criterion may be especially helpful for IT managers.  ©0250 .WUE B-sunise soCsnroitmu

Decision Tools | Desktop Search4 criTEria raTingS The following charts summarize the best tools’ ratings for each criterion. Blinkx and ISYS are versatile tools but struggle to deliver their powerful features in a user-friendly fashion. On the other hand, Ask Jeeves excels in usability, eficiency and security, but lacks versatility. Copernic is excellent in almost all criteria.  1. Usability 2. Versatility Copernic4.80Copernic Archivarius4.75Yahoo! Google4.40Blinkx MSN4.40ISYS Ask Jeeves4.25  3. Accuracy Copernic NSM dtSearch  5. Security Yahoo! Ask Jeeves Google 05.3 92.3 1.34 31.3 05.4 02.4  4. Eficiency Archivarius Copernic Ask Jeeves 41.4 88.3 57.3 57.3 04.4 02.4 08.3  6. Enterprise Readiness Copernic 4.00 ISYS 4.00 Yahoo! 4.00 * Copernic with Coveo, and Yahoo! with X1 Source: UW E-Business Consortium PrODucT rEviEwS This section examines the details for each desktop search tool individually. The benchmark performance for each tool is expressed with a Spider Chart (see description), in order to convey the performance in each one of six criteria as well as the overall balance. Usability Spider charts 05.1 Spider Charts have been used to show how each criterion was scored, as well R E e nt a e d r i p n r e is s e s Ve 5 r . s 0 at 0 ility as the overall balance. For instance, the chart at left indicates that this tool is 3.00 extremely good at Versatility but needs some improvement in Usability and Ef- iciency. SecurityAccuracy 3.503.00 Achieving the maximum scores in all criteria and maintaining a good hexagon  shape are ideal, but that is not required by all users. For instance, if Enterprise Eficiency Readiness is not critical for a speciic user, an unbalanced shape that lacks 2.00 Enterprise Readiness features may still be a solid it. ww.wwube.crog ©0250 .WUE B-sunise soCsnroitmu

Decision Tools | Desktop Search5 copernic Desktop Search 1.5 Beta www.copernic.com Usability Copernic is the most well-balanced desktop search tool among those evaluated. 08.4 The tool is intuitive and easy to use. The new beta version supports FireFox for EnterpriseVersatility Readiness 4.14Web history search, and Thunderbird and Eudora for e-mail (as well as IE, Out- 00.4 look and Outlook Express) “Search as you type” and “dynamic indexing,” which 4.11 detects new and modiied iles/e-mails on the ly, are useful features. Filtering, SecurityAccuracy sorting and grouping search results are well reined. The application has a small 3.004.50 technology footprint and pro- Eficiency vides detailed index controls. 4.20 One potential improvement, Source: UW E-Business Consortium however, is that it should let users choose a default web search engine. It only supports the “all- theweb.com” web search within the application, which is not as popular as Google or Yahoo! For business use, Coveo, a spin-off company from Copernic, provides enterprise desktop search products, which enhance security, manage- ability and network capability. The client applications are identical. However, the enterprise version works with additional server products Copernic Desktop Search 1.5 Beta such as Microsoft SharePoint. yahoo! Desktop Search 1.1 Beta desktop.yahoo.com Usability Yahoo! Desktop Search is based on X1 Desktop Search, so usability will be 00.4 familiar to existing X1 users. Yahoo! integrates X1’s technology into its own portal EnterpriseVersatility Readiness 3.88services such as Yahoo! E-mail and Instant Messaging. It can index Yahoo! IM  00.4 logs as well as Yahoo! Address Book. Versatility is excellent, as Yahoo! claims 3.66 their tool supports more than 200 types of documents. It indexes Adobe Pho- SecurityAccuracy toshop and Illustrator 3.293.20 iles in addition to many Eficiency media iles. Contents of 06.3 Source: UW E-Business Consortium zip iles are examined and displayed in a tree structure. The tool’s preview feature is well reined, but its search results are somewhat clumsy because too many columns are displayed in a vertical view. As opposed to Copernic, there is no dynamic indexing or web history search. X1 offers an enterprise version of the desktop search tool as a server-based product. IT managers may want to check it out. Yahoo! Desktop Search 1.1 Beta www.uwebc.org© 2005. UW E-Business Consortium

Decision Tools | Desktop Search6 likasoft archivarius 3000 3.14 www.likasoft.com Usability In contrast to Copernic or Yahoo!, this is a commercial product, which costs 57.4 EnterpriseVersatility from $25 to $45, depending on your status and purpose of use. Usability and Readiness 3.38eficiency are astonishing. It is a very simple GUI design, yet organizes a lot of  00.3 features compactly. It demonstrated the fastest initial indexing time among all 3.62 products, and memory usage was relatively low in idle time. It does not support SecurityAccuracy any media ile index- 3.003.20 ing (image, audio or Eficiency video) or web/web 4.40 history searches.  On  Source: UW E-Business Consortium the other hand, e-mail support is wide, ranging from Outlook and Outlook Express to Eudora, Thunderbird and Lotus Notes/Domino. One unique feature is that it offers remote search functionality. The ap- plication acts as a small Web server, allowing remote users to search the computer through a web browser. Of course, it has user/group account management capability built-in. Likasoft Archivarius 3000 3.14 mSn Toolbar Suite 2.0 Beta toolbar.msn.com Usability MSN Toolbar Suite is similar to Google Desktop, and has almost the identical 04.4 EnterpriseVersatility functionality and navigation scheme. Moreover, it includes a Popup Blocker and Readiness 3.63“Auto Form Fill” features which are already available with the Google Toolbar.  00.2 MSN performed well in terms of search accuracy. Word accuracy was very good 3.45 in our test. “Shortcut keyword” is a unique feature, which lets users associate SecurityAccuracy a keyword with speciic 2.864.20 iles. With this association, Eficiency users can type a keyword 3.60 in Windows Explorer’s  Source: UW E-Business Consortium address bar to fetch a ile, instead of crawling multiple folders. One challenge is that it does not support PDF iles by default. To index PDF contents, users must download and install an add-in tool called “IFilter.” ww.wwube.crog MSN Toolbar Suite 2.0 Beta  ©0250 .WUE B-sunise soCsnroitmu

Decision Tools | Desktop Search7 google Desktop 1.0 desktop.google.com Usability Google seamlessly integrates desktop search into its popular web search en- 04.4 EnterpriseVersatility gine. The browser-based desktop search tool is easy to use and will be familiar Readiness 3.50to anyone who has used Google. A loating bar is a unique feature and allows  00.2 users to type keywords from anywhere on the screen. Google Desktop API is 3.26 another promising feature, allowing software developers to develop add-ins to SecurityAccuracy enhance the tool’s func- 3.133.20 tionality. OpenOfice and Eficiency ICQ index add-ins are 3.33 already available. Unfortu- Source: UW E-Business Consortium nately, iltering and sorting functions are quite limited. It appears as if Google is so focused on its relevance algorithm that other sorting functions seem to be ignored. Google Desktop 1.0 ask Jeeves Desktop Search 1.7 Beta sp.ask.com/docs/desktop Usability Ask Jeeves’ usability is remarkably simple and well reined. It searches all types 52.4 EnterpriseVersatility of documents simultaneously, and users can look through each type of results Readiness 2.57via tabbed panes. It has a nice preview pane, which even plays Windows Media 00.2 Player. The application is very small and eficient. Users are given index control 3.16 when they can choose either fast or gradual indexing. However, Ask Jeeves’ big- SecurityAccuracy gest challenge is to 3.143.20 improve versatility.  Eficiency Currently, the sup- 3.80 ported document Source: UW E-Business Consortium types are very lim- ited. Also, the preview pane does not render Microsoft Excel or PowerPoint. Web history search is not supported, either. If it would support more ile types, Ask Jeeves has the potential to become one of the top desktop search tools. ww.wwube.crog Ask Jeeves Desktop Search 1.7 Beta  ©0250 .WUE B-sunise soCsnroitmu

Decision Tools | Desktop Search8 Enish Professional 6.1 www.enish.com Usability Enish is a commercial software product. We tested the Professional version, 04.3 EnterpriseVersatility which costs $199.95. Enish is a lot different from the other 11 tools in terms of Readiness 3.38usability. It gives users full customization of views. It provides calendar, contacts  00.3 and weather views in addition to search/preview views. It almost simulates a 3.10 Microsoft Outlook SecurityAccuracy environment. Users 3.002.40 can create multiple  Eficiency index iles and associ- 04.3 Source: UW E-Business Consortium ate them with differ- ent views. However, this customization creates a steep learning curve. It takes some time to get used to its operations. Enish has “Relevant Search” features, but its purpose appeared unclear. When we searched “Open Ofice,” for example, the relevant search high- lighted “[e-mail],” which made us wonder why. Enish Professional 6.1 iSyS Desktop 6.0 www.isys-search.com Usability ISYS is a versatile tool. It supports multi-language indexing, FTP indexing, SQL 57.1 EnterpriseVersatility indexing (requiring XML output), and supports many e-mail clients including Readiness 3.75Compuserve, Eudora and VIM. It is also good at enhancing search accuracy.  00.4 It includes a spell checker, synonym rings, fuzzy logic search (which correct 3.05 misspells in indexed documents) and intelligent date/number format (e.g. ind SecurityAccuracy “1/1/05” from “Jan. 1, 2005”). In contrast to those valuable features, however, us- 3.002.80 ability is signiicantly poor. The application creates multiple Windows menus and Eficiency confuses users. Custom query 3.00 syntaxes create another steep Source: UW E-Business Consortium learning curve for users who do not want to memorize them. Toolbar icons don’t have text descrip- tions, yet their symbols are somewhat vague. When we searched MP3 iles, we were stuck because it did not provide any links or enable us to play them inside the application. ISYS must improve its usability to be able to capitalize on its powerful versatility and accuracy features. ww.wwube.crog ISYS Desktop 6.0  ©0250 .WUE B-sunise soCsnroitmu

Decision Tools | Desktop Search9 dtSearch Desktop 6.5 www.dtsearch.com Usability dtSearch Desktop is outstanding in terms of word accuracy features. It provides 05.2 EnterpriseVersatility phonic and fuzzy search, boolean and wildcard keywords, multi language and Readiness 3.25encoding support, noise word list, which excludes common words such as “is” or  00.3 “a,” and case/accent sensitive indexing. The most regrettable aspect, however, 3.02 is that it does not deliver SecurityAccuracy those features very well 2.883.50 to the end user because Eficiency of poor usability. There is 00.3 Source: UW E-Business Consortium no search keyword ield on the main window, and users have to go through the menu. Setting up the above features one by one is also a tedious task. The indexer treats most of the binary iles as text and messes up the index ile with none char- acters. If it enhanced the GUI and reined the usability, it could become a very interesting desktop search tool. dtSearch Desktop 6.5 diskmETa Pro 1.0.1 www.diskmeta.com Usability diskMETA is also a commercial product. We tested the most advanced version, 05.2 EnterpriseVersatility “Professional,” which costs $97.50. The application is one of the simplest, but Readiness 2.43comes with very limited functionality. diskMETA does not support any web history  00.2 or e-mail search. Filtering and sorting are also limited, and there is no preview 2.63 pane. However, it does have remarkable word accuracy features. It includes a SecurityAccuracy dictionary feature that can identify a word, like “criterion” from a keyword “crite- 2.862.60 ria.” Surprisingly, most Eficiency desktop search tools 3.40 cannot do this. Iterating Source: UW E-Business Consortium search results is another nice feature. Most desktop search tools we tested show all search results regardless of its amount (e.g. 2,000 matches), which can often overwhelm users. diskMETA’s page iteration is intuitive and easy to use. ww.wwube.crog diskMETA Pro 1.0.1  ©0250 .WUE B-sunise soCsnroitmu

Decision Tools | Desktop Search10 Blinkx 3.0 www.blinkx.com Usability Blinkx 3.0 has a Metal theme, which makes it look somewhat similar to the Ma- 00.3 EnterpriseVersatility cintosh user interface. The tool’s versatility is somewhat limited, but it does have Readiness 3.75some unique features. “Blinkx Visualizer” produces a tree view of search results,  00.2 and users can see the 2.63 tree growing. “Smart- SecurityAccuracy folder” crawls the web 2.632.60 to ind relevant informa- Eficiency tion to the documents 08.1 Source: UW E-Business Consortium in the folder. Unfortu- nately, the application has major problems with eficiency. Indexing is painfully slow. In fact, in our test, the application couldn’t complete the process. It runs four instances and consumes signiicant memory. In con- sequence, search outputs are slow, and a window often lickers. Hopefully, this will be improved in the future release. Blinkx 3.0 hotBot Desktop Beta www.hotbot.com/tools/desktop Usability HotBot is a toolbar-based desktop search tool and displays output in the brows- 00.2 EnterpriseVersatility er’s left pane, where Favorite and History links are often displayed. The tool Readiness 3.00is very compact and has some unique features. It supports RSS indexing and  00.2 allows users to associate keywords to custom web sites (e.g. “eb <keyword>” 2.34 for eBay search). Unfortunately, we couldn’t complete indexing on this tool for SecurityAccuracy unknown reasons. The soft- 2.862.20 ware seems to have been Eficiency rushed for the beta release. 2.00 HotBot offers a deskbar  Source: UW E-Business Consortium version, but usability is very inconsistent from the toolbar. Coniguration is text ile based, and this is simply not user-friendly. It provides a lot of custom search syntax- es but has a steep learning curve. Search results are automatically saved as HTML iles, and this may cause some security concerns. We expect signiicant improvements in its inal release. ww.wwube.crog HotBot Desktop Beta  ©0250 .WUE B-sunise soCsnroitmu