Insert a single page to the index at any time to keep it current.Include/exclude tags allows choosing which portions of web pages are indexed, so template text can be ignored.A remove common option automatically removes header and footer information.Duplicate detection to index only one copy of identical documents.Crawls a variety of server types to index data served by HTTP, HTTPS, FTP, Gopher or your file server.Indexes PDFs, JavaScript and Flash/Shockwave content and links to ensure all website content is indexed.Crawls any number of web servers by combining all content needed into one or more collections.Preconfigured to your needs to be ready to go as soon as you plug it in.Large capacity with models to handle from 20,000 to tens of millions of documents in a single device, and even more with custom configurations.Fast response with the ability to handle 20+ queries per second, more than enough for peak times.Unlimited collections provide separate searches for different communities, websites or topics.The Thunderstone Search Appliance is available as a plug-and-play device or virtual machine that combines the simplicity of a hosted service with the security and performance of a local solution. Quickly add high quality search results to an internal or external site.

Applian easy duplicate finder