Sphider is described as a lightweight search engine written in php with a mysql database back end. Ideal as a site search script or as a custom search engine, sphider is easy to install and easy to set up. It includes a full featured administration back end where the admin can set search rules and depth, allow sphider to leave the server to crawl the internet etc.
But sphider has been adapated and there are now several versions. Sphider Pro and Sphider Plus. The latest version of Sphider plus claims to have added 370 new features, but if this makes it more of a heavyweight search engine, I'm unsure. It also sells for 25 Euro's, and for anything other than a personal search engine it might be worth the price.
To start with though, I would reccommend users test out the open source versions to see if they are adequate or not, before spending money on something that may be more than required.
In all versions, admin can add and edit categories or create sub-categories, add sites and index them with the ability to browse pages and view stats, set the indexing depth, clean tables i.e. clean keywords that go nowhere.
Settings include: language, admin email, templates, set temp directory and spider output, log spider results in html or text format, set spider rules such as keywords, words per page, types of files to index, and adjust various search settings.
Statistics show top keywords, largest pages and most popular searches. The database can be optimised and backed up from within sphider admin.
I've included three downloads here, Sphider, Sphider plus and Sphider pro, but you might also like to check out the paid version at: http://www.sphider-plus.eu/
Sphider demo and download (from source): http://www.sphider.eu
Sphider-pro demo from Sourceforge: https://sourceforge.net/projects/sphiderpro/
I also include the downloads here except the paid version, for posterity.