This article has been archived. It is offered "as is" and will no longer be updated.
The following message is seen in the searchctrl-15600 logs when the search processes is attempting the load a new index, but it continuously fails after about 60 seconds:
[2007-03-12 02:06:38] INFO : searchctrl-search: Search Node has problems with inactive search processes. Stopping it temporarily.
Issuing a resetindex seems to allow the index to build properly, but search is still unable to start.
By default, fsearch is allowed 60 seconds to start up. If it fails to complete the startup process within that time, the search controller will stop the process, and the cycle will start again. With debug logging enabled (by adding debuglog="true" to $FASTSEARCH/etc/searchrc-1.xml), you see the timeout message explicitly:
[2007-03-12 02:09:35] DEBUG : searchctrl-search: RTFSearch: Failed to wait-for-process. [2007-03-12 02:09:35] DEBUG : searchctrl-search: fsearch (3_0) Process not reporting ready within 60 seconds. [2007-03-12 02:09:35] DEBUG : searchctrl-search: pw(009EDC10): Terminating. [2007-03-12 02:09:36] DEBUG : searchctrl-search: pw(009EDC10): Process terminated with retcode 65535, command is n:\DATASE~1\bin\fsearch -C n:\DATASE~1\var\searchctrl -c fsearchrc-6-15605 -L file:n:\DATASE~1\var\log\searchctrl\fsearch-6-15600.log.
Typically there are two reasons why this may occur:
A complex index-profile with many features or a large number of fields can cause the fsearch process to have to load too many files before the 60 second timeout. Reducing the complexity or number of fields is recommended.
Needed system resources (particularly diskIO throughput) are taken up by other processes (such as active indexing, crawling, or third-party applications) and are unavailable for the search processes. Separating the search processes from other diskIO inextensive processes is the best solution. Alternatively, it may be possible to schedule other processes to run during hours when search is not being used as much.
If neither of the solutions above are possible, extend the timeout period by modifying the file $FASTSEARCH/etc/searchrc-1.xml on each search node. Add the following parameter and increase the value (in seconds) to something larger than 60.
searchtimeout = "300"
After editing the file, restart the search-1 process (nctrl restart search-1) on each search node.
Monitor the debug messages to confirm that the parameter has changed if the behavior continues.
These example logs are from FDS 4.x. The ESP 5.x logs may look different, but the functionality of the parameter should be the same.