---------------------------
LuceneServer Index Marathon
---------------------------

Overview:
Test the luceneserver by creating...
[holds pinky finger to side of mouth]
	1 million documents!
[/holds pinky finger to side of mouth]
Test performance of lucene server.

Conclusion:
The lucene server slows considerably with 1M documents 
and multiple threads.  Single thread times are 
acceptable IMHO.

How:
Create 500,000 documents. Perform tests. Create another
500,000 documents.

Document details:
Using the random advert generator from StuffOnSale.  With 
approximately 1k of advert detail text.  This is taken 
randomly from a text file of the bible (King James version
of course!).


Query Details:
After creating the documents the query returns a count of 
17000 records (with 500,000 documents)
34000 records (with 1,000,000 documents)

The query result set is limited to 5 records.



Results:
No.Queries/Avg Time/Throughput

Indexed 500000 documents (avgtime 43ms)
Index Directory Size: 197Mb in 232 files

10	6626    1.42

Optimize performed. Took 205000ms (~3.5minutes!).
Index Directory Size now 196Mb in 23 files.

13      4309    2

Not very hopeful!

Ran a Garbage Collection Process (took 686ms)
Free memory before: 43040704b (43Mb)
Free memory after: 196178504b (196Mb)

12      4598    1.83

Restarted the dev server
12      4692    1.83

Limited to only one thread the results are better:
10      514     1.82

Indexed [ANOTHER] 500000 documents (avg time 55ms)

Index Directory Size: 394Mb in 802 files.

3       2039    0.46

10      18760   0.50

Optimize performed. Took 283678ms (~4.7minutes!).
Index Directory Size now 391Mb in 23 files.

10      9350    0.99

More hopeful than before with only 500,000 records.

Free memory before: 77986616 (77Mb)
Free memory after: 188225624 (188Mb)
Time taken: 638ms

10      9218    1.05

restart the dev server
10      10091   0.94

(2nd column here is the number of threads)
5       1       5       1009    0.91
5       2       6       2043    0.92
5       3       6       3087    0.86
5       4       8       4045    0.94



