May be you should try searching in the merged indexes to check if the search speed is fast enough?
In my situation, i created three index. the first one is used to store new data (new data), the second is a temp index to store data that are not packed to the final index (temp-old data). the third is the final index (old data).
In every midnight, i move old data from first index to the second, and merge both the indexes. After that, if the data in the second is more than a threshold, the data in this index is moved to the third, and the final index is merged to 1 segments. (In case the move operation needs more than one day, i will create another index to store temp-old data.)
Daily index is useful in case all your queries are passed to the recent days, but if your queries are passed to all the days, such setup is still slow. Assuming you have 100 day's data...
BTW: I just cannot understand why lucene stores all the data in compound indexes (what's more, the maximum size of compound index is 5GB, (So such index cannot handle big data?)), until force merge is required by the user?