Need help with the flush operation and lucene files

Elastic Stack Elasticsearch
1

I insert into 200000 docs into a index(called "test_index",1 primary shard and 0 replica shard) using bulk function, each bulk request has 20000 docs. Then I flush all data into the disk( lucene file) with FlushRequest and close the index.

FlushRequest flushRequest = new FlushRequest("test_index");
 flushRequest.force(true);
 transportClient.admin().indices().flush(flushRequest).get();

Then I use IndexWriter to check the data immediately, I find the lucene directory's doc number sometimes is between 190000 and 200000 after a lot of experiments.

>     Directory directory = FSDirectory.open(Paths.get(shardDataPath));
>     IndexWriterConfig config = new IndexWriterConfig(new StandardAnalyzer());
>     IndexWriter indexWriter = new IndexWriter(directory,config);
>     int docsNumber = indexWriter.numDocs();
>     indexWriter.close();

Why?I guess it's because the flush operation have not finished before returning the flush response? Thus, I got the incomplete data? Thanks a lot.

2

I think you want to refresh not flush.

But why are you doing low level Lucene code?

4

Hi,I need to copy the lucene file to hdfs for another program. I think it is flush operation, because I wanna make sure all the data are written into the lucene file after the flush.The refresh operation just makes the doc available for search,it does not make sure the doc are written to disk(lucene file).Right?Thanks a lot.

5

Did you try the refresh I mentioned?

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.