Flushing small index is corrupting index file

Elastic Stack Elasticsearch
1

Well, this is odd. I'm doing some testing, so I created an index, and added
one doc to it. Later I used Luke to read it (get some analytics on the
fields and ngrams). But luke could not read, I assumed since there was only
one record it probably was not even on the disk yet, so I did a flush on
the index, and after that, luke or even ES after a restart could not read
the index anymore: read past EOF errors:

The gist of the commands: https://gist.github.com/3217914

Is this expected?

Regards

2

I have received the same errors when attempting to do the same steps.
For me, closing and re-opening makes the index load in Luke. I would
assume calling optimize/refresh would have the same benefit. But flush
does have issues.

Cheers,

Ivan

On Tue, Jul 31, 2012 at 8:41 AM, Vinicius Carvalho
viniciusccarvalho@gmail.com wrote:

Well, this is odd. I'm doing some testing, so I created an index, and added
one doc to it. Later I used Luke to read it (get some analytics on the
fields and ngrams). But luke could not read, I assumed since there was only
one record it probably was not even on the disk yet, so I did a flush on the
index, and after that, luke or even ES after a restart could not read the
index anymore: read past EOF errors:

The gist of the commands: ES corrupt index example · GitHub

Is this expected?

Regards

3

Well, I tried to re-open, refresh, optimize, nothing works. Index is gone :frowning:

When I restart ES it complains about not being able to open the index.

I managed to get my tests working, by creating the index, adding 1
document, shutting down ES, starting it again, and with that I was able to
load the results from luke.

Kinda odd, but I really don't care, as I don't expect to have such small
index :smiley:

Cheers

On Tuesday, July 31, 2012 4:54:39 PM UTC-4, Ivan Brusic wrote:

I have received the same errors when attempting to do the same steps.
For me, closing and re-opening makes the index load in Luke. I would
assume calling optimize/refresh would have the same benefit. But flush
does have issues.

Cheers,

Ivan

On Tue, Jul 31, 2012 at 8:41 AM, Vinicius Carvalho
viniciusccarvalho@gmail.com wrote:

Well, this is odd. I'm doing some testing, so I created an index, and
added
one doc to it. Later I used Luke to read it (get some analytics on the
fields and ngrams). But luke could not read, I assumed since there was
only
one record it probably was not even on the disk yet, so I did a flush on
the
index, and after that, luke or even ES after a restart could not read
the
index anymore: read past EOF errors:

The gist of the commands: ES corrupt index example · GitHub

Is this expected?

Regards

4

I should have explain myself better. My scenario is identical to
yours. Executing a flush (via BigDesk) renders the index useless. If I
want to test via Luke, I simple close and re-open the index, no need
to restart Elasticsearch. Always wanted to open an issue, but I wanted
to come up with some better test cases and I never have.

--
Ivan

On Tue, Jul 31, 2012 at 2:03 PM, Vinicius Carvalho
viniciusccarvalho@gmail.com wrote:

Well, I tried to re-open, refresh, optimize, nothing works. Index is gone :frowning:

When I restart ES it complains about not being able to open the index.

I managed to get my tests working, by creating the index, adding 1 document,
shutting down ES, starting it again, and with that I was able to load the
results from luke.

Kinda odd, but I really don't care, as I don't expect to have such small
index :smiley:

Cheers

On Tuesday, July 31, 2012 4:54:39 PM UTC-4, Ivan Brusic wrote:

I have received the same errors when attempting to do the same steps.
For me, closing and re-opening makes the index load in Luke. I would
assume calling optimize/refresh would have the same benefit. But flush
does have issues.

Cheers,

Ivan

On Tue, Jul 31, 2012 at 8:41 AM, Vinicius Carvalho
viniciusccarvalho@gmail.com wrote:

Well, this is odd. I'm doing some testing, so I created an index, and
added
one doc to it. Later I used Luke to read it (get some analytics on the
fields and ngrams). But luke could not read, I assumed since there was
only
one record it probably was not even on the disk yet, so I did a flush on
the
index, and after that, luke or even ES after a restart could not read
the
index anymore: read past EOF errors:

The gist of the commands: ES corrupt index example · GitHub

Is this expected?

Regards