Compare solr generated index with elastic

[abhishekash]

Hello,

We had solr as our search back end and have been moving towards elastic search .

Now, while generating index on a dataset with

1. elastic 5.2.2 generated index of size 17 GB
2. solr on lucene 4.7.x generated index of size 57 GB

There is no obvious error in the logs.

But, i am worried on the index size. If i have ingested everything. Searching everything is not possible.

So, i wanted to compare if the same number of document are there or the number of ids are same or not to be sure.

I tried luke but seems like the index structure is different in solr vs elastic and does not seem obvious on how to use it.

Please help

[dadoonet (David Pilato)]

elasticsearch 5 does not use Lucene 4 behind the scene. So that might explain some few things, no?

But at the end of the day, I don't understand if you are able to make elasticsearch works or not?

[abhishekash]

Elastic 5.2.2 is using its default lucene version . I am not trying to run an older version of lucene with elastic.

Here i am trying to find out if i missing some data in my indexes considering the index size is drastically less in elastic

Are you suggesting that since it is much more recent version of lucene in elastic , we cannot compare the index sizes ?

[dadoonet (David Pilato)]

@abhishekash

Are you suggesting that since it is much more recent version of lucene in elastic , we cannot compare the index sizes ?

Yes. Exactly. It's like comparing oranges with apples IMHO.

Here i am trying to find out if i missing some data in my indexes considering the index size is drastically less in elastic

I'd first just run a GET _search?size=0 and count the number of documents.
If they are all there, I think it's already good.