I guess it would be easier if you have a quiet cluster while moving your data to a new index.
If not and if you have for example a modification date in your document fields or something which could help to identify new creation/update you could run a query based on this and scroll this query to extract all results. But for deletion it won't work.
May be using an alias on top of oldindex and newindex could help while scrolling the second time. I mean that all new index/create and delete operation should be send from a client level to newindex.
But older updates should be fetched from oldindex.
As I said, it would be much easier to do that on a quiet cluster.
If you have your source data around (database, filesystem, whatever), may be simply reindexing is the easiest option?
Do other readers have another idea?
--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr
Le 3 décembre 2013 at 21:22:08, Alex Cojocaru (acojocaru@pingidentity.com) a écrit:
Yes, David, it makes sense. I have done some testing myself with a scan search and I saw that it basically puts the existing segments in read-only mode, so all new changes will go into new ones.
But now I am more confused on how a scan and scroll will help me reindex. How do I get access to the new segments which are being created while the scan search is active?
alex
On Tue, Dec 3, 2013 at 4:12 AM, David Pilato david@pilato.fr wrote:
No a new document will belong to a new segment. So it won't be added when you scroll.
Same for delete.
Update is behind the scene a delete + new document. Same here.
Make sense?
--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr
Le 2 décembre 2013 at 23:14:08, Alex Cojocaru (acojocaru@pingidentity.com) a écrit:
That would make sense.
A new document will be at the end, so it will be included in the scan.
An updated document will be at the end too - assuming the update happened after the scan processed the document, it will be picked up again at the end and the old version of the document already indexed in the new index will be rewritten with the new version.
But a document delete which happens after the scan processed the document (and while it's still running) will not be 'replicated' into the new index - at least I don't see how it would be possible. The delete operation works by marking the document in the old index as deleted, by using an extra bit set.
alex
On Mon, Dec 2, 2013 at 4:55 PM, David Pilato david@pilato.fr wrote:
You basically scan a bunch of segments which are kept around until the scan is over.
New documents, deletion, updates won't be part of the scan because they are written in new segments.
My 2 cents
--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr
Le 2 décembre 2013 at 21:48:16, AlexC (acojocaru@pingidentity.com) a écrit:
Fair enough - I assume the reindex operation will use the scan/scroll API or, to make it easier, the reindex plugin written by karussell.
I haven't been able to figure out what happens with the document(s) updated/deleted/created after the scan process is initiated.
alex
On Friday, November 29, 2013 2:55:08 PM UTC-5, David Pilato wrote:
No. You need to reindex.
--
David 
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 29 nov. 2013 à 20:46, AlexC acoj...@pingidentity.com a écrit :
Just wondering if the API supports deleting a field from a mapping.
When a mapping is updated, the new definition is merged with the existing one, which makes me believe there is no support for deleting an existing field.
But, if I really really need this functionality, is there a better way other than creating the type under a new index and reindex all the documents?
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/dFkuWEHMdTI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.529d01d6.7fdcc233.b39a%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHUBgW_ok9ZpdrmJF3vnMkf%3DXC-U42P7qkKRQbLW9ET0C6h18g%40mail.gmail.com.
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/dFkuWEHMdTI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.529da066.238e1f29.bd3d%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHUBgW9JJ%2BX82hrc5fJwu9rRpBoz9faKLkv1V91mWo_FpqoUrw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.529ef86d.628c895d.bd3d%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.
