I have an index of ~500 million documents, which I'd like to "scroll"
through either sorted by a common field (across multiple types), or by
clustering parent-child documents together.
Here's how a typical set of documents is laid out:
PARENT: {"id" : "PARENT_ID1", "index" => "CommonIndex", "type" => "TypeA",
"data" => {"parent_id" : "PARENT_ID1", "name" : ... } }
CHILD1: {"id" : "CHILD_ID1A", "index" => "CommonIndex", "type" => "TypeB",
"parent" : "PARENT_ID1", "data" => {"parent_id" : "PARENT_ID1", "name" :
... } }
CHILD2: {"id" : "CHILD_ID1B", "index" => "CommonIndex", "type" => "TypeB",
"parent" : "PARENT_ID1", "data" => {"parent_id" : "PARENT_ID1", "name" :
... } }
CHILD3: {"id" : "CHILD_ID1C", "index" => "CommonIndex", "type" => "TypeC",
"parent" : "PARENT_ID1", "data" => {"parent_id" : "PARENT_ID1", "name" :
... } }
I'd like to retrieve parent and child documents clustered together, merge
them, and store the merged result in a separate index.
Which leads me to my question: is there a way to scan/scroll and retrieve
parent-child documents bundled / clustered together?
--
Alternative approach tried with partial success:
FYI, I tried using "sort" with "scroll" (without "scan" set) on the
"parent_id" field as follows: {sort => [{"parent_id" => "desc"}]}, as a
shot in the dark. While the results did appear sorted at first glance, upon
closer examination, I noticed that the results were at times out of order.
Here's an extract of just the parent_id fields printed:
7n3uxOf1dYUq
7n3uxOf1dYUq
7n3Zj2oadUKM
7n3Zj2oadUKM
7n3U2l0DkeYg
7n3U2l0DkeYg
7n3SOHKjAG2K --> out of order
7n3U2l0DkeYg
7n3U2l0DkeYg
Thanks!
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/de41eda5-0b33-44e9-9951-b72dde7d7f85%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.