Understanding search response

Elastic Stack Elasticsearch
1

I am writing perf tool to load test elasticsearch. I have both index and
search working properly. I noticed that when I query for a term I get the
response in pretty decent time considering I just have 1 shard with 200M
docs in it. However, when I set size to 1000 it takes around 10 secs. In
both cases when I use "size" or not I still get same number of hits.total
value, which means it is searching through entire shard. Does it mean that
the extra time, when size is set to 1000, is coming from the fact that it
has to fetch the source of the document? In other words time to do just
"search" is always the same for that specific term in both cases?

{

  • took: 2671
  • timed_out: false
  • _shards: {
    • total: 3
    • successful: 3
    • failed: 0
      }
  • hits: {
    • total: 3047482
    • max_score: 2.6712089
    • hits: [
      • {
        • _index: test
        • _type: test
        • _id: 81CGcKoHRdCozyo1MRObGg
        • _score: 2.6712089
        • _source: {
          • title: Wikipedia: Lynedoch railway station
            }
            }
      • {
        • _index: test

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Hi,
queries are always executed against the relevant shards. Every shard
returns "size" documents, thus a reduce is needed to resort and return only
the top "size" documents. With a single shard the reduce is not needed
though, which means that the additional time is spent on loading the lucene
fields that need to be returned from disk (_source by default). From your
search response I see 3 shards though, not just one.

On Wednesday, November 20, 2013 1:46:54 AM UTC+1, Mo wrote:

I am writing perf tool to load test elasticsearch. I have both index and
search working properly. I noticed that when I query for a term I get the
response in pretty decent time considering I just have 1 shard with 200M
docs in it. However, when I set size to 1000 it takes around 10 secs. In
both cases when I use "size" or not I still get same number of hits.total
value, which means it is searching through entire shard. Does it mean that
the extra time, when size is set to 1000, is coming from the fact that it
has to fetch the source of the document? In other words time to do just
"search" is always the same for that specific term in both cases?

{

  • took: 2671
  • timed_out: false
  • _shards: {
    • total: 3
    • successful: 3
    • failed: 0
      }
  • hits: {
    • total: 3047482
    • max_score: 2.6712089
    • hits: [
      • {
        • _index: test
        • _type: test
        • _id: 81CGcKoHRdCozyo1MRObGg
        • _score: 2.6712089
        • _source: {
          • title: Wikipedia: Lynedoch railway station
            }
            }
      • {
        • _index: test

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.