Msearch Map request and response

Elastic Stack Elasticsearch
1

Hi

I'm exploring multi search capability to see if it can cater to our requirement. My requirement simply is to perform single bulk search, rather than making series of individual calls. Once we get the response payload, we would want to map the response to the individual search queries. This is required for further processing.

Example illustration

curl -H 'Content-Type: application/x-ndjson' -XGET 'localhost:9200/products/_msearch?pretty=true' --data-binary @"multiSearch.q" ; echo

multiSearch.q: Content
{ "query": { "match": { "product": "Jeans" } } }
{ "query": { "match": { "product": "Men Shoes" } } }

This is the response we get :

{
  "responses" : [
	{
	  "took" : 4,
	  "timed_out" : false,
	  "_shards" : {
		"total" : 5,
		"successful" : 5,
		"skipped" : 0,
		"failed" : 0
	  },
	  "hits" : {
		"total" : 1,
		"max_score" : 11.156035,
		"hits" : [
		  {
			"_index" : "product",
			"_type" : "coverage",
			"_id" : "TWlKeGIBtL1jpnPeSMhq",
			"_score" : 11.156035,
			"_source" : {
			  "product_id" : "100",
			  "name" : "Jeans",
			  "available" : "Y"
			}
		  }
		]
	  },
	  "status" : 200
	},
	{
	  "took" : 2,
	  "timed_out" : false,
	  "_shards" : {
		"total" : 5,
		"successful" : 5,
		"skipped" : 0,
		"failed" : 0
	  },
	  "hits" : {
		"total" : 1,
		"max_score" : 12.604036,
		"hits" : [
		  {
			"_index" : "product",
			"_type" : "coverage",
			"_id" : "8WZJeGIBtL1jpnPe8lR2",
			"_score" : 12.604036,
			"_source" : {
			  "product_id" : "200",
			  "name" : "Men Shoes",
			  "available" : "N"
			}
		  }
		]
	  },
	  "status" : 200
	}
  ]
}

My question is how to we identify the response and match it to the request.

Option 1: Based on ordering

One way to look at this is based on the ordering of the request-search and response. Is it guaranteed that the response will always be in the same order as that of the request. Meaning in the example above since the request had search for "Jeans" above "Men Shoes", can we assume that the response output will always be "Search-output-for-Jeans" before "Search-output-for-Men Shoes".

Option 2: Tagging each search request

Is it possible to search each search request, such that the tag is passed over in the response. This would help to map the search request to the responses.

Something like this : Note the string "query-name" in the request and response

{ "query-name" : "Search1", "query": { "match": { "product": "Jeans" } } }
{ "query-name" : "Search2", "query": { "match": { "product": "Men Shoes" } } }

And the response carries the search name which would help us to map -

{
  "responses" : [
	{
	  "query-name" : "Search1",
	  "took" : 4,
	  "timed_out" : false,
	  "_shards" : {
		"total" : 5,
		"successful" : 5,
		"skipped" : 0,
		"failed" : 0
	  },
	  "hits" : {
		"total" : 1,
		"max_score" : 11.156035,
		"hits" : [
		  {
			"_index" : "product",
			"_type" : "coverage",
			"_id" : "TWlKeGIBtL1jpnPeSMhq",
			"_score" : 11.156035,
			"_source" : {
			  "product_id" : "100",
			  "name" : "Jeans",
			  "available" : "Y"
			}
		  }
		]
	  },
	  "status" : 200
	},
	{
	  "query-name" : "Search2",		
	  "took" : 2,
	  "timed_out" : false,
	  "_shards" : {
		"total" : 5,
		"successful" : 5,
		"skipped" : 0,
		"failed" : 0
	  },
	  "hits" : {
		"total" : 1,
		"max_score" : 12.604036,
		"hits" : [
		  {
			"_index" : "product",
			"_type" : "coverage",
			"_id" : "8WZJeGIBtL1jpnPe8lR2",
			"_score" : 12.604036,
			"_source" : {
			  "product_id" : "200",
			  "name" : "Men Shoes",
			  "available" : "N"
			}
		  }
		]
	  },
	  "status" : 200
	}
  ]
}

Apart from these, are there any other ideas ?

Thanks

2

Appreciate any inputs on this.

3

Once last try before I give up.

What I'm asking for is similar to what is possible with percolate where name is provided for the search.

GET /my-index/_search
{
    "query" : {
        "bool" : {
            "should" : [
                {
                    "percolate" : {
                        "field" : "query",
                        "document" : {
                            "message" : "bonsai tree"
                        },
                        "name": "query1" 
                    }
                },
                {
                    "percolate" : {
                        "field" : "query",
                        "document" : {
                            "message" : "tulip flower"
                        },
                        "name": "query2" 
                    }
                }
            ]
        }
    }
}

Documented here : https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-percolate-query.html#_specifying_multiple_percolate_queries

4

The documentation for the multi search API specifies that: "The response returns a responses array, which includes the search response and status code for each search request matching its order in the original multi search request."

So yes, going back to your original question: the ordering in the response is guaranteed to be the same as in the request.

As far as I know there is no way to name the individual queries in the request.

1 Like
5

Thanks Abdon.

6

I just figured out that it is actually possible to name queries, if you would still wish to do so, using named queries. Each filter and query can accept a _name that is returned with the search results as matched_queries. For example, your _msearch would look like this:

GET /_msearch
{}
{"query":{"match":{"product":{"_name":"Query for jeans","query":"Jeans"}}}}
{}
{"query":{"match":{"product":{"_name":"Query for shoes","query":"Men Shoes"}}}}
7

Oh this is really great. Thanks Abdon.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.