How to pass raw query to elastic search and handle the raw JSON Response

Elastic Stack Elasticsearch
1

NEST version: NEST.2.1.1
Elastic Search Version : Elasticsearch 2.3.1

I have a query like below.

{
        "size" : 10,    
        "query" : {
            "match" : {
                "bodyContent" : "singapore" 
            }
        }
 }

I converted it into a string.

string query = @{
        ""size"" : 10,    
        ""query"" : {
            ""match"" : {
                ""bodyContent"" : ""singapore"" 
            }
        }
    };

then im sending this as a parameter.

 var result = client.Search<Class1>(s => s
                         .From(0)
                         .Size(10)
                         .QueryRaw(query)
                     );

But the compiler says it Class1 doesnt contain QueryRaw.

Class1.

 public class Class1
    {
        [JsonProperty("key")]
        public string key { get; set; }

        [JsonProperty("doc_count")]
        public int doc_count { get; set; }
    }

Can you please help me in solving this.

Russcam reply:

There is no QueryRaw in NEST 2.x (as the compiler indicates :smile: ). You can use .Raw() on QueryContainerDescriptor

client.Search<Class1>(s => s
    .Size(10)
    .Query(q => q
        .Raw(@"{ 
            ""match"" : {
                ""bodyContent"" : ""singapore"" 
            } 
        }")
    )
);

Note that this is only the query element of the query, so .From() and .Size() etc need to be set outside.

i tried to modify my code after the suggestion given by Russ. But still, im getting the same error.
Modified code:

var result = client.Search<Class1>(s => s
        .From(0)
        .Size(100)
        .Query(q => q
            .Raw(query)
        )
    );

output: INVALID NEST RESPONSE BUILT FROM AN UNSUCCESSFUL LOW LEVEL CALL ON POST.

But if the call is made without any query(similar to below code), the output says VALID NEST RESPONSE BUILT FROM A SUCCESSFUL LOW LEVEL CALL ON POST

var res = client.Search<Class1>();

2

A couple of observations:

  1. You don't have a BodyContent property on Class1 type that correlates to the field bodyContent on which you are trying to run the match query. Should that property exist on the type and is simply missed? Does that field exist in the mapping for Class1 in the index you are targeting?
  2. You can get more details around what the problem is by looking at .DebugInformation on the result instance. For the most amount of information that NEST can provide, also set .DisableDirectStreaming() on the ConnectionSettings passed to the IElasticClient constructor, for example
// use the right kind of connection pool for your environment
var connectionPool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));

var settings = new ConnectionSettings(connectionPool)
    .DisableDirectStreaming();

var client = new ElasticClient(settings);
3

Hi Russ,

Thanks for the reply.

That class1 is just for explanation purpose. but i have a body content in my real class.

what im confused about is,
for example:
localhost:9200/my_tweet/tweets/_search without query gave me one result
client.search<tweets>();
where as
localhost:9200/my_tweet/tweets/_search with a query gave me another result
client.search<tweets>(q=>q.Query(s=>s.raw(query)))

so how can i write a class which can handle both the responses?
I hope u understood my question.

4

I'm not sure I understand your question.

Two different queries can obviously give you two different results but the type of the response for both .Search<T>() calls when using NEST will be ISearchResponse<T>.

5

this is my class...

 public class kb_post
    {
       
        private static PropertyChangingEventArgs emptyChangingEventArgs = new PropertyChangingEventArgs(String.Empty);

        [JsonProperty("id")]
        private int _Id { get; set; }

        [JsonProperty("threadId")]
        private System.Nullable<int> _ThreadId { get; set; }

        [JsonProperty("title")]
        private string _Title { get; set; }

       [JsonProperty("bodyContent")]
       private string _BodyContent { get; set; }
}

this is my query:

 {
    "size" : 0,
    "aggs" : {
        "words" : {
            "terms" : { 
                "field" : "bodyContent" ,
                "size" : 80,
				"exclude" : ["is","an"]
            }
        }
    }
}

i changed it to string.
string query = @"{
""size"" : 0,
""aggs"" : {
""words"" : {
""terms"" : {
""field"" : ""bodyContent"" ,
""size"" : 80,
""exclude"" : [""is"",""an"" ]
}
}
}
}";

so when i just call the service without any query (http://192.168.7.200:9200/kbpost/kb_post/_search) the output is as below.

client.Search<kb_post>();

{
  "took": 4,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "hits": {
    "total": 37077,
    "max_score": 1,
    "hits": [
      {
        "_index": "kbpost",
        "_type": "kb_post",
        "_id": "72",
        "_score": 1,
        "_source": {
          "id": 72,
          "threadId": 9,
          "title": "lemurs!",
          "bodyContent": "Look at this"
        }
      }

But when i make the call using the query (http://192.168.7.200:9200/kbpost/kb_post/_search) below is the output.

 var result = client.Search<kb_post>(s => s
                         .From(0)
                         .Size(100)
                         .Query(q => q
                         .Raw(query)
                         )
                       );

Im getting an error when i call the above code. Its saying invalid NEST call on low level post method.

Expected output:

{
  "took": 94,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "hits": {
    "total": 37077,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "words": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 544072,
      "buckets": [
        {
          "key": "singapore",
          "doc_count": 4498
        },
        {
          "key": "more",
          "doc_count": 3429
        },
        {
          "key": "need",
          "doc_count": 2382
        },
        {
          "key": "many",
          "doc_count": 2033
        }
      ]
    }
  }
}
6

Take a look at the previous replies:

i.e. they should not be part of the json passed to Query.Raw()

and

The string representing your query includes size and aggs (aggregations) which are not part of the query so should not be passed to .Raw().

If you want to pass raw json that represents the entirety of your query then you should take a look at the low level client

	string query = @"
	{
		""size"" : 0,
		""aggs"" : {
			""words"" : {
				""terms"" : { 
					""field"" : ""bodyContent"" ,
					""size"" : 80,
					""exclude"" : [""is"",""an"" ]
				}
			}
		}
	}";


	client.LowLevel.Search<TResponse>(query);

You can specify what type you want the response as using the generic type parameter TResponse, for example,

client.LowLevel.Search<string>(query);

Will return the response as a string. Take a look at the documentation for more details.

NOTE: The low level client accessed via client.LowLevel, returns an ElasticsearchResponse<TResponse>, NOT an ISearchResponse<T> as per the high level client

2 Likes
7

thanks a lot...Finally got it !!!