Elasticsearch javaApi

Elastic Stack Elasticsearch
1

hi , when i use searsh-api to retrieve data from one index using this code :

Settings settings = Settings.settingsBuilder().put("cluster.name", "elasticsearch-cluster").build();
client = TransportClient.builder().settings(settings).build().addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("10.206.206.82"), 9300));

SearchResponse sr = client.prepareSearch("logstash-2016.05.31").get();

i got a json contains a data but also non-useful data at the beginning of json like this

{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 110,
"max_score" : 1.0,
"hits" : [

plz how eliminate it ?

2

This? https://www.elastic.co/guide/en/elasticsearch/reference/2.3/common-options.html#_response_filtering

3

You can do response filtering using REST api, but I'm not sure the java api supports this. According to How to use filter_path in java api? you can't use it.

4

You might be interested in this comment too: https://github.com/elastic/elasticsearch/pull/10980#issuecomment-190107941

5

i need to eliminate the header

{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
}

and informations about returned results

"hits" : {
"total" : 110,
"max_score" : 1.0,

so i want to display just the hits array, which includes the returned data

this is a part of the response

{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 25,
"successful" : 10,
"failed" : 0
},
"hits" : {
"total" : 200,
"max_score" : 1.0,
"hits" : [ {
"_index" : "logstash-2016.05.30",
"_type" : "log4j",
"_id" : "AVUCj6LyBTC2ElQBKf9k",
"_score" : 1.0,
"_source" : {
"message" : " retrieveValidGatewayTransmissionData :put dupsClosedMap !",
"@version" : "1",
"@timestamp" : "2016-05-30T16:46:54.391Z",
"timestamp" : 1464626814678,
"path" : "com.esprit.HelloWorldServlet",
"priority" : "WARN",
"logger_name" : "com.esprit.HelloWorldServlet",
"thread" : " [task-scheduler-5] ",
"class" : "?",
"file" : "?:?",
"method" : "?",
"host" : "10.206.206.72:51303",
"type" : "log4j",
"date" : "016-03-23 16:32:02.613 ",
"lvl" : " W ",
"gateway" : "GTW2",
"device" : "Device2",
"source" : " LogService:14 "
}
}, {
"_index" : "logstash-2016.05.30",
"_type" : "log4j",
"_id" : "AVUCj6LyBTC2ElQBKf9l",
"_score" : 1.0,
"_source" : {
"message" : " retrieveValidGatewayTransmissionData :put dupsClosedMap !",
"@version" : "1",
"@timestamp" : "2016-05-30T16:46:54.395Z",
"timestamp" : 1464626814678,
"path" : "com.esprit.HelloWorldServlet",
"priority" : "ERROR",
"logger_name" : "com.esprit.HelloWorldServlet",
"thread" : " [task-scheduler-6] ",
"class" : "?",
"file" : "?:?",
"method" : "?",
"host" : "10.206.206.72:51303",
"type" : "log4j",
"date" : "016-03-23 17:32:02.613 ",
"lvl" : " E ",
"gateway" : "GTW3",
"device" : "Device5",
"source" : " LogService:23 "
}
}]
}

6

i need to returns just the _source , without metadata
:slight_smile:

7

As @tanguy pointed out, you can take the search response (sr) and use its #toXContent() method to write it to an XContentBuilder that allies filters. Here is an example that gives you hits.hits._source

XContentBuilder filteredBuilder = XContentBuilder.builder(JsonXContent.jsonXContent, new String[]{"hits.hits._source"}).prettyPrint();
filteredBuilder.startObject();
sr.toXContent(filteredBuilder, ToXContent.EMPTY_PARAMS);
filteredBuilder.endObject();
System.out.println(filteredBuilder.string());

The result looks something like this:

{
  "hits" : {
    "hits" : [ {
      "_source" : {
        "value" : 2,
        "text" : "something"
      }
    }, {
      "_source" : {
        "value" : 4,
        "text" : "something else"
      }, 
     ...
   } ]
  }
}

Note however this only changes presentation of the result. Since the filtering is done on the search response, all data that is filtered out is still transported to the client.

8

thanks chuescher ,

i used this code to extract only the data

> SearchResponse sr = client.prepareSearch("logstash-*")

          .setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
  	 	.setFetchSource(true)
          .setSize(1000)
          .setFetchSource( new String[] {"lvl","message","date","device","thread","source","gateway"}, null)		
         // .addFields("lvl")
          .execute()
          .actionGet();
  
  
  for (SearchHit hit : sr.getHits()){
  	  map = hit.getSource();
  	}