How to get only one field from elasticsearch in the output

Elastic Stack Beats
1

I am having my log data pushed into elasticssearch looks like given below:

{
     "took": 5,
   "timed_out": false,
   "_shards": {
       "total": 1,
       "successful": 1,
       "skipped": 0,
       "failed": 0
   },
   "hits": {
       "total": {
           "value": 1999,
           "relation": "eq"
       },
       "max_score": 1.0,
       "hits": [
           {
               "_index": "logstash-2021.01.13-000001",
               "_type": "_doc",
               "_id": "lVef-3YBI8ZVMz0vOphU",
               "_score": 1.0,
               "_source": {
                   "host": {
                       "name": "AAD-W1PF14DMMK"
                   },
                   "@timestamp": "2021-01-13T12:01:19.794Z",
                   "log": {
                       "file": {
                           "path": "C:\\elk\\test.log"
                       },
                       "offset": 158
                   },
                   "type": "test",
                   "tags": [
                       "beats_input_codec_plain_applied"
                   ],
                   "ecs": {
                       "version": "1.6.0"
                   },
                   "agent": {
                       "hostname": "AAD-W1PF14DMMK",
                       "type": "filebeat",
                       "name": "AAD-W1PF14DMMK",
                       "id": "4aa46436-264c-40ba-a24a-17af072c8363",
                       "version": "7.10.1",
                       "ephemeral_id": "18c7451e-78a8-4806-b43f-5ebae812b533"
                   },
                   "@version": "1",
                   "message": "2015-10-18 18:01:48,963 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens:"
               }
           },
}

I want to get output like

{  "message": "2015-10-18 18:01:48,963 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens:" }

I tried using

GET localhost:9200/_search?filter_path=hits.hits._source 
 {
    "_source": {
        "includes": ["message"]
    },
    "query": {
        "multi_match" : {
        "query": "ERROR",
        "fields": [ "message"] 
        }
    }
}

It worked but giving warning that it will get deprecated by default.
WARNING :

#! Deprecation: this request accesses system indices: [.apm-agent-configuration, .apm-custom-link, .async-search, .kibana_1, .kibana_task_manager_1], but in a future major version, direct access to system indices will be prevented by default

What would be the alternative solution??

2

Right now your query is executing against every index, including indices that are used "behind the scenes" by various Elasticsearch components. The warning is telling you that in the future, those indices will not be available for search by default.

Fortunately, the data you are looking for is not in that sort of index. Instead of using the general search endpoint:

GET localhost:9200/_search

...try searching against an endpoint for your logstash indices only:

GET localhost:9200/logstash-*/_search?filter_path=hits.hits._source

This should return the results you are looking for without showing the deprecation warning.

2 Likes
3

Thank you!
Got my mistake!!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.