# Filebeat send json data to Elastiksearch isuue

**URL:** https://discuss.elastic.co/t/filebeat-send-json-data-to-elastiksearch-isuue/244860
**Category:** Logs
**Created:** [August 13, 2020, 10:45am UTC](https://discuss.elastic.co/t/filebeat-send-json-data-to-elastiksearch-isuue/244860 "2020-08-13T10:45:12Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![Partha\_Biswas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/partha_biswas/32/47685_2.png) [@Partha\_Biswas](https://discuss.elastic.co/u/Partha_Biswas)
#### Post date: [August 13, 2020, 10:45am UTC](https://discuss.elastic.co/t/filebeat-send-json-data-to-elastiksearch-isuue/244860/1 "2020-08-13T10:45:12Z")

</div>

I am using filebeat to send log data in ELK kibana.

Log example:- error\_log.log

```auto
{"type":"ERROR","errorType":"ERROR","message":"Error in add","others":{"error": "something went wrong","source":"/public/new/user"}}
{"type":"ERROR","errorType":"ERROR","message":"Error in update","others":{"error": "something went wrong","source":"/public/users/edit"}}

```

filebeat.yml:-

```auto
   filebeat.inputs:
    - type: log
         enable: true
         paths:
             - error_log.log
         json.keys_under_root: true

    processors:
        - decode_json_fields:
              fields: ["others"]

    filebeat.config.modules:
        path: ${path.config}/modules.d/*.yml

    output.elasticsearch:
        hosts: ["elastic.xyz.xyz:9200"]

```

kiban logging not working but if I remove `json.keys_under_root: true` & `processors` then it's work but only as text format not as json

---

<div class="post-metadata">

### Author: ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)
#### Post date: [August 13, 2020, 11:01am UTC](https://discuss.elastic.co/t/filebeat-send-json-data-to-elastiksearch-isuue/244860/2 "2020-08-13T11:01:33Z")

</div>

Hi @Partha_Biswas,

the `decode_json_fields` processor decodes JSON values from string fields. Looking at the example log lines you gave, the value assigned to the `others` key is not a string, but already an object. That means that the initial JSON parsing via `json.keys_under_root` should have parsed it already and the that the processor likely fails.

Have you tried removing the processor but leaving the `json.keys_under_root` in?

ℹ Unrelated to the indexing, but important for querying later, in order to properly search the resulting documents you will likely want to declare the `others` field as a `nested`-type field in your mapping.

---

<div class="post-metadata">

### Author: ![Partha\_Biswas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/partha_biswas/32/47685_2.png) [@Partha\_Biswas](https://discuss.elastic.co/u/Partha_Biswas)
#### Post date: [August 14, 2020, 8:17am UTC](https://discuss.elastic.co/t/filebeat-send-json-data-to-elastiksearch-isuue/244860/3 "2020-08-14T08:17:48Z")

</div>

Yes, I tried with `json.keys_under_root` and removed `processors`  
But same kibana logging not work

Is there any problem with my `error_log.log` because it's json formats but it's file extention not `.json` and it's all log is not warped with `{` `}`

---

<div class="post-metadata">

### Author: ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)
#### Post date: [August 17, 2020, 9:36am UTC](https://discuss.elastic.co/t/filebeat-send-json-data-to-elastiksearch-isuue/244860/4 "2020-08-17T09:36:21Z")

</div>

I don't expect the file name extension to be of any significance. Could you check if there is anything in the Filebeat logs or Elasticsearch logs that correlates with the parsing failures?

---

<div class="post-metadata">

### Author: ![Partha\_Biswas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/partha_biswas/32/47685_2.png) [@Partha\_Biswas](https://discuss.elastic.co/u/Partha_Biswas)
#### Post date: [August 21, 2020, 6:38am UTC](https://discuss.elastic.co/t/filebeat-send-json-data-to-elastiksearch-isuue/244860/5 "2020-08-21T06:38:34Z")

</div>

Thanks,  
I got my error and now it's working

---

<div class="post-metadata">

### Author: ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)
#### Post date: [August 21, 2020, 9:32am UTC](https://discuss.elastic.co/t/filebeat-send-json-data-to-elastiksearch-isuue/244860/6 "2020-08-21T09:32:01Z")

</div>

That's good to hear. Is it something you can share with the community so we can all learn from it?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 18, 2020, 9:32am UTC](https://discuss.elastic.co/t/filebeat-send-json-data-to-elastiksearch-isuue/244860/7 "2020-09-18T09:32:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
