# Filebeat.yml include\_lines and decode\_json can't work together

**URL:** <https://discuss.elastic.co/t/filebeat-yml-include-lines-and-decode-json-cant-work-together/131388>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [May 11, 2018, 3:52am UTC](https://discuss.elastic.co/t/filebeat-yml-include-lines-and-decode-json-cant-work-together/131388 "2018-05-11T03:52:15Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Qing](https://avatars.discourse-cdn.com/v4/letter/q/e99b99/32.png) [@Qing](https://discuss.elastic.co/u/Qing)\
**Post date:** [May 11, 2018, 3:52am UTC](https://discuss.elastic.co/t/filebeat-yml-include-lines-and-decode-json-cant-work-together/131388/1 "2018-05-11T03:52:15Z")

</div>

Hi:  
Can anyone help me ?  
I meet problem using filebeat, error message:  
Exiting: Error in initing prospector: When using the JSON decoder and line filtering together, you need to specify a message\_key value accessing 'filebeat.prospectors.0' (source:'filebeat.yml')  
Exiting: Error in initing prospector: When using the JSON decoder and line filtering together, you need to specify a message\_key value accessing 'filebeat.prospectors.0' (source:'filebeat.yml')

----------------------------------------------filebeat.yml----------------------------------------------------------------  
filebeat.prospectors:

```
- type: log
  enabled: true
  json.keys_under_root: true
  paths:
    - /data/weblog/nginx/nginx.log
	
  # Include lines. A list of regular expressions to match. It exports the lines that are
  # matching any regular expression from the list.
  include_lines: ['/api/datasources/proxy/']

processors:
    - decode_json_fields:
        fields: ["message"]
        target: json
        max_depth: 1

```

----------------------------------------------filebeat debug message----------------------------------------  
2018-05-11T10:58:43.129+0800 DEBUG [publish] pipeline/processor.go:275 Publish event: {  
"@timestamp": "2018-05-11T02:58:43.129Z",  
"@metadata": {  
"beat": "filebeat",  
"type": "doc",  
"version": "6.2.4"  
},  
"source": "/data/weblog/nginx/nginx.log",  
"beat": {  
"version": "6.2.4",  
"name": "[8.ops.com](http://8.ops.com)",  
"hostname": "[8.ops.com](http://8.ops.com)"  
},  
"method": "GET",  
"time": "11/May/2018:09:36:17 +0800",  
"cost": "0.010",  
"prospector": {  
"type": "log"  
},  
"body": "",  
"status": "200",  
"uri": "GET /d/000000321/neng?refresh=1m\u0026orgId=43 HTTP/1.1",  
"offset": 1780916  
}

ths ahead.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [May 11, 2018, 7:07am UTC](https://discuss.elastic.co/t/filebeat-yml-include-lines-and-decode-json-cant-work-together/131388/2 "2018-05-11T07:07:18Z")

</div>

I can see how the log message might be confusing here. It actually does not refer to the processor but to `json.*` config inside the prospector. You have to set `json.message_key`: [https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#config-json](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#config-json)

As I see you are using the processor and the json option I wonder if that is really what you want? There are use cases for decoded json in json but normally that is not needed.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 8, 2018, 7:07am UTC](https://discuss.elastic.co/t/filebeat-yml-include-lines-and-decode-json-cant-work-together/131388/3 "2018-06-08T07:07:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.