Filebeat.yml include_lines and decode_json can't work together


(Qing) #1

Hi

Can anyone help me ?
I meet problem, error info:

Exiting: Error in initing prospector: When using the JSON decoder and line filtering together, you need to specify a message_key value accessing 'filebeat.prospectors.0' (source:'filebeat.yml')
Exiting: Error in initing prospector: When using the JSON decoder and line filtering together, you need to specify a message_key value accessing 'filebeat.prospectors.0' (source:'filebeat.yml')

-------------------------------------------filebeat.yml------------------------------------------------------------------
filebeat.prospectors:

- type: log
  enabled: true
  json.keys_under_root: true
  paths:
    - /data/weblog/nginx/nginx.log
	
  # Include lines. A list of regular expressions to match. It exports the lines that are
  # matching any regular expression from the list.
  include_lines: ['/api/datasources/proxy/']

processors:
    - decode_json_fields:
        fields: ["message"]
        target: json
        max_depth: 1


------------------------------------------filebeat debug log---------------------------------------------------------------------------------		
2018-05-11T10:58:43.129+0800	DEBUG	[publish]	pipeline/processor.go:275	Publish event: {
  "@timestamp": "2018-05-11T02:58:43.129Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "doc",
    "version": "6.2.4"
  },
  "source": "/data/weblog/nginx/nginx.log",
  "beat": {
    "version": "6.2.4",
    "name": "8.ops.com",
    "hostname": "8.ops.com"
  },
  "method": "GET",
  "time": "11/May/2018:09:36:17 +0800",
  "cost": "0.010",
  "prospector": {
    "type": "log"
  },
  "body": "",
  "status": "200",
  "uri": "GET /d/000000321/neng?refresh=1m\u0026orgId=43 HTTP/1.1",
  "offset": 1780916
}

ths


(ruflin) #2

Can you please explain why you post the same question 3 times?

Please stop doing that.


(Qing) #3

I'm so sorry
I can't delete other two topic.
I send the first and the second topic which were hiden, I thought them lost.

Can you help me to delete them?


(ruflin) #4

I prefer not deleting post especially when links exists. Best is that we close the topics as soon as they are answered.


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.