Error loading config file: yaml: line 21: did not find expected key

Elastic Stack Beats
1

I referred other similar discussions but none of those helped. i'm getting this error after adding second prospector. and the error line specified is also at the beginning of the second prospector. Please take a look at it let me know if i have gone wrong in my config file.

filebeat_config
filebeat_config.PNG896×531 21.4 KB

2

You you please share you config as text formatted using </>?

1 Like
4 
filebeat.prospectors:
- type: log
  enabled: true
  setup.template.overwrite: true
  paths:
    - /usr/share/filebeat/logs/50lk_1cr_lines.txt
  encoding: plain
  include_lines: ['\d{4}\-\d{2}\-\d{2}']
  multiline.pattern: '^\d{4}\-\d{2}\-\d{2}'
  multiline.negate: false
  multiline.match: after
  fields:
    document_type: partial_zuul_log
  clean_removed: true
  registry: /usr/share/filebeat/data/registry
output.elasticsearch:
  hosts: [""]
  index: "%{[fields.document_type]}-%{[beat.version]}-%{+yyyy.MM.dd}"
setup.template:
  name: "partial_zuul_log"
  pattern: "partial_zuul_log-*"
- type: log
  enabled: true
  setup.template.overwrite: true
  paths:
    - "/usr/share/filebeat/dockerlogs/3780b5604149475b7911122dfe734760e4e4cbeb70d37350ae112a34f3cf6936/*.log"
  encoding: plain
  multiline.max_lines: 500
  fields:
    document_type: jenkins_container_log
  clean_removed: true
  registry: /usr/share/filebeat/data/registry
setup.template:
  name: "jenkins_container_log"
  pattern: "jenkins_container_log-*"
1 Like
5

Filebeat version: 6.0.1

Filebeat container log:

Exiting: error loading config file: yaml: line 21: did not find expected key
filebeat2018/09/03 13:01:32.470913 beat.go:625: CRIT Exiting: error loading config file: yaml: line 21: did not find expected key
Exiting: error loading config file: yaml: line 21: did not find expected key
filebeat2018/09/03 13:04:57.622215 beat.go:625: CRIT Exiting: error loading config file: yaml: line 21: did not find expected key
Exiting: error loading config file: yaml: line 21: did not find expected key
filebeat2018/09/03 13:11:47.566024 beat.go:625: CRIT Exiting: error loading config file: yaml: line 21: did not find expected key
Exiting: error loading config file: yaml: line 21: did not find expected key
filebeat2018/09/03 13:25:27.110224 beat.go:625: CRIT Exiting: error loading config file: yaml: line 21: did not find expected key
Exiting: error loading config file: yaml: line 21: did not find expected key
1 Like
6

Prospector configuration must be under the same filebeat.prospectors block. Also, yo ucan only have one setup.template.* field in a configuration.

filebeat.prospectors:
- type: log
  enabled: true
  paths:
    - /usr/share/filebeat/logs/50lk_1cr_lines.txt
  encoding: plain
  include_lines: ['\d{4}\-\d{2}\-\d{2}']
  multiline.pattern: '^\d{4}\-\d{2}\-\d{2}'
  multiline.negate: false
  multiline.match: after
  fields:
    document_type: partial_zuul_log
  clean_removed: true
  registry: /usr/share/filebeat/data/registry
- type: log
  enabled: true
  paths:
    - "/usr/share/filebeat/dockerlogs/3780b5604149475b7911122dfe734760e4e4cbeb70d37350ae112a34f3cf6936/*.log"
  encoding: plain
  multiline.max_lines: 500
  fields:
    document_type: jenkins_container_log
  clean_removed: true
  registry: /usr/share/filebeat/data/registry

setup.template.overwrite: true
setup.template:
  name: "jenkins_container_log"
  pattern: "jenkins_container_log-*"

output.elasticsearch:
  hosts: [""]
  index: "%{[fields.document_type]}-%{[beat.version]}-%{+yyyy.MM.dd}"
1 Like
7

Thank you very much for the reply,

But what if i wanted to have different templates for each indices i define? (As the format of log may be significantly different from one another)

1 Like
8

I corrected the config file as you suggested. but the second prospector isn't sending log to output. I don't see entry for the second log file in 'registry' file because, from filebeat container log, it seems like there's no open harvester for second log file. could you please help me?

The second log file is docker container log. I can't use docker input because my filebeat version is 6.0.1 and docker.input is supported from only 6.3 i think.

below is my current config file:

filebeat.prospectors:
- type: log
  enabled: true
  setup.template.overwrite: true
  paths:
    - /usr/share/filebeat/logs/50lk_1cr_lines.txt
  encoding: plain
  include_lines: ['\d{4}\-\d{2}\-\d{2}']
  multiline.pattern: '^\d{4}\-\d{2}\-\d{2}'
  multiline.negate: false
  multiline.match: after
  fields:
    document_type: partial_zuul_log
  clean_removed: true
  registry: /usr/share/filebeat/data/registry
- type: log
  enabled: true
  paths:
    - /usr/share/filebeat/dockerlogs/3780b5604149475b7911122dfe734760e4e4cbeb70d37350ae112a34f3cf6936/*.log
  encoding: plain
  include_lines: ['failed']
  fields:
    document_type: jenkins_container_log
  clean_removed: true
  registry: /usr/share/filebeat/data/registry
  json.keys_under_root: true
  json.add_error_key: true
  json.message_key: log

setup.template.overwrite: true
setup.template:
  name: "filebeat"
  pattern: "filebeat-*"
  settings:
    index.number_of_shards: 1
    index.number_of_replicas: 1
output.elasticsearch:
  hosts: [""]
  index: "filebeat-%{[fields.document_type]}-%{+yyyy.MM.dd}"

And below is the filebeat container log:

2018/09/04 07:18:19.637058 registrar.go:119: INFO States Loaded from registrar: 1
2018/09/04 07:18:19.637134 crawler.go:44: INFO Loading Prospectors: 2
2018/09/04 07:18:19.637273 registrar.go:150: INFO Starting Registrar
2018/09/04 07:18:19.638225 prospector.go:103: INFO Starting prospector of type: log; id: 4955260964263621248
2018/09/04 07:18:19.640368 prospector.go:103: INFO Starting prospector of type: log; id: 16936726131415518943
2018/09/04 07:18:19.640443 crawler.go:78: INFO Loading and starting Prospectors completed. Enabled prospectors: 2
2018/09/04 07:18:49.635649 metrics.go:39: INFO Non-zero metrics in the last 30s: beat.memstats.gc_next=4194304 beat.memstats.memory_alloc=1407256 beat.memstats.memory_total=3187120 filebeat.events.added=1 filebeat.events.done=1 filebeat.harvester.open_files=0 filebeat.harvester.running=0 libbeat.config.module.running=0 libbeat.output.type=elasticsearch libbeat.pipeline.clients=2 libbeat.pipeline.events.active=0 libbeat.pipeline.events.filtered=1 libbeat.pipeline.events.total=1 registrar.states.current=1 registrar.states.update=1 registrar.writes=1
2018/09/04 07:19:19.635542 metrics.go:39: INFO Non-zero metrics in the last 30s: beat.memstats.gc_next=4194304 beat.memstats.memory_alloc=1433560 beat.memstats.memory_total=3213424 filebeat.harvester.open_files=0 filebeat.harvester.running=0 libbeat.config.module.running=0 libbeat.pipeline.clients=2 libbeat.pipeline.events.active=0 registrar.states.current=1
2018/09/04 07:19:49.635453 metrics.go:39: INFO Non-zero metrics in the last 30s: beat.memstats.gc_next=4194304 beat.memstats.memory_alloc=1458328 beat.memstats.memory_total=3238192 filebeat.harvester.open_files=0 filebeat.harvester.running=0 libbeat.config.module.running=0 libbeat.pipeline.clients=2 libbeat.pipeline.events.active=0 registrar.states.current=1
2018/09/04 07:20:19.635678 metrics.go:39: INFO Non-zero metrics in the last 30s: beat.memstats.gc_next=4194304 beat.memstats.memory_alloc=1477224 beat.memstats.memory_total=3257088 filebeat.harvester.open_files=0 filebeat.harvester.running=0 libbeat.config.module.running=0 libbeat.pipeline.clients=2 libbeat.pipeline.events.active=0 registrar.states.current=1
2018/09/04 07:20:49.635523 metrics.go:39: INFO Non-zero metrics in the last 30s: beat.memstats.gc_next=4194304 beat.memstats.memory_alloc=1406960 beat.memstats.memory_total=3283576 filebeat.harvester.open_files=0 filebeat.harvester.running=0 libbeat.config.module.running=0 libbeat.pipeline.clients=2 libbeat.pipeline.events.active=0 registrar.states.current=1
2018/09/04 07:21:19.635517 metrics.go:39: INFO Non-zero metrics in the last 30s: beat.memstats.gc_next=4194304 beat.memstats.memory_alloc=1431424 beat.memstats.memory_total=3308040 filebeat.harvester.open_files=0 filebeat.harvester.running=0 libbeat.config.module.running=0 libbeat.pipeline.clients=2 libbeat.pipeline.events.active=0 registrar.states.current=1

Registry file:

{"source":"/usr/share/filebeat/logs/50lk_1cr_lines.txt","offset":343297578,"timestamp":"2018-09-04T07:18:19.638361432Z","ttl":-1,"type":"log","FileStateOS":{"inode":940025752,"device":64768}}]

9

Could you share the debug logs (filebeat -e -d "*")? It would provide more info on what's happening on the prospector level.

10

this is the debug log:

2018/09/04 11:16:09.704538 registrar.go:253: DBG Registry file updated. 7 states written.
2018/09/04 11:16:09.704599 registrar.go:200: DBG Processing 1 events
2018/09/04 11:16:09.704617 registrar.go:195: DBG Registrar states cleaned up. Before: 7, After: 7
2018/09/04 11:16:09.704627 registrar.go:228: DBG Write registry file: /usr/share/filebeat/data/registry
2018/09/04 11:16:09.704791 harvester.go:353: DBG Drop line as it does not match any of the include patterns     at java.net.Socket.connect(Socket.java:538)
2018/09/04 11:16:09.705710 registrar.go:253: DBG Registry file updated. 7 states written.
2018/09/04 11:16:09.705758 registrar.go:200: DBG Processing 1 events
2018/09/04 11:16:09.705777 registrar.go:195: DBG Registrar states cleaned up. Before: 7, After: 7
2018/09/04 11:16:09.705788 registrar.go:228: DBG Write registry file: /usr/share/filebeat/data/registry
2018/09/04 11:16:09.706034 harvester.go:353: DBG Drop line as it does not match any of the include patterns     at java.net.Socket.<init>(Socket.java:434)
2018/09/04 11:16:09.706774 registrar.go:253: DBG Registry file updated. 7 states written.
2018/09/04 11:16:09.706821 registrar.go:200: DBG Processing 1 events
2018/09/04 11:16:09.706839 registrar.go:195: DBG Registrar states cleaned up. Before: 7, After: 7
2018/09/04 11:16:09.706849 registrar.go:228: DBG Write registry file: /usr/share/filebeat/data/registry
2018/09/04 11:16:09.707005 harvester.go:353: DBG Drop line as it does not match any of the include patterns     at java.net.Socket.<init>(Socket.java:211)
2018/09/04 11:16:09.707956 registrar.go:253: DBG Registry file updated. 7 states written.
2018/09/04 11:16:09.707997 registrar.go:200: DBG Processing 1 events
2018/09/04 11:16:09.708015 registrar.go:195: DBG Registrar states cleaned up. Before: 7, After: 7
2018/09/04 11:16:09.708026 registrar.go:228: DBG Write registry file: /usr/share/filebeat/data/registry
2018/09/04 11:16:09.708165 harvester.go:353: DBG Drop line as it does not match any of the include patterns     at com.jcraft.jsch.Util.createSocket(Util.java:338)
2018/09/04 11:16:09.709185 registrar.go:253: DBG Registry file updated. 7 states written.
2018/09/04 11:16:09.709224 registrar.go:200: DBG Processing 1 events
2018/09/04 11:16:09.709246 registrar.go:195: DBG Registrar states cleaned up. Before: 7, After: 7
2018/09/04 11:16:09.709259 registrar.go:228: DBG Write registry file: /usr/share/filebeat/data/registry

It says: Drop line as it does not match any of the include patterns,

But my log has the pattern "failed". following is the output when i grep for keyword "failed" in my log:

{"log":"[08/30/18 02:48:27] SSH Launch of yyy on xxx failed in 5 ms\n","stream":"stdout","time":"2018-08-30T02:48:27.204380049Z"}
{"log":"[08/30/18 03:12:27] SSH Launch of yyy on xxx failed in 3 ms\n","stream":"stdout","time":"2018-08-30T03:12:27.205670953Z"}
{"log":"[08/30/18 03:12:27] SSH Launch of node01 on xxxx failed in 4 ms\n","stream":"stdout","time":"2018-08-30T03:12:27.205727879Z"}
11

There seems to be problem with include_lines field. But i have used json.message_key to specify the key on which the pattern should be matched.

Can you please help me?

12

Sorry to have bothered you. Include_lines with json.message_key does work. My log was too big so it was taking too much time i guess. when i pointed the path to part of that file it could filter out 'failed' lines.

1 Like
13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.