Filebeat error while reading multiline json lines log file

Elastic Stack Beats
1

I am facing issues while ingesting logs to logstash using filebeat. both of filebeat and logstash versions are 7.5.2 and please find the details below
filebeat config

filebeat.inputs:
- type: log
  enabled: true
  encoding: utf-8
  ignore_older: 3h
  json.keys_under_root: true
  json.add_error_key: true
  fields:
    index_env: prod
    index_name: vault
    index_dc: ho
  paths:
    - /var/log/vault_audit.log
output.logstash:
 enabled: true
 hosts: ["<IP ADDRESS>:5045"]

actual log from /var/log/vault_audit.log: which is causing the error, below is the log with new lines added for convenience

 {
	"time": "2020-05-12T13:21:27.468042577Z",
	"type": "response",
	"auth": {
		"client_token": "hmac-sha256:21549fef0d44b7b880b021e4d80b966bc59383393a9cf1695a8d849602e7e19d",
		"accessor": "hmac-sha256:6f5cab54378cb18d10f067763dca36215d150e5198939b24ac6c20349a5f0c82",
		"display_name": "github_abc-def-provisioning",
		"policies": ["abc", "default", "admin-configuration_vault"],
		"token_policies": ["abc", "default", "admin-configuration_vault"],
		"metadata": {
			"org": "ABC-Provisioning-Configuration-DEF",
			"username": "admin123"
		},
		"entity_id": "21549fe-fef0-e7e19d-9ec9-f1695a8d8496",
		"token_type": "service"
	},
	"request": {
		"id": "7d50810b-e7dd-5430-1c85-ba496e98c1ed",
		"operation": "read",
		"client_token": "hmac-sha256:2cab76cb0ba20f1c2ecc60241c02d4cc5f7593b36bd0b120b7a153f2695a1258",
		"client_token_accessor": "hmac-sha256:2a45f24b436b93f5a5a85ad2f0fff0c1500f6d18c6dd7e67ff303d8b4d89d76a",
		"namespace": {
			"id": "root"
		},
		"path": "secret/abc-def-provisioning-configuration/my-own-service/prd-r2",
		"remote_address": "127.0.0.1"
	},
	"response": {
		"secret": {},
		"data": {
			"vault.npp.authServer.publicKey": "hmac-sha256:15a36c8272ca984212df5f3fed206ac53cefdc63d359fbb8d3d66b16d965351f",
			"vault.npp.soapclient.def.customerServiceClient.wss4j.password": "hmac-sha256:707d59c031e29e29e8cc8fa93946b828989f4d56ce031056fbad5249aaa862e7",
			"vault.npp.soapclient.def.customerServiceClient.wss4j.username": "hmac-sha256:1bb69b75bc1667d2809261c41e481bedf8b566fe41404ed3c3c8cc194ee39699",
			"vault.npp.xsp.security.authServer.configs[0].secret": "hmac-sha256:1bb69b75bc1667d2809261c41e481bedf8b566fe41404ed3c3c8cc194ee39699",
			"vault.npp.xsp.security.authServer.configs[0].username": "hmac-sha256:707d59c031e29e29e8cc8fa93946b828989f4d56ce031056fbad5249aaa862e7",
			"vault.npp.xsp.security.authServer.configs[1].secret": "hmac-sha256:707d59c031e29e29e8cc8fa93946b828989f4d56ce031056fbad5249aaa862e7",
			"vault.npp.xsp.security.authServer.configs[1].username": "hmac-sha256:707d59c031e29e29e8cc8fa93946b828989f4d56ce031056fbad5249aaa862e7",
			"vault.npp.xsp.security.authServer.configs[2].secret": "hmac-sha256:1bb69b75bc1667d2809261c41e481bedf8b566fe41404ed3c3c8cc194ee39699",
			"vault.npp.xsp.security.authServer.configs[2].username": "hmac-sha256:707d59c031e29e29e8cc8fa93946b828989f4d56ce031056fbad5249aaa862e7"
		}
	}
}

and error from filebeat is below

|2020-05-19T15:25:56.543Z|ERROR|[logstash]|logstash/async.go:279|Failed to publish events caused by: read tcp <IP>:34870-><IP>:5045: read: connection reset by peer|
|---|---|---|---|---|
|2020-05-19T15:25:56.543Z|ERROR|[logstash]|logstash/async.go:279|Failed to publish events caused by: read tcp <IP>:34870-><IP>: read: connection reset by peer|
|2020-05-19T15:25:56.543Z|ERROR|[logstash]|logstash/async.go:279|Failed to publish events caused by: read tcp <IP>:34870-><IP>: read: connection reset by peer|

error from the logstash is below

[INFO ] 2020-05-19 15:25:56.523 [defaultEventExecutorGroup-4-3] BeatsHandler - [local: 96.112.245.149:5045, remote: <IP>:34870] Handling exception: Invalid FieldReference: `vault.npp.xsp.security.authServer.configs[0].username`
[WARN ] 2020-05-19 15:25:56.531 [nioEventLoopGroup-2-4] DefaultChannelPipeline - An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
org.logstash.FieldReference$IllegalSyntaxException: Invalid FieldReference: `vault.npp.xsp.security.authServer.configs[0].username`
	at org.logstash.FieldReference$StrictTokenizer.tokenize(FieldReference.java:303) ~[logstash-core.jar:?]
	at org.logstash.FieldReference.parse(FieldReference.java:204) ~[logstash-core.jar:?]
	at org.logstash.FieldReference.parseToCache(FieldReference.java:195) ~[logstash-core.jar:?]
	at org.logstash.FieldReference.from(FieldReference.java:127) ~[logstash-core.jar:?]
	at org.logstash.ConvertedMap.put(ConvertedMap.java:95) ~[logstash-core.jar:?]
	at org.logstash.ConvertedMap.newFromMap(ConvertedMap.java:75) ~[logstash-core.jar:?]
	at org.logstash.Valuefier.lambda$initConverters$13(Valuefier.java:172) ~[logstash-core.jar:?]
	at org.logstash.Valuefier.convert(Valuefier.java:94) ~[logstash-core.jar:?]
	at org.logstash.ConvertedMap.newFromMap(ConvertedMap.java:75) ~[logstash-core.jar:?]
	at org.logstash.Valuefier.lambda$initConverters$13(Valuefier.java:172) ~[logstash-core.jar:?]
	at org.logstash.Valuefier.convert(Valuefier.java:94) ~[logstash-core.jar:?]
	at org.logstash.ConvertedMap.newFromMap(ConvertedMap.java:75) ~[logstash-core.jar:?]
	at org.logstash.ext.JrubyEventExtLibrary$RubyEvent.initializeFallback(JrubyEventExtLibrary.java:323) ~[logstash-core.jar:?]
	at org.logstash.ext.JrubyEventExtLibrary$RubyEvent.ruby_initialize(JrubyEventExtLibrary.java:97) ~[logstash-core.jar:?]
	at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_beats_minus_6_dot_0_dot_9_minus_java.lib.logstash.inputs.beats.message_listener.RUBY$method$onNewMessage$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-beats-6.0.9-java/lib/logstash/inputs/beats/message_listener.rb:40) ~[?:?]
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.