I am trying to input logs from Azure file share, so I created a volume mount in the Beats yaml file and was able to get the logs mounted under "/mnt/smbshare/Logs/*.log".
But filebeat is not able to read those logs with my below given input code
- type: filestream
id: ***-logs
enabled: true
paths:
- /mnt/smbshare/Logs/*.log
parsers:
- ndjson:
keys_under_root: true
overwrite_keys: true
processors:
- decode_json_fields:
fields: ["message"]
target: ""
overwrite_keys: true
add_error_key: true
- drop_fields:
fields: ["LogNumber"]
- rename:
fields:
- from: "message"
to: "log_message"
- dissect:
tokenizer: "%{Timestamp} %{Level} %{MessageTemplate} %{Properties}"
field: "log_message"
target_prefix: ""
- timestamp:
field: "Timestamp"
target_field: "@timestamp"
layouts:
- "2006-01-02T15:04:05.0000000-07:00"
I have purposely dropped the field "Properties" which is a nested json block and was having parsing error.
please let me know if I have chosen the correct way to get these logs ?