Hello Team and @john.akash,
Hope you are doing good.
I am also trying to push log message data to ES so that I can filter the data and create visualization.
Here is my sample log
"@timestamp" => 2021-09-28T04:51:22.169Z,
logstash_1 | "message" => "2021-03-17:00:00:54 +0000 | GET | 301 | 0 | 261 | check_http/v2.2 (monitoring-plugins 2.2) | 80",
logstash_1 | "host" => {
logstash_1 | "name" => "xxxxxxxx"
logstash_1 | }
Sample data
2021-03-17:00:00:54 +0000 | GET | 301 | 0 | 261 | check_http/v2.2 (monitoring-plugins 2.2) | 80
2021-03-17:00:00:54 +0000 | GET | 404 | 0 | 261 | check_http/v2.2 (monitoring-plugins 2.2) | 80
2021-03-17:00:00:54 +0000 | GET | 501 | 0 | 261 | check_http/v2.2 (monitoring-plugins 2.2) | 80
File beat input file.
filebeat.config:
modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
processors:
- add_cloud_metadata: ~
filebeat.inputs:
- type: log
enabled: true
paths:
- "/var/log/test.log"
exclude_files: ['\.gz$']
json.message_key: GET
json.keys_under_root: true
json.add_error_key: true
#output.elasticsearch:
#hosts: '${ELASTICSEARCH_HOSTS:elasticsearch:9200}'
output.logstash:
hosts: ["logstash:5044"]
Still I am getting error decode json issue. Can you please suggest/assist me on this?
Regards,
Sabil.