Hello,
I'm new to this platform and I need your help for my ELK stack
Indeed logstash has not been able to write to elasticsearch for a while
below is my logstash/conf.d flow management configuration
<
input {
tcp {
port => 5144
type => syslog
}
udp {
port => 5144
type => syslog
}
}
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
}
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
}
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
output {
if [type] == "syslog" {
elasticsearch { hosts => ["172.22.20.190:9200", "172.22.20.191:9200"] index => "netapp-%{+YYYY.MM.dd}" }
stdout { codec => rubydebug }
}
}
/>
However when I restart logstash the below errors in logstash-plain.log file
<
[2023-09-04T13:14:39,798][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"http://172.22.20.190:9200/"}
[2023-09-04T13:14:39,808][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch version determined (7.17.4) {:es_version=>7}
[2023-09-04T13:14:39,810][WARN ][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the type event field won't be used to determine the document _type {:es_version=>7}
[2023-09-04T13:14:39,860][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"http://172.22.20.191:9200/"}
[2023-09-04T13:14:39,958][INFO ][logstash.outputs.elasticsearch][main] Config is not compliant with data streams. data_stream => auto resolved to false
[2023-09-04T13:14:40,009][INFO ][logstash.outputs.elasticsearch][main] Using a default mapping template {:es_version=>7, :ecs_compatibility=>:disabled}
/>
And I have another file (logstash-deprecation.log) with the error below:
<
[2023-09-04T13:14:41,503][WARN ][deprecation.logstash.codecs.plain][main][941e8458b1648478d146bdf3375031ae2f2d2499b916047d2511464488cb2041] Relying on default value of pipeline.ecs_compatibility, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
/>
Please can you help me to understand and solve the problem?
THANKS.
Best regards.