We have logstash to push the logs from one server to other, where all the traps are collected. Through which we plot graphs in "Grafana". The port number in which logs parsing is "7546".
until "logstash-7.16.3-1.x86_64" it was worked without any issues and I have updated with "logstash-8.5.3-1.x86_64", then it stopped parsing the logs.
could you please help us to fix the issue?
Below are our configurations.
[root@localhost logstash]# ls -ltrh
total 48K
-rw------- 1 root root 1.7K Nov 30 08:40 startup.options
-rw-r--r-- 1 root root 285 Nov 30 08:40 pipelines.yml
-rw-r--r-- 1 root root 342 Nov 30 08:40 logstash-sample.conf
-rw-r--r-- 1 root root 7.3K Nov 30 08:40 log4j2.properties
-rw-r--r-- 1 root root 1.9K Nov 30 08:40 jvm.options
-rw-r--r-- 1 root root 15K Jan 3 09:07 logstash.yml
-rw-r--r-- 1 root root 300 Jan 3 09:07 monit_alert.yml
drwxr-xr-x 2 root root 4.0K Jan 5 10:59 conf.d
we have below "deprecation" log. Does this got any effect that the issue that am facing now ?
[2023-01-05T00:01:14,032][WARN ][deprecation.logstash.codecs.line][main][37a9313707e5221038d742efc461ec902fd1d7138c0a83d0998e19b6e8d4abb4] Relying on default value of pipeline.ecs_compatibility, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
[2023-01-05T00:01:14,039][WARN ][deprecation.logstash.codecs.line][main][37a9313707e5221038d742efc461ec902fd1d7138c0a83d0998e19b6e8d4abb4] Relying on default value of pipeline.ecs_compatibility, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
In-fact, you are correct. but, we are an engineering team and we ship the RPMs with minimal ISO and customized them with our own third-party applications (one of them is Logstash) to customers. We can't ask them to do these changes after the ISO is installed (ISO will have the logstash RPM). Hence, it's important for us to fix it before shipping the product to the customer.
You have a custom RPM for Logstash, is that what you mean? It is not clear
In any case, your issue is probably caused by the default configuration of pipeline.ecs_compatibility, you need to test if changing it to disabled in logstash.yml wil solve.
If this solve, you need to have this config in logstash.yml, how you will manage that is entirely dependent on your infrastructure and is out of the scope of the forum.
Another option is to have the pipeline.ecs_compatibility: disabled setting in each pipeline in pipelines.yml, which I understand your customers can edit.
Custom RPM, I meant that we download the logstash RPM and build it with the ISO image. In this ISO we'll have many other 3rd party tools like logstash, zookeeper, etc...
we use "*.src.rpm" file and using rpmbuild we customize the RPM with changes like what in in this particular case we need to add an entry in the config file "logstash.yml".
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.