I have 2 conf files and they are sending data at the same time to the 2 index (when I would like each conf to send the information to the specific index)
If you can help me, I can provide more information if needed.
[Unit]
Description=logstash
[Service]
Type=simple
User=logstash
Group=logstash
# Load env vars from /etc/default/ and /etc/sysconfig/ if they exist.
# Prefixing the path with '-' makes it try to load, but if the file doesn't
# exist, it continues onward.
EnvironmentFile=-/etc/default/logstash
#EnvironmentFile=-/etc/sysconfig/logstash
ExecStart=/usr/share/logstash/bin/logstash --path.settings /etc/logstash --path.config /etc/logstash/conf.d/*.conf
Restart=always
WorkingDirectory=/
Nice=19
LimitNOFILE=16384
# When stopping, how long to wait before giving up and sending SIGKILL?
# Keep in mind that SIGKILL on a process can cause data loss.
TimeoutStopSec=infinity
[Install]
WantedBy=multi-user.target
when some information is sent , by any of the conf, it duplicates in the 2 index
Curious did you create / edit the service file yourself or is that the default?
Seems to me the line above is the problem... that will concatonate the 2 files together which would explain what you are seeing ... Or your pipelines.yml not being read and therefore the files and the directory are being concatenated... Either way, pretty sure that's the issue
Seems like that should be
--path.settings /etc/logstash
I will need to triple check, but I would test with that first.
If you intentionally want to concatenate them together which I don't think you want to, but if you do we can just put in logic so that the input will get a tag and then the output will be conditional on that tag
a) Your pipeline.yml has the correct path and it is still not working?
b) Or now it is fixed and it is working?
if a) That would indicate that If the path is correct and it is still not working there is a different issue... like logstash can not find / access the .conf file or there is an error in the conf file. You need to show the logstash startup logs...
I would comment out the mikrotik pipeline in the pipeline.yml so we can focus on the other one...
Things to check check list the actual file ... is is there... are the permissions correct?
ls -l /etc/logstash/conf.d/srvvmfs01-log.conf
You can also just start logstash in the foreground with the following command and watch the logs....
There is something simple at this point... most likely has to do with the pipelines.yml or the path... because when you used *.conf 2 pipelines ran which indicates there is not an error in the actual .conf file.
Minor I don't think you need this on the beats
codec => plain
Do you have a line feed? You know blank line after the last line in the pipeline.yml
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.