Is it possible to send logs to Elasticsearch & Logstash both from a server ? For example I want to send some logs to Elasticsearch only say Apache access logs howwever I want o send and parse apache error logs using Logstash. Is it possible?
It is possible, but not necessarily encouraged. Can you describe your setup in more detail? What is sending the logs from the server?
I am using filebeat to send Apache Logs.
Filebeat doesn't support multiple outputs, so you have two options:
- Have two separate Filebeat instances running on the server, one is talking directly to Elasticsearch and another is talking to Logstash.
- Send everything to Logstash and parse only error logs, pass access logs to Elasticsearch.
I'd go with second option for easier administration and maintenance
yep in this case its better to go through Logstash. So logstash process the config file in what order? for example I have below conf files in logstash conf directory
ssh.conf - for ssh success/ failed attempts with geoip
Apache.conf - for Apache access/error l ogs with geoip
which one logstash process first Apache.conf ?
Not sure which one of those because ssh.conf starts with lowercase and Apache.conf uppercase. Logstash reads configs in alphabetical order.
Previously, I used 001-input-name.conf scheme in config file naming.
Nowadays I can mostly get away with using different pipelines, but still follow the naming if there's a need.
ok so its better to name them in an order like 01.apache.conf 02.ssh.conf 03.mysql.conf ?
now it will process the conf in order of 01,02 & 03
Logstash concanates all configs into one starting from the first alphanumerically.
First should come inputs, then filters and last outputs.