How separate the logs each device

[ramindia (R!)]

Hi

I have Centralized SYSLOG-NG Installed in my network.

I have few hundred devices in the network, including Cisco Switch, Routers, FW and other kind of devices.

All the devices send logs to centralized Logging server that is SYSLOG-NG.

Each Device store in separate folder with the respective IP address as folder name.

Now installed Filebeat on the same server to ship the logs to Logstash.

Problem i have here is i have mentioned filebeat to ship all the logs from source folder /var/log/syslog-ng///*.logs

I am able to see the logs in LogStash and inturn the logs can visulaise in Elasticsearch and Kibana.

But what i was looking here is to configure is filebeat config, how to tag each device information with respective IP before shipping to Logstash, so i can search based on the tag name in Kibana and make alerts.

appreciate guidance and help.

R!

[exekias (Carlos Pérez Aradros)]

Hi @ramindia,

All filebeat log events have a source field, they will give you the source file for the log message. Would that be enough to tell you what's the device sending it in your case?

Best regards

[ramindia (R!)]

Hi

yes i have seen that source have different source filename

but is there any way i can tag each log file name like cisco-3750, cisco-nexus so on.

Example : like below

Cisco-3750
/var/log/syslog-ng/192.168.1.63/2017/05/192.168.1.63-2017-05-01.log

Cisco-Nexus
/var/log/syslog-ng/192.168.1.64/2017/05/192.168.1.64-2017-05-01.log

appreciate help.

R!

[ruflin]

Are you looking for tags / fields config option? https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#configuration-fields

[steffens (Steffen Siering)]

given all information you need are available in the source field, you can use Logstash or Elasticsearch Ingest node grok-processor to extract some information. Just having IPs might be somewhat 'rough' though. If hostname is in the log message, you can extract it from the log message, otherwise you will need some kind of lookup filter adding the missing metadata to the events. e.g. via conditionals in LS or scriptable filters in ES/LS or gsub....