Mutli-log output config help

Elastic Stack Logstash
1

I have multiple logs I want to output to a index with the computer name and filename if its from filebeat and with computername if anything else. Not sure on the best way to accomplish below is my attempt however not sure how to grab the filename and computer name. I setup a custom pattern for teamviewer_id and info.

# Beats -> Logstash -> Elasticsearch pipeline.

input {

 beats {
          	port => 5044
          	host => "10.10.10.10"
       }
      }          

filter {
        grok { patterns_dir => ["./patterns"]

              if [LOGFILENAME] == "Teamviewer11_Logfile.log" {
               match => { "message" => 
                          [
                           "%{DATESTAMP:logtime}%{SPACE}%{INFO:id1}%{SPACE}%{INFO:id2}%{SPACE}%{GREEDYDATA:messagestart}%{SPACE}(?<teamviewerid>%{TEAMVIEWER_ID})%{GREEDYDATA:messageend}", 
                           "%{DATESTAMP:logtime}%{SPACE}%{INFO:id1}%{SPACE}%{INFO:id2}%{GREEDYDATA:message}"    
                          ] 
                        }
                    
                                                             }

              else  if [LOGFILENAME] == "Connections_incoming.txt" {
                         match => { "message" => 
                         [
                          "%{DATESTAMP:logtime}%{GREEDYDATA:message}" 
                         ]
                                  }
                                                                   }


                      date { match => [ "[@metadata][timestamp]", "YYYY/MM/dd HH:mm:ss.SSS" ] 
                           }
             }
       }

output {
          if %{[@metadata][beat]} == "filebeat" {
            elasticsearch {
          hosts => ["http://10.10.10.10:9200"]
          index => "%{[@metadata][beat]}-%{[@metadata][version]}-COMPUTERNAME-LOGFILENAME"
          #user => "elastic"
          #password => "changeme"
                          }
                                    }

          else {
            elasticsearch {
          hosts => ["http://10.10.10.10:9200"]
          index => "%{[@metadata][beat]}-%{[@metadata][version]}-COMPUTERNAME"
          #user => "elastic"
          #password => "changeme"
                         }
                                          }                                
       }
2

Here is one of the log entries from grafana

{"_id":"HNyummsB1EHn_IFjgHK9","_type":"_doc","_index":"filebeat-7.0.0-2019.06.27","@timestamp":["2019-06-27T20:43:18.280Z"],"log":{"file":{"path":"C:\\Program Files (x86)\\TeamViewer\\TeamViewer11_Logfile.log"},"offset":214630},"@version":"1","agent":{"ephemeral_id":"2862da84-cb0f-41ac-9c1a-a412a2d7e111","type":"filebeat","hostname":"VIST","id":"1d6e7573-8d12-4c44-9f51-5312a0faccfd","version":"7.0.0"},"host":{"architecture":"x86_64","hostname":"VIST","name":"VIST","os":{"family":"windows","build":"9600.0","kernel":"6.3.9600.19000 (winblue_ltsb.180410-0600)","platform":"windows","name":"Windows Server 2012 R2 Standard","version":"6.3"},"id":"fe47919d-537a-44d0-9f95-8a2b14c377b2"},"message":"2019/06/27 16:43:16.857 17064 18016 G2 VoIP: Receiver: Terminate","ecs":{"version":"1.0.0"},"tags":["beats_input_codec_plain_applied","_grokparsefailure"],"input":{"type":"log"}}

3

You do not have a field called [LOGFILENAME] so your conditional will not work. The conditional has to be outside of the filter itself, so

filter {
    if (condition) {
         grok {
             [....]
         }
    } else {
        grok {

etc. And if you are getting _grokparsefailure with custom patterns you need to show us what the patterns are if you expect help fixing them.

4

Not asking for help on the existing custom patterns they work. logfilename and computername were just place holders I wasn't sure if there was a metadata option that would just grab those. Thanks I will adjust the condition location.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.