Logstash, Filebeat and Winlogbeat


(Marcos Felix) #1

Is there any good tutorials out there with good configuration to install these tools so that I can ship event logs to my Kibana interface?
Thanks in advance


(Bill McConaghy) #2

Moved to Logstash forum as you will likely get a better answer there.


(Magnus Bäck) #3

I believe there's pretty good starter documentation on elastic.co. If it's incomplete please tell us what's missing.


(Marcos Felix) #4

do you know how I can update ELK without having to reinstall everything?


(Marcos Felix) #5

I am following the conf file in elastic website:

input {
  file {
    path => "/tmp/access_log"
    start_position => "beginning"
  }
}

filter {
  if [path] =~ "access" {
    mutate { replace => { "type" => "apache_access" } }
    grok {
      match => { "message" => "%{COMBINEDAPACHELOG}" }
    }
  }
  date {
    match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
  }
  stdout { codec => rubydebug }
}

I don't have a access_log dir, should I make one? this is the dir that will keep all apache logs right? if someone doesn't have an apache_log is there an alternative?


(Magnus Bäck) #6

do you know how I can update ELK without having to reinstall everything?

The easiest way of upgrading the Elastic stack components depends on your operating system and how you've installed the software.


(Magnus Bäck) #7

I don't have a access_log dir, should I make one? this is the dir that will keep all apache logs right? if someone doesn't have an apache_log is there an alternative?

What are you trying to do here?


(Marcos Felix) #8

I have installed using wget, I tried with the key but didn't work.
I am following the logstash config file. It works, I created the file and added the log samples there. Logstash is processing it and adding to Kibana. I will update here if anything else comes up


(Marcos Felix) #9

I am getting this error attempt to configtest the syslog from: https://www.elastic.co/guide/en/logstash/current/config-examples.html

root@root1-virtual-machine:/etc/logstash/conf.d# /usr/share/logstash/bin/logstash -f logstash-syslog.conf
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Sending Logstash's logs to /usr/share/logstash/logs which is now configured via log4j2.properties[2018-07-06T14:02:42,979][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-07-06T14:02:43,121][FATAL][logstash.runner] Logstash could not be started because there is already another instance using the configured data directory.  If you wish to run multiple instances, you must change the "path.data" setting.
[2018-07-06T14:02:43,148][ERROR][org.logstash.Logstash] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

There are no instance running, I only telnet from another shell as the tutorial said.


(Magnus Bäck) #10

As the error message

Logstash could not be started because there is already another instance using the configured data directory. If you wish to run multiple instances, you must change the "path.data" setting.

indicates you can't run multiple Logstash instances that share the same data directory. The path.data setting in logstash.yml or the --path.data command line option can change the data directory.


(Marcos Felix) #11

I have used that, but it only works once and then the error persists.


(Marcos Felix) #12

I have been following the tutorial to get Winlogbeat installed, and I got stuck on this bit:
https://www.elastic.co/guide/en/beats/winlogbeat/master/winlogbeat-template.html#load-template-manually.
Whenever I run this command:

PS > .\winlogbeat.exe setup --template -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]'

I get this error:

Get-Process : Parameter cannot be processed because the parameter name 'E' is ambiguous. Possible matches include:
-ErrorAction -ErrorVariable.
At line:1 char:40
+ PS > .\winlogbeat.exe setup --template -E output.logstash.enabled=fal ...
+                                        ~~
    + CategoryInfo          : InvalidArgument: (:) [Get-Process], ParameterBindingException
    + FullyQualifiedErrorId : AmbiguousParameter,Microsoft.PowerShell.Commands.GetProcessCommand

Any clues?


(Magnus Bäck) #13

No idea. Asking in the Winlogbeat category might be more fruitful.


(system) #14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.