Hi @st1988 Welcome the community and thanks for trying the Elastic Stack.
It can be a bit challenging the first time you enter into the stack we are working to make this easier with our new feet
So here is what I do
Macro I make Filebeat to Elasticsearch Work direct first then I route thought Logstash.... if you can not do that thats ok but that is the best way to check if every thing works before routing through logstash
1st) clean up if you have any filbeat indices as the may be not setup correctly .
2nd) Setup Filebeat and the modules dashboards etc. and run it direct t
Follow the instructions on this page Up THROUGH Step 4
You will need to set and configure the PANOS module
This will setup the index templates, dashboards etc.
If you can (fws etc are good) I would run Filebeat directly to elasticsearch using Step 5 just to see if everything works .... if it does Great, take a look see if see the data correctly dashboards etc... if so go ahead and stop. If you can not run directly fine proceed to step 3
3rd) Setup logstash I will put a working sample beats-logstash.conf below.
So install and setup logstash use the conf below, this will basically act as a pass through but will still using any pipelines, index templates, dashboards etc. You can put this pipeline in the conf.d folder if you have installed via a package manager (deb, rpm etc)
start logstash using the conf below. Data should be flowing.
4th) Go back into filebeat and comment out the Kibana and Elasticsearch output sections and enable the logstash output sections in filebeat and then start filebeat... this should the pump the data via logstash to elasticsearch.
Filebear -> Logstash -> Elasticsearch ....
################################################
# beats->logstash->es default config.
################################################
input {
beats {
port => 5044
}
}
output {
if [@metadata][pipeline] {
elasticsearch {
hosts => "http://localhost:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{[@metadata][version]}"
pipeline => "%{[@metadata][pipeline]}"
user => "elastic"
password => "secret"
}
} else {
elasticsearch {
hosts => "http://localhost:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{[@metadata][version]}"
user => "elastic"
password => "secret"
}
}
}