How to configure Elasticsearch output

Elastic Stack Beats
1

Hi All,

I am trying to load the log data over logstash to elasticsearch, so i have done the below steps,

  1. installed filebeat and configured "filebeat.yml" --> like below

#----------------------------- Logstash output --------------------------------
output.logstash:
# The Logstash hosts
hosts: ["localhost:9600"]

  1. after I have enabled logstash module over cmd,
  2. then trying to setup the filebeat in cmd but getting the below error in cmd

C:\Program Files\Filebeat>filebeat.exe setup
"Exiting: Template loading requested but the Elasticsearch output is not configured/enabled"

but I want to transfer the log file over logstash pipeline to elasticsearch,

Thanks,
Nagaraj,

2

To send data to Logstash you need to add a beats input into your Logstash config and send data to the port this is configured with. Post 9600 is not the correct one.

In order to upload dashboards and templates I believe Filebeat need to connect directly to Elasticsearch during the setup. Once this has completed you should be able to send data via Logstash.

1 Like
3

You could set setup.template.enabled to false, so it won't attempt to load it.

Documentation links:

https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-template.html
https://www.elastic.co/guide/en/beats/filebeat/current/configuration-template.html

1 Like
4

yeah I have added the beat port 5044 in "logstash.config" file, like below

input {
beats {
port => "5044"
}
}

that "port:9600" I am using for logstash api console,

And filebeat can able to connect elasticsearch directly for posting log file and others, but when I am trying to load the log file over logstash at the time I am getting error,

5

The Logstash output in the Filebeat config should specify port 5044, not 9600.

6

Are you actually going to enrich your log data in Logstash? If you just use Logstash as a proxy to pass the data through, you might want to consider using HAProxy instead to tunnel the connection from the beat to elasticsearch.

This is what I just did on one of my setups, where I had to have a jump host between my machines running their beats and the elasticsearch endpoint. Logstash would have been overkill, since I didn't want to do anything with the data but pass it through.

1 Like
7

I am just filtering some entry in source log file over logstash then put the filtered entry in to elasticsearch, for example below is the log file content,

"2019-03-20 01:55:54, Info DISM API: PID=3720 TID=8956 Leave CCommandThread::CommandThreadProcedureStub - CCommandThread::CommandThreadProcedureStub
2019-03-20 01:55:54, Info DISM API: PID=3720 TID=10736 Deleted g_internalDismSession - DismShutdownInternal
2019-03-20 01:55:54, Info DISM API: PID=3720 TID=10736 Shutdown SessionTable - DismShutdownInternal
2019-03-20 01:55:54, Info DISM API: PID=3720 TID=10736 Leave DismShutdownInternal - DismShutdownInternal
2019-03-20 01:55:54, Info DISM API: PID=3720 TID=10736 DismApi.dll: - DismShutdownInternal
2019-03-20 01:55:54, Info DISM API: PID=3720 TID=10736 DismApi.dll: <----- Ending DismApi.dll session -----> - DismShutdownInternal
2019-03-20 01:55:54, Info DISM API: PID=3720 TID=10736 DismApi.dll: - DismShutdownInternal"

so I am filtering like content message = Ending DismApi.dll session -----> - DismShutdownInternal

the only entry need to feed it in to elastic search over logstash,

This is what I am trying,

@Christian_Dahlqvist I have changed the port to "5044", now also getting the same error " Exiting: Template loading requested but the Elasticsearch output is not configured/enabled"

8

When you run setup, you can not connect through Logstash but must configure an Elasticsearch output.

1 Like
9

in filebeat.yml file if I use output as "elasticserach" then the "filebeat.exe setup" command is working fine, but when I use output "logstash" with hostname and port [localhost:5044] it is throwing the error like "elasticsearch output is not enabled"

10

yeah at the beginning of configuration step it was going over elasticsearch, so now I am trying to load data from logstash,

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.