No outputs are defined. Please define one under the output section

Elastic Stack Beats
1

Hi i'm very new to filebeat previously i had use log-courier to process my log. i have created the filebeat.yml like this

filebeat:
prospectors:
-
paths:
- C:\elkenter\Data\sample*.csv
document_type: w3c_csv2
config_dir:
registry_file: C:\ProgramData\filebeat\registry
output:
elasticsearch:
enabled: false
hosts: ["localhost:9200"]
logstash:
enabled: true
hosts: ["localhost:10544"]

while i'm start the filbeat its show the error and get stopped. Error is "Error Initialising publisher: No outputs are defined. Please define one under the output section."

This is my logstash file
"input {
beats {
port => 10544
}
}

filter{

if [type] == "w3c_csv2" {
    
	# drop comment lines
if ([message] =~ "Name") {
  drop{}
}
csv {
    columns => ["Name", "volume", "Dept","Unit"]
    separator => ","
}

}
}

output{

if [type] == "w3c_csv2" {
elasticsearch {
index => "logstash-%{[type]}-%{+YYYY.MM.dd}"
hosts => "localhost:9200"
document_type => "%{[@metadata][type]}"
}
}

stdout{
  codec => rubydebug
}

}"

Kindly help me to resolve the issue and unable to send the value from filebeat to logstash.

2

Please see my answer in this thread: CRIT No outputs are defined

3

HI,
I have indented properly. This is my new code its running but no data is sending to logstash
filebeat:
prospectors:
-
paths:
- C:\elkenter\Data\sample*.csv
document_type: w3c_csv2
config_dir:
registry_file: C:\ProgramData\filebeat\registry
output:
elasticsearch:
enabled: false
hosts: ["localhost:9200"]
logstash:
enabled: true
hosts: ["localhost:10544"]

4

Would you mind pasting your filebeat config into http://pastie.org/?lang=yaml and sharing the link here?

5

Filebeat config file pasted URL http://pastie.org/10677824

6

Thanks. Would you mind adding this section to your filebeat config for debugging?

output:
  console:
    pretty: true

This will print any events to the console in addition to sending them to other outputs (e.g. Logstash).

Then, if you run filebeat, do you see any rows from your CSV files being logged to the console as JSON documents?

7

where i can find that in my command prompt itself or else somewhere

8

Remove enabled: true and enabled: false from your configuration file. Those are not valid configuration options and have no effect. Also remove the disabled elasticsearch output from your config file. Like so...

filebeat:
  prospectors:
    -
      paths:
        - C:\elkenter\Data\sample\*.csv
      document_type: w3c_csv2
      config_dir:
  registry_file: C:\ProgramData\filebeat\registry
  config_dir:
output:
  logstash:
     hosts: ["localhost:10544"]
9

I have made andrewkroh even though it showing same thing nothing is sending to logstash. Its cracking my mind.

I have one doubt, my filebeat is right or not?

1 Like
10

If you start Filebeat from the command line with the -e -v -d "*" it will generate debug information that will help identify the problem. This is similar to what Shaunak was suggesting, just another way of getting the events logged to the console.

Open a command prompt.
Start Filebeat in the foreground .\filebeat.exe -c path\to\config.yml -e -v -d "*"

1 Like
11

HI Andrewkroh,
Thank you so much now it working and now i can find where i made mistake.

12

Can you share where you made your mistake so that others might learn from that as well? Thanks

1 Like