Why newest versions of logstash are not processing any message

Elastic Stack Logstash
1

Hello I've been using logstash version 6.2.4 for some time and today I came with the idea to upgrade to the 6.6.0

Version 6.2.4 Is working fine with any issue. But today when I wanted to do the same stuff of the v 6.2.4 with the v 6.6.0, I showed with the surprise that v 6.6.0 is not doing anything. For example I want to send this csv file with this conf to terminal output:

input{
	file{
			path => "C:\Users\JOSEGA\Documents\Elastic\1DISTCYBERFINANCIAL.csv" 
			start_position => "beginning"
			sincedb_path => "nul"
	}

}

filter {
	csv{
			separator => ","
			columns => ["server_name","application","sub_application_name","job_member_name","order_date","start_time","end_time","run_time",
						"average_runtime","host","agent_elapsed_time","completion_status"]
	}

	mutate{	convert =>  ["run_time", "integer"]}
	mutate{	convert =>  ["average_runtime", "integer"]}
	mutate{	convert =>  ["agent_elapsed_time", "integer"]}
	
	date{
	
			match => ["start_time", "MMMM d yyyy H mm ss"]
	
	}

}

output {
stdout { codec => rubydebug}
}

And version 6.6.0 is not processing any, but when I execute it with v 6.2.4 It works fine. I tried the same thing with v 6.5.4 and the same it doens't work.

It seems logstash starts, but it get stuck with this message in both versions with v 6.6.0 and v 6.5.4,

[2019-01-30T13:25:25,236][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2019-01-30T13:25:25,455][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.5.4"}
[2019-01-30T13:25:31,142][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2019-01-30T13:25:32,131][INFO ][logstash.pipeline        ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x1b709b run>"}
[2019-01-30T13:25:32,179][INFO ][filewatch.observingtail  ] START, creating Discoverer, Watch with file and sincedb collections
[2019-01-30T13:25:32,192][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2019-01-30T13:25:32,823][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

What could be happening?

2

I would start by changing the backslashes in the path to forward slash. If that does not help then enable "--log.level debug" and see what filewatch has to say.

1 Like
3

Hello Badger, you as always rock!!

Thanks so much! it works with the forward slash. But now I'm confuse I have two questions:

  1. It's no suppose that in Windows OS, you need to specify the path with backward slashes? In the cmd terminal I specified the path of the .conf files with backward slashes and it read the file fine.

  2. Why in the version 6.2.4 works with the backward slashes?

4

Yes, cmd.exe expects backslash, but in most other places forward slash works.

My guess is this changed in 6.4.0.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.