Log-stash not showing output in stdout

Elastic Stack Logstash
1

config -> my config file

logstash -f "C:\busapps\drsb\gbl0\logstash\7.0.0\bin\pipelines\logstash.conf"

above is my command

below is the output :

[2020-03-14T10:43:12,670][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2020-03-14T10:43:12,701][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.0.0"}
[2020-03-14T10:43:19,873][INFO ][logstash.javapipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, :thread=>"#<Thread:0x1ad1dde5 run>"}
[2020-03-14T10:43:20,433][INFO ][logstash.inputs.file ] No sincedb_path set, generating one based on the "path" setting {:sincedb_path=>"C:/busapps/drsb/gbl0/logstash/7.0.0/data/plugins/inputs/file/.sincedb_4f82f2aa7ccc5e22dc845052d595113f", :path=>["C:\busdata\samplelogs\2020-03-10_Robot.log"]}
[2020-03-14T10:43:20,519][INFO ][logstash.javapipeline ] Pipeline started {"pipeline.id"=>"main"}
[2020-03-14T10:43:20,605][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>}
[2020-03-14T10:43:20,605][INFO ][filewatch.observingtail ] START, creating Discoverer, Watch with file and sincedb collections
[2020-03-14T10:43:21,137][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

ideally its should process / simply display the output file (As there is no filter ) . but nothing is happening.. only pipeline started !

2

Hi there,

my guess is this is not the first time you try to process that file and since you didn't specify in your input a null parameter for the sincedb_path (sincedb_path => "/dev/null" for linux and sincedb_path => "NUL" for windows if I'm not wrong), Logstash thinks it has already processed that file, that's why you don't see anything.

Try cloning that file, changing its name and changing the filename in the input section of logstash, adding the sincedb_path => "NUL" and see if it works.

1 Like
3

thanks first of all , i changed the file name and included the code for "NUL" still the same . any other thoughts.

Sending Logstash logs to C:/busapps/drsb/gbl0/logstash/7.0.0/logs which is now configured via log4j2.properties
[2020-03-14T14:00:37,917][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2020-03-14T14:00:37,932][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.0.0"}
[2020-03-14T14:00:45,006][INFO ][logstash.javapipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, :thread=>"#<Thread:0x46dce278 run>"}
[2020-03-14T14:00:45,540][INFO ][logstash.javapipeline ] Pipeline started {"pipeline.id"=>"main"}
[2020-03-14T14:00:45,629][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>}
[2020-03-14T14:00:45,645][INFO ][filewatch.observingtail ] START, creating Discoverer, Watch with file and sincedb collections
[2020-03-14T14:00:46,063][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

4

infact i deleted the sincedb file , and copied the same log file and named it differently. i went on to even reduce the content of the log file( assuming offset will affect the execution ) .included a "ignore_older=>" 0 but still

C:\busapps\drsb\gbl0\logstash\7.0.0\bin>logstash -f "C:\busapps\drsb\gbl0\logstash\7.0.0\bin\pipelines\logstash.conf"
Sending Logstash logs to C:/busapps/drsb/gbl0/logstash/7.0.0/logs which is now configured via log4j2.properties
[2020-03-14T15:16:36,820][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2020-03-14T15:16:36,836][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.0.0"}
[2020-03-14T15:16:43,958][INFO ][logstash.javapipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, :thread=>"#<Thread:0x3f200de7 run>"}
[2020-03-14T15:16:44,618][INFO ][logstash.javapipeline ] Pipeline started {"pipeline.id"=>"main"}
[2020-03-14T15:16:44,713][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>}
[2020-03-14T15:16:44,728][INFO ][filewatch.observingtail ] START, creating Discoverer, Watch with file and sincedb collections
[2020-03-14T15:16:45,185][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

@yaauie any thoughts ? i am tagging you as i found your reply on a similar issue (in git gub) :slight_smile: i hope u dont mind.

5

Do not use backslash in the path option of a file input. It is interpreted as an escape. Use forward slash (or else use \\).

1 Like
6

ok new path ,

path => "C:/Users/G703065T1/Desktop/log.txt"

new output

C:\busapps\drsb\gbl0\logstash\7.0.0\bin>logstash -f "C:\busapps\drsb\gbl0\logstash\7.0.0\bin\pipelines\logstash.conf"
Sending Logstash logs to C:/busapps/drsb/gbl0/logstash/7.0.0/logs which is now configured via log4j2.properties
[2020-03-14T18:28:40,221][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2020-03-14T18:28:40,237][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.0.0"}
[2020-03-14T18:28:47,850][INFO ][logstash.javapipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, :thread=>"#<Thread:0x35ffeae2 run>"}
[2020-03-14T18:28:48,411][INFO ][logstash.javapipeline ] Pipeline started {"pipeline.id"=>"main"}
[2020-03-14T18:28:48,512][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>}
[2020-03-14T18:28:48,543][INFO ][filewatch.observingtail ] START, creating Discoverer, Watch with file and sincedb collections
[2020-03-14T18:28:49,259][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

content of the file log.txt is "abcd" thats all. its a simple string!

7

my wonder is , in the output section there is no reference of the input file or its path etc., is it first of all processing or not ? infact when i gave "//" it threw some error

File paths must be absolute, relative path specified: C:\Users\G703065T1\Desktop\log.txt>, :backtrace=>["C:/busapps/drsb/gbl0/logstash/7.0.0/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/logstash/inputs/file.rb:269:in block in register'", "org/jruby/RubyArray.java:1792:in each'", "C:/busapps/drsb/gbl0/logstash/7.0.0/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/logstash/inputs/file.rb:267:in register'", "C:/busapps/drsb/gbl0/logstash/7.0.0/logstash-core/lib/logstash/java_pipeline.rb:191:in block in register_plugins'", "org/jruby/RubyArray.java:1792:in each'", "C:/busapps/drsb/gbl0/logstash/7.0.0/logstash-core/lib/logstash/java_pipeline.rb:190:in register_plugins'", "C:/busapps/drsb/gbl0/logstash/7.0.0/logstash-core/lib/logstash/java_pipeline.rb:280:in start_inputs'", "C:/busapps/drsb/gbl0/logstash/7.0.0/logstash-core/lib/logstash/java_pipeline.rb:244:in start_workers'", "C:/busapps/drsb/gbl0/logstash/7.0.0/logstash-core/lib/logstash/java_pipeline.rb:145:in run'", "C:/busapps/drsb/gbl0/logstash/7.0.0/logstash-core/lib/logstash/java_pipeline.rb:104:in block in start'"], :thread=>"#<Thread:0x1721e33d run>"}

8 
input
{


stdin{}

}

filter{
  grok{
    match => {"message"=>"%{LOGLEVEL:loglevel}"}
    overwrite => ["message"]
  }
}
output
{


  stdout{}

}

above configuration file solved the problem .

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.