I am trying to execute logstash.bat with version 5.6 and 6.0 on a windows 10 machine.
Following is the conf file:
input {
file {
path => "D:/access_log113.log"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
grok{
match=>{
"message"=>"%{IP:clientip} - - [%{NOTSPACE:date} -%{INT}] "%{WORD:action} /%{WORD}/%{WORD}/%{NOTSPACE:login} %{WORD:protocol}/%{NUMBER:protocolNum}" %{NUMBER:status} %{NUMBER} "%{NOTSPACE}" "%{NOTSPACE:client} (%{WORD}; %{WORD:clientOs}%{GREEDYDATA}"
}
add_field=>{
"eventName"=>"grok"
}
}
geoip {
source => "clientip"
}
}
output {
stdout { codec => json }
}
Following is the log file I am trying to parse
172.26.214.3 - - [14/Feb/2017:11:35:56 +0530] "POST /perfservice/j_spring_cas_security_check HTTP/1.1" 200 -
172.26.214.3 - - [14/Feb/2017:11:48:38 +0530] "GET /perfservice/ HTTP/1.1" 200 4050
172.26.214.3 - - [14/Feb/2017:11:48:38 +0530] "GET /perfservice/images/TaaS 2.jpg HTTP/1.1" 200 30037
172.26.214.3 - - [14/Feb/2017:11:48:38 +0530] "GET /perfservice/images/background-homenew.png HTTP/1.1" 200 8631
172.26.214.3 - - [14/Feb/2017:11:48:38 +0530] "GET /perfservice/images/PerformanceBanner.jpg HTTP/1.1" 200 100516
172.26.214.3 - - [14/Feb/2017:11:48:44 +0530] "GET /perfservice/newlauncherExecution HTTP/1.1" 302 -
172.36.214.4 - - [14/Feb/2017:11:35:56 +0530] "POST /perfservice/j_spring_cas_security_check HTTP/1.1" 200 -
172.36.214.5 - - [14/Feb/2017:11:48:38 +0530] "GET /perfservice/ HTTP/1.1" 200 4050
172.36.214.3 - - [14/Feb/2017:11:48:38 +0530] "GET /perfservice/images/TaaS 2.jpg HTTP/1.1" 200 30037
172.36.217.3 - - [14/Feb/2017:11:48:38 +0530] "GET /perfservice/images/background-homenew.png HTTP/1.1" 200 8631
172.36.214.3 - - [14/Feb/2017:11:48:38 +0530] "GET /perfservice/images/PerformanceBanner.jpg HTTP/1.1" 200 100516
173.36.215.3 - - [14/Feb/2017:11:48:44 +0530] "GET /perfservice/newlauncherExecution HTTP/1.1" 302 -
The pipeline main starts but there is no output. Neither does it print to stdout, nor it creates an output file when i mention file path in conf file output field.
Following is the debug command line info I am receiving:
D:\ELKStack\logstash-6.0.1\logstash-6.0.1\bin>logstash.bat -f logstash1.conf --debug
Sending Logstash's logs to D:/ELKStack/logstash-6.0.1/logstash-6.0.1/logs which is now configured via log4j2.properties
[2017-12-14T11:45:09,207][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"D:/ELKStack/logstash-6.0.1/logstash-6.0.1/modules/fb_apache/configuration"}
[2017-12-14T11:45:09,210][DEBUG][logstash.plugins.registry] Adding plugin to the registry {:name=>"fb_apache", :type=>:modules, :class=>#<LogStash::Modules::Scaffold:0x28e9c8f @module_name="fb_apache", @directory="D:/ELKStack/logstash-6.0.1/logstash-6.0.1/modules/fb_apache/configuration", @kibana_version_parts=["6", "0", "0"]>}
[2017-12-14T11:45:09,212][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"D:/ELKStack/logstash-6.0.1/logstash-6.0.1/modules/netflow/configuration"}
[2017-12-14T11:45:09,212][DEBUG][logstash.plugins.registry] Adding plugin to the registry {:name=>"netflow", :type=>:modules, :class=>#<LogStash::Modules::Scaffold:0x1b556461 @module_name="netflow", @directory="D:/ELKStack/logstash-6.0.1/logstash-6.0.1/modules/netflow/configuration", @kibana_version_parts=["6", "0", "0"]>}
[2017-12-14T11:45:09,270][DEBUG][logstash.config.source.multilocal] Reading pipeline configurations from YAML {:location=>"D:/ELKStack/logstash-6.0.1/logstash-6.0.1/config/pipelines.yml"}
[2017-12-14T11:45:09,278][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2017-12-14T11:45:09,380][DEBUG][logstash.agent ] Agent: Configuring metric collection
[2017-12-14T11:45:09,389][DEBUG][logstash.instrument.periodicpoller.os] PeriodicPoller: Starting {:polling_interval=>5, :polling_timeout=>120}
[2017-12-14T11:45:09,419][DEBUG][logstash.instrument.periodicpoller.jvm] PeriodicPoller: Starting {:polling_interval=>5, :polling_timeout=>120}
[2017-12-14T11:45:09,485][DEBUG][logstash.instrument.periodicpoller.persistentqueue] PeriodicPoller: Starting {:polling_interval=>5, :polling_timeout=>120}
[2017-12-14T11:45:09,492][DEBUG][logstash.instrument.periodicpoller.deadletterqueue] PeriodicPoller: Starting {:polling_interval=>5, :polling_timeout=>120}
[2017-12-14T11:45:09,507][DEBUG][logstash.agent ] starting agent
[2017-12-14T11:45:09,520][DEBUG][logstash.agent ] Starting puma
[2017-12-14T11:45:09,523][DEBUG][logstash.agent ] Trying to start WebServer {:port=>9600}
[2017-12-14T11:45:09,546][DEBUG][logstash.config.source.local.configpathloader] Skipping the following files while reading config since they don't match the specified glob pattern {:files=>["D:/ELKStack/logstash-6.0.1/logstash-6.0.1/bin/cpdump", "D:/ELKStack/logstash-6.0.1/logstash-6.0.1/bin/ingest-convert.sh", "D:/ELKStack/logstash-6.0.1/logstash-6.0.1/bin/logstash", "D:/ELKStack/logstash-6.0.1/logstash-6.0.1/bin/logstash-plugin", "D:/ELKStack/logstash-6.0.1/logstash-6.0.1/bin/logstash-plugin.bat", "D:/ELKStack/logstash-6.0.1/logstash-6.0.1/bin/logstash.bat", "D:/ELKStack/logstash-6.0.1/logstash-6.0.1/bin/logstash.lib.sh", "D:/ELKStack/logstash-6.0.1/logstash-6.0.1/bin/ruby", "D:/ELKStack/logstash-6.0.1/logstash-6.0.1/bin/setup.bat", "D:/ELKStack/logstash-6.0.1/logstash-6.0.1/bin/system-install"]}
[2017-12-14T11:45:09,551][DEBUG][logstash.api.service ] [api-service] start
[2017-12-14T11:45:09,554][DEBUG][logstash.config.source.local.configpathloader] Reading config file {:config_file=>"D:/ELKStack/logstash-6.0.1/logstash-6.0.1/bin/logstash1.conf"}
[2017-12-14T11:45:09,585][DEBUG][logstash.agent ] Converging pipelines
[2017-12-14T11:45:09,587][DEBUG][logstash.agent ] Needed actions to converge {:actions_count=>1}
[2017-12-14T11:45:09,592][DEBUG][logstash.agent ] Executing action {:action=>LogStash::PipelineAction::Create/pipeline_id:main}
[2017-12-14T11:45:09,665][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
Need help on what the possible issue could be. Pipeline main starts but I receive no output.