Unable to load Configuration file into Logstash

Elastic Stack Logstash
1

I'm trying to load basic configuration file into logstash but logstash doesn't seem to load the files and it's showing error as

Sending Logstash logs to D:/elk/logstash-6.4.1/logs which is now configured via log4j2.properties
[2018-09-20T19:38:54,963][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-09-20T19:38:57,878][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.4.1"}
[2018-09-20T19:39:07,806][ERROR][logstash.inputs.file ] Unknown setting 'position_start' for file
[2018-09-20T19:39:08,377][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Something is wrong with your configuration.", :backtrace=>["D:/elk/logstash-6.4.1/logstash-core/lib/logstash/config/mixin.rb:86:in config_init'", "D:/elk/logstash-6.4.1/logstash-core/lib/logstash/inputs/base.rb:60:ininitialize'", "org/logstash/plugins/PluginFactoryExt.java:233:in plugin'", "org/logstash/plugins/PluginFactoryExt.java:166:inplugin'", "D:/elk/logstash-6.4.1/logstash-core/lib/logstash/pipeline.rb:71:in plugin'", "(eval):8:in'", "org/jruby/RubyKernel.java:994:in eval'", "D:/elk/logstash-6.4.1/logstash-core/lib/logstash/pipeline.rb:49:ininitialize'", "D:/elk/logstash-6.4.1/logstash-core/lib/logstash/pipeline.rb:90:in initialize'", "D:/elk/logstash-6.4.1/logstash-core/lib/logstash/pipeline_action/create.rb:38:inexecute'", "D:/elk/logstash-6.4.1/logstash-core/lib/logstash/agent.rb:309:in `block in converge_state'"]}
[2018-09-20T19:39:09,648][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

I'm currently using Windows 10- 64 bit and JDK 1.8.0_144 and Logstash-6.4.1, Elasticsearch-6.4.1 and Kibana-6.4.1 and my config file is as follows

input{
file{
path => "D:\elk\Logstash-input.log"
position_start => "beginning"
}
}
output{
file{
path => "D:\elk\Logstash-output.log"

}
}

and not only this, even basic command also not executing like

D:\elk\logstash-6.4.1\bin>logstash -e "input{stdin{}} output{stdout()}"

Sending Logstash logs to D:/elk/logstash-6.4.1/logs which is now configured via log4j2.properties
[2018-09-21T12:28:26,506][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
ERROR: Settings 'path.config' (-f) and 'config.string' (-e) can't be used simultaneously.
usage:
bin/logstash -f CONFIG_PATH [-t] [-r] [] [-w COUNT] [-l LOG]
bin/logstash --modules MODULE_NAME [-M "MODULE_NAME.var.PLUGIN_TYPE.PLUGIN_NAME.VARIABLE_NAME=VALUE"] [-t] [-w COUNT] [-l LOG]
bin/logstash -e CONFIG_STR [-t] [--log.level fatal|error|warn|info|debug|trace] [-w COUNT] [-l LOG]
bin/logstash -i SHELL [--log.level fatal|error|warn|info|debug|trace]
bin/logstash -V [--log.level fatal|error|warn|info|debug|trace]
bin/logstash --help
[2018-09-21T12:28:26,818][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

please help with the same, and thanks in advance

2

[2018-09-20T19:39:07,806][ERROR][logstash.inputs.file ] Unknown setting 'position_start' for file

As the error message says, the file input doesn't have a position_start option. You can confirm this by consulting the file input documentation. In there you'll see that there is an option named start_position.

3

thank you,
I changed conf file and executed it again, it showing error like

D:\elk\logstash-6.4.1\bin>logstash -f logstash.conf
Sending Logstash logs to D:/elk/logstash-6.4.1/logs which is now configured via log4j2.properties
[2018-09-21T12:56:57,981][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-09-21T12:56:58,058][FATAL][logstash.runner ] Logstash could not be started because there is already another instance using the configured data directory. If you wish to run multiple instances, you must change the "path.data" setting.
[2018-09-21T12:56:58,078][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

and it will be helpful, if u reply for other command too which I mentioned earlier,
thank you in advance

4

There's a lock file in the data directory. It might be left behind by a Logstash instance that was killed. If you really don't have a Logstash process running in the background you can delete the file.

1 Like
5

we deleted the lock file and ran the configuration file through command prompt, it ran but no logs were generated into the output file specified in the path of config file.

D:\elk\logstash-6.4.1\bin>logstash -f logstash.conf
Sending Logstash logs to D:/elk/logstash-6.4.1/logs which is now configured via log4j2.properties
[2018-09-21T15:39:55,845][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-09-21T15:39:56,816][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.4.1"}
[2018-09-21T15:40:00,397][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-09-21T15:40:01,326][INFO ][logstash.inputs.file ] No sincedb_path set, generating one based on the "path" setting {:sincedb_path=>"D:/elk/logstash-6.4.1/data/plugins/inputs/file/.sincedb_e7a9d45a1a411d9c708db547f359e5f9", :path=>["D:\elk\Logstash-input.log"]}
[2018-09-21T15:40:01,399][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x63a88cfc run>"}
[2018-09-21T15:40:01,559][INFO ][filewatch.observingtail ] START, creating Discoverer, Watch with file and sincedb collections
[2018-09-21T15:40:01,569][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2018-09-21T15:40:02,572][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

is there any wrong with config file?
thanks in advance.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.