Sudo bin/logstash -f /etc/logstash/conf.d/logstash.conf --path.settings /etc/logstash

I am getting error.
Logstash is not really working - Elasticsearch has no feed.
Ubuntu 16.04 LTS,
Java 1.8, ES 6., LS 6.

/usr/share/logstash/bin$ ls -l
total 68
-rwxr-xr-x 1 logstash logstash 152 Aug 17 17:21 benchmark.sh
-rwxr-xr-x 1 logstash logstash 377 Aug 17 17:21 cpdump
-rwxr-xr-x 1 logstash logstash 1086 Aug 17 17:21 dependencies-report
-rwxr-xr-x 1 logstash logstash 155 Aug 17 17:21 ingest-convert.sh
-rwxr-xr-x 1 logstash logstash 2354 Aug 17 17:21 logstash
-rw-r--r-- 1 logstash logstash 2439 Aug 17 17:21 logstash.bat
-rwxr-xr-x 1 logstash logstash 447 Aug 17 17:21 logstash-keystore
-rw-r--r-- 1 logstash logstash 257 Aug 17 17:21 logstash-keystore.bat
-rwxr-xr-x 1 logstash logstash 4871 Aug 17 17:21 logstash.lib.sh
-rwxr-xr-x 1 logstash logstash 448 Aug 17 17:21 logstash-plugin
-rw-r--r-- 1 logstash logstash 260 Aug 17 17:21 logstash-plugin.bat

usr/share/logstash/bin$ logstash --path.settings /etc/logstash -f /etc/logstash/conf.d/logstash.conf
logstash: command not found
Even tried with sudo . Still getting same error.

You need to run ./logstash ....

Thank you... I think I am getting there.. Here is the new screenshot. Elasticsearch not having the logstash feed.
Here is the elastic feed:

curl -XGET 127.0.0.1:9200/_cat/indices?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open ratings L7klfQ_0S4ezn8iWPiSCTA 5 1 100004 0 12.4mb 12.4mb
yellow open movies qkHy6aPDQESEPQEygQCD7Q 5 1 9125 0 1.3mb 1.3mb
yellow open tags RVWyQcYCQHq-HbADA0PRjg 5 1 1296 0 300.8kb 300.8kb

Here is the logstash script run:
/usr/share/logstash/bin$ sudo ./logstash --path.settings /etc/logstash -f /etc/logstash/conf.d/logstash.conf
Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties
[2018-09-05T10:16:11,704][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-09-05T10:16:13,483][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.4.0"}
[2018-09-05T10:16:21,802][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-09-05T10:16:23,043][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2018-09-05T10:16:23,064][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
[2018-09-05T10:16:23,750][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2018-09-05T10:16:23,894][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>6}
[2018-09-05T10:16:23,909][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the type event field won't be used to determine the document _type {:es_version=>6}
[2018-09-05T10:16:23,985][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[2018-09-05T10:16:24,044][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2018-09-05T10:16:24,093][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"default"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2018-09-05T10:16:25,218][INFO ][logstash.inputs.file ] No sincedb_path set, generating one based on the "path" setting {:sincedb_path=>"/var/lib/logstash/plugins/inputs/file/.sincedb_a41128b87621637bbba893f760026e5e", :path=>["/home/sidraj/access_log*"]}
[2018-09-05T10:16:25,296][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0xc159253 run>"}
[2018-09-05T10:16:25,457][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2018-09-05T10:16:25,526][INFO ][filewatch.observingtail ] START, creating Discoverer, Watch with file and sincedb collections
[2018-09-05T10:16:26,456][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

My crystal ball tells me you haven't set the start_position option for the file input. I strongly recommend that you read the file input documentation thoroughly.

Thanks for the reply!
looking into it...in the meanwhile my logstash.conf file has following entry. The input file access_log looks alright.
input
{
file {
path =>["/home/sidraj/access_log"]
start_position => "beginning"
ignore_older => 0
}
}

filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}"}
}
date { match => ["timestamp", "dd/MMM/yyyy:HH:mm:ss Z"]
}
}

output { elasticsearch {
hosts => [ "localhost:9200" ]
}
stdout {
codec => rubydebug
}
}

Tried a whole new conf file. Still getting the error. Below is the sample file from Logstash configuration examples | Logstash Reference [8.11] | Elastic

input { stdin { } }

filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
}
date {
match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
}
}

output {
elasticsearch { hosts => ["localhost:9200"] }
stdout { codec => rubydebug }
}

/usr/share/logstash/bin$ ./logstash -f logstash_f.conf
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[FATAL] 2018-09-05 15:58:12.617 [main] runner - An unexpected error occurred! {:error=>#<ArgumentError: Path "/usr/share/logstash/data" must be a writable directory. It is not writable.>, :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/settings.rb:447:in validate'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:229:in validate_value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:140:in block in validate_all'", "org/jruby/RubyHash.java:1343:in each'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:139:in validate_all'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:278:in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in run'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:237:in run'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in run'", "/usr/share/logstash/lib/bootstrap/environment.rb:73:in '"]}
[ERROR] 2018-09-05 15:58:12.658 [main] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
ubuntusrv:/usr/share/logstash/bin$
Adding path didn't help either:

./logstash --path.settings /etc/logstash -f logstash_f.conf

Saw error:
Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties
2018-09-05 17:00:44,879 main ERROR RollingFileManager (/var/log/logstash/logstash-plain.log) java.io.FileNotFoundException: /var/log/logstash/logstash-plain.log (Permission denied) java.io.FileNotFoundException: /var/log/logstash/logstash-plain.log (Permission denied)

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.