Not able see data in Kibana? Is Logstash pushing to ES

Elastic Stack Logstash
1

I am new to ELK and working on the whole stack

Below are my configs:

vivsingh@blr-lvg2e:/etc/logstash/conf.d$ ls
filter.conf input.conf output.conf

vivsingh@blr-lvg2e:/etc/logstash/conf.d$ cat input.conf output.conf filter.conf
input
{
file {
path => "/tmp/access_log.log"
start_position => "beginning"
}
}

output {
elasticsearch {
hosts => ["localhost:9200"]
}
stdout { codec => rubydebug }
}

filter {
if [path] =~ "access" {
mutate { replace => { "type" => "apache_access" } }
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
}
}
date {
match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
}
}

output {
elasticsearch {
hosts => ["localhost:9200"]
}
stdout { codec => rubydebug }
}

I am checking if something is pushed via
vivsingh@blr-lvg2e:/etc/logstash/conf.d$ curl http://localhost:9200/_cat/indices?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size

What am I doing wrong the file has data in it:

vivsingh@blr-lvg2e:/etc/logstash/conf.d$ head /tmp/access_log.log
i64.242.88.10 - - [07/Mar/2004:16:05:49 -0800] "GET /twiki/bin/edit/Main/Double_bounce_sender?topicparent=Main.ConfigurationVariables HTTP/1.1" 401 12846
64.242.88.10 - - [07/Mar/2004:16:06:51 -0800] "GET /twiki/bin/rdiff/TWiki/NewUserTemplate?rev1=1.3&rev2=1.2 HTTP/1.1" 200 4523
64.242.88.10 - - [07/Mar/2004:16:10:02 -0800] "GET /mailman/listinfo/hsdivision HTTP/1.1" 200 6291
64.242.88.10 - - [07/Mar/2004:16:11:58 -0800] "GET /twiki/bin/view/TWiki/WikiSyntax HTTP/1.1" 200 7352
64.242.88.10 - - [07/Mar/2004:16:20:55 -0800] "GET /twiki/bin/view/Main/DCCAndPostFix HTTP/1.1" 200 5253
64.242.88.10 - - [07/Mar/2004:16:23:12 -0800] "GET /twiki/bin/oops/TWiki/AppendixFileSystem?template=oopsmore¶m1=1.12¶m2=1.12 HTTP/1.1" 200 11382
64.242.88.10 - - [07/Mar/2004:16:24:16 -0800] "GET /twiki/bin/view/Main/PeterThoeny HTTP/1.1" 200 4924
64.242.88.10 - - [07/Mar/2004:16:29:16 -0800] "GET /twiki/bin/edit/Main/Header_checks?topicparent=Main.ConfigurationVariables HTTP/1.1" 401 12851
64.242.88.10 - - [07/Mar/2004:16:30:29 -0800] "GET /twiki/bin/attach/Main/OfficeLocations HTTP/1.1" 401 12851
64.242.88.10 - - [07/Mar/2004:16:31:48 -0800] "GET /twiki/bin/view/TWiki/WebTopicEditTemplate HTTP/1.1" 200 3732

2

Present stats..

vivsingh@blr-lvg2e:~$ curl http://localhost:9200/_cat/indices?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .kibana SjYX9nKTRm2uJ2TumYeM2A 1 0 0 0 259b 259b

Do the output states that logstash is pushing data to ES? Doc count is zero

3

strace shows that logstash fails to send bulk to ES ans raised transport error on ruby:

872 write(5, "{:timestamp=>"2018-05-04T09:33:46.439000-0400", :message=>"Attempted to send a bulk request to Elasticsearch configured at '[\"http://localhost:9200/\"]', but an error occurred and it failed! Are you sure you can reach elasticsearch from this machine using the configuration provided?", :error_message=>"[406] {\"error\":\"Content-Type header [text/plain; charset=UTF-8] is not supported\",\"status\":406}", :error_class=>"Elasticsearch::Transport::Transport::Errors::NotAcceptable", :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/base.rb:146:in __raise_transport_error'\", \"/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/base.rb:256:inperform_request'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/http/manticore.rb:54:in perform_request'\", \"/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/client.rb:125:inperform_request'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-api-1.0.15/lib/elasticsearch/api/actions/bulk.rb:87:in bulk'\", \"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.5.5-java/lib/logstash/outputs/elasticsearch/http_client.rb:53:innon_threadsafe_bulk'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.5.5-java/lib/logstash/outputs/elasticsearch/http_client.rb:38:in bulk'\", \"org/jruby/ext/thread/Mutex.java:149:insynchronize'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.5.5-java/lib/logstash/outputs/elasticsearch/http_client.rb:38:in bulk'\", \"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.5.5-java/lib/logstash/outputs/elasticsearch/common.rb:163:insafe_bulk'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.5.5-java/lib/logstash/outputs/elasticsearch/common.rb:101:in submit'\", \"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.5.5-java/lib/logstash/outputs/elasticsearch/common.rb:86:inretrying_submit'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.5.5-java/lib/logstash/outputs/ela:

vivsingh@blr-lvg2e:~$ telnet localhost 9200
Trying ::1...
Connected to localhost.
Escape character is '^]'.

4

What versions of Logstash and Elasticsearch are you using?

5

vivsingh@blr-lvg2e:~$ curl "http://localhost:9200"
{
"name" : "dYZlRnx",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "nC3fodoPSUq93jhvtofAmw",
"version" : {
"number" : "6.2.4",
"build_hash" : "ccec39f",
"build_date" : "2018-04-12T20:37:28.497551Z",
"build_snapshot" : false,
"lucene_version" : "7.2.1",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}

Kibana version 6.2.4

6

What about Logstash version?

7

Okay guess here is the issue :slight_smile:

vivsingh@blr-lvg2e:~$ /opt/logstash/bin/logstash --version
logstash 2.2.4

Do i need to upgrade ?

And also i am going to upgrade in few mins, could you please check if the other input filter and output looks fine?

Once all pieces are set Kibana should auto populate the data right?

8

Upgraded to vivsingh@blr-lvg2e:~$ /usr/share/logstash/bin/logstash --version
logstash 6.2.4

vivsingh@blr-lvg2e:~$ curl "http://localhost:9200/_cat/indices?v"
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .kibana SjYX9nKTRm2uJ2TumYeM2A 1 0 2 1 10.7kb 10.7kb
yellow open logstash-2018.05.04 GpnkOrS9SXW_cxguPIFm4A 5 1 52 0 42.6kb 42.6kb
yellow open apache OPizXnbOSl6tLrcYW6n6rQ 5 1 0 0 1.1kb 1.1kb

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.