Issue on collectd setup!

Elastic Stack Logstash
1

Tried to setup collectd to push my network related stuff to Logstash/Elasticsearch and Kibana. Collectd is pushing logs to logstash. I have verified that by set output to stdout{} option.

But when I redirect the output to elasticsearch, no index/types are created. Means, no data is created on Elasticsearch.

Logstash output config :
output {
elasticsearch {
host => localhost
}
stdout { }
}

I couldn't see any error either in logstash or elasticsearch to move further on this case. Please someone give idea on this.

This is the log when I redirect the output to logstash stdout file:
{"host":"test-srv1","@timestamp":"2015-06-27T14:44:30.247Z","plugin":"cpu","plugin_instance":"0","collectd_type":"cpu","type_instance":"user","value":388319601,"@version":"1","type":"collectd"}{"host":"test-srv1","@timestamp":"2015-06-27T14:44:30.247Z","plugin":"cpu","plugin_instance":"0","collectd_type":"cpu","type_instance":"nice","value":8215544,"@version":"1","type":"collectd"}{"host":"test-srv1","@timestamp":"2015-06-27T14:44:30.247Z","plugin":"cpu","plugin_instance":"0","collectd_type":"cpu","type_instance":"system","value":504912201,"@version":"1","type":"collectd"}{"host":"test-srv1","@timestamp":"2015-06-27T14:44:30.247Z","plugin":"cpu","plugin_instance":"0","collectd_type":"cpu","type_instance":"idle","value":7398660093,"@version":"1","type":"collectd"}{"host":"test-srv1","@timestamp":"2015-06-27T14:44:30.247Z","plugin":"cpu","plugin_instance":"0","collectd_type":"cpu","type_instance":"wait","value":68402900,"@version":"1","type":"collectd"}{"host":"test-srv1","@timestamp":"2015-06-27T14:44:30.247Z","plugin":"cpu","plugin_instance":"0","collectd_type":"cpu","type_instance":"interrupt","value":14303643,"@version":"1","type":"collectd"}{"host":"test-srv1","@timestamp":"2015-06-27T14:44:30.247Z","plugin":"cpu","plugin_instance":"0","collectd_type":"cpu","type_instance":"softirq","value":76156622,"@version":"1","type":"collectd"}{"host":"test-srv1","@timestamp":"2015-06-27T14:44:30.247Z","plugin":"cpu","plugin_instance":"0","collectd_type":"cpu","type_instance":"steal","value":0,"@version":"1","type":"collectd"}

Not sure, why the records are not created in Elasticsearch.

2

That looks fine.

Can you curl localhost:9200 to see the response from ES?

3

@warkolm this is my ELS output,

{
"status" : 200,
"name" : "Node 1",
"version" : {
"number" : "1.3.7",
"build_hash" : "3042293e4b219dfb855a4e6c64241c530d1abeb0",
"build_timestamp" : "2014-12-16T13:59:32Z",
"build_snapshot" : false,
"lucene_version" : "4.9"
},
"tagline" : "You Know, for Search"
}

curl http://localhost:9200/_search?pretty
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 0,
"successful" : 0,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : 0.0,
"hits" : [ ]
}
}

Logstash output config:
output {
elasticsearch {
host => localhost
}
stdout { codec => json }
}

4

Any chance to check this ? .

Not sure why Index are not created in ELS. From the Collectd output, it doesn't mention the index name where the data going to store. Am I missing anything in the config file ? . Please someone help me on this

5

You're running a bit of an old version there, I'd encourage you to upgrade.

It looks like LS is not talking to ES, try using the HTTP protocol in the output and see what you get.

6

Thanks @warkolm . Now index are created in ELS. :slight_smile: , I will test further.