Create button do not appear

Elastic Stack Kibana
1

Hello expert,

I have a issue, no create button appear although i fill in the blank is logstash-*.
I install ELK via repo with the most latest version and on CentOS 6 final.
Here is the log when elasticsearch and logstash started:

Logstash:

[root@vmvnsyslog02 logstash]# ./bin/logstash -f
/etc/logstash/conf.d/logstash-syslog.conf

Jul 20, 2015 11:55:41 AM org.elasticsearch.node.internal.InternalNode

INFO: [logstash-vmvnsyslog02.domain.com-8188-11624] version[1.5.1], pid[8188],
build[5e38401/2015-04-09T13:41:35Z]

Jul 20, 2015 11:55:41 AM org.elasticsearch.node.internal.InternalNode

INFO: [logstash-vmvnsyslog02.domain.com-8188-11624] initializing ...

Jul 20, 2015 11:55:41 AM org.elasticsearch.plugins.PluginsService

INFO: [logstash-vmvnsyslog02.domain.com-8188-11624] loaded , sites

Jul 20, 2015 11:55:43 AM org.elasticsearch.node.internal.InternalNode

INFO: [logstash-vmvnsyslog02.domain.com-8188-11624] initialized

Jul 20, 2015 11:55:43 AM org.elasticsearch.node.internal.InternalNode start

INFO: [logstash-vmvnsyslog02.domain.com-8188-11624] starting ...

Jul 20, 2015 11:55:43 AM org.elasticsearch.transport.TransportService doStart

INFO: [logstash-vmvnsyslog02.domain.com-8188-11624] bound_address
{inet[/0:0:0:0:0:0:0:0:9301]}, publish_address {inet[/10.126.122.27:9301]}

Jul 20, 2015 11:55:43 AM org.elasticsearch.discovery.DiscoveryService doStart

INFO: [logstash-vmvnsyslog02.domain.com-8188-11624]
elasticsearch/Hxcfd68aQJSiCN_HJ7CNAQ

Jul 20, 2015 11:55:47 AM
org.elasticsearch.cluster.service.InternalClusterService$UpdateTask run

INFO: [logstash-vmvnsyslog02.domain.com-8188-11624] detected_master [Dragon
Man][RUmrn41rR-iy_6feEmyelA][vmvnsyslog02.domain.com][inet[/127.0.0.1:9300]],
added {[Man-Bull][XQS0OWWuQfmPsnhBEtZ4Kg][vmvnsyslog02.domain.com][inet[/127.0.0.1:9302]],[Dragon
Man][RUmrn41rR-iy_6feEmyelA][vmvnsyslog02.domain.com][inet[/127.0.0.1:9300]],},
reason: zen-disco-receive(from master [[Dragon
Man][RUmrn41rR-iy_6feEmyelA][vmvnsyslog02.domain.com][inet[/127.0.0.1:9300]]])

Jul 20, 2015 11:55:47 AM org.elasticsearch.node.internal.InternalNode start

INFO: [logstash-vmvnsyslog02.domain.com-8188-11624] started

Logstash startup completed

Elasticsearch:

[root@vmvnsyslog02
elasticsearch]# ./bin/elasticsearch start

[2015-07-20 11:44:55,001][INFO
][node
] [Man-Bull] version[1.7.0], pid[7620], build[929b973/2015-07-16T14:31:07Z]

[2015-07-20 11:44:55,001][INFO
][node
] [Man-Bull] initializing ...

[2015-07-20 11:44:55,095][INFO
][plugins
] [Man-Bull] loaded , sites [kopf]

[2015-07-20 11:44:55,135][INFO
][env
] [Man-Bull] using [1] data paths, mounts [[/ (/dev/mapper/VolGroup-lv_root)]],
net usable_space [43gb], net total_space [49gb], types [ext4]

[2015-07-20 11:44:57,923][INFO
][node
] [Man-Bull] initialized

[2015-07-20 11:44:57,923][INFO
][node
] [Man-Bull] starting ...

[2015-07-20 11:44:58,066][INFO
][transport
] [Man-Bull] bound_address {inet[/127.0.0.1:9302]}, publish_address
{inet[localhost/127.0.0.1:9302]}

[2015-07-20 11:44:58,090][INFO
][discovery
] [Man-Bull] elasticsearch/XQS0OWWuQfmPsnhBEtZ4Kg

[2015-07-20 11:45:01,168][INFO
][cluster.service ]
[Man-Bull] detected_master [Dragon Man][RUmrn41rR-iy_6feEmyelA][vmvnsyslog02.domain.com][inet[/127.0.0.1:9300]],
added {[logstash-vmvnsyslog02.domain.com-5883-11624][Y9Xo-KuyTXyUkQ8sRtUBNA][vmvnsyslog02.domain.com][inet[/10.126.122.27:9301]]{client=true,
data=false},[Dragon Man][RUmrn41rR-iy_6feEmyelA][vmvnsyslog02.domain.com][inet[/127.0.0.1:9300]],},
reason: zen-disco-receive(from master [[Dragon
Man][RUmrn41rR-iy_6feEmyelA][vmvnsyslog02.domain.com][inet[/127.0.0.1:9300]]])

[2015-07-20 11:45:01,244][INFO
][http
] [Man-Bull] bound_address {inet[/127.0.0.1:9201]}, publish_address
{inet[localhost/127.0.0.1:9201]}

[2015-07-20 11:45:01,244][INFO
][node
] [Man-Bull] started

[2015-07-20 11:55:27,104][INFO
][cluster.service ]
[Man-Bull] removed {[logstash-vmvnsyslog02.domain.com-5883-11624][Y9Xo-KuyTXyUkQ8sRtUBNA][vmvnsyslog02.domain.com][inet[/10.126.122.27:9301]]{client=true,
data=false},}, reason: zen-disco-receive(from master [[Dragon
Man][RUmrn41rR-iy_6feEmyelA][vmvnsyslog02.domain.com][inet[/127.0.0.1:9300]]])

[2015-07-20 11:55:47,026][INFO ][cluster.service
] [Man-Bull] added {[logstash-vmvnsyslog02.domain.com-8188-11624][Hxcfd68aQJSiCN_HJ7CNAQ][vmvnsyslog02.domain.com][inet[/10.126.122.27:9301]]{client=true,
data=false},}, reason: zen-disco-receive(from master [[Dragon
Man][RUmrn41rR-iy_6feEmyelA][vmvnsyslog02.domain.com][inet[/127.0.0.1:9300]]])

Any help is appreciated,

Best regards,

2

Did you pick the timestamp field as well?

3

Hello,

I even cannot login to kibana to setting timestamp,
Kibana webpage showing: Kibana: Unable to create Kibana index "logstash-*"

kibana.PNG1217×614 46.8 KB

Any help is appreciated,

4

Hello,

I just fresh installed ELK with the latest version via repo, but still no luck to see the create button?
All files configuration is default as guide on https://www.elastic.co.
I do not know where i am wrong.
Any one can help me?

kibana.PNG1286×616 30.6 KB

5

Do you have data in Elasticsearch?

6

Hello,

How it look like?
When i show:
[root@vmsyslog02 ~]# curl 'localhost:9200/_cat/indices?v' health status index pri rep docs.count docs.deleted store.size pri.store.size yellow open .kibana 1 1 1 0 2.5kb 2.5kb
All files configuration are default.

How can i do now? I am newbie.

Thanks & best regards,

7

There's no LS indices which is why you cannot create that in KB.

Have you pushed data through LS?

8

Hello,

I created configuration file in: /etc/logstash/conf.d/logstash-syslog.conf as below:

    input {
  tcp {
    port => 5514
    type => syslog
  }
  udp {
    port => 5514
    type => syslog
  }
}

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:sysl$
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    }
    syslog_pri { }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
  }
}

output {
  elasticsearch { host => localhost }
  stdout { codec => rubydebug }
}

Do i miss any configuration file?

Best regards,

9

Ok, did you send data to those ports?
Is there anything in stdout?

10

Hello,

I send log with syslog type from my Firewall to this port on ELK server. If i testing with stdout i think it fine:

    bin/logstash -e 'input { stdin { } } output { stdout                                                    { codec => rubydebug } }'

Logstash startup completed
{
       "message" => "",
      "@version" => "1",
    "@timestamp" => "2015-07-28T07:24:07.075Z",
          "host" => "vmsyslog02.domain.com"
}
goodnight moon
{
       "message" => "goodnight moon",
      "@version" => "1",
    "@timestamp" => "2015-07-28T07:24:30.839Z",
          "host" => "vmsyslog02.domain.com"
}

But when i checking the file configuration:

[root@vmsyslog02 logstash]# ./bin/logstash -t -f /etc/logstash/conf.d/logstash-syslog.conf
'[DEPRECATED] use `require 'concurrent'` instead of `require 'concurrent_ruby'`
Configuration OK

is this normal error?

And when i check the logstash log, it's showing:

[root@vmsyslog02 logstash]# tail -f logstash.log-20150728
{:timestamp=>"2015-07-27T08:39:30.038000+0700", :message=>"The error reported is: \n  Permission denied - bind(2)"}
{:timestamp=>"2015-07-27T11:42:15.839000+0700", :message=>"The error reported is: \n  Permission denied - bind(2)"}
{:timestamp=>"2015-07-27T13:25:23.173000+0700", :message=>"The error reported is: \n  Permission denied - bind(2)"}
{:timestamp=>"2015-07-27T13:51:31.860000+0700", :message=>"The error reported is: \n  Permission denied - bind(2)"}
{:timestamp=>"2015-07-27T14:40:53.736000+0700", :message=>"SIGTERM received. Shutting down the pipeline.", :level=>:warn}

Seem i had many error?

Any help is appreciated,

11

Ignore the deprecated warning.

I wonder what it is erroring on, is anything else using either of those ports?

12

Ok, i changed to port 5000, here is the port used

[root@vmsyslog02 elasticsearch]# netstat -nulpt | grep :5000
tcp        0      0 :::5000                     :::*                        LISTEN      19194/java
udp        0      0 :::5000                     :::*                                    19194/java

LS configuration file:

input {
  tcp {
    port => 5000
    type => syslog
  }
  udp {
    port => 5000
    type => syslog
  }
}

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSI$
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    }
    syslog_pri { }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
  }
}

output {
  elasticsearch { host => localhost }
  stdout { codec => rubydebug }
}

Also restarted Kibana, Elasticsearch, Logstash, but still no luck have a create button.

Regards,

13

Ignore the Kibana button for now and comment out the elasticsearch output. Are you getting what you expect to stdout? If yes, re-add the elasticsearch output and try again. Pay close attention to the Logstash logs.

14

Hello,

I just comment elasticsearch output on logstash configuration as below:

input {
  tcp {
    port => 5000
    type => syslog
  }
  udp {
    port => 5000
    type => syslog
  }
}

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_prog$
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    }
    syslog_pri { }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
  }
}

output {
#  elasticsearch { host => localhost }
  stdout { codec => rubydebug }
}

After that i checked by:

  #curl 'localhost:9200/_cat/indices?v'
health status index   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   .kibana   1   1          1            0      2.5kb          2.5kb

But still no luck, although restarted ELK.

Any help is appreciated,

15

If you comment out the elasticsearch output you'll (obviously) not get anything into ES. The point of that exercise is to see if messages are flowing through Logstash at all. Are you getting anything on stdout when syslog messages are sent to port 5000?

16

Hello,

Now i can see the Create button.
I did as you recommended and checked configuration file to change something output.
I also get logs from our device. Great job.

Thanks and regards,

17

i have the same problem i dont see the create botton can you explain to me how did you solve it ??