ELK stack elasticsrearch index issue

Elastic Stack Elasticsearch
1

I have installed ELK to analyze our log files in ubuntu 14, using nginx - kibana - elasticsearch -logstsh --- logstash client server (filebeate) nginx connected to kibana with port 5601 ,and elasticsearch.yml is running with settings

network.host: localhost
logstsh working with below configaration , also i have Loaded Kibana Dashboards with beats-dashboards-1.1.0.zip, its loaded [filebeat-]YYYY.MM.DD still i did not getting any logs in kibana, please check the screen short .

ELK
ELK.png1344×529 36.4 KB

</>input {
beats {
port => 5043
type => syslog
ssl => true
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}
}

filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}

output {
elasticsearch {
host => "localhost:9200"
}
}
#I have tried below also
#output {

elasticsearch {

host => "localhost:9200"

sniffing => true

manage_template => false

index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"

document_type => "%{[@metadata][type]}"

}

#}
client system running with filebeat data type syslog file type /var/log/auth-log

filebeat:
prospectors:
paths:
- /var/log/auth.log
input_type: log
document_type: syslog
registry_file: /var/lib/filebeat/registry
output:
logstash:
hosts: ["ELK_server_IP:5043"]
bulk_max_size: 2048
tls:
certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]
shipper:
logging:
files:
rotateeverybytes: 10485760 </>

2

Please format your code using </> icon as explained in this guide and not the citation button. It will make your post more readable.

Or use markdown style like:

```
CODE
```

Please edit your post.

Looks like very old versions... Why did you choose to install that instead of 6.1.2?

3

Thanks for reply , i have updated elastcsreach version from 4.2 to 6.x , and kibana to 5,
still am stuck in Create index pattern, please check the screenshot .

kibana
kibana.png1119×413 30.1 KB

4

Looks like a Kibana 6, no?

5

Did you index anything?

What gives GET _cat/indices?v

6

currently i did not index anything , how can i do index?, basically am in new in ELK
curl -I http://localhost:5601/status
HTTP/1.1 200 OK
kbn-name: kibana
kbn-version: 6.1.2
cache-control: no-cache
Date: Wed, 24 Jan 2018 17:00:51 GMT
Connection: keep-alive

7

You can go in the developer Console in Kibana and do:

DELETE test
PUT test/doc/1
{
  "message": "Elasticsearch is cool"
}
GET test/_search?q=cool

May be read
https://www.elastic.co/guide/en/elasticsearch/reference/6.1/getting-started.html

And specifically
https://www.elastic.co/guide/en/elasticsearch/reference/6.1/_exploring_your_data.html

HTH

8

yes i added default index in elasticsreach using below url reference, and kibana now working fine
https://www.elastic.co/guide/en/elasticsearch/reference/6.1/_exploring_your_data.html

But how can i get my filebeate client logs ??

9

Did you read https://www.elastic.co/guide/en/beats/filebeat/current/index.html ?