Hi I have got a server installed with ELK
kibana.service - Kibana
Loaded: loaded (/etc/systemd/system/kibana.service; disabled; vendor preset: disabled)
Active: active (running) since Wed 2018-02-14 06:48:43 IST; 8s ago
Main PID: 32630 (node)
CGroup: /system.slice/kibana.service`Preformatted text`
└─32630 /usr/share/kibana/kibana-6.1.3-linux-x86_64/bin/../node/bin/node --no-warnings /usr/share/kibana/kibana-6.1.3-linux-x86_64...
logstash.service - logstash
Loaded: loaded (/etc/systemd/system/logstash.service; enabled; vendor preset: disabled)
Active: active (running) since Wed 2018-02-14 06:46:09 IST; 16min ago
Main PID: 32553 (java)
CGroup: /system.slice/logstash.service
└─32553 /bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnl...
Feb 14 06:46:09 wpace.in systemd[1]: Started logstash.
Feb 14 06:46:09 wpace.in systemd[1]: Starting logstash...
Feb 14 06:46:16 wpace.in logstash[32553]: Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties
elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2018-02-01 00:36:18 IST; 1 weeks 6 days ago
Docs: http://www.elastic.co
Main PID: 31416 (java)
CGroup: /system.slice/elasticsearch.service
├─31416 /bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -...
└─31463 /usr/share/elasticsearch/plugins/x-pack/platform/linux-x86_64/bin/controller
MY logstash conf.d has one one file in the directory
input{
beats{port => "5044"}
}
filter{
}
output{
if [beat][hostname] == "ip-192.168.16.11"{
elasticsearch {
hosts => ["https://wpace.in:9200"]
ssl => true
ssl_certificate_verification => false
user => "elastic"
password => "solution"
manage_template => false
index => "phoneme99__log-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
}
my Logstash log is showing something likethis
[2018-02-14T07:06:17,026][INFO ][org.logstash.beats.BeatsHandler] [local: 0.0.0.0:5044, remote: 192.168.16.11:45345] Handling exception: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 84
I have got Filebeats installed on another server and trying to import all apache logs from there
filebeat.yml
filebeat.prospectors:
# Each - is a prospector. Most options can be set at the prospector level, so
# you can use different prospectors for various configurations.
# Below are the prospector specific configurations.
- type: log
# Change to true to enable this prospector configuration.
enabled: true
# Paths that should be crawled and fetched. Glob based paths.
paths:
- /var/log/httpd/*.log
#- c:\programdata\elasticsearch\logs\*
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
# Array of hosts to connect to.
# hosts: ["localhost:9200"]
# Optional protocol and basic auth credentials.
#protocol: "https"
#username: "elastic"
#password: "changeme"
#----------------------------- Logstash output --------------------------------
#output.logstash:
# The Logstash hosts
hosts: ["192.168.16.12:5044"]
# Optional SSL. By default is off.
# List of root certificates for HTTPS server verifications
#ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
# Certificate for SSL client authentication
#ssl.certificate: "/etc/pki/client/cert.pem"
Filebeat logs on this server are as follows
2018-02-14T06:55:44.430+0530 ERROR pipeline/output.go:74 Failed to connect: Get http://192.168.16.12:5044: read tcp 192.168.16.11:45345->192.168.16.12:5044: read: connection reset by peer
2018-02-14T06:56:40.786+0530 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":460,"time":467},"total":{"ticks":1720,"time":1735,"value":1720},"user":{"ticks":1260,"time":1268}},"info":{"ephemeral_id":"75e6c9f2-b2e0-481c-8bb9-2a8b6f3e1842","uptime":{"ms":3720015}},"memstats":{"gc_next":10905456,"memory_alloc":6569848,"memory_total":60958824}},"filebeat":{"harvester":{"open_files":10,"running":10}},"libbeat":{"config":{"module":{"running":2}},"pipeline":{"clients":9,"events":{"active":4118}}},"registrar":{"states":{"current":7}},"system":{"load":{"1":0.94,"15":0.72,"5":0.74,"norm":{"1":0.235,"15":0.18,"5":0.185}}}}}}
2018-02-14T06:56:44.432+0530 ERROR pipeline/output.go:74 Failed to connect: Get http://192.168.16.12:5044: read tcp 192.168.16.11:45386->192.168.16.12:5044: read: connection reset by peer
2018-02-14T06:57:10.786+0530 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":470,"time":470},"total":{"ticks":1750,"time":1759,"value":1750},"user":{"ticks":1280,"time":1289}},"info":{"ephemeral_id":"75e6c9f2-b2e0-481c-8bb9-2a8b6f3e1842","uptime":{"ms":3750014}},"memstats":{"gc_next":10905328,"memory_alloc":5559944,"memory_total":61312496}},"filebeat":{"harvester":{"open_files":10,"running":10}},"libbeat":{"config":{"module":{"running":2}},"output":{"read":{"errors":1},"write":{"bytes":125}},"pipeline":{"clients":9,"events":{"active":4118,"retry":50}}},"registrar":{"states":{"current":7}},"system":{"load":{"1":0.89,"15":0.73,"5":0.75,"norm":{"1":0.2225,"15":0.1825,"5":0.1875}}}}}}
2018-02-14T06:57:40.786+0530 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":470,"time":472},"total":{"ticks":1760,"time":1766,"value":1760},"user":{"ticks":1290,"time":1294}},"info":{"ephemeral_id":"75e6c9f2-b2e0-481c-8bb9-2a8b6f3e1842","uptime":{"ms":3780013}},"memstats":{"gc_next":10905328,"memory_alloc":5887248,"memory_total":61639800,"rss":-8192}},"filebeat":{"harvester":{"open_files":10,"running":10}},"libbeat":{"config":{"module":{"running":2}},"pipeline":{"clients":9,"events":{"active":4118}}},"registrar":{"states":{"current":7}},"system":{"load":{"1":1.05,"15":0.75,"5":0.8,"norm":{"1":0.2625,"15":0.1875,"5":0.2}}}}}}
But after restarting the service I couldnt find anything in elastic search indices or in Kibana ?
https://wpace.in:9200/_cat/indices?v
It doesnt show up the indices as phoneme99*
Please guide me where i have gone wrong .