Hi,
I am using filebeat with Logstash output. Please find below config files:
filebeat.yml
#=========================== Filebeat prospectors =============================
filebeat.prospectors:
Each - is a prospector. Most options can be set at the prospector level, so
you can use different prospectors for various configurations.
Below are the prospector specific configurations.
-
input_type: log
Paths that should be crawled and fetched. Glob based paths.
paths:
- /abc/xyz/authenticateb_metrics.log*
#================================ Outputs =====================================
Configure what outputs to use when sending the data collected by the beat.
Multiple outputs may be used.
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
Array of hosts to connect to.
#hosts: ["localhost:9200"]
Optional protocol and basic auth credentials.
#protocol: "https"
#username: "elastic"
#password: "changeme"
#----------------------------- Logstash output --------------------------------
#output.logstash:
The Logstash hosts
#hosts: ["localhost:5044"]
hosts: ["10.xx.xx.xx:96xx"]
Optional SSL. By default is off.
List of root certificates for HTTPS server verifications
#ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
Certificate for SSL client authentication
#ssl.certificate: "/etc/pki/client/cert.pem"
Client Certificate Key
#ssl.key: "/etc/pki/client/cert.key"
Logstash config file:
input {
beats {
type => abc
port => 96xx
client_connectivity_timeout => 120
}
}
filter {
grok { match => ["message", "%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{GREEDYDATA}%{SPACE}TrxId:%{UUID:id}%{SPACE}%{GREEDYDATA}%{SPACE}%{WORD}.%{WORD:method}.%{WORD:class} %{WORD}: %{INT:milliseconds}"]}
mutate {
convert => {"milliseconds" => "integer"}
}
date {
locale => "en"
match => [ "timestamp", "yyyy-MM-dd mm:ss:SS,ZZZ", "ISO8601" ]
target => "timestamp"
}
}
output {
elasticsearch {
hosts => ["10.xx.xx.xx:92xx", "10.xx.xx.xx:92xx"]
index => "xyz-%{+YYYY.MM.dd}"
}
stdout { codec => rubydebug }
}
I am getting error (/var/log/filebeat) :
2017-10-05T13:45:02-07:00 ERR Connecting error publishing events (retrying): Get http://10.xx.xx.xx:96xx: read tcp 10.xx.xx.xx:51576->10.xx.xx.xx:xxxx: read: connection reset by peer
2017-10-05T13:45:31-07:00 INFO Non-zero metrics in the last 30s: libbeat.es.publish.read_errors=1 libbeat.es.publish.write_bytes=124
2017-10-05T13:45:34-07:00 ERR Connecting error publishing events (retrying): Get http://10.xx.xx.xx:96xx: read tcp 10.xx.xx.xx:51588->10.xx.xx.xx:96xx: read: connection reset by peer
2017-10-05T13:46:01-07:00 INFO Non-zero metrics in the last 30s: libbeat.es.publish.read_errors=1 libbeat.es.publish.write_bytes=124
I have configured same port on Filebeat and Logstash congig file.
Please help me to solve this...
Thanks!
-Amit