Hi,
I'm trying to parse a log file with logstash to put it in an elasticsearch database, but I have the following error:
Elasticsearch::Transport::Transport::Error: Cannot get new connection from pool.
I'm using elasticsearch 2.4.0 and logstash 2.4.0
This is a line example of my log file:
[Wed Oct 05 18:01:10 CEST 2016] [IP:109.7.92.64] [ID:spersyn_] [TICKET:ST-66537-ZP7ClBVOTMW4ffq3CRCD-cas.univ-valenciennes.fr] [SERVICE:https://portail.univ-valenciennes.fr/Login] [USER-AGENT:Mozilla/5.0 (Windows NT 10.0; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0]
This is my logstash conf file:
input {
stdin { }
}
filter {
grok {
match => [ "message", "[%{DATA:requestdate}] [IP:%{IPORHOST:clientip}] [ID: {0,10}%{NOTSPACE:uid} {0,10}] [TICKET:%{NOTSPACE}] [SERVICE:%{DATA:[service][request]}] [USER-AGENT:%{DATA:[agent]}]" ]
}
grok {
match => [ "message", "[%{DATA}] [IP:%{IPORHOST}] [ID: {0,10}%{NOTSPACE} {0,10}] [TICKET:%{NOTSPACE}] [SERVICE:%{URIPROTO:[service][proto]}://(?:%{USER:[service][user]}(?::[^@]*)?@)?(?:%{URIHOST:[service][host]}$
}
if "_grokparsefailure" not in [tags] {
if [uid] =~ /^http.? {
mutate {
add_field => { "not_user" => "%{uid}" }
}
} else {
mutate {
lowercase => [ "uid" ]
}
elasticsearch {
hosts => ["agimus1", "agimus2", "agimus3"]
query => "_type:ldap AND _id:%{uid}"
sort => ""
fields => {
'eduPersonPrimaryAffiliation' => 'eduPersonPrimaryAffiliation'
'supannEtuCursusAnnee' => 'supannEtuCursusAnnee'
'supannEtuSecteurDisciplianire' => 'supannEtuSecteurDisciplianire'
'supannAffectation' => 'supannAffectation'
}
}
}
}
}
output {
}
If you have any solution can you share it to me?
Thanks