Hi all
I have the following configuration:
input {
beats {
port => "5044"
host => "0.0.0.0"
}
rabbitmq {
host => "localhost"
port => 5672
user => "yyyyyyy"
password => "xxxxxxx"
vhost => "lala"
queue => "lolCalls"
add_field => {
"[@metadata][beat]" => "lolCalls"
"[@metadata][type]" => "event"
}
}
}
filter {
# Data gets transformed here, shouldn't have impact however
}
output {
elasticsearch {
hosts => localhost
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
#stdout { codec => rubydebug { metadata => true } }
}
Everything coming in from either filebeat or metricbeat gets created just fine and is accepting incoming data, status is yellow, but this is my development test server, so that's normal as I have only one node:
GET _cat/indices
yellow open metricbeat-2017.09.05 YVASOlvETmONgcfF88mITg 5 1 134374 0 43.7mb 43.7mb
yellow open filebeat-2017.09.05 Zwn4HLKaQYKec5pA_-fJPw 5 1 1471 0 1006.2kb 1006.2kb
yellow open .kibana hRbZQrUpTh-TBEQmcRG8-A 1 1 35 0 115kb 115kb
However, my lolCalls-(date) index just won't be created. If I enable the stdout as well, I get to see my data just fine:
{
"basicInformation" => {
"endpoint" => "XXXXX",
"siteClassification" => "XXXXX",
"machineId" => "XXXXX",
"siteVariant" => "XXXXX",
"ruid" => "XXXXX",
"frontendId" => 999
},
"curl" => {
"headerSize" => 346,
"preTransferTime" => 0.062061,
"nameLookupTime" => 0.031032,
"downloadSize" => 3417,
"redirectTime" => 0,
"totalTime" => 0.191588,
"connectTime" => 0.037858,
"httpCode" => 200,
"requestSize" => 358,
"startTransferTime" => 0.191565
},
"@timestamp" => 2017-09-05T13:44:54.027Z,
"@metadata" => {
"beat" => "lolCalls",
"type" => "event"
},
"@version" => "1",
"tags" => [
[0] "_geoip_lookup_failure"
],
"APICallResponse" => [
[0] {
"XXXXX" => {
"XXXXX" => [
// A lot of data
]
},
"direct_payment" => 1,
"date" => "2017-09-22"
}
]
}
Where can I begin to look? The rabbitmq connection seems to go fine, as I can filter it and the entry actually gets deleted from the queue.
The only difference between the file- and metricbeat entries is that I had already a template for , but that shouldn't impact the creation of an index right?
Greetings.