JDBC SQL and Inner and aggregate plug in


(Dave) #1

Hi, first of all congratulations on your site and documentation

I have a problem with my configuration, everything works fine, but there is something I can't accomplish

I have this SQL results

|clipid|ClientCode|Title|

where

clipid is an int value, unique
clientcode is a value where can be many
text is the text of the clipid

I need something like this

clipid = 345
clientcodes {
clientcode = 2
clientcode = 3
cliencode = 4
}
text = "blablabla"

for some reason only I can save 2 clientcodes, and no more

this is my config

   filter {
 aggregate {
   task_id => "%{clipid}"
   code => "
     map['clipid'] = event.get('clipid')
     map['clientcodes'] ||= []
     map['clientcodes'] << {'clientcode' => event.get('clientcode')}
     event.cancel()
   "
   push_previous_map_as_event => true
   timeout => 10
 }

}

output {

elasticsearch {
hosts => ["http://192.168.2.24:9200"]
index => "clipsx"
document_type => "jdbccon"
document_id => "%{clipid}"
doc_as_upsert => true
action => "update"
}

stdout { codec => rubydebug }
}

and this is what Kibana show me , notice I do not have the single text

{
"_index": "clipsx",
"_type": "jdbccon",
"_id": "45143427",
"_version": 2,
"_score": null,
"_source": {
"@version": "1",
"clipid": 45143427,
"@timestamp": "2018-11-22T18:10:57.811Z",
"clientcodes": [
{
"clientcode": 64
},
{
"clientcode": 50181 #this is a number that needs to be updated
}
],
"tags": [
"_aggregatefinalflush"
]
},
"fields": {
"@timestamp": [
"2018-11-22T18:10:57.811Z"
]
},
"sort": [
1542910257811
]
}

Can you help me please?