I've been working in this problem for 2 days now and I can't seem to find
the information I need to figure out a solution.
I'm using logstash to parse AWS cloudtrail logs. I'm having trouble
creating an index template that does what I want. Here is the doc in
elasticsearch. The contents of the requestParameters element is dynamic.
Because of this fact, after several hours of docs are created I have tons
of fields for different requestParameters that I don't really want. I
would like the contents of that element to be stored as a single field that
is a string representation of the json blob for that element (i.e.
requestParameters = { "filterSet": { "items": [ { "name": "resource-id",
"valueSet": { "items": [ { "value": "i-3bd4461a"} ] } ] }). (if that is
not possible, I would like the field to not be created at all).
{
"_index": "logstash-2014.01.30",
"_type": "logs",
"_id": "JOGAYVL5RNy5UX7fIVG45w",
"_score": null,
"_source": {
...
"requestParameters": {
"filterSet": {
"items": [
{
"name": "resource-id",
"valueSet": {
"items": [
{
"value": "i-3bd4461a"
}
]
}
}
]
}
},
...
},
...
}
The template I'm currently using is the default logstash template (below).
{
"logstash" : {
"template" : "logstash-",
"order" : 0,
"settings" : {
"index.analysis.analyzer.default.stopwords" : "none",
"index.refresh_interval" : "5s",
"index.analysis.analyzer.default.type" : "standard"
},
"mappings" : {
"default" : {
"dynamic_templates" : [ {
"string_fields" : {
"mapping" : {
"type" : "multi_field",
"fields" : {
"raw" : {
"index" : "not_analyzed",
"ignore_above" : 256,
"type" : "string"
},
"{name}" : {
"index" : "analyzed",
"omit_norms" : true,
"type" : "string"
}
}
},
"match_mapping_type" : "string",
"match" : ""
}
} ],
"properties" : {
"geoip" : {
"dynamic" : true,
"path" : "full",
"properties" : {
"location" : {
"type" : "geo_point"
}
},
"type" : "object"
},
"@version" : {
"index" : "not_analyzed",
"type" : "string"
}
},
"_all" : {
"enabled" : true
}
}
}
}
}
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/23b35b57-e959-4087-a937-2d61df56841a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.