Is this an OK addition of doc_values : true in elasticsearch template with logstash?

Elastic Stack Elasticsearch
1

Hi,

got a little tired of memory issues and figured I'd try enabling doc_value and see if this helps (eventually as indexes are created).

This is for a central log ELK stack, kibana3 & 4. ES 1.7.

Should I add it in more / less places in the template?

The template (pastebin: elasticsearch-template.json from logstash with doc_value - Pastebin.com ):

{
"template" : "logstash-",
"aliases" : {
"alias1" : {},
"dcachebilling" : {
"filter" : {
"term" : {
"type" : "dcache_billing"
}
}
}
},
"settings" : {
"index.refresh_interval" : "5s",
"number_of_shards" : 4
},
"mappings" : {
"default" : {
"_all" : {"enabled" : true, "omit_norms" : true, "doc_value" : true},
"dynamic_templates" : [ {
"message_field" : {
"match" : "message",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string", "index" : "analyzed", "omit_norms" : true, "doc_value" : true
}
}
}, {
"string_fields" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string", "index" : "analyzed", "omit_norms" : true, "doc_value" : true,
"fields" : {
"raw" : {"type": "string", "index" : "not_analyzed", "ignore_above" : 256, "doc_value" : true}
}
}
}
} ],
"properties" : {
"@version": { "type": "string", "index": "not_analyzed" },
"type": { "type": "string", "index": "not_analyzed" },
"geoip" : {
"type" : "object",
"dynamic": true,
"path": "full",
"properties" : {
"location" : { "type" : "geo_point" }
}
}
}
}
}
}

// mart

2

IIRC doc values only work for non-analyzed fields.

3

@magnusbaeck is correct.

4

Thanks.

I tried to parse the template, are these statements true ?:

the _all mapping - all fields will with this one get doc_value enabled and whatever the default of index: is.

  • Except that within this mapping there's an exception for the "message" field - setting it to analyzed.
    the string_fields mapping sets "*" to analyzed strings (and doc_value but that's probably without effect).
  • Has an exception for the raw fields which are not_analyzed.

Making only these fields analyzed fields with doc_value enabled:

  • .raw fields
  • the two not_analyzed fields in the "properties" section of the template

?