Customize watch and/or watch_history index template

Elastic Stack Elasticsearch
1

Is there a reason why these templates have an order set to Integer.MAX? I'd like to customize these a little bit. Specifically would like to index certain fields in the input.payload so that I can later search on them or set some fields to doc_values.

Was hoping I could just add a higher order template for this but doesn't seem that this will be possible. Are there any other alternatives?

2

The reason for hardening some parts of the watch history schema is to prevent conflicts between different watches. Take the input payload as an example. A payload can be anything, and can change from one watch execution to another... even 2 executions of the same watch... think search inputs yielding 2 different search results, and it gets even worse with more generic inputs like http and simple. For this reason we disable the mappings for the input payload. There's currently no alternative. What we could consider for future feature is to throw the input as text into a single catch all field, say payload_data and make that searchable. Would that work for you?

3

I see your point. Your proposal could work I think for the payload scenario.

What about metadata? That's really my data structures, so I'd like to be able to override the mappings (make them not_analyzed and/or use doc_values for example). Is there way to do that (via dynamic_templates of a lower ordered template maybe?)?

Thanks

4

yea... customizing the metadata mapping totally makes sense. I'll add that to the roadmap. Thx for the suggestion!

5

I have having the watcher mapping issue for the same version of watcher- 1.0.1, ES version is 1.5.2

In one server the Watcher Mapping the Strings are not analysed whereas in the other one they are. So below Mapping exists in one instance and not another. These two ES instances are running independently. This behavior is causing our term based queries to not return the correct results.
What could be causing this and what actions can be taken to ensure that we see consistent mapping in production when we go live?

"watch": {
"dynamic_date_formats": [
"date_optional_time"
],
"dynamic": "strict",
"dynamic_templates": [{
"strings": {
"mapping": {
"index": "not_analyzed",
"type": "string"
},
"match_mapping_type": "string",
"match": "s*"
}
},