Watch-history fieldname error

Elastic Stack Elasticsearch
1

Version: v2.4.3

Hi,

I'm currently running into a problem where the watch-history record for a particular watcher isn't index due complains about the fieldname ctx.payload.hits.total:

[2017-01-31 11:33:51,687][ERROR][watcher.execution ] failed to update watch record     [recovery_ceased_log_source_***_dev_wa_userapi_cus_1689-2017-01-31T11:33:51.268Z]     MapperParsingException[Field name [ctx.payload.hits.total] cannot contain '.']

This is unfortunate, as I need my watch-history-* indices to contain 100% accurate data. I have a series of "recovery" watchers that read through the execution history, and send a 'recovered' message when things are back to normal (e.g excess disk usage watcher stopped triggering)

Is there any easy way to fix this?

The watcher that triggered this error is included below. It is a chained-input watcher that checks:

  • did the ceased_log_source watcher execute?
  • did the ceased_log_source watcher "recover"?
  • did this watcher trigger?
    • if so, don't pass this watcher's condition
  • otherwise, was the input-order the order executed -> recovered?
    • if so, trigger this watcher
2

Hey,

can you paste the output of an _execute run, so that we can see the output that is supposed to be stored in the watch history?

--Alex

3

Hi Alex,

Sure; here's

and with "ignore-condition": true

4

Hey,

After a quick glance I think there is an issue with the mapping of chained inputs, where fields are accidentally mapped that should not be... will keep you posted

--Alex

1 Like
5

Hey,

so the the default setup looks ok to me (tested with a fresh installation, no old data). Can you show me

  • The output of GET /_template/watch_history
  • The output of a mapping of the affected watch history index

Thanks!

--Alex

6

Sure;

Just for clarity, I used today's mapping since that's when the error I provided occurred ( 2017-01-31 11:33:51,687 )

7

Hey,

it looks as if you are using an old watcher history template can you run

DELETE _template/watch_history
# wait a moment and the new template should be added
GET _template/watch_history

The new template should have a different path_math value for the disabled_payload_fields in the dynamic templates.

P.S. This also means, that the new template is only applied the next day, unless you delete todays watch history.

--Alex

8

Thanks a million for helping. I recreated the template as described, I'll comment here tomorrow letting you know if this resolved the problem

Edit: or tomorrow, as my cluster went into a red-state today & needed to be restored, ...

9

Thanks, that worked perfectly

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.