Unit test watcher

Elastic Stack Elasticsearch
1

Is it possible to unit test watcher scripts?

If so how are people doing this?

There doesn't seem to be a framework in place and not a lot of information on it, if there is.

Thanks

2

Hey,

there is no way to unit test it by loading a mocked watch execution environment or something. I took some time to write down a lengthy blog post, how to use the existing APIs to exactly test that, so if you have a watch instance to test against, this should be good enough.

See at https://www.elastic.co/blog/watching-the-watches-writing-debugging-and-testing-watches

--Alex

1 Like
3

Hi Alex/@spinscale ,

This might be relevant to @SKing

I'm working on our alert-tests at the moment, and I

  • I have a simple alert that detects whether any tests have failed based on an exit code. It's triggered when
    at least one log is detected with exit-code '1' listed.

  • I have a list of hits and assertions as test cases. For example, I have one case where there's a single log with exit-code 1, and an assertion stating that upon running '_execute' the state should be 'executed'.

  • 'alternative_input' allows me to specify the results of a query as watcher input. I want to provide the hits to the query, and have the query provide the results to the watcher condition. I'm effectively looking for a field like 'alternative_index_content' or 'alternative_source_hits'.

We discussed this before, and you suggested using some scripting to accomplish this. Do you have any suggestions on how I could implement this. Ideally I could:

  • Give a list of hits to the watcher query directly* without indexing the data first. Hopefully there's a single REST call available to do this, it would be a useful way of system-testing ES queries too.
  • Get back the results back from this query, and as suggested in your blog post pass these as alternative_input.

Is there any method of running an ES query with literal input data provided in the same REST call? This would make testing a lot easier, I'd prefer not to set up dedicated test-indices if possible.

Hey,

a workaround for this issue could be, that you use an alias in your watch to query, and that points to your test data or your live data, but using the alternative_input with some scripting on the client side might be easier, to be honest.

--Alex

1 Like
4

@Ryan_Grannell I dont think there is anything like getting a list of hits from a query, but all in-memory (it sounds to me, that this is what you are searching for). If you have those hits precreated (or maybe only create them once per test suite), that passing it to an alternative input would be pretty easy - but not sure if if that works for you.

1 Like
5

@spinscale Yes, I would like to do this in memory (or at least have the _execute API wrap all the index / hit writing). I've actually implemented automated tests for watchers now, but it wasn't too easy

  • set up type mappings e.g *primary-*
  • delete an existing test-index
  • wait
  • create a test index (test-primary-1)
  • wait
  • repeatedly retry posting hits & the alert to ElasticSearch. Poll for the hit ID's being available in ElasticSearch.

Things like snapshotting need to be accounted for during deletion, and it's important to do search polling to make sure the correct hits are set. This method doesn't have any false-positives / random errors, but it is slow (~40 seconds per test-case) and requires about 300 lines of setup code.

On the bright side, our Watcher's are now reasonably well tested :slight_smile:

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.