In order to create a testbed for anomaly detection, I need to create some tasks that may never finish (unless you manually kill them) in ES. Is there any way to do that? I do find some problems with storage hardware may result in such tasks, and probably I can rely on some known bugs and ES of the corresponding versions to do that. But is there any regular approach to creating such tasks?
Maybe I can use Groovy scripting to send a task that does nothing but sleep for a long time to occupy the thread pool?
Oh, Groovy is not supported since ES 6... So maybe I can use expert_scripts to do that?
I am not sure this is possible. Why does the task need to run within Elasticsearch? Can you not instead monitor an external process that is easier to manipulate?
1 Like
Sadly, I cannot... Have to build a long-running task that only consumes a little CPU time within ES. 
Why?
Sorry for the late reply...
We are going to develop some anomaly detection techniques and use ES to validate these developed techniques.
Overall, we want to implement an example where one task may considerably degrade the performance of the whole ES system. In this case, an operator could manually cancel the corresponding task in order to improve the performance of ES.
One such case I can imagine is that if a long-running task somehow occupies a thread from the thread pool for a long time but does nothing, the whole system may suffer.
A better example could be: if an important task t1, such as create index, is somehow blocked for a while due to an unimportant task t2, and an operator could improve the system performance by canceling t2 and then re-send t2.
Such a case has been found in MySQL, where a backup task that needs to get the global read lock could be blocked by a long-running query. Reference: Handling long-running queries in MySQL with Percona XtraBackup
But I cannot find such a case in ES. I'm now reading the source codes of these important tasks in ES and hopefully I can find a satisfying case.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.