Hello,
I have running now 3 different jobs for their respective index. In one of them, docs are being processed but on the other 2, they are not.
The thing is that they 3 have the exactly same configuration for the roll up, they only differ in the amount of terms selected for the job:
- Roll up A has only 2 terms
- Roll up B has 7 terms
- Roll up C has 7 terms
They are all being rolled up under the same index pattern. Non of the terms have infinite (of tendency to infinite) amounts of possibilities (such as would be with a client_ip
kind of term)
Roll ups configs for jobs A, B and C are the following:
But I noticed that for some reason jobs B and C stopped processing documents.. the stats are the very same since day ago:
- Roll up A: (is the only roll up that hasn't stopped processing documents )
- Roll up B:
- Roll up C
Roll ups B and C seems being "limitated" by something for some reason. Can you tell that "limitation" that is making stop the processed documents for those jobs?
I only suspect of the page_size
, which is the same for the three jobs: the 1000 default. I find curious about the fact that the number of pages processed in B and C is a multiple *1000 of the rollups indexed. Is it related to the amount of terms? Is there any recommended page size related to the amount of terms selected for a job?
EDIT: I realized that indeces B and C have "search_failures"s:
Logs shows:
[2020-10-19T09:00:00,227][WARN ][o.e.x.r.j.RollupJobTask ] [QyJqSRr] Rollup job [A] failed with an exception:
java.lang.RuntimeException: Shard failures encountered while running indexer for job [A]: [shard [[QyJqSRrVS1alvBduaEDp5w][filebeat-corso-6.6.2-2020.09.05][0]], reason [RemoteTransportException[[QyJqSRr][172.18.0.2:9300][indices:data/read/search[phase/query]]]; nested: NumberFormatException[For input string: "tracker-tag-absent"]; ], cause [java.lang.NumberFormatException: For input string: "tracker-tag-absent"
tracker-tag-absent
is the output of the log
Thank for reading,
Caro.