Rollup Problem

[darkwarrior (Amey)]

Hello, I am having some problems in Rollup,

1. We have daywise indices and in one index we are having millions of documents.

2. So we are having such type of indices from june 2018 to current date.

3. And I am trying to perform rollup on all of them in single rollup job using wildcard *

   With delay = 30days and interval as 1day and cron is set to be hit at once every day at fix time

4. So the problem here is for the range 21st September to 9th October the data is not rolled up and buckets for this range are showing 0 documents in rollup index and the boundary indices are also not rolledup properly.

5. But when I tried to rollup them individually it worked fine (on one day index in the above mentioned range like for eg. one rollup index for 21st September, one more for 22nd September, ... and so on till 9th Of october ).

6. So we are not able to understand what is happening and on internet also no queries have been asked related to this.

   So please give me some advice on this problem. I am waiting for your response .Thanks in advance.

[polyfractal (Zachary Tong)]

Hi there, sorry for the delay (just saw this thread). Some questions:

• What version of ES are you using?
• Were there any errors in your logs while the data was being rolled up?
• What are the index names, and have you verified the wildcard pattern matches all of them?
• When you saw an issue with 21st September through 9th October, was that when using the RollupSearch API, or just a regular search?
• Are you using timezones in the rollup job?

Sorry for the barrage of questions, there might be a number of different possibilities so I'm asking questions to cover a wide range of thoughts

[darkwarrior (Amey)]

First of all thanks for your reply sir and no worries for the questions.

1. Es Version = 6.3.1
2. Sorry but the system logs are not available.
3. Yes the wildcard pattern matches all of them.
4. We saw the issue for both type of searches (rollup_search and normal).
5. No we are not using timezones in our rollup-job.
6. And for solution we tried to run another job for the missing range with keeping rollup index same as of the main job and then we are facing one more difficulty in that we are getting data in a result for the newly running jobs only and not for our main job (we are able to see the data for the whole range in normal search, (after running 2nd job) but in rollup_search we are getting data of newly running jobs only not the old-main jobs data).