My last record in the destination index is for the minute, 2020-08-11 20:05:00.
It is a continuous transform and it's status=indexing, but no new destination logs are being generated.
I have other transforms in continuous mode; I don't know why this one is failing.
see messages below... and stats attached
2020-08-11 20:05:42
df-elastic-es-data-nodes-1
Finished indexing for transform checkpoint [0].
2020-08-12 03:23:28
df-elastic-es-data-nodes-1
Finished indexing for transform checkpoint [1].
hmmmm stopping it and starting it again has got it outputting more destination logs again, but that defeats the point a bit!
Do I have a conflict here?
recv_timestamp is the originating source time of the msg
ingest_timestamp is added by Elasticsearch to the index at the time the doc is indexed
"frequency": "1s",
"sync": {
"time": {
"field": "ingest_timestamp",
"delay": "10s"
}
},
"pivot": {
"group_by": {
"recv_timestamp": {
"date_histogram": {
"field": "recv_timestamp",
"calendar_interval": "1m"
}
},
and after the restart, it only reached the first checkpoint.
so I've stopped and restarted again, and I expect it will pause at the next checkpoint again
I still have a problem here.
The status will be indexing, but no data is coming out. Then after a few hours there are a splurge of new destination documents output