Find missing documents in a time series


#1

I run a daily import of hotels with their offer count.
The goal is to see when the offer count of a hotel "provider" changes significantly.
If this happens, I need to know, what hotels have gone missing. My mapping:

"mappings": {
    "offer": {
        "properties": {
            "offerCount": { "type": "integer" },
            "insertDate": { "type": "date" },
            "hotelCode": {
                "type": "text",
                "fields": { "keyword": { "ignore_above": 256, "type": "keyword" } }
            },
            "provider": {
                "type": "text",
                "fields": { "keyword": { "ignore_above": 256, "type": "keyword" } }
            }
        }
    }
}

I am able to sum up offerCount by provider and find changes with a sequential diff.
But I do not know how I am able to figure out what hotelCodes have been dropped from this to the last daily import.

Is this something I can do with elasticsearch, or is it simply the wrong tool for that?
One way to achieve this would be to index all missing hotelCodes with a value of 0 for the offerCount. But it would be appreciated if this could be done by elasticsearch itself.

I am using ES 5.5.


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.