Running an _update_by_query statement dropped the field in unmatched elements?

[cdav (Davide)]

I've run the following query:

POST hits-*/_update_by_query
{
  "script": {
    "source": "ctx._source.type = 'se';",
    "lang": "painless"
  },
  "query": {
    "match": {
      "type": "qa"
    }
  }
}

Everything seemed good, but I noticed too late that every document that didn't match the query got updated to a null "type" value.

Is this a problem that can be solved with a simple operation, or is the data lost forever?

Many thanks,

[dadoonet (David Pilato)]

Welcome!

I doubt that documents have been updated with a null value when running the exact request you shared.

But to answer your question, there's no way to restore the original values unless you did a snapshot before.

[cdav (Davide)]

Thank you for the welcome

I'm almost certain that this is the correct query, I know because it took an entire night to run and I saved a copy of what was on my browser before going to sleep in case it timed out.

What's making me lead towards a problem other than the query is that only ~half the documents got updated, and as far as I can tell data up to ~6 months ago had the query run on it but didn't lose any data.

I should be able to rebuild the field from a different source, but it might take close to forever.

[cdav (Davide)]

Ok, I believe I'm at fault here. There is no bug, just an unattentive user. The data from July onwards was corrupted because I was uploading it to elastic as such, and everything functions as intended.