Is it possible to index data partially and don't loose current data?


(mehmet) #1

Hi,
In my index, i have field which is optional. So, while i am indexing, i don't index this field. If it required then i add this field with update. This work just fine for me but problem is if i need to reindex that doc, i loose optional field.
Let me explain it with an example;

let's say i index this,

{
    "id":"123"
    "user":"xyz",
    "message":"sample message"
}

then an update request came:

{
    "id":"123"
    "user":"xyz",
    "message":"sample message"
    "optional_field":"some data"
}

then when an reindex request came for this id, it became like this

  {
        "id":"123"
        "user":"xyz",
        "message":"updated sample message"
  }

i look at upsert in order to if doc is exist then update id, user and message field only but with upsert i couldn't use version control.
I can't put optional field to index definition because then i have to set some default value when i am indexing and if doc is already exist then it will override it

So, i am wondering if it is possible to index data partially, so i wouldn't loose optional field.


(Mark Walkom) #2

Have you seen https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update.html#_updates_with_a_partial_document?


(mehmet) #3

Yeah, i check that but it's not what i am looking for.
I looking for partially indexing.
I mean if doc not exist then create it,
if doc exist then only update only given field (or merge whole exist data with new data end then index)
As i said i check upsert but it doesn't have version control.


(system) #4