Copy field value to a new field in existing index

Elastic Stack Elasticsearch
1

Hey all,
I had a field name with a typo. Let's say it was called Helo.keyword.
I have created a new field with the correct name Hello.keyword.
New data is populating the new field.
How can I copy the old values to the new field created?
This is a one-time operation. After the values are copied I will delete the old field from the index.
I tried the following script, but it does not update any document

POST /test/text/_update_by_query
{
"query": {
  "match_all": {}
},
"script": "ctx._source.Hello.keyword = ctx._source.Helo.keyword"
}

Result

#! Deprecation: [types removal] Specifying types in search requests is deprecated.
{
  "took" : 4,
  "timed_out" : false,
  "total" : 0,
  "updated" : 0,
  "deleted" : 0,
  "batches" : 0,
  "version_conflicts" : 0,
  "noops" : 0,
  "retries" : {
    "bulk" : 0,
    "search" : 0
  },
  "throttled_millis" : 0,
  "requests_per_second" : -1.0,
  "throttled_until_millis" : 0,
  "failures" : [ ]
}

Thanks!

2

Could you provide a full recreation script as described in About the Elasticsearch category. It will help to better understand what you are doing. Please, try to keep the example as simple as possible.

A full reproduction script will help readers to understand, reproduce and if needed fix your problem. It will also most likely help to get a faster answer.

Without that it's hard to know but let's make some guesses. May be the type is actually _doc and not test.
You can look also at the rename processor and the reindex API. So you reindex in a new clean index instead of keeping the old mapping around.

3

Hi @dadoonet
Thank you for the reply.
I do not have a full recreation scripts as I was not involved in the entire process.
Let me know what info I should provide, and I'll do it with pleasure.

The script, as I understand it:
POST /test/text/_update_by_query - test is the index name
POST /test/text/_update_by_query - text is the field type
POST /test/text/_update_by_query - update by query is the method

If my understanding is wrong, please let me know.
I took the script from here - elasticsearch - Add a field copying data from another fields - Stack Overflow

Reading the documentation here, makes me think I have a syntax problem?!

POST twitter/_update_by_query
{
  "script": {
    "source": "ctx._source.likes++",
    "lang": "painless"
  },
  "query": {
    "term": {
      "user": "kimchy"
    }
  }
}

I do not want to use the re-index API, as there are multiple indices with multiple fields I need to update and I am afraid it will cause issues/a mess with the data. I prefer the approach of updating and then deleting 1 field at a time.

4

Could you create a sample script with some fake data so we can start from there to help you fixing the problem?

If you look at the link I provided, a sample script is something we can paste into Kibana dev console and play.
look at the sample script provided in the link.

5

Hi @dadoonet
This are the steps, more or less
Create an index

PUT test-24-03-2020

Add mappings. Need also to add the keyword type, not sure how to do that.

PUT /test-24-03-2020/_mapping
{
        "properties" : {
            "helo" : { "type" : "text" }
        }
}

Populate values

PUT test-24-03-2020/_doc/1
{
  "helo": "test123"
}

Add mappings. Need also to add the keyword type, not sure how to do that.

PUT /test-24-03-2020/_mapping
{
        "properties" : {
            "hello" : { "type" : "text" }
        }
}

Now I want to copy all the values from helo to hello

Hope this explains a bit more what is required

Thanks

6

OK.
I managed to copy the values from one field to another.
I had a typo(?) error or logic(?) issue.

The solution is

POST index-patter-name-*/_update_by_query
{
  "script": {
    "source": "ctx._source.destination_field= ctx._source.origin_field",
    "lang": "painless"
  },
  "query": {
          "term": {
                  "somefield.keyword": "somevalue"
          }
  }
}
  • Construct the query according to your specific logic.

Originally I tried to use the keyword and the process failed.

    "source": "ctx._source.destination_field.keyword= ctx._source.origin_field.keyword",

Cheers!

7

Great. It's because the keyword field does not exist in the _source field but is generated at index time.

1 Like
8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.