Command to change fieldtype of an index

[pathri (Vinit Pathri)]

I am trying changing the fieldtype using of a field from string to integer and not able to find out to correct syntax.
preferably using _mapping API
Please help

Thanks in advance

[Dominic_Page]

Hi @pathri

Once you a field has a fieldtype you can't change this.

If you don't have data in the index, try this:

1. Get the mapping using

GET <my_index>/_mapping

and copy the JSON to a safe place

2. delete the index, see:

https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-delete-index.html

(I gave a link to avoid accidental deletion)

3. Create the index using this API

https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html

you can modify the mapping you saved and use that in the create request.

Alternatively, if you already have data, you will need to create a new index, following steps 1 and 3

and then use the _reindex API, see https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html

to transfer your data to the new index.

Hope this helps!

• Dom

[pathri (Vinit Pathri)]

thanks for your help Dom

But this is a big process and also the time consuming one.
and we are aware that the reindex takes long time and if the data in the index will be in TBs then it is going to take much longer time

no other way to handle this scenario, like using _ingest API ??

[Dominic_Page]

Hi @pathri

The _ingest api will allow you to add a pipeline which can change the type of incoming data

What it won't do is allow you to change the already-existing fieldtype on the already existing field or fields. That's because existing indexes will already have data structures in place to store the original fieldtype.

You could also add (the same) pipeline to a reindex request

I understand your concern about the cost in time of a reindex. It will also require space. You might consider adding a query to a reindex to break it down into smaller operations

[pathri (Vinit Pathri)]

alright
thanks for clearing all doubts,
just 1 question left which is : i was going through ingest API documentation and saw that i can run the created pipeline on a particular document but is it possible to run it on a complete index in 1 go.

Simple example:
i have created a pipeline to delete the "message" field from apache log, i am able to do it for a particular document but wanted to do for all documents, what should be the query for that?

Thanks in advance

[Dominic_Page]

Hi @pathri

If you are deleting a field from an existing document or documents you can do this in-place in the same index using the "Update by Query" API, see

https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update-by-query.html

If you don't specify a query to scope the operation, this will act on ALL the documents in the index.

You can specify a pipeline on the command line, for example:

`POST twitter/_update_by_query?pipeline=set-foo``

and this will run the pipeline on all the documents in the index

[pathri (Vinit Pathri)]

cool

this query is working but getting time out
{
"statusCode": 504,
"error": "Gateway Time-out",
"message": "Client request timeout"
}

this index is having 924851 documents in it and documents are nothing but apache log.

[Dominic_Page]

Hi @pathri is this relating to an update_by_query?

[Dominic_Page]

Hi @pathri please can you raise this as a new Topic as I'll be out of touch in the short term

Cheers

[pathri (Vinit Pathri)]

Hi @Dominic_Page: yes it was for the update query only

POST twitter/_update_by_query?pipeline=delete-message

sorry for the delayed response got stuck with something

[Dominic_Page]

Hi @pathri

you might want to take a look at this thread

Update API timeout

@spinscale i'm getting the following exception { "statusCode": 504, "error": "Gateway Time-out", "message": "Client request timeout" }

where the user had the same issue while using the same API

Hope this helps!