Hi all,
Thanks in advance for the help!
I want to rename a field in an index 'everywhere'. By everywhere, I mean that I don't want my graphs and dashboard based on this field to be impacted, I don't know if there is any possibility to do it.
I found this like on the web : https://stackoverflow.com/questions/43120430/elasticsearch-mapping-rename-existing-field
It's using an Ingest pipeline, a Rename processor and the Reindex API but I don't want to create a new index because I have more than hundred graphs included in many dashboards (means : very long to update everything with the new name of my field)
Thanks again for your help !
Regards
Guillaume
[EDIT] - quick edit just to say that i'm running elasticsearch 6.4.3 and also to say why do I want to rename a field. For the moment I have something like :
logs before 10th Dec I have into them myOldFieldName = user1
logs after 10th Dec I have into them myNewFieldName = user1
and I'd like the same myNewFieldName = user1 for all my logs
Hello @dadoonet and thank you for your answer,
I will take a look at it as soon as I can but first of all, I didn't see your reply before editing my post.
Did you see in my post the edit below :
Hi @dadoonet and all,
I've been reading the page that you told me but I'm not sure if it can work in my case ...
My command GET /_search returns :
{
"my-index-name":{
"aliases":{},
"mappings":{
"spread_doctype":{
"properties":{
"@timestamp":{
"type":"date"
},
"@version":{
"type":"text",
"fields":{
"keyword":{
"type":"keyword",
"ignore_above":256
}
}
},
"my-old-field-name":{
"type":"text",
"fields":{
"keyword":{
"type":"keyword",
"ignore_above":256
}
}
},
"my-new-field-name":{
"type":"text",
"fields":{
"keyword":{
"type":"keyword",
"ignore_above":256
}
}
}
}
}
},
"settings":{
"index":{
//some settings
}
}
}
}
As you can see, my-old-field-name and my-new-field-name are two independent fields in this index but the old one for the old logs and the new one for the new logs. I don't want to create a third field as alias here, I just wanna merge these two fields into a unique one.
Sorry If I'm not clear
Thank you
While you are saying that this is the output of GET /_search it looks like a mapping to me?
You said earlier that "logs before 10th Dec I have into them myOldFieldName = user1". Do you have a daily / weekly / ... index pattern? Then you would just add the alias to indices before the 10th of December. If it's one big index you'll probably need to do an _update_by_query on all fields that don't have the field myNewFieldName. Shout if the query with the script to rename the field is not obvious. At the end of https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html there is a very similar script, but you won't need a full reindex — update by query on the old format will do it.
My logs are imported by logstash to elasticearch every minute from a csv file. Before this 10th Dec the header of this csv file was with myOldFieldName field and after this 10th Dec I modified this header with myNewFieldName. So now what happened ? My logs before and after this 10th Dec don't have the same field name for the same "attribute". It's very painful in the case I want to create a dashboard and apply a filter based on this field ...
So your solution would be to add an alias on my logs that don't already contain myNewFieldName ? What do you mean by "big index" ? Mine has almost 30M logs ... I won't get any issue by adding an alias with the same name than a real field name contained in other (newer) logs ?
So the way I read this is that you have a single index spread with all the data, right? Then you will need the _update_by_query approach for the old data. Otherwise you'd have the mapping problem of colliding a concrete field and the alias field.
Hi,
I hope you re doing well,
I've found how to create an alias but not how to say that I want to create this alias for the logs before a specific date or where a field doesn't exist or exist ...
I mean, I would like to do : create my alias
PUT myIndex
{
"mappings": {
"_doc": {
"properties": {
"myOldFieldName": {
"type": "keyword"
},
"myNewFieldName": {
"type": "alias",
"path": "myOldFieldName"
}
}
}
}
} only for those logs
GET /myIndex/_search
{
"query": {
"range" : {
"@timestamp": {
"lte" : "dateOfTheUpdate"
}
}
}
}
You are using an index alias. I think what you actually would need is a field alias if the new field doesn't exist anywhere yet:
DELETE field_alias
PUT field_alias/_doc/1
{
"foo": "test"
}
GET field_alias/_mapping
GET field_alias/_search
{
"query": {
"match": {
"bar": "test"
}
}
}
PUT field_alias/_mapping/_doc
{
"properties": {
"bar": {
"type": "alias",
"path": "foo"
}
}
}
GET field_alias/_mapping
GET field_alias/_search
{
"query": {
"match": {
"bar": "test"
}
}
}
If the field already exists in the index you will need to run an update query (though this is relatively heavy depending on how many documents need to be reindexed):
DELETE update_index
PUT update_index/_doc/1
{
"foo": "test"
}
PUT update_index/_doc/2
{
"bar": "test"
}
GET update_index/_mapping
GET update_index/_search
{
"query": {
"match": {
"foo": "test"
}
}
}
# This will fail with: mapper [bar] of different type, current_type [text], merged_type [FieldAliasMapper]
PUT update_index/_mapping/_doc
{
"properties": {
"bar": {
"type": "alias",
"path": "foo"
}
}
}
POST update_index/_update_by_query
{
"query": {
"bool": {
"must": {
"exists": {
"field": "bar"
}
},
"must_not": {
"exists": {
"field": "foo"
}
}
}
},
"script": {
"source": """
ctx._source.foo = ctx._source.bar;
"""
}
}
GET update_index/_search
{
"query": {
"match": {
"foo": "test"
}
}
}
Depending on your scenario you could also solve the problem at query time by simply searching both fields:
DELETE query_time
PUT query_time/_doc/1
{
"foo": "test"
}
PUT query_time/_doc/2
{
"bar": "test"
}
GET query_time/_search
{
"query": {
"multi_match": {
"query": "test",
"fields": [ "foo", "bar"]
}
}
}
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
