Elasticsearch remove fields

Hello!
i'm using:
Elasticsearch 7.13.2
kibana 7.12.1

I am trying to remove index pattern fields in kibana -> dev tools using the following command

POST /mdaemon_test_array_2_step-2021.11.16_test_reindex/_update_by_query?conflicts=proceed
{
"script" : "ctx._source.remove("new_field3")"
}

But it didn't work. Although, if I try to add a field, then I can do it using the following command

POST /mdaemon_test_array_2_step-2021.11.16_test_reindex/_update_by_query?conflicts=proceed
{
"script" : "ctx._source.new_field66 = 'value_of_new_field6'"
}

Please tell me what I'm doing wrong

What is it doing when you try to run it?

("new_field3") needs to be ('new_field3') since it's wrapped in ".

POST /mdaemon_test_array_2_step-2021.11.16_test_reindex/_update_by_query?conflicts=proceed
{
"script" : "ctx._source.remove('new_field3')"
}

Thanks for the answer. I tried your command, but the field remains

What do you see when you run this?

GET mdaemon_test_array_2_step-2021.11.16_test_reindex/_search
{
  "_source": [
    "new_field3"
  ]
}

#! Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See Set up minimal security for Elasticsearch | Elasticsearch Guide [7.13] | Elastic to enable security.
{
"took" : 10,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 10000,
"relation" : "gte"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "mdaemon_test_array_2_step-2021.11.16_test_reindex",
"_type" : "_doc",
"_id" : "YUwrKX0B6KcywX5P2CJp",
"_score" : 1.0,
"_source" : { }
},
{
"_index" : "mdaemon_test_array_2_step-2021.11.16_test_reindex",
"_type" : "_doc",
"_id" : "YkwrKX0B6KcywX5P2CJp",
"_score" : 1.0,
"_source" : { }
},
{
"_index" : "mdaemon_test_array_2_step-2021.11.16_test_reindex",
"_type" : "_doc",
"_id" : "Y0wrKX0B6KcywX5P2CJp",
"_score" : 1.0,
"_source" : { }
},
{
"_index" : "mdaemon_test_array_2_step-2021.11.16_test_reindex",
"_type" : "_doc",
"_id" : "ZEwrKX0B6KcywX5P2CJp",
"_score" : 1.0,
"_source" : { }
},
{
"_index" : "mdaemon_test_array_2_step-2021.11.16_test_reindex",
"_type" : "_doc",
"_id" : "ZUwrKX0B6KcywX5P2CJp",
"_score" : 1.0,
"_source" : { }
},
{
"_index" : "mdaemon_test_array_2_step-2021.11.16_test_reindex",
"_type" : "_doc",
"_id" : "ZkwrKX0B6KcywX5P2CJp",
"_score" : 1.0,
"_source" : { }
},
{
"_index" : "mdaemon_test_array_2_step-2021.11.16_test_reindex",
"_type" : "_doc",
"_id" : "Z0wrKX0B6KcywX5P2CJp",
"_score" : 1.0,
"_source" : { }
},
{
"_index" : "mdaemon_test_array_2_step-2021.11.16_test_reindex",
"_type" : "_doc",
"_id" : "aEwrKX0B6KcywX5P2CJp",
"_score" : 1.0,
"_source" : { }
},
{
"_index" : "mdaemon_test_array_2_step-2021.11.16_test_reindex",
"_type" : "_doc",
"_id" : "aUwrKX0B6KcywX5P2CJp",
"_score" : 1.0,
"_source" : { }
},
{
"_index" : "mdaemon_test_array_2_step-2021.11.16_test_reindex",
"_type" : "_doc",
"_id" : "akwrKX0B6KcywX5P2CJp",
"_score" : 1.0,
"_source" : { }
}
]
}
}

So it worked.

If you are looking at deleted: 0 of an indication it didn't work that's not what that means. That's the count of entire records deleted which should be 0. You were just deleting 1 field in the record which would be an update. All of your records were updated so it's a good indication it worked.

If I remove them, then why then do I see these fields in discover and index patterns?

image

Refresh or delete/create your index pattern to see if it's still there. Not sure if your version requires that, newest ones auto updates.

I tried "delete/create" in "index patterns" but it gave no results.
I tried the "refresh index" in the "index managment" but it gave no results.
If I delete index and create it (in the index managment), then it deletes all the fields that I manually created, and I only need to delete a specific field. Also it will delete all the data that was in it, but it doesn't suit me

I think you might have something else going on then.

If you run this query in dev tools and you don't find any field with that name then that field does not exist in the data.

GET mdaemon_test_array_2_step-2021.11.16_test_reindex/_search
{
  "_source": [
    "new_field3"
  ]
}

Can you post what your index pattern is named and how you deleted and recreated?

I delete it in kibana (as in the screenshot), create it in "Index Patterns" -> "Create index patters"
p.s.
There is no data in the "new_field3" field, if it matters

Is there a data pipeline that is constantly feeding data to this index?

No, this is a copied inidex. No information goes there

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.