Elasticsearch remove fields

Elastic Stack Kibana
1

Hello!
i'm using:
elasticsearch 7.13.2
kibana 7.12.1

I am trying to remove index pattern fields in kibana -> dev tools using the following command

POST /mdaemon_test_array_2_step-2021.11.16_test_reindex/_update_by_query?conflicts=proceed
{
"script" : "ctx._source.remove("new_field3")"
}

But it didn't work. Although, if I try to add a field, then I can do it using the following command

POST /mdaemon_test_array_2_step-2021.11.16_test_reindex/_update_by_query?conflicts=proceed
{
"script" : "ctx._source.new_field66 = 'value_of_new_field6'"
}

Please tell me what I'm doing wrong

2

What is it doing when you try to run it?

("new_field3") needs to be ('new_field3') since it's wrapped in ".

POST /mdaemon_test_array_2_step-2021.11.16_test_reindex/_update_by_query?conflicts=proceed
{
"script" : "ctx._source.remove('new_field3')"
}
3

Thanks for the answer. I tried your command, but the field remains

image
image1558×422 28.6 KB

4

What do you see when you run this?

GET mdaemon_test_array_2_step-2021.11.16_test_reindex/_search
{
  "_source": [
    "new_field3"
  ]
}
5

#! Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See Set up minimal security for Elasticsearch | Elasticsearch Guide [7.13] | Elastic to enable security.
{
"took" : 10,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 10000,
"relation" : "gte"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "mdaemon_test_array_2_step-2021.11.16_test_reindex",
"_type" : "_doc",
"_id" : "YUwrKX0B6KcywX5P2CJp",
"_score" : 1.0,
"_source" : { }
},
{
"_index" : "mdaemon_test_array_2_step-2021.11.16_test_reindex",
"_type" : "_doc",
"_id" : "YkwrKX0B6KcywX5P2CJp",
"_score" : 1.0,
"_source" : { }
},
{
"_index" : "mdaemon_test_array_2_step-2021.11.16_test_reindex",
"_type" : "_doc",
"_id" : "Y0wrKX0B6KcywX5P2CJp",
"_score" : 1.0,
"_source" : { }
},
{
"_index" : "mdaemon_test_array_2_step-2021.11.16_test_reindex",
"_type" : "_doc",
"_id" : "ZEwrKX0B6KcywX5P2CJp",
"_score" : 1.0,
"_source" : { }
},
{
"_index" : "mdaemon_test_array_2_step-2021.11.16_test_reindex",
"_type" : "_doc",
"_id" : "ZUwrKX0B6KcywX5P2CJp",
"_score" : 1.0,
"_source" : { }
},
{
"_index" : "mdaemon_test_array_2_step-2021.11.16_test_reindex",
"_type" : "_doc",
"_id" : "ZkwrKX0B6KcywX5P2CJp",
"_score" : 1.0,
"_source" : { }
},
{
"_index" : "mdaemon_test_array_2_step-2021.11.16_test_reindex",
"_type" : "_doc",
"_id" : "Z0wrKX0B6KcywX5P2CJp",
"_score" : 1.0,
"_source" : { }
},
{
"_index" : "mdaemon_test_array_2_step-2021.11.16_test_reindex",
"_type" : "_doc",
"_id" : "aEwrKX0B6KcywX5P2CJp",
"_score" : 1.0,
"_source" : { }
},
{
"_index" : "mdaemon_test_array_2_step-2021.11.16_test_reindex",
"_type" : "_doc",
"_id" : "aUwrKX0B6KcywX5P2CJp",
"_score" : 1.0,
"_source" : { }
},
{
"_index" : "mdaemon_test_array_2_step-2021.11.16_test_reindex",
"_type" : "_doc",
"_id" : "akwrKX0B6KcywX5P2CJp",
"_score" : 1.0,
"_source" : { }
}
]
}
}

6

So it worked.

If you are looking at deleted: 0 of an indication it didn't work that's not what that means. That's the count of entire records deleted which should be 0. You were just deleting 1 field in the record which would be an update. All of your records were updated so it's a good indication it worked.

7

If I remove them, then why then do I see these fields in discover and index patterns?

image
image1183×195 9.48 KB

image

8

Refresh or delete/create your index pattern to see if it's still there. Not sure if your version requires that, newest ones auto updates.

9

I tried "delete/create" in "index patterns" but it gave no results.
I tried the "refresh index" in the "index managment" but it gave no results.
If I delete index and create it (in the index managment), then it deletes all the fields that I manually created, and I only need to delete a specific field. Also it will delete all the data that was in it, but it doesn't suit me

10

I think you might have something else going on then.

If you run this query in dev tools and you don't find any field with that name then that field does not exist in the data.

GET mdaemon_test_array_2_step-2021.11.16_test_reindex/_search
{
  "_source": [
    "new_field3"
  ]
}

Can you post what your index pattern is named and how you deleted and recreated?

11

I delete it in kibana (as in the screenshot), create it in "Index Patterns" -> "Create index patters"
p.s.
There is no data in the "new_field3" field, if it matters

delete
delete1308×793 54.4 KB

12

Is there a data pipeline that is constantly feeding data to this index?

13

No, this is a copied inidex. No information goes there

14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.