# Not able to remove tag from xml

**URL:** <https://discuss.elastic.co/t/not-able-to-remove-tag-from-xml/326771>\
**Category:** Logstash\
**Created:** [March 1, 2023, 2:30pm UTC](https://discuss.elastic.co/t/not-able-to-remove-tag-from-xml/326771 "2023-03-01T14:30:24Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![Navya\_04](https://avatars.discourse-cdn.com/v4/letter/n/b4bc9f/32.png) [@Navya\_04](https://discuss.elastic.co/u/Navya_04)\
**Post date:** [March 1, 2023, 2:30pm UTC](https://discuss.elastic.co/t/not-able-to-remove-tag-from-xml/326771/1 "2023-03-01T14:30:24Z")

</div>

I am trying to load xml through logstash. I have an unwnated tag which needs to be removed from xml while parsing. Used remove\_tag but not able to remove the tag while indexing to Elasticsearch  
xml File

```auto
<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="Report" time="212.715" tests="7" failures="1" errors="3">
   <testsuite name="Report" tests="7" failures="1" errors="3" time="212.715" skipped="0" timestamp="2023-02-23 16:45:10" id="Test Suites/Report">
      <testcase name="Test Cases/01" time="34.627" classname="Test Cases/01" status="FAILED">
        <system-err><![CDATA[2023-02-23 16:45:10 - [TEST_SUITE][FAILED] - Report: Test Cases/TC01 FAILED.
Reason:
com.kms.katalon.core.exception.StepFailedException: Unable to scroll to object 'Object Repository/ScrollToEle'
]]></system-err>
      </testcase>
      <testcase name="Test Cases/02" time="34.627" classname="Test Cases/02" status="PASSED">
       
      </testcase>
      <system-err><![CDATA[2023-02-23 16:45:10 - [TEST_SUITE][FAILED] - Report: Test Cases/TC01 FAILED.
Reason:
com.kms.katalon.core.exception.StepFailedException: Unable to scroll to object 'Object Repository/ScrollToEle'
]]></system-err>
   </testsuite>
</testsuites>

```

logstash Conf file

```auto
input {
  file 
{
    path => "C/log-sample.xml"
    start_position => "beginning"
     codec => multiline 
    {
        pattern => "<testsuites>"
        negate => true
        what => "previous"
        max_lines => 50000
    }
    sincedb_path => "NULL"
  }
}

filter {
xml {
source => "message"
target => "theXML"
force_array => false
remove_field => ["message"]
}
mutate {remove_tag => ["[testsuites][testsuite][testcase][system-err]"]}
ruby {
code => '
event.get("theXML").each { |k, v|
event.set(k,v)
}
event.remove("theXML")
'
}
}

output 
{
    elasticsearch {
hosts => "localhost:9200"
index => "xml_testing"
}
    stdout 
    {
        codec => rubydebug
    }
}

```

output I got. I still see system-err tag

```auto
{
"took": 13,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "xml_testing",
"_type": "_doc",
"_id": "v7GTnYYBhrMoCAz3NLRv",
"_score": 1,
"_source": {
"tags": [
"multiline"
],
"name": "Report",
"time": "212.715",
"tests": "7",
"host": "hdc3-l-JNRPMW2",
"path": "C:/Users/navya.krishna.voggu/Downloads/sample/sample.xml",
"failures": "1",
"@timestamp": "2023-03-01T14:28:33.127Z",
"@version": "1",
"errors": "3",
"testsuite": {
"skipped": "0",
"failures": "1",
"timestamp": "2023-02-23 16:45:10",
"testcase": [
{
"time": "34.627",
"classname": "Test Cases/01",
"status": "FAILED",
"system-err": "2023-02-23 16:45:10 - [TEST_SUITE][FAILED] - Report: Test Cases/TC01 FAILED.\nReason:\ncom.kms.katalon.core.exception.StepFailedException: Unable to scroll to object 'Object Repository/ScrollToEle'\n",
"name": "Test Cases/01"
},
{
"name": "Test Cases/02",
"time": "34.627",
"classname": "Test Cases/02",
"status": "PASSED",
"content": "\n \n "
}
],
"name": "Report",
"id": "Test Suites/Report",
"system-err": "2023-02-23 16:45:10 - [TEST_SUITE][FAILED] - Report: Test Cases/TC01 FAILED.\nReason:\ncom.kms.katalon.core.exception.StepFailedException: Unable to scroll to object 'Object Repository/ScrollToEle'\n",
"tests": "7",
"time": "212.715",
"errors": "3"
}
}
}
]
}
}

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [March 1, 2023, 3:34pm UTC](https://discuss.elastic.co/t/not-able-to-remove-tag-from-xml/326771/2 "2023-03-01T15:34:54Z")

</div>

The `remove_tag` is used to remove tags from the `tags` field, what you want is to remove a field.

You need to use the `remove_field` .

```auto
mutate {
    remove_field => ["field-name"]
}

```

---

<div class="post-metadata">

**Author:** ![Navya\_04](https://avatars.discourse-cdn.com/v4/letter/n/b4bc9f/32.png) [@Navya\_04](https://discuss.elastic.co/u/Navya_04)\
**Post date:** [March 1, 2023, 3:45pm UTC](https://discuss.elastic.co/t/not-able-to-remove-tag-from-xml/326771/3 "2023-03-01T15:45:36Z")

</div>

Tried with remove\_field too but still it did not help. Still able to see system-err field

```auto
mutate {
    remove_field => ["system-err"]
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 1, 2023, 4:34pm UTC](https://discuss.elastic.co/t/not-able-to-remove-tag-from-xml/326771/4 "2023-03-01T16:34:18Z")

</div>

You need the full field name, which appears to be "[testsuite][testcase][0][system-err]"

---

<div class="post-metadata">

**Author:** ![Navya\_04](https://avatars.discourse-cdn.com/v4/letter/n/b4bc9f/32.png) [@Navya\_04](https://discuss.elastic.co/u/Navya_04)\
**Post date:** [March 1, 2023, 5:10pm UTC](https://discuss.elastic.co/t/not-able-to-remove-tag-from-xml/326771/5 "2023-03-01T17:10:24Z")

</div>

No luck with this too. I have given filter as below still able to see system-err

```auto
filter {
xml {
source => "message"
target => "theXML"
force_array => false
remove_field => ["message"]
}
mutate {remove_field => ["[testsuite][testcase][0][system-err]"]}
ruby {
code => '
event.get("theXML").each { |k, v|
event.set(k,v)
}
event.remove("theXML")
'
}
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 1, 2023, 5:31pm UTC](https://discuss.elastic.co/t/not-able-to-remove-tag-from-xml/326771/6 "2023-03-01T17:31:37Z")

</div>

What does your event look like if you use `output { stdout { codec => rubydebug } } `

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [March 1, 2023, 5:34pm UTC](https://discuss.elastic.co/t/not-able-to-remove-tag-from-xml/326771/7 "2023-03-01T17:34:32Z")

</div>

You are using the `target` option in the `xml` filter, so you won't have your field in the root of the document.

Try to use `[theXML][testsuite][testcase][0][system-err]`.

---

<div class="post-metadata">

**Author:** ![Navya\_04](https://avatars.discourse-cdn.com/v4/letter/n/b4bc9f/32.png) [@Navya\_04](https://discuss.elastic.co/u/Navya_04)\
**Post date:** [March 1, 2023, 6:04pm UTC](https://discuss.elastic.co/t/not-able-to-remove-tag-from-xml/326771/8 "2023-03-01T18:04:02Z")

</div>

Thanks, that worked!  
If there are multiple testcase fields which consists of multiple system-err field as below xml, how can we remove all the fields with name system-err at once?

```auto
<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="Report" time="212.715" tests="7" failures="1" errors="3">
   <testsuite name="Report" tests="7" failures="1" errors="3" time="212.715" skipped="0" timestamp="2023-02-23 16:45:10" id="Test Suites/Report">
      <testcase name="Test Cases/01" time="34.627" classname="Test Cases/01" status="FAILED">
        <system-err><![CDATA[2023-02-23 16:45:10 - [TEST_SUITE][FAILED] - Report: Test Cases/TC01 FAILED.
Reason:
com.kms.katalon.core.exception.StepFailedException: Unable to scroll to object 'Object Repository/ScrollToEle'
]]></system-err>
      </testcase>
      <testcase name="Test Cases/02" time="34.627" classname="Test Cases/02" status="FAILED">
       <system-err><![CDATA[2023-02-23 16:45:10 - [TEST_SUITE][FAILED] - Report: Test Cases/TC02 FAILED.
Reason:
com.kms.katalon.core.exception.StepFailedException: Unable to scroll to object 'Object Repository/ScrollToEle'
]]></system-err>
      </testcase>
	  <testcase name="Test Cases/03" time="34.627" classname="Test Cases/03" status="FAILED">
       <system-err><![CDATA[2023-02-23 16:45:10 - [TEST_SUITE][FAILED] - Report: Test Cases/TC03 FAILED.
Reason:
com.kms.katalon.core.exception.StepFailedException: Unable to scroll to object 'Object Repository/ScrollToEle'
]]></system-err>
      </testcase>
   </testsuite>
</testsuites>

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [March 1, 2023, 6:08pm UTC](https://discuss.elastic.co/t/not-able-to-remove-tag-from-xml/326771/9 "2023-03-01T18:08:43Z")

</div>

You will need to do this with a ruby code.

---

<div class="post-metadata">

**Author:** ![Navya\_04](https://avatars.discourse-cdn.com/v4/letter/n/b4bc9f/32.png) [@Navya\_04](https://discuss.elastic.co/u/Navya_04)\
**Post date:** [March 2, 2023, 9:29am UTC](https://discuss.elastic.co/t/not-able-to-remove-tag-from-xml/326771/11 "2023-03-02T09:29:22Z")

</div>

I have tried using this ruby code but it is not removing system-err.

```auto
ruby {
code => '
event.get("theXML").each_with_index { |b,index|
event.remove("[theXML][testsuite][testcase][#{index}][system-err]")
}
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 2, 2023, 5:36pm UTC](https://discuss.elastic.co/t/not-able-to-remove-tag-from-xml/326771/12 "2023-03-02T17:36:36Z")

</div>

```
    xml { source => "message" target => "theXML" force_array => false remove_field => ["message"] }
    ruby { code => 'event.get("theXML").each_with_index { |b,index| event.remove("[theXML][testsuite][testcase][#{index}][system-err]") }' }
    ruby { code => 'event.remove("theXML").each { |k, v| event.set(k, v) }' }

```

works just fine for me.

---

<div class="post-metadata">

**Author:** ![Navya\_04](https://avatars.discourse-cdn.com/v4/letter/n/b4bc9f/32.png) [@Navya\_04](https://discuss.elastic.co/u/Navya_04)\
**Post date:** [March 3, 2023, 11:23am UTC](https://discuss.elastic.co/t/not-able-to-remove-tag-from-xml/326771/13 "2023-03-03T11:23:22Z")

</div>

When I am using this code it is working for 6 indices i.e, we are able to remove system-err upto 6th index(testcase[0] to [5]) after that from 7th index(testcase[6]) we are able to see system-err field. Is there any max limit for indices?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 3, 2023, 5:23pm UTC](https://discuss.elastic.co/t/not-able-to-remove-tag-from-xml/326771/14 "2023-03-03T17:23:17Z")

</div>

> [@Navya\_04](#):
>
> Is there any max limit for indices?

No, there is not.

---

<div class="post-metadata">

**Author:** ![Navya\_04](https://avatars.discourse-cdn.com/v4/letter/n/b4bc9f/32.png) [@Navya\_04](https://discuss.elastic.co/u/Navya_04)\
**Post date:** [March 6, 2023, 11:02am UTC](https://discuss.elastic.co/t/not-able-to-remove-tag-from-xml/326771/15 "2023-03-06T11:02:41Z")

</div>

But I see it is only removing the field for few indices. Any idea why this is happening?

---

<div class="post-metadata">

**Author:** ![Navya\_04](https://avatars.discourse-cdn.com/v4/letter/n/b4bc9f/32.png) [@Navya\_04](https://discuss.elastic.co/u/Navya_04)\
**Post date:** [March 8, 2023, 8:32am UTC](https://discuss.elastic.co/t/not-able-to-remove-tag-from-xml/326771/16 "2023-03-08T08:32:22Z")

</div>

> [@Navya\_04](#):
>
> When I am using this code it is working for 6 indices i.e, we are able to remove system-err upto 6th index(testcase[0] to [5]) after that from 7th index(testcase[6]) we are able to see system-err field.

Hi @Badger,

Can you suggest a solution for this?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 5, 2023, 8:32am UTC](https://discuss.elastic.co/t/not-able-to-remove-tag-from-xml/326771/17 "2023-04-05T08:32:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.