# Extract information from field

**URL:** <https://discuss.elastic.co/t/extract-information-from-field/145376>\
**Category:** Logstash\
**Created:** [August 21, 2018, 11:25am UTC](https://discuss.elastic.co/t/extract-information-from-field/145376 "2018-08-21T11:25:48Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![saisimo02](https://avatars.discourse-cdn.com/v4/letter/s/edb3f5/32.png) [@saisimo02](https://discuss.elastic.co/u/saisimo02)\
**Post date:** [August 21, 2018, 11:25am UTC](https://discuss.elastic.co/t/extract-information-from-field/145376/1 "2018-08-21T11:25:49Z")

</div>

Hello,  
I am using Logstash to parse an XML file, I have something like that:

`<Value Obj="SPM=med48610,RGN=region1,AZ=zone1,VCM=med-4861-0-storage-vm0,Link=eth1">`

I am looking for a way to get a new field with `SPM` which contains the username and another field to indicate the number of my VM `vm0` and also the type `storage` (VCM=med-4861-0-storage-vm0)  
For the moment I can just get one field that contains all the `Obj`

Thanks for help

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 21, 2018, 11:30am UTC](https://discuss.elastic.co/t/extract-information-from-field/145376/2 "2018-08-21T11:30:11Z")

</div>

Use a kv filter to parse the field containing the "SPM=med48610,RGN=region1,AZ=zone1,VCM=med-4861-0-storage-vm0,Link=eth1" string.

---

<div class="post-metadata">

**Author:** ![saisimo02](https://avatars.discourse-cdn.com/v4/letter/s/edb3f5/32.png) [@saisimo02](https://discuss.elastic.co/u/saisimo02)\
**Post date:** [August 21, 2018, 11:37am UTC](https://discuss.elastic.co/t/extract-information-from-field/145376/3 "2018-08-21T11:37:04Z")

</div>

I have no idea how I can use the kv filter, which option can I use? thanks  
I think it's something like that

> kv { field\_split =\> "=" }

But How can I indicate that I am trying to split the "SPM=med48610,RGN=region1,AZ=zone1,VCM=med-4861-0-storage-vm0,Link=eth1"

Because my XML contains a lot of others elements

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 21, 2018, 12:32pm UTC](https://discuss.elastic.co/t/extract-information-from-field/145376/4 "2018-08-21T12:32:49Z")

</div>

Set the kv filter's `source` option to the name of the field containing the string. This probably works:

```nohighlight
kv {
  source => "some-fieldname"
  field_split => ","
}

```

---

<div class="post-metadata">

**Author:** ![saisimo02](https://avatars.discourse-cdn.com/v4/letter/s/edb3f5/32.png) [@saisimo02](https://discuss.elastic.co/u/saisimo02)\
**Post date:** [August 21, 2018, 3:08pm UTC](https://discuss.elastic.co/t/extract-information-from-field/145376/5 "2018-08-21T15:08:35Z")

</div>

Thank you Magnus, but still have a problem how I can I get a field that contains only vm0 like my example but it could be vm1... and another field with storage (and I can have different element depending on my Obj)  
I would like to use that to filter my data when I use Kibana by adding a DropDown so the user can only choose the number of vm and also the type (storage or something else)  
Thank you again, and waiting for your suggestions

---

<div class="post-metadata">

**Author:** ![saisimo02](https://avatars.discourse-cdn.com/v4/letter/s/edb3f5/32.png) [@saisimo02](https://discuss.elastic.co/u/saisimo02)\
**Post date:** [August 21, 2018, 3:13pm UTC](https://discuss.elastic.co/t/extract-information-from-field/145376/6 "2018-08-21T15:13:18Z")

</div>

In fact uising your solution I will get only a field VCM, and unfortunately sometimes this VMC is VFM or another name. I though that there is a way to look for string vm in the "SPM=med48610,RGN=region1,AZ=zone1,VCM=med-4861-0-storage-vm0,Link=eth1" and then add only vm0 (before `','`) and for the type I know that there is only 3 types, so using a condition (if) I can store the appropriate type.

---

<div class="post-metadata">

**Author:** ![rijinmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rijinmp/32/24634_2.png) [@rijinmp](https://discuss.elastic.co/u/rijinmp)\
**Post date:** [August 21, 2018, 3:38pm UTC](https://discuss.elastic.co/t/extract-information-from-field/145376/7 "2018-08-21T15:38:41Z")

</div>

Try Grok Filter

grok {  
match =\> { "message" =\> '\<%{DATA:Info}"SPM=%{DATA:SPM},RGN=%{DATA:RGN},AZ=%{DATA:AZ},VCM=%{DATA:VCM}-%{DATA:VCM}-%{DATA:VCM}-%{DATA:VCM}-%{DATA:VM},Link=%{GREEDYDATA:Link}"\>'}

}

---

<div class="post-metadata">

**Author:** ![rijinmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rijinmp/32/24634_2.png) [@rijinmp](https://discuss.elastic.co/u/rijinmp)\
**Post date:** [August 21, 2018, 3:40pm UTC](https://discuss.elastic.co/t/extract-information-from-field/145376/8 "2018-08-21T15:40:48Z")

</div>

Output will be like this

{  
"Info": [  
[  
"Value Obj="  
]  
],  
"SPM": [  
[  
"med48610"  
]  
],  
"RGN": [  
[  
"region1"  
]  
],  
"AZ": [  
[  
"zone1"  
]  
],  
"VCM": [  
[  
"med",  
"4861",  
"0",  
"storage"  
]  
],  
"VM": [  
[  
"vm0"  
]  
],  
"Link": [  
[  
"eth1"  
]  
]  
}

---

<div class="post-metadata">

**Author:** ![saisimo02](https://avatars.discourse-cdn.com/v4/letter/s/edb3f5/32.png) [@saisimo02](https://discuss.elastic.co/u/saisimo02)\
**Post date:** [August 21, 2018, 3:50pm UTC](https://discuss.elastic.co/t/extract-information-from-field/145376/9 "2018-08-21T15:50:25Z")

</div>

Thank you, your "message" is "fieldname" ?

---

<div class="post-metadata">

**Author:** ![rijinmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rijinmp/32/24634_2.png) [@rijinmp](https://discuss.elastic.co/u/rijinmp)\
**Post date:** [August 21, 2018, 3:54pm UTC](https://discuss.elastic.co/t/extract-information-from-field/145376/10 "2018-08-21T15:54:32Z")

</div>

No .. Message is a keyword . Its part of the grok filter . Fields are inside the message.

grok {  
match =\> { "message" =\> ' Here we enter the parsing pattern and fields '}

}

---

<div class="post-metadata">

**Author:** ![rijinmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rijinmp/32/24634_2.png) [@rijinmp](https://discuss.elastic.co/u/rijinmp)\
**Post date:** [August 21, 2018, 3:58pm UTC](https://discuss.elastic.co/t/extract-information-from-field/145376/11 "2018-08-21T15:58:09Z")

</div>

Field names are

Info , SPM ,RGN, AZ, VCM ,VM ,Link

And SPM field's value is med48610 , VM field's value is vm0

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 21, 2018, 4:46pm UTC](https://discuss.elastic.co/t/extract-information-from-field/145376/12 "2018-08-21T16:46:30Z")

</div>

> match =\> { "message" =\> '\<%{DATA:Info}"SPM=%{DATA:SPM},RGN=%{DATA:RGN},AZ=%{DATA:AZ},VCM=%{DATA:VCM}-%{DATA:VCM}-%{DATA:VCM}-%{DATA:VCM}-%{DATA:VM},Link=%{GREEDYDATA:Link}"\>'}

This grok expression is _extremely_ inefficient. All occurrences of DATA and GREEDYDATA should be replaced with more exact patterns.

And I still think the kv-based solution is better, and the issue with the VCM field can be solved with an additional grok or dissect filter that only looks at the VCM value.

---

<div class="post-metadata">

**Author:** ![saisimo02](https://avatars.discourse-cdn.com/v4/letter/s/edb3f5/32.png) [@saisimo02](https://discuss.elastic.co/u/saisimo02)\
**Post date:** [August 21, 2018, 6:18pm UTC](https://discuss.elastic.co/t/extract-information-from-field/145376/13 "2018-08-21T18:18:02Z")

</div>

Yes kv is working very good but still having a problem with VCM and as I mentioned this name could be different from an xml to another, so it's hard to do it for each file. That's why I thought that adding if could be a solution. But not really sure is the best way.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 21, 2018, 6:52pm UTC](https://discuss.elastic.co/t/extract-information-from-field/145376/14 "2018-08-21T18:52:01Z")

</div>

Using conditionals to run different filters depending on which fields are present sounds like an okay idea as long as the number of possible field names is reasonably small.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 18, 2018, 6:52pm UTC](https://discuss.elastic.co/t/extract-information-from-field/145376/15 "2018-09-18T18:52:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.