# Parsing Logstash Data

**URL:** https://discuss.elastic.co/t/parsing-logstash-data/98685
**Category:** Logstash
**Created:** [August 29, 2017, 12:30pm UTC](https://discuss.elastic.co/t/parsing-logstash-data/98685 "2017-08-29T12:30:34Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![paul1243](https://avatars.discourse-cdn.com/v4/letter/p/e99b99/32.png) [@paul1243](https://discuss.elastic.co/u/paul1243)
#### Post date: [August 29, 2017, 12:30pm UTC](https://discuss.elastic.co/t/parsing-logstash-data/98685/1 "2017-08-29T12:30:35Z")

</div>

Hi -  
I have the below message coming to the ELK, and I'd like to parse those values and create a mapping (Key =\> Value) and add as separate field in Elastic Search.

![image](https://us1.discourse-cdn.com/elastic/original/3X/e/c/ec773656f544ad55053d7605a9491916148dcd99.png)

Please let me know how could I do this.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 29, 2017, 1:41pm UTC](https://discuss.elastic.co/t/parsing-logstash-data/98685/2 "2017-08-29T13:41:42Z")

</div>

You'll have to use a ruby filter that loops over the items in the `[json][Terminal]` field and adds the new fields. Something like this should work:

```nohighlight
event.get('[json][Terminal]').each { |h|
  event.set(h['Name'], h['Value'])
}

```

---

<div class="post-metadata">

### Author: ![paul1243](https://avatars.discourse-cdn.com/v4/letter/p/e99b99/32.png) [@paul1243](https://discuss.elastic.co/u/paul1243)
#### Post date: [August 30, 2017, 11:40am UTC](https://discuss.elastic.co/t/parsing-logstash-data/98685/3 "2017-08-30T11:40:02Z")

</div>

Thanks @magnusbaeck.

That worked, Things I wanted to know:

How do I change the data type of the fields getting created. We'd expect this to be Integer.

And I've a field ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/1/51b073e1ff51f2020170391b5619f6452f57168e.png)  
I'd need to split the values with the (Colan(';')) separated and push it to 4 different fields.  
I've a ruby script created:

filter {  
ruby {  
code =\> "  
data = event.get('CpuLoad').split(';')  
data.each\_index {|i| event[i.to\_s]=times[i]  
if times[i] == '4'  
event['IndexValue'] = event['id'].index(4)}  
end  
"  
}  
}

It would be really a great help if you could help me out on this. Thanks !

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 30, 2017, 11:46am UTC](https://discuss.elastic.co/t/parsing-logstash-data/98685/4 "2017-08-30T11:46:06Z")

</div>

See [https://code-maven.com/convert-string-to-number-in-ruby](https://code-maven.com/convert-string-to-number-in-ruby). Keep in mind that if the destination field exists in ES as a string field you'll have to reindex the current index or create a new index to actually map the field as an integer.

---

<div class="post-metadata">

### Author: ![paul1243](https://avatars.discourse-cdn.com/v4/letter/p/e99b99/32.png) [@paul1243](https://discuss.elastic.co/u/paul1243)
#### Post date: [August 30, 2017, 11:49am UTC](https://discuss.elastic.co/t/parsing-logstash-data/98685/5 "2017-08-30T11:49:12Z")

</div>

Thanks so much for your quick reply @magnusbaeck.  
Could you please help me out with the other question:

And I’ve a field ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/1/51b073e1ff51f2020170391b5619f6452f57168e.png)  
I’d need to split the values with the (Colan(’;’)) separated and push it to 4 different fields.  
I’ve a ruby script created but it doesn't seems to be alright: Could you please have a look:

filter {  
ruby {  
code =\> “  
data = event.get(‘CpuLoad’).split(’;’)  
data.each\_index {|i| event[i.to\_s]=times[i]  
if times[i] == '4’  
event[‘IndexValue’] = event[‘id’].index(4)}  
end  
”  
}  
}

Thank in advance.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 30, 2017, 11:51am UTC](https://discuss.elastic.co/t/parsing-logstash-data/98685/6 "2017-08-30T11:51:55Z")

</div>

Except that I don't know what `times` comes from the script looks more or less correct. Replace `times[i]` with `data[i]`?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 27, 2017, 11:52am UTC](https://discuss.elastic.co/t/parsing-logstash-data/98685/7 "2017-09-27T11:52:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
