# Logstash parsing

**URL:** <https://discuss.elastic.co/t/logstash-parsing/339929>\
**Category:** Logstash\
**Created:** [August 2, 2023, 10:35am UTC](https://discuss.elastic.co/t/logstash-parsing/339929 "2023-08-02T10:35:29Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![dilipchiru](https://avatars.discourse-cdn.com/v4/letter/d/7ea924/32.png) [@dilipchiru](https://discuss.elastic.co/u/dilipchiru)\
**Post date:** [August 2, 2023, 10:35am UTC](https://discuss.elastic.co/t/logstash-parsing/339929/1 "2023-08-02T10:35:29Z")

</div>

Hi Team,

I have 2 Fields which is From and TO which contains set of values which is comma separated.  
For example:

"from" : "Loin, Elephant, cat, movie, John"  
"to" : "Loin, Elephant, cat, movie, John, **USA**"

Now we would like to get the difference/unique values between the From & TO field and dynamically assign in it new field. As we can " **USA**" is the difference/unique .

Please suggest me to get this in the logstash using ruby

Thanks  
DILIP BK

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [August 2, 2023, 3:32pm UTC](https://discuss.elastic.co/t/logstash-parsing/339929/2 "2023-08-02T15:32:07Z")

</div>

Try something like this.

**Conf**

```auto
input {
  generator {
    lines => [
      '{"from": "Loin, Elephant, cat, butter, movie, John", "to": "Loin, Elephant, cat, movie, John, USA"}'
    ]
    codec => "json"
    count => 1
  }
}

filter {
  ruby {
    code => '
      from_array = event.get("from").split(", ").map(&:strip).to_set
      to_array = event.get("to").split(", ").map(&:strip).to_set

      difference = from_array ^ to_array

      event.set("unique_fields", difference.to_a.join(", "))
    '
  }
}

output {
  stdout { codec => json_lines }
}

```

**Output**

```auto
{
    "unique_fields": "USA, butter",
    "from": "Loin, Elephant, cat, butter, movie, John",
    "to": "Loin, Elephant, cat, movie, John, USA"
}

```

---

<div class="post-metadata">

**Author:** ![dilipchiru](https://avatars.discourse-cdn.com/v4/letter/d/7ea924/32.png) [@dilipchiru](https://discuss.elastic.co/u/dilipchiru)\
**Post date:** [August 8, 2023, 5:16am UTC](https://discuss.elastic.co/t/logstash-parsing/339929/3 "2023-08-08T05:16:01Z")

</div>

Hi ,

Thanks for the help. small suggestion need , where the values "from" & "to" are dynamic. can we ignore the input Generator plugin and directly we can add the grok pattern with the ruby script ??

Please help me with this

Thanks  
DILIP BK

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [August 8, 2023, 11:02am UTC](https://discuss.elastic.co/t/logstash-parsing/339929/4 "2023-08-08T11:02:17Z")

</div>

The input generator is just to test the pipeline with the data. Replace the input with your input and as long as the data that's coming in looks like the data I used it should work.

---

<div class="post-metadata">

**Author:** ![dilipchiru](https://avatars.discourse-cdn.com/v4/letter/d/7ea924/32.png) [@dilipchiru](https://discuss.elastic.co/u/dilipchiru)\
**Post date:** [August 14, 2023, 5:44am UTC](https://discuss.elastic.co/t/logstash-parsing/339929/5 "2023-08-14T05:44:21Z")

</div>

Hi @aaron-nimocks ,

Thanks for the help, It is working as required. Solved my issue

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 11, 2023, 5:44am UTC](https://discuss.elastic.co/t/logstash-parsing/339929/6 "2023-09-11T05:44:23Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.