# Logstash - Removing duplicity from array

**URL:** <https://discuss.elastic.co/t/logstash-removing-duplicity-from-array/226164>\
**Category:** Logstash\
**Created:** [April 2, 2020, 7:12am UTC](https://discuss.elastic.co/t/logstash-removing-duplicity-from-array/226164 "2020-04-02T07:12:23Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![vasek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vasek/32/136636_2.png) [@vasek](https://discuss.elastic.co/u/vasek)\
**Post date:** [April 2, 2020, 7:12am UTC](https://discuss.elastic.co/t/logstash-removing-duplicity-from-array/226164/1 "2020-04-02T07:12:24Z")

</div>

Hello,  
I would like to share the experience with removing duplicity from array in Logstash.

I used this approach before:  
I was checking whether value was already being present in array.

```auto
  if ( [host][name] and ([host][name] not in [host][address])) {
    mutate {
      add_field => { "[host][address]" => "%{[host][name]}" }
    }
  }

  if ( [agent][hostname] and ([agent][hostname] not in [host][address])) {
    mutate {
      add_field => { "[host][address]" => "%{[agent][hostname]}" }
    }
  }  

```

**With ruby we can remove duplicity by `uniq` function very easily.**  
We can remove checks for testing presence of some value in array,

```auto
  if ( [host][name] ) {
    mutate {
      merge => { "[host][address]" => "[host][name]" }
    }
  }

  if ( [agent][hostname] ) {
    mutate {
      merge => { "[host][address]" => "[agent][hostname]" }
    }
 } 

ruby {
    code => "

       array = event.get('[host][address]')
       is_array = array.kind_of?(Array)

       if is_array
         event.set('[host][address]', event.get('[host][address]').uniq)
       end

     "
  }

```

I hope it will save you some time.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 30, 2020, 7:16am UTC](https://discuss.elastic.co/t/logstash-removing-duplicity-from-array/226164/2 "2020-04-30T07:16:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.