# Exclude if not in array

**URL:** <https://discuss.elastic.co/t/exclude-if-not-in-array/105552>\
**Category:** Logstash\
**Created:** [October 27, 2017, 10:35am UTC](https://discuss.elastic.co/t/exclude-if-not-in-array/105552 "2017-10-27T10:35:42Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![henrilabarre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/henrilabarre/32/16909_2.png) [@henrilabarre](https://discuss.elastic.co/u/henrilabarre)\
**Post date:** [October 27, 2017, 10:35am UTC](https://discuss.elastic.co/t/exclude-if-not-in-array/105552/1 "2017-10-27T10:35:42Z")

</div>

Hi here is my source:

```
{
"telephone" => {
        "fine" => {
             "date" => "2017-10-26T16:54:28.477Z",
            "value" => "0387931080"
        }
    },
"TEL" => "0387931080",
}

```

I use this conf file to add the field telephone/fine/value and TEL to the same field:

```
filter {
  if "TEL" not in [TEL2] {
	  mutate {
		 merge => { "TEL2" => "TEL" }
	  }
  }
  if "telephone[fine][value]" not in [TEL2] {
	  mutate {
		 merge => { "TEL2" => "telephone[fine][value]" }
	  }
  }
  mutate {
    join => { "TEL2" => "," }
  }
}
output {
  stdout { codec => rubydebug }
}

```

The idea is to have a field with unique value of TEL, but I get duplicate:

```
"TEL2" => "0387931080,0387931080",

```

Thanks for point me what I miss!

---

<div class="post-metadata">

**Author:** ![paz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/paz/32/28003_2.png) [@paz](https://discuss.elastic.co/u/paz)\
**Post date:** [October 30, 2017, 12:44pm UTC](https://discuss.elastic.co/t/exclude-if-not-in-array/105552/3 "2017-10-30T12:44:44Z")

</div>

Your field reference is slightly malformed. Try this instead:

```auto
filter {
    if [TEL] not in [TEL2] {
	    mutate {
		    merge => { "TEL2" => "TEL" }
	    }
    }
    if [telephone][fine][value] not in [TEL2] {
	    mutate {
		    merge => { "TEL2" => "telephone[fine][value]" }
	    }
    }
    mutate {
        join => { "TEL2" => "," }
    }
}
output {
    stdout { codec => rubydebug }
}
```

You can also replicate that exact function with some ruby code, where you can use Sets (that by default only contain unique values), in order to avoid checks so the code is more compact.

```auto
filter {
    ruby {
        init => "require 'set'"
        code => "
            event.set('TEL2', Set.new([event.get('TEL'), event.get('[telephone][fine][value]')]).to_a.join(','))
        "
    }
}
```

---

<div class="post-metadata">

**Author:** ![henrilabarre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/henrilabarre/32/16909_2.png) [@henrilabarre](https://discuss.elastic.co/u/henrilabarre)\
**Post date:** [October 30, 2017, 1:26pm UTC](https://discuss.elastic.co/t/exclude-if-not-in-array/105552/4 "2017-10-30T13:26:48Z")

</div>

Bravo and thanks for your help! It's working now

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 27, 2017, 1:27pm UTC](https://discuss.elastic.co/t/exclude-if-not-in-array/105552/5 "2017-11-27T13:27:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.