# Compare incoming string/data to keyword list, if in keyword list, flag?

**URL:** <https://discuss.elastic.co/t/compare-incoming-string-data-to-keyword-list-if-in-keyword-list-flag/212527>\
**Category:** Logstash\
**Created:** [December 19, 2019, 5:02pm UTC](https://discuss.elastic.co/t/compare-incoming-string-data-to-keyword-list-if-in-keyword-list-flag/212527 "2019-12-19T17:02:32Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![stcdarrell](https://avatars.discourse-cdn.com/v4/letter/s/a183cd/32.png) [@stcdarrell](https://discuss.elastic.co/u/stcdarrell)\
**Post date:** [December 19, 2019, 5:02pm UTC](https://discuss.elastic.co/t/compare-incoming-string-data-to-keyword-list-if-in-keyword-list-flag/212527/1 "2019-12-19T17:02:32Z")

</div>

hi,  
i'm not really sure the best way to describe what i'm trying to do.. but i will have data bring streamed into logstash. i have a growing keyword list, that is a text file.

is there a way i can compare the values/data coming in to this keyword list. if the data coming in contains values from the keyword list it creates a new boolean field? keyword=true?

does that make sense?

any suggestions would be appreciated

---

<div class="post-metadata">

**Author:** ![Transrian](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/transrian/32/58436_2.png) [@Transrian](https://discuss.elastic.co/u/Transrian)\
**Post date:** [December 19, 2019, 6:09pm UTC](https://discuss.elastic.co/t/compare-incoming-string-data-to-keyword-list-if-in-keyword-list-flag/212527/2 "2019-12-19T18:09:23Z")

</div>

Do you have control of the input keyword file? (Can you modify the format?)

If so, you could probably make use of the filter translate.

If each line of the dictionary is a regex like. '.\*mykeyword.\*', and the value associated is 'true', it could probably work (even if I'm not sure it would be the best in term of performances).

---

<div class="post-metadata">

**Author:** ![stcdarrell](https://avatars.discourse-cdn.com/v4/letter/s/a183cd/32.png) [@stcdarrell](https://discuss.elastic.co/u/stcdarrell)\
**Post date:** [December 19, 2019, 6:34pm UTC](https://discuss.elastic.co/t/compare-incoming-string-data-to-keyword-list-if-in-keyword-list-flag/212527/3 "2019-12-19T18:34:44Z")

</div>

i've looked at that approach, and the problem is, the keywords will have match WITHIN the datastream.

example: keyword: "[blogspot.com](http://blogspot.com)"  
[androidlover.blogspot.com](http://androidlover.blogspot.com) MATCH  
[iphonelover.blogspot.com](http://iphonelover.blogspot.com) MATCH  
[nokia.blogspots.com](http://nokia.blogspots.com)  
[samsung.blogspot.com](http://samsung.blogspot.com) MATCH

---

<div class="post-metadata">

**Author:** ![Transrian](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/transrian/32/58436_2.png) [@Transrian](https://discuss.elastic.co/u/Transrian)\
**Post date:** [December 19, 2019, 6:46pm UTC](https://discuss.elastic.co/t/compare-incoming-string-data-to-keyword-list-if-in-keyword-list-flag/212527/4 "2019-12-19T18:46:59Z")

</div>

That's why I talked about dictionnary using regex keys 😄

Using your example (and assuming we read the dic value from a file), we would have something like :

```ruby
filter {
  translate {
    field => "[message]"
    destination => "[match]"

    dictionary => {
      "blogspot\.com" => true
    }

    fallback => false
    regex => true
  }

}

```

Become :

```json
{
  "@timestamp": "2019-12-19T18:43:57.977Z",
  "@version": "1",
  "host": "localhost",
  "match": "true",
  "message": "androidlover.blogspot.com"
}
{
  "@timestamp": "2019-12-19T18:43:57.978Z",
  "@version": "1",
  "host": "localhost",
  "match": "true",
  "message": "iphonelover.blogspot.com"
}
{
  "@timestamp": "2019-12-19T18:43:57.978Z",
  "@version": "1",
  "host": "localhost",
  "match": "false",
  "message": "nokia.blogspots.com"
}
{
  "@timestamp": "2019-12-19T18:43:57.978Z",
  "@version": "1",
  "host": "localhost",
  "match": "true",
  "message": "samsung.blogspot.com"
}

```

Did I understand you correctly ?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 16, 2020, 6:47pm UTC](https://discuss.elastic.co/t/compare-incoming-string-data-to-keyword-list-if-in-keyword-list-flag/212527/5 "2020-01-16T18:47:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.