Copy a field without making it a hash value?

Elastic Stack Logstash
1

Hi I have an array field. And i want to copy it without turning it to a hash data type, which is what "add_field" does. Is there any ideas? Tks!

PS: the reason i want to do it is because i now have field search_terms = [0]good, [1] day; and i want to ALSO have a new field search_query = "good day". Both fields at same time.

2

Well, you could always use a ruby filter to make low-level manipulations of events. But I don't get what you mean by this:

And i want to copy it without turning it to a hash data type, which is what "add_field" does

Could you give an example?

3

Sure...for example, i have an array field search_terms = [0]good, [1] day. If i add_field "temp" => "%{search_terms}", this field temp will not be able to join, since it is a hash data type. It can not be converted in to any other format, only the hash format as: good,day.

4

Your use of the word "hash" is incorrect. There is no hash involved here. This shows why concrete examples with configuration snippets and copy/paste from terminal windows is so useful and reduces confusion.

If you want to join the search_terms array with spaces instead of commas without overwriting the original field you can use a ruby filter. Or, maybe using the mutate filter's gsub option to replace the commas in the joined string with spaces would work.

5

Hi I want to have search_terms both in an array format and as a phrase. If i join search_terms, i would not have it as array format. If I copy search_term using add_field, the copied value is a hash data type, based on the online documentation, and become "good,day" only, not changeable to any other format...(https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html)

Below is my config FYI:
originally k = good++day
mutate {
gsub => [ "k", "[++]", " "]
split => { "k" => " " }
add_field => { "search_terms"=> "%{k}" }
join => { "search_terms" => " "} ## search_terms = good,day, as a hash data type, per the documentation
gsub => [ "search_terms", "[,]", " "] ## nothing will be done since gsub doesn't work on hash type
}

6

As for Ruby, is it similar like below? Would you point out the syntax error pls...coz i never write ruby:

Ruby{
init => "search_terms" = "%{k}"
}

7

The reason your mutate filter doesn't work as expected is that the options aren't applied in the order you specify them. The order is instead defined here:

In other words, the gsub will happen first, then the join (and finally the add_field). You have to use multiple mutate filters.

If I copy search_term using add_field, the copied value is a hash data type, based on the online documentation

You are interpreting the documentation incorrectly.

As for Ruby, is it similar like below?

No. Something like

ruby {
  code => "
    event['search_terms'] = event['k'].join(' ')
  "
}

should work though.

1 Like