Combining two inputs how to access the fields

[ericohtake]

Hi,
I have 2 inputs in a config file where I need to send a field from one to another if a condition is met. However I'm failing to find documentation on how to reference the fields living in different input events.

Below I put a simple example on what I'm trying to do to not overcomplicate the question, since my real requirement is to combine DB2 and Elasticsearch inputs.

input {
  generator {
    lines => ["1234"]
    count => 1
    type => "db"
  }

  generator {
    lines => ["5678"]
    count => 1
    type => "es"
  }
}

filter {
  if [type] == "db" {
    mutate {
      add_field => { "from_es_event" => "%{[type:es][message]}" }
    }
  }
}

This is the result expected:

      "sequence" => 0,
    "@timestamp" => 2017-11-29T01:57:02.933Z,
"from_es_event"  => "5678",
       "message" => "1234",
          "type" => "db"

How can I accomplish this?

[magnusbaeck (Magnus Bäck)]

Maybe the aggregate filter could help? This isn't a use case that Logstash handles very well.

[ericohtake]

Thank you for your reply Magnus. I was analyzing the aggregate filter, but I'm not sure how to give the plugin the fields addresses (depending on the events) so it can do the lookup and add the field in the other event. Any idea how to address the field for a given event?

[magnusbaeck (Magnus Bäck)]

No, sorry. I've never used that filter.

[ericohtake]

No problem. I will index both datasets and reindex using LS query with joins.