# \[@metadata\]\[field\] not behaving as expected

**URL:** https://discuss.elastic.co/t/metadata-field-not-behaving-as-expected/111554
**Category:** Logstash
**Created:** [December 13, 2017, 12:20pm UTC](https://discuss.elastic.co/t/metadata-field-not-behaving-as-expected/111554 "2017-12-13T12:20:50Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![tgdesrochers](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tgdesrochers/32/51322_2.png) [@tgdesrochers](https://discuss.elastic.co/u/tgdesrochers)
#### Post date: [December 13, 2017, 12:20pm UTC](https://discuss.elastic.co/t/metadata-field-not-behaving-as-expected/111554/1 "2017-12-13T12:20:50Z")

</div>

I am using [@metadata] fields for filtering and outputs. I have an issue with how the field is behaving when there are two @metadata fields with different names. Below is a stripped down config I am using for testing:

```
input {
  tcp {
    codec => json_lines { charset => "CP1252" }
    port => 80
    type => "eventlog"
    add_field => { "[@metadata][source]" => "sysmon_raw" }
    ssl_enable => false
  }
}
filter{
  if [@metadata][source] == "sysmon_raw" {
    if [Channel] != "Microsoft-Windows-Sysmon/Operational" {
      mutate {
        add_field => { "[@metadata][eventlog]" => "eventlog_raw" }
      }
    }
  }
}
output {
  if [@metadata][eventlog] == "eventlog_raw" {
    stdout { codec => rubydebug { metadata => true } }
  }
}
```

This works as expected and data with [@metadata][eventlog] gets printed to screen. But when I use an output to account for both @metadata fields, the data goes into both outputs.

Example output:

```
output {
  if [@metadata][source] == "sysmon_raw" {
    kafka {
      bootstrap_servers => "host1.com:9092"
      topic_id => "sysmon_raw"
      compression_type => "lz4"
      codec => "json"
    }
  }
  if [@metadata][eventlog] == "eventlog_raw" {
    kafka {
      bootstrap_servers => "host2.com:9092"
      topic_id => "win_eventlog_raw"
      compression_type => "lz4"
      codec => "json"
    }
  }
}
```

The data from the [@metadata][eventlog] field foes into the [@metadata][source] output, but not the opposite way around. How can I use the @metadata field to ensure only the data from each field goes into the correct output. I have tried removing the [@metadata][source] field but no luck removing it.

Example of the output of the data:

```
{
                  "Task" => 0,
              "Keywords" => -9223372036854775808,
               "Message" => "%SYSTEM32%\\LOCATIONNOTIFICATIONWINDOWS.EXE was allowed to run.",
             "EventType" => "INFO",
              "UserData" => "",
                "Opcode" => "Info",
                  "type" => "eventlog",
         "SeverityValue" => 2,
               "Version" => 0,
                "UserID" => "S-1-5-21-3710767279-2211127119-3867209129-1003",
              "@version" => "1",
                  "host" => "10.223.57.0",
     "ExecutionThreadID" => 25072,
           "OpcodeValue" => 0,
      "SourceModuleType" => "im_msvistalog",
             "@metadata" => {
        "eventlog" => "eventlog_raw",
          "source" => "sysmon_raw"
    },
               "Channel" => "Microsoft-Windows-AppLocker/EXE and DLL",
              "Hostname" => "hostname.com",
            "SourceName" => "Microsoft-Windows-AppLocker",
    "ExecutionProcessID" => 996,
              "Severity" => "INFO",
           "AccountType" => "User"
     "EventReceivedTime" => "2017-12-12 14:38:32",
      "SourceModuleName" => "eventlogs",
          "ProviderGuid" => "{CBDA4DBF-8D5D-4F69-9578-BE14AA540D22}",
            "@timestamp" => 2017-12-12T19:40:58.149Z,
                  "port" => 65524,
             "EventTime" => "2017-12-12 14:38:30",
               "EventID" => 8002,
                "Domain" => "domain.name",
          "RecordNumber" => 334,
           "AccountName" => "user"
```

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [December 13, 2017, 12:47pm UTC](https://discuss.elastic.co/t/metadata-field-not-behaving-as-expected/111554/2 "2017-12-13T12:47:53Z")

</div>

Perhaps

```nohighlight
if [@metadata][eventlog] == "eventlog_raw" {
  ...
} else if [@metadata][source] == "sysmon_raw" {
  ...
}

```

is what you're looking for, i.e. send eventlog\_raw events to one output, otherwise send it to another output if it's a sysmon\_raw event, but never sent events to both outputs.

---

<div class="post-metadata">

### Author: ![tgdesrochers](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tgdesrochers/32/51322_2.png) [@tgdesrochers](https://discuss.elastic.co/u/tgdesrochers)
#### Post date: [December 13, 2017, 1:00pm UTC](https://discuss.elastic.co/t/metadata-field-not-behaving-as-expected/111554/3 "2017-12-13T13:00:09Z")

</div>

Yes I am looking to do that exact thing, send each to their own output but never to both. What that logic above souldn't it still out `eventlog_raw` data in `sysmon_raw` since all `[@metadata][source]` data contains `sysmon_raw` and `eventlog_raw` fields.

```auto
if [@metadata][source] == "sysmon_raw" { 
  if [Channel] != "Microsoft-Windows-Sysmon/Operational" { 
    mutate { add_field => { "[@metadata][eventlog]" => "eventlog_raw" } } 
  } 
}

```

So when it outputs any event with `sysmon_raw` it will contain all `eventlog_raw` data. How can I drop the `[@metadata][source]` for any data that meets the criteria above to receive the `[@metadata][eventlog]` field so that the value `sysmon_raw` doesn't even exist in that data.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [December 13, 2017, 1:41pm UTC](https://discuss.elastic.co/t/metadata-field-not-behaving-as-expected/111554/4 "2017-12-13T13:41:05Z")

</div>

> How can I drop the `[@metadata][source]` for any data that meets the criteria above to receive the `[@metadata][eventlog]` field so that the value `sysmon_raw` doesn't even exist in that data.

That's of course easy to do, but what's the point? If `[@metadata][eventlog]` is "eventlog\_raw" then the event won't reach the sysmon\_raw topic. Removing `[@metadata][source]` won't make a difference.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 10, 2018, 1:41pm UTC](https://discuss.elastic.co/t/metadata-field-not-behaving-as-expected/111554/5 "2018-01-10T13:41:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
