ECS expect `target` value

Elastic Stack Logstash
1

Hello boys and girls,

I'm playing with logstash and sflow codec input

Trying to enrich the data with geoip and get this error:

[WARN ][logstash.filters.geoip   ][3_sflow] ECS expect `target` value `destination.geo.ip` in ["client", "destination", "host", "observer", "server", "source"]

The ip fields in the index are in this format:
src_ip and dst_ip

 geoip {
     add_tag => "GeoIP-DST"
     source => "dst_ip"
     target => "destination.geo.ip"
     }

     geoip {
     add_tag => "GeoIP-SRC"
     source => "src_ip"
     target => "source.geo.ip"
     }

How do I get this to work so it's ECS compatible?

Thanks

2

Try [destination][geo][ip].

3

Hi Badger,

Thanks for your help!

I've tried it and the log message complains about the same thing:

[WARN ][logstash.filters.geoip   ][3_sflow] ECS expect `target` value `[source][geo][ip]` in ["client", "destination", "host", "observer", "server", "source"]

I'm not sure what format the ip fields should be in, can't really find an example.

I've tried renaming dst_ip and src_ip fields to source.ip and destination.ip - the reasoning behind is that filebeat netflow module uses this naming convention but no luck with that either.

I've seen this post already and if i disable ecs compatibility I'm guessing it will work, but I'd like to get it working with ECS, if possible :slight_smile:

4

If you set target => "destination" then you will not get an error message and the destination field will be

"destination" => {
    "geo" => {
        "country_iso_code" => "US",
                "timezone" => "America/Chicago",
            "country_name" => "United States",
          "continent_code" => "NA",
                "location" => {
            "lon" => -97.822,
            "lat" => 37.751
        }
    },
     "ip" => "131.2.88.4"
},

I have to assume that is ECS compliant.

5

I've changed it to this and the message is gone:

  geoip {
     add_tag => [ "GeoIP-DST" ]
     source => "dst_ip"
     target => "destination"
     }

     geoip {
     add_tag => [ "GeoIP-SRC"]
     source => "src_ip"
     target => "source"
     }

However, I must have another issue, the geo ip fields are not showing

[2022-07-02T09:19:23,822][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"8.3.1", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.15+10 on 11.0.15+10 +indy +jit [linux-x86_64]"}
[2022-07-02T09:19:23,824][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED]
[2022-07-02T09:19:25,069][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2022-07-02T09:19:26,097][INFO ][org.reflections.Reflections] Reflections took 127 ms to scan 1 urls, producing 124 keys and 408 values
[2022-07-02T09:19:27,063][INFO ][logstash.javapipeline    ] Pipeline `1_syslog` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
[2022-07-02T09:19:27,068][INFO ][logstash.javapipeline    ] Pipeline `2_beat` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
[2022-07-02T09:19:27,181][INFO ][logstash.outputs.elasticsearch][2_beat] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["https://localhost:9200"]}
[2022-07-02T09:19:27,214][INFO ][logstash.outputs.elasticsearch][1_syslog] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["https://localhost:9200"]}
[2022-07-02T09:19:27,775][INFO ][logstash.outputs.elasticsearch][1_syslog] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://elastic:xxxxxx@localhost:9200/]}}
[2022-07-02T09:19:27,777][INFO ][logstash.outputs.elasticsearch][2_beat] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://elastic:xxxxxx@localhost:9200/]}}
[2022-07-02T09:19:28,250][WARN ][logstash.outputs.elasticsearch][1_syslog] Restored connection to ES instance {:url=>"https://elastic:xxxxxx@localhost:9200/"}
[2022-07-02T09:19:28,253][WARN ][logstash.outputs.elasticsearch][2_beat] Restored connection to ES instance {:url=>"https://elastic:xxxxxx@localhost:9200/"}
[2022-07-02T09:19:28,265][INFO ][logstash.outputs.elasticsearch][1_syslog] Elasticsearch version determined (8.3.1) {:es_version=>8}
[2022-07-02T09:19:28,267][WARN ][logstash.outputs.elasticsearch][1_syslog] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>8}
[2022-07-02T09:19:28,268][INFO ][logstash.outputs.elasticsearch][2_beat] Elasticsearch version determined (8.3.1) {:es_version=>8}
[2022-07-02T09:19:28,269][WARN ][logstash.outputs.elasticsearch][2_beat] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>8}
[2022-07-02T09:19:28,322][WARN ][logstash.filters.translate] You are using a deprecated config setting "destination" set in translate. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. Use `target` option instead. If you have any questions about this, please visit the #logstash channel on freenode irc. {:name=>"destination", :plugin=><LogStash::Filters::Translate destination=>"protocol", dictionary=>{"1"=>"ETHERNET", "11"=>"IP"}, override=>true, id=>"0d33e9f1edb51994de81a78c5d9678ed3974d3ac1f58f3f3a0968c4c6b0d13ca", field=>"protocol", fallback=>"UNKNOWN", enable_metric=>true, periodic_flush=>false, refresh_interval=>300, exact=>true, regex=>false, refresh_behaviour=>"merge">}
[2022-07-02T09:19:28,323][WARN ][logstash.filters.translate] You are using a deprecated config setting "field" set in translate. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. Use `source` option instead. If you have any questions about this, please visit the #logstash channel on freenode irc. {:name=>"field", :plugin=><LogStash::Filters::Translate destination=>"protocol", dictionary=>{"1"=>"ETHERNET", "11"=>"IP"}, override=>true, id=>"0d33e9f1edb51994de81a78c5d9678ed3974d3ac1f58f3f3a0968c4c6b0d13ca", field=>"protocol", fallback=>"UNKNOWN", enable_metric=>true, periodic_flush=>false, refresh_interval=>300, exact=>true, regex=>false, refresh_behaviour=>"merge">}
[2022-07-02T09:19:28,335][WARN ][logstash.filters.translate] You are using a deprecated config setting "destination" set in translate. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. Use `target` option instead. If you have any questions about this, please visit the #logstash channel on freenode irc. {:name=>"destination", :plugin=><LogStash::Filters::Translate destination=>"eth_type", dictionary=>{"2048"=>"IP", "33024"=>"802.1Q VLAN"}, override=>true, id=>"aee2848c28519d4cdfb68ea328ce6c13349fef552e6de9fb5771a9cede96580a", field=>"eth_type", fallback=>"UNKNOWN", enable_metric=>true, periodic_flush=>false, refresh_interval=>300, exact=>true, regex=>false, refresh_behaviour=>"merge">}
[2022-07-02T09:19:28,336][WARN ][logstash.filters.translate] You are using a deprecated config setting "field" set in translate. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. Use `source` option instead. If you have any questions about this, please visit the #logstash channel on freenode irc. {:name=>"field", :plugin=><LogStash::Filters::Translate destination=>"eth_type", dictionary=>{"2048"=>"IP", "33024"=>"802.1Q VLAN"}, override=>true, id=>"aee2848c28519d4cdfb68ea328ce6c13349fef552e6de9fb5771a9cede96580a", field=>"eth_type", fallback=>"UNKNOWN", enable_metric=>true, periodic_flush=>false, refresh_interval=>300, exact=>true, regex=>false, refresh_behaviour=>"merge">}
[2022-07-02T09:19:28,342][WARN ][logstash.filters.translate] You are using a deprecated config setting "destination" set in translate. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. Use `target` option instead. If you have any questions about this, please visit the #logstash channel on freenode irc. {:name=>"destination", :plugin=><LogStash::Filters::Translate destination=>"vlan_type", dictionary=>{"2048"=>"IP"}, override=>true, id=>"a7d2fc5dc0c5e732199919176787a78a9d3ea5518789d1766a6e2e47c00acc60", field=>"vlan_type", fallback=>"UNKNOWN", enable_metric=>true, periodic_flush=>false, refresh_interval=>300, exact=>true, regex=>false, refresh_behaviour=>"merge">}
[2022-07-02T09:19:28,343][WARN ][logstash.filters.translate] You are using a deprecated config setting "field" set in translate. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. Use `source` option instead. If you have any questions about this, please visit the #logstash channel on freenode irc. {:name=>"field", :plugin=><LogStash::Filters::Translate destination=>"vlan_type", dictionary=>{"2048"=>"IP"}, override=>true, id=>"a7d2fc5dc0c5e732199919176787a78a9d3ea5518789d1766a6e2e47c00acc60", field=>"vlan_type", fallback=>"UNKNOWN", enable_metric=>true, periodic_flush=>false, refresh_interval=>300, exact=>true, regex=>false, refresh_behaviour=>"merge">}
[2022-07-02T09:19:28,343][INFO ][logstash.outputs.elasticsearch][2_beat] Config is not compliant with data streams. `data_stream => auto` resolved to `false`
[2022-07-02T09:19:28,349][WARN ][logstash.filters.translate] You are using a deprecated config setting "destination" set in translate. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. Use `target` option instead. If you have any questions about this, please visit the #logstash channel on freenode irc. {:name=>"destination", :plugin=><LogStash::Filters::Translate destination=>"ip_protocol", dictionary=>{"6"=>"TCP", "17"=>"UDP", "50"=>"Encapsulating Security Payload"}, override=>true, id=>"e8b589bbc3147104e4d18eabf9f908e2cb60617c8c38299e372afdc27b2e851d", field=>"ip_protocol", fallback=>"UNKNOWN", enable_metric=>true, periodic_flush=>false, refresh_interval=>300, exact=>true, regex=>false, refresh_behaviour=>"merge">}
[2022-07-02T09:19:28,350][WARN ][logstash.filters.translate] You are using a deprecated config setting "field" set in translate. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. Use `source` option instead. If you have any questions about this, please visit the #logstash channel on freenode irc. {:name=>"field", :plugin=><LogStash::Filters::Translate destination=>"ip_protocol", dictionary=>{"6"=>"TCP", "17"=>"UDP", "50"=>"Encapsulating Security Payload"}, override=>true, id=>"e8b589bbc3147104e4d18eabf9f908e2cb60617c8c38299e372afdc27b2e851d", field=>"ip_protocol", fallback=>"UNKNOWN", enable_metric=>true, periodic_flush=>false, refresh_interval=>300, exact=>true, regex=>false, refresh_behaviour=>"merge">}
[2022-07-02T09:19:28,354][INFO ][logstash.outputs.elasticsearch][1_syslog] Config is compliant with data streams. `data_stream => auto` resolved to `true`
[2022-07-02T09:19:28,357][WARN ][logstash.filters.translate] You are using a deprecated config setting "destination" set in translate. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. Use `target` option instead. If you have any questions about this, please visit the #logstash channel on freenode irc. {:name=>"destination", :plugin=><LogStash::Filters::Translate destination=>"interface_direction", dictionary=>{"0"=>"Ingress", "1"=>"Egress"}, override=>true, id=>"a87ed469614b5a31ef89e8ca1409bd1e1edf89aa104560c9b6f9f3ead91b8d76", field=>"interface_direction", fallback=>"UNKNOWN", enable_metric=>true, periodic_flush=>false, refresh_interval=>300, exact=>true, regex=>false, refresh_behaviour=>"merge">}
[2022-07-02T09:19:28,358][WARN ][logstash.filters.translate] You are using a deprecated config setting "field" set in translate. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. Use `source` option instead. If you have any questions about this, please visit the #logstash channel on freenode irc. {:name=>"field", :plugin=><LogStash::Filters::Translate destination=>"interface_direction", dictionary=>{"0"=>"Ingress", "1"=>"Egress"}, override=>true, id=>"a87ed469614b5a31ef89e8ca1409bd1e1edf89aa104560c9b6f9f3ead91b8d76", field=>"interface_direction", fallback=>"UNKNOWN", enable_metric=>true, periodic_flush=>false, refresh_interval=>300, exact=>true, regex=>false, refresh_behaviour=>"merge">}
[2022-07-02T09:19:28,381][INFO ][logstash.outputs.elasticsearch][1_syslog] Config is compliant with data streams. `data_stream => auto` resolved to `true`
[2022-07-02T09:19:28,382][WARN ][logstash.outputs.elasticsearch][1_syslog] Elasticsearch Output configured with `ecs_compatibility => v8`, which resolved to an UNRELEASED preview of version 8.0.0 of the Elastic Common Schema. Once ECS v8 and an updated release of this plugin are publicly available, you will need to update this plugin to resolve this warning.
[2022-07-02T09:19:28,383][INFO ][logstash.outputs.elasticsearch][2_beat] Config is not compliant with data streams. `data_stream => auto` resolved to `false`
[2022-07-02T09:19:28,388][WARN ][logstash.outputs.elasticsearch][2_beat] Elasticsearch Output configured with `ecs_compatibility => v8`, which resolved to an UNRELEASED preview of version 8.0.0 of the Elastic Common Schema. Once ECS v8 and an updated release of this plugin are publicly available, you will need to update this plugin to resolve this warning.
[2022-07-02T09:19:28,399][WARN ][logstash.filters.grok    ][1_syslog] ECS v8 support is a preview of the unreleased ECS v8, and uses the v1 patterns. When Version 8 of the Elastic Common Schema becomes available, this plugin will need to be updated
[2022-07-02T09:19:28,464][INFO ][logstash.outputs.elasticsearch][2_beat] Using a default mapping template {:es_version=>8, :ecs_compatibility=>:v8}
[2022-07-02T09:19:28,470][INFO ][logstash.outputs.elasticsearch][1_syslog] Using a default mapping template {:es_version=>8, :ecs_compatibility=>:v8}
[2022-07-02T09:19:28,551][INFO ][logstash.javapipeline    ] Pipeline `3_sflow` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
[2022-07-02T09:19:28,574][INFO ][logstash.outputs.elasticsearch][3_sflow] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["https://localhost:9200"]}
[2022-07-02T09:19:28,579][INFO ][logstash.javapipeline    ][2_beat] Starting pipeline {:pipeline_id=>"2_beat", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["/etc/logstash/conf.d/2_beat.conf"], :thread=>"#<Thread:0x17f401ed@/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52 run>"}
[2022-07-02T09:19:28,637][INFO ][logstash.outputs.elasticsearch][3_sflow] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://elastic:xxxxxx@localhost:9200/]}}
[2022-07-02T09:19:28,676][INFO ][logstash.javapipeline    ][1_syslog] Starting pipeline {:pipeline_id=>"1_syslog", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["/etc/logstash/conf.d/1_syslog.conf"], :thread=>"#<Thread:0x25268b65 run>"}
[2022-07-02T09:19:28,677][WARN ][logstash.outputs.elasticsearch][3_sflow] Restored connection to ES instance {:url=>"https://elastic:xxxxxx@localhost:9200/"}
[2022-07-02T09:19:28,681][INFO ][logstash.outputs.elasticsearch][3_sflow] Elasticsearch version determined (8.3.1) {:es_version=>8}
[2022-07-02T09:19:28,681][WARN ][logstash.outputs.elasticsearch][3_sflow] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>8}
[2022-07-02T09:19:28,701][INFO ][logstash.outputs.elasticsearch][3_sflow] Config is not compliant with data streams. `data_stream => auto` resolved to `false`
[2022-07-02T09:19:28,701][INFO ][logstash.outputs.elasticsearch][3_sflow] Config is not compliant with data streams. `data_stream => auto` resolved to `false`
[2022-07-02T09:19:28,704][WARN ][logstash.outputs.elasticsearch][3_sflow] Elasticsearch Output configured with `ecs_compatibility => v8`, which resolved to an UNRELEASED preview of version 8.0.0 of the Elastic Common Schema. Once ECS v8 and an updated release of this plugin are publicly available, you will need to update this plugin to resolve this warning.
[2022-07-02T09:19:28,712][INFO ][logstash.outputs.elasticsearch][3_sflow] Using a default mapping template {:es_version=>8, :ecs_compatibility=>:v8}
[2022-07-02T09:19:29,295][INFO ][logstash.javapipeline    ][2_beat] Pipeline Java execution initialization time {"seconds"=>0.7}
[2022-07-02T09:19:29,296][INFO ][logstash.javapipeline    ][1_syslog] Pipeline Java execution initialization time {"seconds"=>0.62}
[2022-07-02T09:19:29,319][INFO ][logstash.javapipeline    ][1_syslog] Pipeline started {"pipeline.id"=>"1_syslog"}
[2022-07-02T09:19:29,323][INFO ][logstash.inputs.beats    ][2_beat] Starting input listener {:address=>"0.0.0.0:5044"}
[2022-07-02T09:19:29,455][INFO ][logstash.inputs.udp      ][1_syslog][7296050127100992961da3de5418467673f9ef434a3938cb67c0127676bdf079] Starting UDP listener {:address=>"0.0.0.0:5045"}
[2022-07-02T09:19:29,566][INFO ][logstash.inputs.udp      ][1_syslog][7296050127100992961da3de5418467673f9ef434a3938cb67c0127676bdf079] UDP listener started {:address=>"0.0.0.0:5045", :receive_buffer_bytes=>"106496", :queue_size=>"2000"}
[2022-07-02T09:19:29,641][INFO ][logstash.javapipeline    ][2_beat] Pipeline started {"pipeline.id"=>"2_beat"}
[2022-07-02T09:19:29,699][INFO ][org.logstash.beats.Server][2_beat][6394aac8289bfe9010abc7e28a3a89205ad89e48e073dccfd4819805b4e14d5d] Starting server on port: 5044
[2022-07-02T09:19:30,614][INFO ][logstash.filters.geoip.downloadmanager] new database version detected? false
[2022-07-02T09:19:30,716][INFO ][logstash.filters.geoip.databasemanager][3_sflow] By not manually configuring a database path with `database =>`, you accepted and agreed MaxMind EULA. For more details please visit https://www.maxmind.com/en/geolite2/eula
[2022-07-02T09:19:30,717][INFO ][logstash.filters.geoip   ][3_sflow] Using geoip database {:path=>"/var/lib/logstash/plugins/filters/geoip/1656666572/GeoLite2-City.mmdb"}
[2022-07-02T09:19:30,760][INFO ][logstash.filters.geoip.databasemanager][3_sflow] By not manually configuring a database path with `database =>`, you accepted and agreed MaxMind EULA. For more details please visit https://www.maxmind.com/en/geolite2/eula
[2022-07-02T09:19:30,760][INFO ][logstash.filters.geoip   ][3_sflow] Using geoip database {:path=>"/var/lib/logstash/plugins/filters/geoip/1656666572/GeoLite2-City.mmdb"}
[2022-07-02T09:19:30,784][INFO ][logstash.javapipeline    ][3_sflow] Starting pipeline {:pipeline_id=>"3_sflow", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["/etc/logstash/conf.d/3_sflow.conf"], :thread=>"#<Thread:0x3dbe2d12@/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:51 run>"}
[2022-07-02T09:19:30,937][INFO ][logstash.javapipeline    ][3_sflow] Pipeline Java execution initialization time {"seconds"=>0.15}
[2022-07-02T09:19:30,941][INFO ][logstash.javapipeline    ][3_sflow] Pipeline started {"pipeline.id"=>"3_sflow"}
[2022-07-02T09:19:30,962][INFO ][logstash.inputs.udp      ][3_sflow][517f8687ff474c25f5e5d3b0fbad50d0f1f54924cdfbe7b757cf82d1ad64f934] Starting UDP listener {:address=>"0.0.0.0:6343"}
[2022-07-02T09:19:30,964][INFO ][logstash.inputs.udp      ][3_sflow][517f8687ff474c25f5e5d3b0fbad50d0f1f54924cdfbe7b757cf82d1ad64f934] UDP listener started {:address=>"0.0.0.0:6343", :receive_buffer_bytes=>"106496", :queue_size=>"2000"}
[2022-07-02T09:19:30,973][INFO ][logstash.agent           ] Pipelines running {:count=>3, :running_pipelines=>[:"1_syslog", :"2_beat", :"3_sflow"], :non_running_pipelines=>[]}

my sflow template

{
  "sflow" : {
    "order" : 0,
    "version" : 60001,
    "index_patterns" : [
      "sflow-*"
    ],
    "settings" : {
      "index" : {
        "number_of_shards" : "1",
        "refresh_interval" : "5s"
      }
    },
    "mappings" : {
      "properties" : {
        "agent_ip" : {
          "type" : "ip"
        },
        "src_ip" : {
          "type" : "ip"
        },
        "src_port" : {
          "type" : "integer"
        },
        "host.ip" : {
          "type" : "ip"
        },
        "frame_length_times_sampling_rate" : {
          "type" : "long"
        },
        "@timestamp" : {
          "type" : "date"
        },
        "geoip" : {
          "dynamic" : true,
          "properties" : {
            "ip" : {
              "type" : "ip"
            },
            "latitude" : {
              "type" : "half_float"
            },
            "location" : {
              "type" : "geo_point"
            },
            "longitude" : {
              "type" : "half_float"
            }
          }
        },
        "@version" : {
          "type" : "keyword"
        },
        "dst_port" : {
          "type" : "integer"
        },
        "dst_ip" : {
          "type" : "ip"
        }
      }
    },
    "aliases" : { }
  }
}
6

Try enabling --log.level debug. It may be going through one of the exception pathways. If the source field does not exist the event would get tagged, so I don't think it is that.

7

Thanks so much Badger,
I think I might have made an error in the config file

I had

if [type] == "sflow"

in the geoip filter but that type is not referenced anywhere so it was just getting ignored?

input {
  udp {
    port => 6343
    codec => sflow
    tags => ["sflow"]
    }
}


filter {

      if [type] == "sflow" {

  geoip {
     add_tag => [ "GeoIP-DST" ]
     source => "dst_ip"
     target => "destination"
     }

     geoip {
     add_tag => [ "GeoIP-SRC"]
     source => "src_ip"
     target => "source"
     }

     }# if type sflow
}

output {
  elasticsearch { hosts => ["https://localhost:9200"]
  index => "sflow-%{+YYYY.MM.dd}"
  cacert => '/etc/logstash/certs/ca.pem'
  user => elastic
  password => "${ES_PWD}"
  }

}
8

You have added an "sflow" tag, but are testing the [type] field. Try

if "sflow" in [tags] { ...
1 Like
9

Thanks Badger, I have a feeling you've originally helped me with this issue years back

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.