# After update to 8.1.3 Logstash in cluster no longer functioning

**URL:** <https://discuss.elastic.co/t/after-update-to-8-1-3-logstash-in-cluster-no-longer-functioning/303439>\
**Category:** Logstash\
**Created:** [April 27, 2022, 7:48pm UTC](https://discuss.elastic.co/t/after-update-to-8-1-3-logstash-in-cluster-no-longer-functioning/303439 "2022-04-27T19:48:04Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![hueyg](https://avatars.discourse-cdn.com/v4/letter/h/8c91f0/32.png) [@hueyg](https://discuss.elastic.co/u/hueyg)\
**Post date:** [April 27, 2022, 7:48pm UTC](https://discuss.elastic.co/t/after-update-to-8-1-3-logstash-in-cluster-no-longer-functioning/303439/1 "2022-04-27T19:48:04Z")

</div>

Running an 8.1.3 cluster with all components updated. Filebeat agent is also latest.

This particular host has multiple log files that I was shipping, then creating a different index on each logfile based on the "type" field that I specify in the filebeat.yml. I was under the impression that this option was going away, but I am still unclear. Here is the current setup:

```auto
- type: log
  enabled: true
  paths:
    - /u01/wso2is-5.7.0/repository/logs/audit.log*
  fields:
    type: ssoaudit
  fields_under_root: true

- type: log
  enabled: true
  paths:
    - /u01/wso2is-5.7.0/repository/logs/wso2carbon.log*
  fields:
    type: carbonlog
  fields_under_root: true

```

Here are some of the errors from the logstash nodes, but I am not sure all which are relevant:

```auto
[2022-04-27T15:32:07,882][WARN][logstash.filters.grok][main] ECS v8 support is a preview of the unreleased ECS v8, and uses the v1 patterns. When Version 8 of the Elastic Common Schema becomes available, this plugin will need to be updated

[2022-04-27T15:32:09,112][ERROR][logstash.javapipeline][main] Pipeline error {:pipeline_id=>"main", :exception=>#<LogStash::ConfigurationError: GeoIP Filter in ECS-Compatiblity mode requires a `target` when `source` is not an `ip` sub-field, eg. [client][ip]>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-geoip-7.2.12-java/lib/logstash/filters/geoip.rb:143:in `auto_target_from_source!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-geoip-7.2.12-java/lib/logstash/filters/geoip.rb:133:in `setup_target_field'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-geoip-7.2.12-java/lib/logstash/filters/geoip.rb:108:in `register'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:75:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:232:in `block in register_plugins'", "org/jruby/RubyArray.java:1821:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:231:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:594:in `maybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:244:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:189:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:141:in `block in start'"], "pipeline.sources"=>["/etc/logstash/conf.d/001-input.conf", "/etc/logstash/conf.d/200-ldap-filter.conf", "/etc/logstash/conf.d/201-haproxy-filter.conf", "/etc/logstash/conf.d/202-windns-filter.conf", "/etc/logstash/conf.d/203-wso2-filter.conf", "/etc/logstash/conf.d/204-ssoaudit-filter.conf", "/etc/logstash/conf.d/205-carbonlog-filter.conf", "/etc/logstash/conf.d/206-windhcp-filter.conf", "/etc/logstash/conf.d/207-asa-filter.conf", "/etc/logstash/conf.d/900-output.conf"], :thread=>"#<Thread:0x592df691 run>"}

[2022-04-27T15:32:09,134][ERROR][logstash.agent] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}

[2022-04-27T15:32:09,288][FATAL][org.logstash.Logstash] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:94) ~[?:?]

```

Finally here are my filters if it is even related:

```auto
filter {
 if [type] == "wso2" {
    grok {
      match => {"message" => "%{COMBINEDAPACHELOG}"}
    }
    geoip {
      source => "clientip"
    }
  }
}

filter {
  if [type] == "ssoaudit" {
    grok {
        match => {
         "message" => [
"%{TIMESTAMP_ISO8601:date}\]\s\sINFO {AUDIT_LOG}-\s\sInitiator=%{NOTSPACE:user}\sAction=%{NOTSPACE:action}\sTarget=%{WORD:framework}\sData=%{NOTSPACE:data}\sOutcome=%{WORD:result}",
"%{TIMESTAMP_ISO8601:date}\]\s\sINFO {AUDIT_LOG}-\s\sInitiator : %{EMAILADDRESS:user} \| Action : %{WORD:action} \| Target : %{WORD:framework} \| Data : { \"ContextIdentifier\" : %{QUOTEDSTRING:
context},\"AuthenticatedUser\" : %{QUOTEDSTRING:fulluser},\"AuthenticatedUserTenantDomain\" : %{QUOTEDSTRING:tenant},\"ServiceProviderName\" : %{QUOTEDSTRING:serviceprovider},\"RequestType\" : %
{QUOTEDSTRING:request},\"RelyingParty\" : %{QUOTEDSTRING:relyparty},\"AuthenticatedIdPs\" : %{QUOTEDSTRING:authenticatedidps} } \| Result : %{WORD:result}",
"%{TIMESTAMP_ISO8601:date}\]\s\sINFO {AUDIT_LOG}-\s\sInitiator : %{EMAILADDRESS:user} \| Action : %{WORD:action} \| Target : %{WORD:framework} \| Data : { \"ContextIdentifier\" : %{QUOTEDSTRING:
context},\"LoggedOutUser\" : %{QUOTEDSTRING:loggedoutuser},\"LoggedOutUserTenantDomain\" : %{QUOTEDSTRING:tenant},\"ServiceProviderName\" : %{QUOTEDSTRING:serviceprovider},\"RequestType\" : %{QU
OTEDSTRING:request},\"RelyingParty\" : %{QUOTEDSTRING:relyparty},\"AuthenticatedIdPs\" : %{QUOTEDSTRING:authenticatedidps} } \| Result : %{WORD:result}",
"%{TIMESTAMP_ISO8601:date}\]\s\sINFO {AUDIT_LOG}-\s\sInitiator : %{WORD:user} \| Action : %{WORD:action} \| Target : %{WORD:framework} \| Data : { \"ContextIdentifier\" : %{QUOTEDSTRING:context}
,\"AuthenticatedUser\" : %{QUOTEDSTRING:fulluser},\"AuthenticatedUserTenantDomain\" : %{QUOTEDSTRING:tenant},\"ServiceProviderName\" : %{QUOTEDSTRING:serviceprovider},\"RequestType\" : %{QUOTEDS
TRING:request},\"RelyingParty\" : %{QUOTEDSTRING:relyparty},\"AuthenticatedIdP\" : %{QUOTEDSTRING:authenticatedidps} } \| Result : %{WORD:result}",
"%{TIMESTAMP_ISO8601:date}\]\s\sINFO {AUDIT_LOG}-\s\sInitiator : %{WORD:user} \| Action : %{WORD:action} \| Target : %{WORD:framework} \| Data : { \"ContextIdentifier\" : %{QUOTEDSTRING:context}
,\"AuthenticatedUser\" : %{QUOTEDSTRING:fulluser},\"AuthenticatedUserTenantDomain\" : %{QUOTEDSTRING:tenant},\"ServiceProviderName\" : %{QUOTEDSTRING:serviceprovider},\"RequestType\" : %{QUOTEDS
TRING:request},\"RelyingParty\" : %{QUOTEDSTRING:relyparty},\"AuthenticatedIdPs\" : %{QUOTEDSTRING:authenticatedidps} } \| Result : %{WORD:result}"
                       ]
    }
    }
    grok { match => { "user" => '%{USERNAME:uid}@' } }

    mutate {
         remove_field => ["framework","tenant","relyparty","authenticatedips"]
       }
  }
}

filter {
  if [type] == "carbonlog" {
    grok {
        match => ["message" , "TID: \[%{INT:tid}\] \[\] \[%{TIMESTAMP_ISO8601:date}\]\s+%{WORD:result} \{%{USERNAME:resultsource}\} - %{GREEDYDATA:value}"]
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 27, 2022, 8:51pm UTC](https://discuss.elastic.co/t/after-update-to-8-1-3-logstash-in-cluster-no-longer-functioning/303439/2 "2022-04-27T20:51:56Z")

</div>

> [@hueyg](#):
>
> :exception=\>#\<LogStash::ConfigurationError: GeoIP Filter in ECS-Compatiblity mode requires a `target` when `source` is not an `ip` sub-field, eg. [client][ip]\>

In 8.x ecs\_compatability is enabled by default, and when it is enabled the geoip filter no longer has geoip as the default target. You can disable ecs\_compatability at the filter, pipeline, or process level to get the old behaviour.

---

<div class="post-metadata">

**Author:** ![hueyg](https://avatars.discourse-cdn.com/v4/letter/h/8c91f0/32.png) [@hueyg](https://discuss.elastic.co/u/hueyg)\
**Post date:** [April 28, 2022, 7:35pm UTC](https://discuss.elastic.co/t/after-update-to-8-1-3-logstash-in-cluster-no-longer-functioning/303439/3 "2022-04-28T19:35:48Z")

</div>

Thanks Badge, I didn't think this would prevent logstash from running, but apparently I was wrong.

I disabled the ecs\_compatability and switched from using type to [tags] when differentiating the different log files. Everything is working now.

Thank you again for assistance.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 26, 2022, 7:36pm UTC](https://discuss.elastic.co/t/after-update-to-8-1-3-logstash-in-cluster-no-longer-functioning/303439/4 "2022-05-26T19:36:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.