Logstash Error

Elastic Stack Logstash
1

I am trying run the logstash with the below pattern:
.* PulseSecure: .*%{IP:[additionalinfo][client_public_ip]}.*\\%{USERNAME:[additionalinfo][suser]}.*%{IP:[additionalinfo][client_ip]}

for the below sample log:

<182>1 2021-01-05T11:41:56+05:30 welconnect.welspun.com PulseSecure: - - - 2021-01-05 11:41:56 - PRIMARY-VPN - [1.186.78.90] WELSPUNGRP\Ninea_SanketC(WELSPUN_AUTH_REALM)[NINEA_SANKETC] - VPN Tunneling: User with IP 10.20.24.165 connected with SSL transport mode.

when I run this pattern, I am getting the below error:

[2021-01-27T15:51:39,227][ERROR][logstash.javapipeline ] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<RegexpError: unmatched close parenthesis: /.* PulseSecure: .(?IP:additionalinfo][client_public_ip(?:(?:((([0-9A-Fa-f]{1,4}:){7}([0-9A-Fa-f]{1,4}|:))|(([0-9A-Fa-f]{1,4}:){6}(:[0-9A-Fa-f]{1,4}|((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){5}(((:[0-9A-Fa-f]{1,4}){1,2})|:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){4}(((:[0-9A-Fa-f]{1,4}){1,3})|((:[0-9A-Fa-f]{1,4})?:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){3}(((:[0-9A-Fa-f]{1,4}){1,4})|((:[0-9A-Fa-f]{1,4}){0,2}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){2}(((:[0-9A-Fa-f]{1,4}){1,5})|((:[0-9A-Fa-f]{1,4}){0,3}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){1}(((:[0-9A-Fa-f]{1,4}){1,6})|((:[0-9A-Fa-f]{1,4}){0,4}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(:(((:[0-9A-Fa-f]{1,4}){1,7})|((:[0-9A-Fa-f]{1,4}){0,5}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:)))(%.+)?)|(?:(?<![0-9])(?:(?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])...)(?![0-9]))))](?:.?)(?USERNAME:additionalinfo][suser[a-zA-Z0-9._-]+).*(?IP:additionalinfo][client_ip(?:(?:((([0-9A-Fa-f]{1,4}:){7}([0-9A-Fa-f]{1,4}|:))|(([0-9A-Fa-f]{1,4}:){6}(:[0-9A-Fa-f]{1,4}|((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){5}(((:[0-9A-Fa-f]{1,4}){1,2})|:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){4}(((:[0-9A-Fa-f]{1,4}){1,3})|((:[0-9A-Fa-f]{1,4})?:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){3}(((:[0-9A-Fa-f]{1,4}){1,4})|((:[0-9A-Fa-f]{1,4}){0,2}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){2}(((:[0-9A-Fa-f]{1,4}){1,5})|((:[0-9A-Fa-f]{1,4}){0,3}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){1}(((:[0-9A-Fa-f]{1,4}){1,6})|((:[0-9A-Fa-f]{1,4}){0,4}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(:(((:[0-9A-Fa-f]{1,4}){1,7})|((:[0-9A-Fa-f]{1,4}){0,5}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:)))(%.+)?)|(?:(?<![0-9])(?:(?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])...)(?![0-9]))))/m>, :backtrace=>["org/jruby/RubyRegexp.java:940:in initialize'", "/usr/local/seceon/logstash-7.2.0-SECEON/vendor/bundle/jruby/2.5.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:127:in compile'", "/usr/local/seceon/logstash-7.2.0-SECEON/logstash/filters/grok_and_formalize.rb:321:in block in register'", "org/jruby/RubyArray.java:1792:in each'", "/usr/local/seceon/logstash-7.2.0-SECEON/logstash/filters/grok_and_formalize.rb:318:in block in register'", "org/jruby/RubyHash.java:1419:in each'", "/usr/local/seceon/logstash-7.2.0-SECEON/logstash/filters/grok_and_formalize.rb:300:in register'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:56:in register'", "/usr/local/seceon/logstash-7.2.0-SECEON/logstash-core/lib/logstash/java_pipeline.rb:192:in block in register_plugins'", "org/jruby/RubyArray.java:1792:in each'", "/usr/local/seceon/logstash-7.2.0-SECEON/logstash-core/lib/logstash/java_pipeline.rb:191:in register_plugins'", "/usr/local/seceon/logstash-7.2.0-SECEON/logstash-core/lib/logstash/java_pipeline.rb:463:in maybe_setup_out_plugins'", "/usr/local/seceon/logstash-7.2.0-SECEON/logstash-core/lib/logstash/java_pipeline.rb:204:in start_workers'", "/usr/local/seceon/logstash-7.2.0-SECEON/logstash-core/lib/logstash/java_pipeline.rb:146:in run'", "/usr/local/seceon/logstash-7.2.0-SECEON/logstash-core/lib/logstash/java_pipeline.rb:105:in `block in start'"], :thread=>"#<Thread:0xa03c668 run>"}

2

Please edit your post, select the grok pattern, and click on </> in the toolbar above the edit pane. That will prevent characters from your pattern being consumed as formatting.

3

That is done, can someone please help answering on urgent basis. I have to submit my assignment I tried multiple things but not getting successful. Actually I know the problem arising here is:
If you see my log sample above that contain

in the log, and I need to use this as the username. When I am using my grok pattern as mentioned above it is showing correctly but logstash is throwing error bacause of \ that I used in grok pattern. But I am not sure how to replace this \ with other grok pattern.

4

When I run logstash with

input { generator { count => 1 lines => [ '<182>1 2021-01-05T11:41:56+05:30 welconnect.welspun.com PulseSecure: - - - 2021-01-05 11:41:56 - PRIMARY-VPN - [1.186.78.90] WELSPUNGRP\Ninea_SanketC(WELSPUN_AUTH_REALM)[NINEA_SANKETC] - VPN Tunneling: User with IP 10.20.24.165 connected with SSL transport mode.' ] } }
filter {
    grok { match => { "message" => ".* PulseSecure: .*%{IP:[additionalinfo][client_public_ip]}.*\\%{USERNAME:[additionalinfo][suser]}.*%{IP:[additionalinfo][client_ip]}" } }
}

I get

"additionalinfo" => {
    "client_public_ip" => "1.186.78.90",
               "suser" => "Ninea_SanketC",
           "client_ip" => "10.20.24.165"
},

so I do not think you are doing what you think you are doing.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.