Hi,
We are having an Elastic cluster for storing logs from various security devices. We are trying to ingest Logs from an Endpoint server through Logstash to elastic cluster. The sample log format is as below.
"message" =>"2024-05-16T18:00:55+05:30 node2 -: CEF:0|Seqrite|EPS|8.2|Virus Scan Event|^|endpointName=ACCPC^domainName=WORKGROUP^ipAddressFromClient=192.168.1.10^macID1=AA-FF-E6-67-90-3G^macID2=^macID3=^groupName=MISC_PC^incidentOn=Thu May 16 17:47:14 IST 2024^fileName=C:\doc\ACC Setups\ACC Project\xyz.zip^fileNameOnly=xyz.zip^virusName=Infected Archive^actionId=VP1^userName=abc^ "
We are facing issue in parsing the logs. Grok filter plugin and key value filter has been used to parse the logs.
The configuration of the logstash pipeline is as below.
input
{
beats {
port => XXXX
}
}
filter
{
grok
{
match => {"message" => "%{TIMESTAMP_ISO8601:timestamp} %{WORD:node} %{GREEDYDATA} LEEF:%{NUMBER}?\|%{WORD:vendor}\|%{WORD:product}\|%{NUMBER:version}\|%{GREEDYDATA:eventtype}\|\^\|%{GREEDYDATA:message}"
overwrite => [ "message" ]
ecs_compatibility => disabled
}
kv
{
ecs_compatibility => disabled
source => "message"
field_split => "^"
value_split => "="
}
}
Output
{
}
On starting the logstash pipeline the following error is generated and pipeline is shutdown.
[ERROR] 2024-05-17 12:53:03.128 [[main]-pipeline-manager] javapipeline - Pipeline error {:pipeline_id=>"main", :exception=>#<RegexpError: empty char-class: /[^]/>, :backtrace=>["org/jruby/RubyRegexp.java:908:in `initialize'", "org/jruby/RubyClass.java:904:in `new'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-filter-kv-4.7.0/lib/logstash/filters/kv.rb:389:in `register'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-mixin-ecs_compatibility_support-1.3.0-java/lib/logstash/plugin_mixins/ecs_compatibility_support/target_check.rb:48:in `register'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:75:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:237:in `block in register_plugins'", "org/jruby/RubyArray.java:1989:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:236:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:611:in `maybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:249:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:194:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:146:in `block in start'"], "pipeline.sources"=>["/etc/logstash/conf.d/pipe1.conf"], :thread=>"#<Thread:0x117bbb81 /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
[INFO ] 2024-05-17 12:53:03.129 [[main]-pipeline-manager] javapipeline - Pipeline terminated {"pipeline.id"=>"main"}
[ERROR] 2024-05-17 12:53:03.143 [Converge PipelineAction::Create<main>] agent - Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
What could be done to resolve this?
Thank you.
