Thanks, Aaron.
I came up with this configuration:
if ( [message] =~ /(?m).*\n\s+at/ ) {
grok {
match => [ "message", "(?m)%{DATA}\n(?<stacktrace>\s+at.*)" ]
}
fingerprint {
method => "SHA1"
source => [ "stacktrace" ]
target => "stacktrace_id"
}
}
The grok part seems to work fine, but the fingerprint plugin fails with these errors in the log:
[2017-05-05T12:04:23,954][ERROR][logstash.pipeline ] Error registering plugin {:plugin=>"#<LogStash::FilterDelegator:0x713e24a @id=\"3a6701b48090e619946678a06eac772f4892e39c-11\", @klass=LogStash::Filters::Fingerprint, @metric_events=#<LogStash::Instrument::NamespacedMetric:0x6f3f6b9f @metric=#<LogStash::Instrument::Metric:0x4c9ee531 @collector=#<LogStash::Instrument::Collector:0x4e6df34c @agent=nil, @metric_store=#<LogStash::Instrument::MetricStore:0x56745979 @store=#<Concurrent::Map:0x4283b3bb @default_proc=nil>, @structured_lookup_mutex=#<Mutex:0x798f61b7>, @fast_lookup=#<Concurrent::Map:0x58d91a59 @default_proc=nil>>>>, @namespace_name=[:stats, :pipelines, :main, :plugins, :filters, :\"3a6701b48090e619946678a06eac772f4892e39c-11\", :events]>, @logger=#<LogStash::Logging::Logger:0x73534ff8 @logger=#<Java::OrgApacheLoggingLog4jCore::Logger:0x3e79490>>, @filter=<LogStash::Filters::Fingerprint method=>\"SHA1\", source=>[\"stacktrace\"], target=>\"stacktrace_id\", id=>\"3a6701b48090e619946678a06eac772f4892e39c-11\", enable_metric=>true, periodic_flush=>false, base64encode=>false, concatenate_sources=>false>>", :error=>"translation missing: en.logstash.agent.configuration.invalid_plugin_register"}
[2017-05-05T12:04:24,452][ERROR][logstash.agent ] Pipeline aborted due to error {:exception=>#<LogStash::ConfigurationError: translation missing: en.logstash.agent.configuration.invalid_plugin_register>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-fingerprint-3.0.2/lib/logstash/filters/fingerprint.rb:60:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:257:in `register_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:268:in `register_plugins'", "org/jruby/RubyArray.java:1613:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:268:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:278:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:207:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:389:in `start_pipeline'"]}
I installed the plugin like this:
user@host:/usr/share/logstash$ sudo bin/logstash-plugin install logstash-filter-fingerprint
Logstash itself is coming from the APT repository.
Any idea what could be wrong?
Cheers,
Peter