Asterisk in Mutate with GSUB

Elastic Stack Logstash
1

@magnusbaeck

Here is the thing, which broken in our production.

mutate {
     gsub => ["message", "%{docNumber}", "%{docNumberMasked}"]
}

I have to type all the configuration, if there is any typo or syntactical errors please dont mind.

docNumber value is *. docNumberMasked is also *

Now because of this our production Logstash went down saying
<RegexpError: target of repeat operator is not specified /*/>

When I used multiple pipelines my understanding is that it should not impact logstash, if anything happens it will only shutdown that particular pipeline, but because of this our entire logstash instance went down. Is this something expected ?
Anyhow I configured only 1 pipeline for this Logstash instance.

Second:
Is there way to inform Logstash such that it wont shutdown but it will keep a track of such events ?
Can I achieve this using dead_letter_queue?

Thanks,
Suman G

2

@magnusbaeck any help here?

3

@Suman_Reddy1 directly pinging individual members when they have not explicitly offered their help, is rude. Please refrain from doing this.

This shouldn't be taking down the whole process, but the mutate plugin explicitly warns about regex special characters in the gsub directive.

That said, I can't think of a reliable way to do string substitution using existing plugins.

Perhaps the Ruby filter may be our best bet here (the following is pseudo-configs and has not been tested):

filter {
  ruby {
    code => "
      message = event.get('message')
      message && begin
        docNumber = event.get('[docNumber]')
        docNumberMasked = event.get('[docNumberMasked]')
        event.set('message', message.tr(docNumber, docNumberMasked))
      end
    "
  }
}
4

Thank you! Will not happen again. Please understand that this is a prod issue,

5

These are community forums, where the community of people who maintain and use these products help each other out, as we have time. The user you tagged is fairly active here and often provides valuable insight, but no one on these forums has any obligation to drop what they are doing to address your issue.

If you would like real-time production support, you may be interested in one of Elastic's subscriptions, which provide dedicated access to a team of support engineers who can provide valuable feedback tailor-fit for your specific use-case.

6

Noted. Thank you @yaauie

I am still trying to scratch my head to find out why logstash process went down. Any idea?

7

I've opened up a ticket on the mutate filter plug in here:

8

Thankyou @yaauie

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.