Using Gsub to replace field values

Elastic Stack Logstash
1

Hi

I'm trying to transform field values in order to get rid of some characters however I do not know the values.
I tried this:
gsub => ["fwdPtPrecision", ".,","."]
but the fwdPtPrecision=0.001 was changed to 0.00. Which is not what I want . I do not want to mess with the values and I'm only aware of the characters I want to filter out.
Can I use standard regex?

2

Do you know what characters you are trying to get rid of? Can you give us an example. That would probably be best, your first description was kind of vague.

3

So I have these fields with the following values

fwdPtPrecision = 0.001,
tickCaptureEnabled = true,

And I want to get rid of the commas in the values. Simple

4

Can you put the config file on here as well? As well as the full log example?

5

Sorry the config is too huge to put here but here's the filter:

filter {
mutate {
remove_field => [ "unstructured_data" ]
gsub => ["fwdPtPrecision", "+d,","+d"]
}
}

And the logs
message:
2017-07-13T00:13:35,878 [INFO ][Client-Push:1:3][PfClientEnrichmentBlock] [BXA.ESP.2#XXX_1#USD/CHF@lold1] BXA.ESP.2#XXX_1#USD/CHF@lold1 client mapping resolved to ClientMappingBean{uniqueKey='BXA.ESP.2|CH02', clientRef='BXA.ESP.2', channelId='CH02', clientName='XXxx ESP 2', spotTemplateId='SPT02', throttleTemplateId='THR02', priceValidationTemplateId='PVT06', liquidityTemplateId='LET02', spotEntitlementTemplateId='SET02', fwdSpreadTemplateId='FST01', fwdEntitlementTemplateId='FET02', skewTemplateId='SKW03', tickCaptureEnabled=true, FwdSpeedBump=false, fwdPtPrecision=0.001, clientDisabled=false}
EventType:
PfClientEnrichmentBlock
@timest

6

Are you breaking these apart into separate fields or are you trying to gsub on the whole log message? Are you going to break the log message apart with groks and such? You said:

`

So I have these fields with the following values

`

But from your configuration file these aren't separated into fields in Logstash/Elasticsearch yet.

7

They're all seperated into fields by grok. I just want to get rid of the commas that appear with some of the field's values.

8

gsub => [“fwdPtPrecision”, “,”,""]

Try this. It will replace all the commas in the field fwdPtPrecision with a blank string, effectively removing the commas from the field.

9

Thanks. That did the trick for all the bits I didn't want in there.

1 Like
10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.