How to remove additional comma in part of message

Elastic Stack Logstash
1

I am trying to use csv filter to separate log message to several fields including message, date, time, module..., they are separated by comma. But when it is a error message, the first part contains some commas, which makes the fields all go wrong. I noticed that in these error message, comma usually live inside { } or ( ). So Is there any way to identify commas inside { } or ( ) and replace them? Thanks!

2

The mutate filter's gsub option can be used for regexp-based string substitution.

3

Thanks Magnus!

I tried in https://regex101.com/ , this finds me all commas inside () " ,(?!(([^(]"){2})[^)]*$)"

with this input Message: The underlying provider failed on Open.; A network-related or instance-specific error occurred while establishing a connection to SQL Server. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server, no connections, nothing, , ,); The system cannot find the file specified; , Date: 2018-02-14, Time: 15:46:00.0986942, Guid: 000, LogSource: WebApi.Repository, ModuleName: GetLogSources, Arguments: , UserId: , Type: Error

So I went add following in my logstash.confi file, but got error when I tried to start logstash
mutate{
gsub =>[
"message", ",(?!(([^(]"){2})[^)]*$)", "."
]
}

Can you give me some advices? Thanks!

4

Never mind Magnus, I've figured out. It just didn't like the double quotation inside that regex expression. I replaced it with # and it works! Thanks!

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.