Using gsub for multi line patterns

Hi,
I am trying to use gsub to remove the prefix for the log line.
I have a filebeat configuration which matches multiline conguration.
The multi line log message (input to logstash) would look line this:

**** Error	Wed Oct 07 02:29:40 UTC 2020	160203778	/		start10error	psk.nucleus.ConfigurationException: lazy:/psk/dynamo/service/jdbc/SQL_production
**** Error	Wed Oct 07 02:29:40 UTC 2020	160203778	/		at psk.nucleus.SimpleComponentState.invokeSetProperty(SimpleComponentState.java:816)
**** Error	Wed Oct 07 02:29:40 UTC 2020	160203778	/		at psk.nucleus.SimpleComponentState$PropertySetterCallback.handleCallback(SimpleComponentState.java:855)
**** Error	Wed Oct 07 02:29:40 UTC 2020	160203778	/		at psk.aop.LazilyResolvedProxyImpl.resolveLazyTarget(LazilyResolvedProxyImpl.java:257)
**** Error	Wed Oct 07 02:29:40 UTC 2020	160203778	/		at psk.aop.LazilyResolvedProxyImpl$1.loadObject(LazilyResolvedProxyImpl.java:110)
**** Error	Wed Oct 07 02:29:40 UTC 2020	160203778	/		at psk.adapter.gsa.GSARepository$$EnhancerByCGLIB$$3bf38308.CGLIB$LOAD_PRIVATE_0(<generated>)
**** Error	Wed Oct 07 02:29:40 UTC 2020	160203778	/		at psk.adapter.gsa.GSARepository$$EnhancerByCGLIB$$3bf38308.getDataSource(<generated>)
**** Error	Wed Oct 07 02:29:40 UTC 2020	160203778	/		at psk.adapter.gsa.GSARepository.doStartService(GSARepository.java:7027)
**** Error	Wed Oct 07 02:29:40 UTC 2020	160203778	/		at psk.nucleus.GenericService.startService(GenericService.java:621)
**** Error	Wed Oct 07 02:29:40 UTC 2020	160203778	/		at psk.nucleus.NucleusNameResolver.startService(NucleusNameResolver.java:1937)
**** Error	Wed Oct 07 02:29:40 UTC 2020	160203778	/		at psk.nucleus.NucleusNameResolver.configureAndStartService(NucleusNameResolver.java:1591)

which should be transformed to:

**** Error	Wed Oct 07 02:29:40 UTC 2020	160203778	/		start10error	psk.nucleus.ConfigurationException: lazy:/psk/dynamo/service/jdbc/SQL_production
		at psk.nucleus.SimpleComponentState.invokeSetProperty(SimpleComponentState.java:816)
		at psk.nucleus.SimpleComponentState$PropertySetterCallback.handleCallback(SimpleComponentState.java:855)
		at psk.aop.LazilyResolvedProxyImpl.resolveLazyTarget(LazilyResolvedProxyImpl.java:257)
		at psk.aop.LazilyResolvedProxyImpl$1.loadObject(LazilyResolvedProxyImpl.java:110)
		at psk.adapter.gsa.GSARepository$$EnhancerByCGLIB$$3bf38308.CGLIB$LOAD_PRIVATE_0(<generated>)
		at psk.adapter.gsa.GSARepository$$EnhancerByCGLIB$$3bf38308.getDataSource(<generated>)
		at psk.adapter.gsa.GSARepository.doStartService(GSARepository.java:7027)
		at psk.nucleus.GenericService.startService(GenericService.java:621)
		at psk.nucleus.NucleusNameResolver.startService(NucleusNameResolver.java:1937)
		at psk.nucleus.NucleusNameResolver.configureAndStartService(NucleusNameResolver.java:1591)

I am not able to come up with the gsub pattern for this.. Can anybody help
Thanks

You could start with

mutate { gsub => [ "message", "(?!\A)\*\*\*\*[\s0-9a-zA-Z:]+/", " " ] }
2 Likes

Hi Badger,
Thanks for the fast response.
I was trying to use the default grok patterns in gsub. But came to know. gsub does not support the default grok patterns like %{SPACE} or %{UNIXPATH}. (I wonder why?)

The regex you suggested perfectly worked. Thanks a bunch.
I just had to tweak it a bit to match any unix like path at the end

gsub => ["message", "(?!\A)\*\*\*\*[\s0-9a-zA-Z:]+(/([\w_%!$@:.,+~-]+|\\.)*)+", "" ]

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.