Mutate - replace field string using another pattern

[Arthur_Francis]

I have been reading through the logstash documentation for a way to chnage the value of a field by running it through a different pattern.

Ref:
replace
Value type is hash
There is no default value for this setting.
Replace a field with a new value. The new value can include %{foo} strings to help you build a new value from other parts of the event.

Example:

filter {
  mutate {
    replace => { "message" => "%{source_host}: My new message" }
  }
}

I tried to use the logic below to rewrite the field but it does not work it only deletes the field and does not run the following grok filter again:

if "Caused by" in [exception] {
        mutate {
            remove_field => "exception"
        }
        grok {
            patterns_dir => ["./patterns"]
            match => ["message", "%{GREEDYDATA}\n%{JAVA_EXCEPTION_LONG:exception}"]
        }
}

Will it work if i do the following instead?

 if "Caused by" in [exception] {
            mutate {
                replace => { "exception" => "%{JAVA_EXCEPTION_LONG}: exception" }
    }

[magnusbaeck (Magnus Bäck)]

Will it work if i do the following instead?

No, that won't work.

What does an example event look like? What does the rest of your configuration look like?

[Arthur_Francis]

Thanks for the reply Magnus,

the sample message I am parsing is this

2016-11-15 05:19:28,801 ERROR [App-Initialisation-Thread] appengine.java:520 Failed to initialize external authenticator myapp Support Access || appuser@vm23-13:/mnt/data/install/assembly app-1.4.12@cad85b224cce11eb5defa126030f21fa867b0dad
java.lang.IllegalArgumentException: Could not check if provided root is a directory
	at com.myapp.io.AbstractRootPrefixedFileSystem.checkAndGetRoot(AbstractRootPrefixedFileSystem.java:67)
	at com.myapp.io.AbstractRootPrefixedFileSystem.<init>(AbstractRootPrefixedFileSystem.java:30)
	at com.myapp.io.s3.S3FileSystem.<init>(S3FileSystem.java:32)
	at com.myapp.io.s3.S3FileSystemDriver.loadFileSystem(S3FileSystemDriver.java:60)
	at com.myapp.io.FileSystems.getFileSystem(FileSystems.java:55)
	at com.myapp.authentication.ldap.S3LdapConfigProvider.initializeCloudFS(S3LdapConfigProvider.java:77)
	at com.myapp.authentication.ldap.S3LdapConfigProvider.loadS3Config(S3LdapConfigProvider.java:51)
	at com.myapp.authentication.ldap.S3LdapConfigProvider.getLdapConfig(S3LdapConfigProvider.java:42)
	at com.myapp.authentication.ldap.DelegatingLdapConfigProvider.getLdapConfig(DelegatingLdapConfigProvider.java:45)
	at com.myapp.authentication.ldap.LdapExternalAuthenticatorFactory.create(LdapExternalAuthenticatorFactory.java:28)
	at com.myapp.authentication.ldap.LdapExternalAuthenticatorFactory.create(LdapExternalAuthenticatorFactory.java:10)
	at com.myapp.frob.appengine.getExternalAuthenticators(appengine.java:516)
	at com.myapp.frob.appengine.startUp(appengine.java:871)
	at com.myapp.frob.appengine.startUp(appengine.java:754)
	at com.myapp.jsp.KewServeInitContextListener$1.run(QServerInitContextListener.java:104)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.nio.file.NoSuchFileException: fh-ldap-config/
	at com.upplication.s3fs.util.S3Utils.getS3ObjectSummary(S3Utils.java:55)
	at com.upplication.s3fs.util.S3Utils.getS3FileAttributes(S3Utils.java:64)
	at com.upplication.s3fs.S3FileSystemProvider.readAttributes(S3FileSystemProvider.java:463)
	at com.myapp.io.AbstractRootPrefixedFileSystem.checkAndGetRoot(AbstractRootPrefixedFileSystem.java:61)

I actually managed to get it working by doing an overwrite instead:

if [exception] =~ "Caused" {
                mutate {
                    add_tag => [ "has_exception" ]
                }
                grok {
                    patterns_dir => ["./patterns"]
                    match => ["exception", "%{JAVA_EXCEPTION_LONG:exception}"]
                    overwrite => ["exception"]
                }
    }