How to change the content of message field

Elastic Stack Beats
1

Hi All,

Input log content:
{"time": "2019-04-03T23:35:17.751Z","stream":"stdout","log":"message content information\n"}

console output:

{
"@timestamp": "2019-04-05T04:21:57.239Z",
"@metadata": {
"beat": "filebeat",
"type": "doc",
"version": "6.7.0"
},
"log": {
"file": {
"path": "c:\Users\I502170\Desktop\savita\code\src\xxxxx\cloudfoundry\public-endpoint-client\integrationtests\please.log"
}
},
"message": "{"time": "2019-04-03T23:35:17.751Z","stream":"stdout","log":"message content information\n"}",
"beat": {
"name": "INLN56586635A",
"version": "6.7.0"
},
"testhost": "INLN56586635A",
"source": "c:\Users\I502170\Desktop\savita\code\src\github.infra.hana.ondemand.com\cloudfoundry\public-endpoint-client\integrationtests\please.log",
"offset": 776,
"prospector": {
"type": "log"
},
"input": {
"type": "log"
},
"host": {
"name": "INLN56586635A"
}
}

Requirement:

Before sending to output(logstash, elasticsearch etc...) we wanted to change
time ---> written_at
log ---> msg

followed all the docs but dint get exact solution for this.

Can please someone suggest for this scenario .

Thank you
Savita

2

Hey @savitaashture, welcome to elastic discuss

maybe rename input processor might be useful to you

Also please make sure to format any configuration you post here. Theres a button </> to do that. Sometimes the issue is with indentation of the configuration which is hard to spot if it is not correctly formatted

3

Hi @Michal_Pristas,

Thanks for the info...
Sure i will take care formating next time.

I tried using rename input processor
But it will change only the fields not the content of field

Ex:
"message": "{"time": "2019-04-03T23:35:17.751Z","stream":"stdout","log":"message content information\n"}",

here
message is a field which stores string info..
and i wanted to change content of message field.

log--->msg
time--->written_at

4

I'm not sure it is possible on beat directly. You can still use decode json field
which will parse your message and add additional fields so it will be

{
    "message": "{"time": "2019-04-03T23:35:17.751Z","stream":"stdout","log":"message content information\n"}",
    "target": {
        "time": "2019-04-03T23:35:17.751Z",
        "stream":"stdout",
        "log":"message content information\n",
    }
}

and then on top of it rename processor to make it:

{
    "message": "{"time": "2019-04-03T23:35:17.751Z","stream":"stdout","log":"message content information\n"}",
    "target": {
        "written_at": "2019-04-03T23:35:17.751Z",
        "stream":"stdout",
        "msg":"message content information\n",
    }
}

other than that you can look at more advanced options provided by LogStash

5

Hey @Michal_Pristas

Thanks a lot for the information.

One more query

Once we modify the fields data using decode json field

That particular field will become object .

So can we again convert it back to strings with the updated data.

Thank you
savita

6

I believe the answer is no at the moment, but it is an interesting idea

7

@Michal_Pristas,

Oh Okey...

Thank you for clearing all the doubts :slight_smile:

Thanks
Savita

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.