I have the below Json submitted to my logstash from kinesis where the message string in the json contains all the required values which I need to store it as a separate field in elasticsearch . Currently its stored as a single message string field, I tried using the mutate and json filter but unable to get the results
{
"@timestamp": "2019-08-29T13:02:47.468Z",
"id": "34947135803057280532301987232169408483366183409373609984",
"message": "{\"@timestamp\": \"2019-08-29 13:02:47.467\", \"priority\": \"INFO\", \"application\": \"authentication\", \"class\": \"com.ecomm.environment.PropertiesEnvironmentConfig\", \"file\": \"PropertiesEnvironmentConfig.java:287\", \"requestId\": \"\", \"correlationId\": \"\", \"clientCorrelationId\": \"\", \"breadcrumb\": \"\", \"principalId\": \"\", \"customerId\": \"\", \"clientUserAgent\": \"\", \"payload\": {\"logType\":\"EVENT\",\"message\":\"Loaded environment properties\",\"details\":{\"environmentType\":\"sandbox\",\"filename\":\"env.properties\"}} }\n",
"@version": "1",
"messageType": "DATA_MESSAGE",
}
I am looking for something like this,
{
"@timestamp": "2019-08-29T13:02:47.468Z",
"id": "34947135803057280532301987232169408483366183409373609984",
"payload": {
"logType":"EVENT",
"message":"Loaded environment properties"
},
"environmentType":"sandbox",
"filename":"env.properties",
"priority": "INFO",
"application": "authentication",
"class": "com.ecomm.environment.PropertiesEnvironmentConfig",
"file": "PropertiesEnvironmentConfig.java:287",
"requestId": "",
"correlationId": "",
"clientCorrelationId": "",
"breadcrumb": "",
"principalId": "",
"customerId": "",
"clientUserAgent": "",
"@version": "1",
"messageType": "DATA_MESSAGE"
}