Json Transform in Logstash

Elastic Stack Logstash
1

I have a json like this
{
"consumer": {
"created_at": 1566912618154,
"username": "dummyapp",
"id": "079c3e6d-ca35-464b-8596-b2ace5e60481"
},
"service": {
"created_at": 1567173649,
"connect_timeout": 60000,
"protocol": "https",
"read_timeout": 60000,
"port": 9090,
"updated_at": 1567173649,
"retries": 1,
"write_timeout": 60000
},
"request": {
"querystring": {},
"size": "361",
"headers": {
"cache-control": "no-cache",
"content-length": "0",
"postman-token": "fb9e10e4-2f66-4126-beec-d5c1f7c06bf7",
"user-agent": "PostmanRuntime/7.15.0",
"accept": "/",
"accept-encoding": "gzip, deflate",
"connection": "keep-alive"
},
"method": "POST"
}
}

I want this to be transformed as

{
"Title":"Sample"
"consumer": {
    "created_at": 1566912618154,
    "username": "dummyapp"
},
"service": {
    "created_at": 1567173649,
    "connect_timeout": 60000
},
"request": {
    "querystring": {},
    "headers": {
        "user-agent": "PostmanRuntime\/7.15.0"
    }
    "method": "POST"
}

}

I am using below filter

input {
file {
path => "Path_to_Log\test.log"
start_position => "beginning"
type => "json"
}
}

filter {
mutate {
add_field => {
"Title" => "Sample"
}
remove_field =>[consumer][id]
...Some other changes
}
}

output {
file {
path => "Output_Path\Output.log"
}
}

But somehow remove_field is not working.How do i get expected output.

2

You need to add either a json codec on the input, or a json filter.

3

Yes i added json codec to input filter,it worked. How will i tranform input json to specific output json format.

4

You can use a json_lines codec on the output.

5

I removed fields but i get output as

 "consumer": [["created_at", 1566912618154], ["id", "07912445-ca35-464b-8596-b2ace5e60481"], ["username", "dummyapp"], 
"{\"created_at\":1566912618154,\"id\":\"07912445-ca35-464b-8596-b2ace5e60481\",\"username\":\"dummyapp\"}"],
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.