Logstash is replacing : with => How can I get : on the output

[Raj_Sekhar]

Logstash is replacing : with => How can I get : on the output

My input file:

{
"customfield_10150": [
{"key": "caanyimi",
"displayName": "Anyimi, Charles"}
]
}

Output:

"name""=>""caanyimi"",
""emailAddress""=>""charles.anyimi@intel.com"

[harshbajaj16 (Harsh Bajaj)]

Hi @Raj_Sekhar,

Could you please explain bit more about your use case what you are trying to do and what you want to achieve.

Regards,
Harsh Bajaj

[Raj_Sekhar]

Hi Harsh,

I have an nested array of values inside the json.
When I am trying to write to a csv file, the colons are getting replaced with =>

"customfield_10150": [
{
"key": "caanyimi",
"displayName": "Anyimi, Charles",
"self": "https://nsg-jira.intel.com/rest/api/2/user?username=caanyimi",
"avatarUrls": {
"16x16": "https://nsg-jira.intel.com/secure/useravatar?size=xsmall&ownerId=caanyimi&avatarId=18136",
"48x48": "https://nsg-jira.intel.com/secure/useravatar?ownerId=caanyimi&avatarId=18136",
"32x32": "https://nsg-jira.intel.com/secure/useravatar?size=medium&ownerId=caanyimi&avatarId=18136",
"24x24": "https://nsg-jira.intel.com/secure/useravatar?size=small&ownerId=caanyimi&avatarId=18136"
},
"active": true,
"name": "caanyimi",
"timeZone": "US/Pacific",
"emailAddress": "charles.anyimi@intel.com"
},
{
"key": "dablunde",
"displayName": "Blunden, David",
"self": "https://nsg-jira.intel.com/rest/api/2/user?username=dablunde",
"avatarUrls": {
"16x16": "https://www.gravatar.com/avatar/28dd9333e6b5dc333179817530ded97e?d=mm&s=16",
"48x48": "https://www.gravatar.com/avatar/28dd9333e6b5dc333179817530ded97e?d=mm&s=48",
"32x32": "https://www.gravatar.com/avatar/28dd9333e6b5dc333179817530ded97e?d=mm&s=32",
"24x24": "https://www.gravatar.com/avatar/28dd9333e6b5dc333179817530ded97e?d=mm&s=24"
},
"active": true,
"name": "dablunde",
"timeZone": "US/Pacific",
"emailAddress": "david.blunden@intel.com"
}
]

[harshbajaj16 (Harsh Bajaj)]

Hi @Raj_Sekhar,

I understood your point. For this please try with below line adding in filter section.

ruby {code => 'open("/tmp/test.json", "w") { |file| file.write(event.get("json").to_json) }' }

Please do let me know if still you are not able achieve the same.

Regards,
Harsh Bajaj

[Raj_Sekhar]

Hi @harshbajaj16, Tried it, but the same result.
May I know where is the file writing in your command ?

[harshbajaj16 (Harsh Bajaj)]

Hi @Raj_Sekhar,

You need to add this in your logstash configuration file which is in /conf.d/ directory.

There are three section in conf file Input, Filter and Output. You need to add this in filter section.

Please find document link for filter ruby plugin.

Ruby filter plugin | Logstash Reference [8.11] | Elastic

Also, i found a discussion for similar problem. You can look into this and can get more idea about filter plugin.

JSON parsing problem

Hi there! My target is very simple (as it seems): I want logstash to receive a json from http, process this json with some scrypt, and bypass it as json next. I've started with json codec, but I've not found any way to properly access the root of event object, so I could not get the whole json content. So, I saw this How to read JSON input sent to Http input plugin in filter section topic and tried to make config like in the answer. Let's say I've this config now: input { http { id => …

File in the command is /tmp/test.json.

Please do let me know if you need more help in this regard.

Regards,
Harsh Bajaj

[Raj_Sekhar]

Hi @harshbajaj16,

Below is my conf file after adding what you have suggested to add.
input {
stdin {
codec => "json"
}
}

filter {

    if "_jsonparsefailure" in [tags] {drop { }} # for last record which comes with text->Impala query scan limit reached

ruby {code => 'open("/tmp/abc.json", "w") { |file| file.write(event.get("json").to_json) }' }
}

mutate {
rename => {"[priority][name]" => "priority_name"}
rename => {"[priority][id]" => "priority_id"}
rename => {"[priority][self]" => "priority_self"}
}
output {
stdout {codec => rubydebug { metadata => true }}
stdout { codec => dots }

file {
codec => "json"
path => ["/tmp/logstash_output/test/%{key}.json"]
write_behavior => "overwrite"
}

csv {fields => ["priority_name","priority_id","priority_self"]
	path => ["/tmp/output_csv/%{key}.csv"]
	write_behavior => "overwrite"
}

}

[Badger]

@Raj_Sekhar

stdout {codec => rubydebug { metadata => true }}

rubydebug always displays data using => but that should not affect the format written to your file or csv output.