How to remove special character from string in logstash

Elastic Stack Logstash
1

I'm getting some fields in the kibana like "attributes.host_mem_file_buffer 1.3747773440e+09|g".
I want to first replace with "|g" to "" and then convert to float value.
I tried below one but it's not replacing "|g" to ""(empty)
mutate {
lowercase => [ "host" ]
gsub => [ "attributes.host_mem_mem_free", "|g", "" ]
gsub => [ "host_mem_mem_free", "|g", "" ]
gsub => [ "host.mem.mem.free", "|g", "" ]
}

Can you please help me how to remove "|g" in the string

2

Since | has a special meaning in regular expressions I'd try replacing \|g instead.

3

Thanks for your reply but not working. Below is my code. Can you please suggest me how to change entire attribute object "|g"
mutate {
lowercase => [ "host" ]
#gsub => [ "m_host_mem_mem_free", "|g", "" ]
gsub => [ "m_host_mem_mem_free", "|g", "" ]
}

My Json object is coming into the kibana
{
"_index": "logstash-metrics-2016.12.07",
"_type": "json",
"_id": "AVjY9S9M",
"_score": null,
"_source": {
"id": "TL4NDQHCOSX4QEJP42C5KCJ23WS74NH====",
"body": "",
"host": "vc2..com",
"source": "",
"ts": 1481089953634,
"location": "RTP",
"service": "metrics",
"event_type_id": 107,
"attributes": {
"m_host_mem_mem_cache": "2.0229201920e+09|g",
"m_host_mem_mem_free": "3.8571212800e+09|g",
"m_host_mem_free_mem_swap": "1.9327344640e+10|g",
"m_host_mem_file_buffer": "8.2237849600e+08|g",
"m_host_mem_mem_total": "8.0994344960e+09|g",
"m_host_mem_mem_swap": "1.9327344640e+10|g"
},
"@version": "1",
"@timestamp": "2016-12-07T11:04:56.335Z",
"type": "json"
},
"fields": {
"@timestamp": [
1481108696335
]
},
"sort": [
1481108696335
]
}

4

The field name isn't m_host_mem_mem_free, it's [attributes][m_host_mem_mem_free].

5

Is below format is correct?
gsub => [ "[attributes][m_host_mem_mem_free]", "|g", "" ]

1 Like
6

That looks correct, yes.

7

Thanks a lot it works.
gsub => [ "[_source][attributes][m_host_mem_mem_free]", "|g", "" ]
convert => { "[_source][attributes][m_host_mem_mem_free]" => "float" }

8

I have another question related to same . I have 100 filed names values like "|g" and field names come like below
"attributes": {
"m_host_mem_mem_cache": "2.0229201920e+09|g",
"m_host_mem_mem_free": "3.8571212800e+09|g",
"m_host_mem_free_mem_swap": "1.9327344640e+10|g",
"m_host_mem_file_buffer": "8.2237849600e+08|g",
"m_host_mem_mem_total": "8.0994344960e+09|g",
"m_host_mem_mem_swap": "1.9327344640e+10|g"
},
How can I use iterator and dynamic replacement "|g" with empty and convert the float. Please give me solution.

9

I'm pretty sure you have to use a ruby filter.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.