How to use Multiline codec and Json at the same time in a config file?

Elastic Stack Logstash
1

Hi, I have logs in the following format,

{
"Name": "xxx",
"Profession": "Developer",
"Designation": "Senior",
"StartDate": "20170317",
"Salary": "xxxx"
} {
"Name": "xxx",
"Profession": "Programmer",
"Designation": "Junior",
"StartDate": "20170519",
"Salary": "xxxx"
}

Basically, I am looking to consider each Json message as single event. I have a config file with multiline codec but after that I don't know how to consider that as Json event so that I can read input based on the field names. Can someone shed light on this?

2

What's your current config look like?

3

Hi Warkolm, here is my config file

input{
file {
path=>"E:/Ex.log"
codec => multiline {
pattern => "}"
negate => true
what => previous
}
}
}
filter {
if [message] =~ /^{.*}$/ {
json { source => message }
}
}
output{
stdout {
codec=>rubydebug
}
}

4

It looks like you have a single line that contains both the closing brace of one JSON object as well as the opening brace of the next. This means that this line would need to be split and included in both, which is not possible with a multiline pattern. You may need to gather the multiline objects and then do some post-processing to clean up/add/remove braces.

5

Hi Christian, I've changed my input file so that it looks like,

{
"Name": "xxx",
"Profession": "Developer",
"Designation": "Senior",
"StartDate": "20170317",
"Salary": "xxxx"
}
{
"Name": "xxx",
"Profession": "Programmer",
"Designation": "Junior",
"StartDate": "20170519",
"Salary": "xxxx"
}

And my config file is the same which I've mentioned above but the output is weird,

{
"path" => "E:/Ex.log",
"@timestamp" => 2017-05-15T14:44:34.982Z,
"@version" => "1",
"host" => "xxxx",
"message" => "{\r"
}
{
"path" => "E:/Ex.log",
"@timestamp" => 2017-05-15T14:44:34.982Z,
"@version" => "1",
"host" => "xxxx",
"message" => "\t"Name": "xxx",\r"
}
{
"path" => "E:/Ex.log",
"@timestamp" => 2017-05-15T14:44:34.982Z,
"@version" => "1",
"host" => "xxxx",
"message" => "\t"Profession": "Developer",\r"
}
{
"path" => "E:/Ex.log",
"@timestamp" => 2017-05-15T14:44:34.982Z,
"@version" => "1",
"host" => "xxxx",
"message" => "\t"Designation": "Senior",\r"
}
{
"path" => "E:/Ex.log",
"@timestamp" => 2017-05-15T14:44:34.982Z,
"@version" => "1",
"host" => "xxxx",
"message" => "\t"Salary": "xxx",\r"
}

Could you please help me with this?

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.