File output plugin add some additional field in output file

Elastic Stack Logstash
1

i m having a issue with logstash
i m getting logs from multiple remote server via
filebeat (INPUT)
use csv filter (FILTER)
and
put it in
elasticsearch & file

the issue is logs which is stored in file having two extra entries before the line
i want same format of logs to be stored in file as in remote server

Input

input {
beats {
port => 5044
}
}

Filter

csv{
separator => ","
columns => ["DateTime" ,"hname" ,"uname" ,"cwd","fname","line"]
}

Output

file{ path => "/var/log/filename"
codec => line
}

2

Use the json_lines codec in your file output.

3

i m not get it how to use it
i go through the various link

but there is no example

the way i use this is like

codec => json_line

but logs say
message=>"Couldn't find any codec plugin named 'json_line

4

json_lines, not json_line.

5

sorry my bad
i wrote wrong

but it is not give me solution i want

my input is
2018-04-17T18:40:17+0530,ip-10-0-1-14,root,loyal,quotes.txt,Delhi
2018-04-17T18:40:17+0530,ip-10-0-1-14,root,honest,quotes.txt,Delhi 06
2018-04-17T18:40:17+0530,ip-10-0-1-14,root,satisfied,quotes.txt,Delhi 53
2018-04-17T18:40:17+0530,ip-10-0-1-14,root,confident,quotes.txt,Mumbai 02
2018-04-17T18:40:17+0530,ip-10-0-1-14,root,success,quotes.txt,Mumbai

my output is like this
2018-04-17T18:41:09.227Z ip-10-0-1-14 2018-04-17T18:40:17+0530,ip-10-0-1-14,root,loyal,quotes.txt,Delhi
2018-04-17T18:41:09.227Z ip-10-0-1-14 2018-04-17T18:40:17+0530,ip-10-0-1-14,root,honest,quotes.txt,Delhi 06
2018-04-17T18:41:09.227Z ip-10-0-1-14 2018-04-17T18:40:17+0530,ip-10-0-1-14,root,satisfied,quotes.txt,Delhi 53
2018-04-17T18:41:09.227Z ip-10-0-1-14 2018-04-17T18:40:17+0530,ip-10-0-1-14,root,confident,quotes.txt,Mumbai 02
2018-04-17T18:41:09.227Z ip-10-0-1-14 2018-04-17T18:40:17+0530,ip-10-0-1-14,root,success,quotes.txt,Mumbai

i got these two field extra which i mention in bold character

file{ path => "/var/log/filename"
codec => line
}

from this

and
json_lines is not give me that

6

Okay, rereading your original post I see what you mean. Use the line codec but configure the message format:

codec => line {
  format => "%{message}"
}
7

Thanks
its working
Thanks a lot

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.