File output plugin add some additional field in output file

(ADG) #1

i m having a issue with logstash
i m getting logs from multiple remote server via
filebeat (INPUT)
use csv filter (FILTER)
put it in
elasticsearch & file

the issue is logs which is stored in file having two extra entries before the line
i want same format of logs to be stored in file as in remote server


input {
beats {
port => 5044


separator => ","
columns => ["DateTime" ,"hname" ,"uname" ,"cwd","fname","line"]


file{ path => "/var/log/filename"
codec => line

(Magnus Bäck) #2

Use the json_lines codec in your file output.

(ADG) #3

i m not get it how to use it
i go through the various link
but there is no example

the way i use this is like

codec => json_line

but logs say
message=>"Couldn't find any codec plugin named 'json_line

(Magnus Bäck) #4

json_lines, not json_line.

(ADG) #5

sorry my bad
i wrote wrong

but it is not give me solution i want

my input is
2018-04-17T18:40:17+0530,ip-10-0-1-14,root,honest,quotes.txt,Delhi 06
2018-04-17T18:40:17+0530,ip-10-0-1-14,root,satisfied,quotes.txt,Delhi 53
2018-04-17T18:40:17+0530,ip-10-0-1-14,root,confident,quotes.txt,Mumbai 02

my output is like this
2018-04-17T18:41:09.227Z ip-10-0-1-14 2018-04-17T18:40:17+0530,ip-10-0-1-14,root,loyal,quotes.txt,Delhi
2018-04-17T18:41:09.227Z ip-10-0-1-14 2018-04-17T18:40:17+0530,ip-10-0-1-14,root,honest,quotes.txt,Delhi 06
2018-04-17T18:41:09.227Z ip-10-0-1-14 2018-04-17T18:40:17+0530,ip-10-0-1-14,root,satisfied,quotes.txt,Delhi 53
2018-04-17T18:41:09.227Z ip-10-0-1-14 2018-04-17T18:40:17+0530,ip-10-0-1-14,root,confident,quotes.txt,Mumbai 02
2018-04-17T18:41:09.227Z ip-10-0-1-14 2018-04-17T18:40:17+0530,ip-10-0-1-14,root,success,quotes.txt,Mumbai

i got these two field extra which i mention in bold character

file{ path => "/var/log/filename"
codec => line

from this

json_lines is not give me that

(Magnus Bäck) #6

Okay, rereading your original post I see what you mean. Use the line codec but configure the message format:

codec => line {
  format => "%{message}"

(ADG) #7

its working
Thanks a lot

(system) #8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.