Hello team,
Is there any way using filebat (7.10), for just forwarding logs ?
The main goals is :
shipping log from differents sources (file log text)
forward them to another (file log text), without do any changes inside
i will also remove the metadata from filebeat !!
here my config file :
filebeat.inputs:
- type: log
paths:
- /ines-logs-nfs/syslog-ng/uaaifnd11rou01/LogFonct.log
- /ines-logs-nfs/syslog-ng/uaaifnd11rou02/LogFonct.log
output.file:
path: "/ines-logs-nfs/syslog-ng/"
filename: all.log
permissions: 0600
Any ideas about this ??
shaunak
(Shaunak Kashyap)
November 26, 2020, 5:42am
2
Sure, you can do this. And you can use the drop_fields
processor to remove any unwanted fields from events before they are sent to the output.
@shaunak thanks for replying !
i have tested this in my config file, but didn't work !! filebeat service failed
processors:
- drop_fields:
when:
has_fields:
fields: ["log.offset", "input.type", "fileset.name"]
ignore_missing: false
any idea ?
Regards
sai
i have changed the configuration (drop_fields) to
processors:
- drop_fields:
fields: ["log.offset", "input.type", "fileset.name", "log.source.address", "log.flags"]
but didn't works, i still show on the output file the medatada (tomestamps version, beat etc ....)
i can summarize here, what i need .
see below the output generated by filebeat :
{"@timestamp":"2020-11-26T13:19:24.149Z","@metadata":{"beat":"filebeat","type":"_doc","version":"7.10.0"},"log":{"file":{"path":"/logs-nfs/syslog-ng/XXXX/appli.log"}},"message":"2020-11-26T14:19:23+01:00 servermaster CORE: TimeStamp=\"2020-11-26 14:19:23.560\" TraceID=\"TIDcb28e32379fd73bdbe896cqsdsdd979ffc\" LogType=\"APPLI_LOG\" MessType=\"QUEING\" StateInd=\"0\" ErrCode=\"0\" ErrMess=\"Operation successful\" Serial=\"TID04b55dab01dsdfdsac13141b\" TicketNBR=\"4\"","input":{},"agent":{"hostname":"filebeatserver","ephemeral_id":"2488c327-71c6-4bcb-a747-eee5b86190a0","id":"91f74f15-737a-42df-b5d7-c629b3dd9e74","name":"filebeatserver","type":"filebeat","version":"7.10.0"},"ecs":{"version":"1.6.0"},"host":{"name":"filebeatserver"}}
the needed is to keep just this part of output , and remove the rest :
2020-11-26T14:19:23+01:00 servermaster CORE: TimeStamp=\"2020-11-26 14:19:23.560\" TraceID=\"TIDcb28e32379fd73bdbe896cqsdsdd979ffc\" LogType=\"APPLI_LOG\" MessType=\"QUEING\" StateInd=\"0\" ErrCode=\"0\" ErrMess=\"Operation successful\" Serial=\"TID04b55dab01dsdfdsac13141b\" TicketNBR=\"4\"
many thanks for helps
sai
shaunak
(Shaunak Kashyap)
November 30, 2020, 3:10am
7
I think you need to include @metadata
in your drop_fields
processor's fields
setting? Also, the @timestamp
field cannot be dropped, as noted in the processor's documentation: https://www.elastic.co/guide/en/beats/filebeat/current/drop-fields.html
hi ,
@metadata in drop_fields didn't resolve the pb
you think it is a bug ?
regards
system
(system)
Closed
December 29, 2020, 6:49pm
10
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.