Filebeat CheckPoint Module

Hi Team,

I'm using filebeat 7.10.0 and am new to filebeat. I have enabled the checkpoint module for parsing the checkpoint logs of different event types. So in checkpoint.yml i have configured input as

    - module: checkpoint
    enabled: true
    # Set which input to use between syslog (default) or file.
    var.input: file
    var.paths: ["/var/Checkpoint_Logs/checkpoint.log"]

I want the events to be parsed and to be recorded in the file as output. In filebeat.yml configured output as file

 path: "/var/Output/"
 filename: checkpoint_filebeat_output.log
       pretty: false     

When am starting the filebeat, it start processing the rawlogs and the output was added to output file. But in the file, i can't see any checkpoint parsed fields in the file.

    "@timestamp":"2021-03-02T04:09:14.184Z","@metadata":{"beat":"filebeat","type":"_doc","version":"7.10.0","pipeline":"filebeat-7.10.0-checkpoint-firewall-pipeline"},"agent":{"version":"7.10.0","hostname":"DELL","ephemeral_id":"1233333-cb34-47bd-aa6c-cb0300abb5b1","id":"56dos23-d933-410f-8f9c-3be44babd4c4","name":"DELL","type":"filebeat"},"message":"<134>1 2021-03-01T02:44:27Z change CheckPoint 14850 - [action:\"Drop\"; flags:\"442361\"; ifdir:\"outbound\"; ifname:\"bond10.713\"; logid:\"0\"; loguid:\"{0x10622cb4,0xc344feff1,0x3c32115a,0x1b6785c8}\"; origin:\"\"; originsicname:\"CN=MDCvfirewall_mgmt,O=change..u74ovr\"; sequencenum:\"11\"; time:\"1614566667\"; version:\"5\"; __policy_id_tag:\"product=VPN-1 & FireWall-1[db_tag={2222EA-F618-D14C-A0F2-4608236D6AC2};mgmt=change;date=1614377438;policy_name=Management\\]\"; drop_reason:\"matched optimized drop\"; dst:\"\"; layer_name:\"Management Network\"; layer_uuid:\"7645aa-4f218-4va1-9b2f-b48s8wc63fd26\"; match_id:\"153\"; parent_rule:\"0\"; rule_action:\"Drop\"; rule_name:\"Cleanup rule\"; rule_uid:\"f9208734-4038-49df-9772-9435d71ee91c\"; product:\"VPN-1 & FireWall-1\"; proto:\"17\"; s_port:\"39\"; service:\"514\"; src:\"\"]","event":{"module":"checkpoint","timezone":"+05:30","dataset":"checkpoint.firewall"},"input":{"type":"log"},"service":{"type":"checkpoint"},"ecs":{"version":"1.5.0"},"log":{"offset":0,"file":{"path":"checkpoint.log"}},"tags":["checkpoint-firewall","forwarded"],"fileset":{"name":"firewall"}}

In the above output there is no parsed fields for checkpoint event.

When try to push the same event instead of output as file i have changed the output to elasticsearch, and now i can able to get the parsed fields for the checkpoint log in elasticsearch index.

please help me, I want to know why am not getting the parsed data in file as output, whether i need to add any other property in the config. Instead of pushing the parsed events to elasticsearch, i need those parsed events to be pushed to file as output.

Thanks in advance.

1 Like

Hello @Kaarthick. In most scenarios filebeat does not actually parse the data itself, the parsing happens once the data is sent to Elasticsearch, and it runs through what we call a Ingest Pipeline on elasticsearch, so this is why you do not see any parsing when the output is file.

There are certain scenarios in which the beat does some processing locally, but the general idea is that parsing is centralized in Elasticsearch itself.

Hi @Marius_Iversen ,

                 Thanks for your response. Will check that.


This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.