Not getting output with CEF codec

Elastic Stack Logstash
1

I've got a working pipeline in Logstash where non-ECS JSON (I have ecs_compatibility disabled in my pipeline) is coming in from SQS, getting transformed using mutate filters, and then output using stdout and the rubydebug codec. What comes out looks like what you'd want in CEF:

{
    "requestClientApplication" => "Chrome",
    "deviceEventClassID" => "user.signin",
    "startTime" => "2022-12-14T20:33:26.818366+00:00",
    "deviceVendor" => "FooCorp",
    "name" => "User signed in.",
    "severity" => 1,
    "sourceAddress" => "1.1.1.1",
    "@timestamp" => 2022-12-15T18:52:58.155290299Z,
    "deviceProduct" => "auth-api",
    "eventId" => "6F93AEDC-9035-43F4-8A50-D4ACD18CDBE5",
    "deviceVersion" => "3143-abcdef12",
    "sourceUserName": "joeschmoe@example.com"
}

But when I change stdout to use the cef codec... nothing comes out. No log message.

When I turn on debug level logging, I see a debug level CEF formatted message that looks like it's from Logstash itself:

CEF:0|Elasticsearch|Logstash|1.0|Logstash|Logstash|6|[2022-12-15T18:57:36,183][DEBUG][org.logstash.execution.PeriodicFlush][main] Pushing flush onto pipeline.

But my log message does not appear. What am I doing wrong? Thanks!

2

I think you are misinterpreting that. The CEF:0|Elasticsearch|Logstash|1.0|Logstash|Logstash|6| (with no newline) is what the cef codec will output if you do not specify the fields option to tell it which fields to add to the cef message. The periodic flush message is just a debug log message.

Try

codec => cef { fields => [ "requestClientApplication", "deviceEventClassID", "startTime", "deviceVendor", "name", "severity", "sourceAddress", "deviceProduct", "eventId", "deviceVersion", "sourceUserName" ] }

which should result in

CEF:0|Elasticsearch|Logstash|1.0|Logstash|Logstash|6|requestClientApplication=Chrome deviceEventClassID=user.signin startTime=2022-12-14T20:33:26.818366+00:00 deviceVendor=FooCorp name=User signed in. severity=1 sourceAddress=1.1.1.1 deviceProduct=auth-api eventId=6F93AEDC-9035-43F4-8A50-D4ACD18CDBE5 deviceVersion=3143-abcdef12 sourceUserName=joeschmoe@example.com
3

That makes so much sense. Thank you! I've got it working now.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.