Can’t collect Check Point FW Logs, via TCP over TLS

[lga1]

Hello,

I’m new in the elastic community. I’d like to collect logs from a Check Point firewall, with Logstash.

I use Elasticsearch, Kibana and Logstash. The infrastructure is running, communications are working between Elasticsearch, Kibana and Logstash. I created a self-signed certification authority with Elasticsearch certutil tool, and create a Logstash certificate signed by this CA.

First, I tried to send logs from my firewall to Logstash, via UDP. It’s works as expected, and by that I mean that logs are sent in real time, by the firewall, and received in real time by Logstash.

But UDP is not secured enough, so I want to use TCP over TLS. For the Logstash configuration, I only put a TCP input and Rubydebug for the output (the configuration is at the end of the message).

To run the config and see the debug output, I use the following command :

/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/tcp.conf

The debug mode starts, with no error and it listens on the port but there is no log in the console. After few minutes I stop it, because nothing is happening, when I send the shutdown signal with the command Ctrl+C, all the logs are coming, concentrated in only one message instead of one message per line.

During the test, I captured the trafic with Wireshark. I can see packets, regularly send by the firewall to the Logstash server. So, it doesn’t seem to be be a firewall problem.

I did two other tests, I start like the first one with the Logstash debug mode :

1. From the firewall, I run the following openssl command :

openssl s_client -connect 192.168.0.1:55555 -showcerts

It opens a communication between the Firewall, and the Logstash server, and when I send a message, immediately I can see it in the Logstash console.

2. On another server, I installed a second instance of Logstash (the configuration is at the end of the message). It has a client role, it takes lines in a file and send them to the Logstash server. Lines are received one by one, as expected.

I can't figured out if the problem comes from the Logstash configuration or from the firewall. I have no error from Logstash.
Does someone could help me please ?

Here some spécifications about configurations :
FW model : Check Point Spark Quantum 1570
Elasticsearch, Kibana and Logstash version : 8.15.0

Firewall configuration, where :

• name is the name of my firewall configuration
• host name is the domain name of the Logstash server, I checked that the firewall can reached it
• port the port where th eLogstash server is listening
• CA is the one used to signed Logstash certificate, so the firewall can trust it.

Logstash configuration to receive firewall logs :

• I force TLSv1.2 because the firewall doesn’t support TLSv1.3
• codec : I tried also « plain », but the behavior is the same.

input {
  tcp {
    port => 55555
    mode => "server"
    codec => line
    ssl_enabled => true
    ssl_certificate => "/path/to/logstash.crt"
    ssl_key => "/path/to/logstash.key"
    ssl_certificate_authorities => ["/path/to/ca.crt"]
    ssl_client_authentication => "none"
    ssl_supported_protocols => "TLSv1.2"
  }
}

output{
  stdout {
    codec => rubydebug
  }
}

Logstash configuration to send data :

input {
    file {
        path => "/path/to/test.txt"
        start_position => "beginning"
        sincedb_path => "/dev/null"
    }
}

output{
    stdout {
        codec => rubydebug
    }

    tcp {
        host => "192.168.0.1"
        port => 55555
        mode => "client"
        ssl_enabled => true
        ssl_certificate_authorities => ["/path/to/ca.crt"]
        ssl_supported_protocols => "TLSv1.2"
    }
}

[Alex_Salgado-Elastic (Alex Salgado)]

Hi @lga1 , welcome to our community.

Have you tried adding delimiter => "\n" to your line codec to explicitly define the end of each log message?

codec => line {
  delimiter => "\n"
}

[lga1]

Hello,

Thank you for your answer.
I tried with the delimiter but it doesn't work either.
Here's the new configuration :

input {
  tcp {
    port => 55555
    mode => "server"
    codec => line {
      delimiter => "\n"
    }
    ssl_enabled => true
    ssl_certificate => "/path/to/logstash.crt"
    ssl_key => "/path/to/logstash.key"
    ssl_certificate_authorities => ["/path/to/ca.crt"]
    ssl_client_authentication => "none"
    ssl_supported_protocols => "TLSv1.2"
  }
}
output{
  stdout {
    codec => rubydebug
  }
}

For UDP, I have only the port in my configuration and it works fine :

input {
  udp {
    port => 55550
  }
}
output{
  stdout {
    codec => rubydebug
  }
}

I don't understand why there are different behaviours with the same TCP configuration, depending on where the data come from.
It seems that only with Check Point Firewall logs (TCP over TLS), Logstash acts like a buffer until I stop it.

Is there something I do wrong or did I forget something in my configurations ?
I didn't do anything more than what I sent previously.

[leandrojmp (Leandro Pereira)]

@lga1

It seems that only with Check Point Firewall logs (TCP over TLS), Logstash acts like a buffer until I stop it.

It works with sources of data? So the issue may be on the sender side.

Did you test if it works with TCP but without TLS?

[Badger]

With a udp input, when a UDP packet arrives whatever data is in it is flushed onto the pipeline as an event.

With a tcp input, the app sees a delimited stream of bytes. The delimiter ("\n") divides the stream into lines, and each line is flushed into the pipeline as an event. If there are no delimiters the input will accumulate data until it is stopped, at which point it will flush all of that data in a single event.

That is exactly consistent with what you are seeing. Your source is not adding "\n" at the end of each line that it sends.

The udp input defaults to a plain codec, and the tcp input defaults to a line codec with "\n" as the delimiter. You could try configuring the tcp input to use a plain codec. However, it would not surprise me if you then sometimes have more than one message combined into a single event.

[lga1]

I checked and the firewall send the log, I can see the packets with Wireshark. I can’t try TCP without TLS because Check Point doesn’t offer this option.

[lga1]

I think you found the cause of my problem, Check Point doesn't seem to add a « \n » at the end of each log.
It doesn’t work in plain codec either, it’s the first configuration I tried. It’s the same behavior for codecs plain and line.

Each line of log starts with « NUMBER », where :

• NUMBER : I don’t know what it is, I think it’s added by Logstash because it’s not in standard Check Point log
• PRIORITY is an integer. It’s the only pattern I can I can see. So I tried this :

    codec => line {
      delimiter => " <"
    }

And now it’s working, I have one log per message.
I’ll try to patch with a delimiter and some filters.

Thank you all for your time !

[Badger]

@lga1

NUMBER : I don’t know what it is, I think it’s added by Logstash because it’s not in standard Check Point log

Could it be a line-length, intended to let you separate lines by counting bytes? (I am not aware of an input codec that supports this.)

[lga1]

I don't think it's a line-height because at a moment I had something like 400 and another times it was more than 1000, messages are not twice longer.
I think I'll cut this number and won't send it to Elasticsearch.