Elastic-Agent and Logstash Integration Issue: FortiGate Logs Not Correctly Parsed in Elasticsearch

Elastic Stack Elastic Agent
1

I am currently trying to integrate Fortinet FortiGate Firewall logs through Elastic-Agent and Logstash, finally directing them into Elasticsearch. Here is my Logstash configuration:

input {
  elastic_agent {
    port => 5044
    ssl_enabled => true
    ssl_certificate_authorities => ["/etc/logstash/certs/elasticsearch-ca.pem"]
    ssl_certificate => "/etc/logstash/certs/logstash.crt"
    ssl_key => "/etc/logstash/certs/logstash.pkcs8.key"
    ssl_client_authentication => "required"
  }
}

filter {
  # This section is left empty to ensure no processing is done on the data
}

output {
    elasticsearch {
      hosts => ["https://192.168.3.171:9200"]
      user => "elastic"
      password => "password"
      ssl => true
      cacert => "/etc/logstash/certs/elasticsearch-ca.pem"
      index => "%{[data_stream][namespace]}-%{+YYYY.MM.dd}"
   }
}

However, when I view the logs in Kibana, I've noticed that the fields are not being parsed as expected by the Elastic-Agent. I am seeking guidance to understand the possible reasons and solutions for this issue.

2

You have your fortigate device configured to send logs to an Elastic Agent with the Fortinet integration and then this agent sends the data to Logstash and after that you send it to Elasticsearch? Is that right?

Please share how the logs looks like in Kibana.

You should remove the index option and add the data_stream => "true" option as mentioned in the configuration example.

3

You have your fortigate device configured to send logs to an Elastic Agent with the Fortinet integration and then this agent sends the data to Logstash and after that you send it to Elasticsearch? Is that right?

Yes, that's correct.
Additionally, I am using the Custom UDP Logs integration to send Windows event logs. The Ingest Pipeline uses logs-winlog.winlog-1.20.0, but the Elastic-Agent is not parsing these fields as expected. What might be the reason for this?

image
image782×775 50.2 KB

image
image1648×527 100 KB

4

The documentation is really bad.

What I ended up doing is installing Elastic Agent on a new Linux Server, pointing the integration there and done. Then, I pointed the Forti to said new Linux Server.

5

You need to share how the message field looks like, also try to not mix up the issues as this make its harder to troubleshoot.

Please share the information about the Fortinet integration, not the windows one, share how the event looks like in Kibana, including the message field and the integration configuration.

Did you change your Logstash output as mentioned in the previous answer?