Filebeat Cisco-FTD: Too many dynamic script compilations

Elastic Stack Beats
,
1

I want to ingest the logs of Cisco Firepowers for use within the SIEM App and custom alerting, but with just one FTD added (out of roughly 30), I'm already receiving the following error:

[script] Too many dynamic script compilations within, max: [1000/5m]; please use indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_rate] setting

As you can see in the error message, I already tweaked the values as recommended by the official module documentation, but without any improvements. Defaults are a maximum of 75 dynamic script compilations within 5 minutes; documentation instructs to go for at least 100/5m. Even with 1000/5m, I receive ~475.000 errors within 24 hrs (of a total of ~1.250.000 events / 24 hrs, so we talk about a "loss" of more than a 3rd of all events).

As I'm hitting the limits of out-of-the-box scalability with just one Firepower added, I think this setting would require a value above sane limits to handle all FTDs.

The only way to get rid of these errors I can think of is routing through Logstash, not using any ES ingest pipelines, and emulating the ingest pipeline behaviour using a Logstash pipeline. But that's a nightmare to maintain, as with every update of the filebeat module the custom logstash pipeline may require adjustments. As would be the case for changes to ECS or Logstash itself. Automatic conversion between the ingest pipeline and a logstash pipeline is (AFAIK) currently impossible.

Does any of you kind folks know of a solution to my problem?

2

Hi @xoh,

You should keep increasing the value of both script.max_compilations_rate and script.cache.max_size until the error is gone.

I'll start with duplicating the cache_max_size. If the cache is too small, already compiled scripts will be evicted and recompiled over and over again.

There's already a work in progress on the Elasticsearch side to disable these limitations for ingest pipelines so that this isn't an issue anymore.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.