Filebeat 7.5 - Suricata module and Elastic cloud ingest pipeline issue

Running Ubuntu 18.04 running Suricata 5.0.1 - I've tried to use the filebeat (7.5.0 and 7.5.1) Suricata module on Elastic cloud but running setup on filebeat keeps giving me the following error:

2020-01-14T15:25:00.230Z INFO elasticsearch/client.go:753 Attempting to connect to Elasticsearch version 7.5.0
2020-01-14T15:25:00.285Z ERROR fileset/setup.go:75 Error loading pipeline: 1 error: Error loading pipeline for fileset suricata/eve: couldn't load pipeline: couldn't load json. Error: 500 Internal Server Error: {"error":{"root_cause":[{"type":"circuit_breaking_exception","reason":"[script] Too many dynamic script compilations within, max: [75/5m]; please use indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_rate] setting","bytes_wanted":0,"bytes_limit":0,"durability":"TRANSIENT"}],"type":"general_script_exception","reason":"Failed to compile inline script [long getOrZero(def map, def key) {\n if (map!=null && map[key]!=null) {\n return map[key];\n }\n return 0;\n} def network=ctx['network'], source=ctx['source'], dest=ctx['destination']; def sp=getOrZero(source,'packets'), sb=getOrZero(source,'bytes'), dp=getOrZero(dest,'packets'), db=getOrZero(dest,'bytes'); if (sb+db+sp+dp > 0) {\n if (network == null) {\n network=new HashMap();\n ctx['network']=network;\n }\n if (sb+db > 0) {\n network['bytes'] = sb+db;\n }\n if(sp+dp>0) {\n network['packets'] = sp+dp;\n }\n}\n] using lang [painless]","processor_type":"script","caused_by":{"type":"circuit_breaking_exception","reason":"[script] Too many dynamic script compilations within, max: [75/5m]; please use indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_rate] setting","bytes_wanted":0,"bytes_limit":0,"durability":"TRANSIENT"},"suppressed":[{"type":"general_script_exception","reason":"Failed to compile inline script [Instant ins(def d) {\n try {\n return Instant.parse(d);\n } catch(Exception e) {\n return null;\n }\n} def ev = ctx['event']; if (ev != null) {\n def start = ins(ev['start']);\n def end = ins(ev['end']);\n if (start != null && end != null && !start.isAfter(end)) {\n ev['duration'] = Duration.between(start,end).toNanos();\n }\n}\n] using lang [painless]","processor_type":"script","caused_by":{"type":"circuit_breaking_exception","reason":"[script] Too many dynamic script compilations within, max: [75/5m]; please use indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_rate] setting","bytes_wanted":0,"bytes_limit":0,"durability":"TRANSIENT"}},{"type":"general_script_exception","reason":"Failed to compile inline script [def t = ctx.suricata?.eve?.event_type; if (t == "stats") {\n ctx['event']['kind'] = "metric";\n} else if (t == "alert") {\n ctx['event']['kind'] = "alert";\n ctx['event']['category'] = "network_traffic";\n} else {\n ctx['event']['kind'] = "event";\n ctx['event']['category'] = "network_traffic";\n}\n] using lang [painless]","processor_type":"script","caused_by":{"type":"circuit_breaking_exception","reason":"[script] Too many dynamic script compilations within, max: [75/5m]; please use indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_rate] setting","bytes_wanted":0,"bytes_limit":0,"durability":"TRANSIENT"}}]},"status":500}. Response body: {"error":{"root_cause":[{"type":"circuit_breaking_exception","reason":"[script] Too many dynamic script compilations within, max: [75/5m]; please use indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_rate] setting","bytes_wanted":0,"bytes_limit":0,"durability":"TRANSIENT"}],"type":"general_script_exception","reason":"Failed to compile inline script [long getOrZero(def map, def key) {\n if (map!=null && map[key]!=null) {\n return map[key];\n }\n return 0;\n} def network=ctx['network'], source=ctx['source'], dest=ctx['destination']; def sp=getOrZero(source,'packets'), sb=getOrZero(source,'bytes'), dp=getOrZero(dest,'packets'), db=getOrZero(dest,'bytes'); if (sb+db+sp+dp > 0) {\n if (network == null) {\n network=new HashMap();\n ctx['network']=network;\n }\n if (sb+db > 0) {\n network['bytes'] = sb+db;\n }\n if(sp+dp>0) {\n network['packets'] = sp+dp;\n }\n}\n] using lang [painless]","processor_type":"script","caused_by":{"type":"circuit_breaking_exception","reason":"[script] Too many dynamic script compilations within, max: [75/5m]; please use indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_rate] setting","bytes_wanted":0,"bytes_limit":0,"durability":"TRANSIENT"},"suppressed":[{"type":"general_script_exception","reason":"Failed to compile inline script [Instant ins(def d) {\n try {\n return Instant.parse(d);\n } catch(Exception e) {\n return null;\n }\n} def ev = ctx['event']; if (ev != null) {\n def start = ins(ev['start']);\n def end = ins(ev['end']);\n if (start != null && end != null && !start.isAfter(end)) {\n ev['duration'] = Duration.between(start,end).toNanos();\n }\n}\n] using lang [painless]","processor_type":"script","caused_by":{"type":"circuit_breaking_exception","reason":"[script] Too many dynamic script compilations within, max: [75/5m]; please use indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_rate] setting","bytes_wanted":0,"bytes_limit":0,"durability":"TRANSIENT"}},{"type":"general_script_exception","reason":"Failed to compile inline script [def t = ctx.suricata?.eve?.event_type; if (t == "stats") {\n ctx['event']['kind'] = "metric";\n} else if (t == "alert") {\n ctx['event']['kind'] = "alert";\n ctx['event']['category'] = "network_traffic";\n} else {\n ctx['event']['kind'] = "event";\n ctx['event']['category'] = "network_traffic";\n}\n] using lang [painless]","processor_type":"script","caused_by":{"type":"circuit_breaking_exception","reason":"[script] Too many dynamic script compilations within, max: [75/5m]; please use indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_rate] setting","bytes_wanted":0,"bytes_limit":0,"durability":"TRANSIENT"}}]},"status":500}
2020-01-14T15:25:00.285Z INFO cfgfile/reload.go:264 Loading of config files completed

Any ideas?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.