Logstash 6.7.0 on Raspberry Pi 3 B+ - java.lang.IllegalStateException

[James_DeHaan]

Howdy All

Quick background, I had tried to get this going:

https:// github.com/TravisFSmith/SweetSecurity

but removed LS 5.5.X completely and installed LS 6.7.0 via dpkg/logstash-6.7.0.deb

• Version:
  logstash-6.7.0

• Operating System:
  Linux raspberrypi 4.14.98-v7+ #1200 SMP Tue Feb 12 20:27:48 GMT 2019 armv7l GNU/Linu

• Config File (if you have sensitive info, please remove it):

root@raspberrypi:~# cat /etc/logstash/conf.d/logstash7.conf
input {
file {
path => ["/var/log/suricata/*.json"]
sincedb_path => ["/var/cache/logstash/sincedbs/since.db"]
codec => json
type => "SELKS"
}

}

filter {
if [type] == "SELKS" {
date {
match => [ "timestamp", "ISO8601" ]
}
ruby {
code => "
if event.get('[event_type]') == 'fileinfo'
event.set('[fileinfo][type]', event.get('[fileinfo][magic]').to_s.split(',')[0])
end
"
}
ruby {
code => "
if event.get('[event_type]') == 'alert'
sp = event.get('[alert][signature]').to_s.split(' group ')
if (sp.length == 2) and /\A\d+\z/.match(sp[1])
event.set('[alert][signature]', sp[0])
end
end
"
}
metrics {
meter => [ "eve_insert" ]
add_tag => "metric"
flush_interval => 30
}
}

if [http] {
useragent {
source => "[http][http_user_agent]"
target => "[http][user_agent]"
}
}
if [src_ip] {
geoip {
source => "src_ip"
target => "geoip"
#database => "/opt/logstash/vendor/geoip/GeoLiteCity.dat"
add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ]
}
mutate {
convert => [ "[geoip][coordinates]", "float" ]
}
if ![geoip.ip] {
if [dest_ip] {
geoip {
source => "dest_ip"
target => "geoip"
#database => "/opt/logstash/vendor/geoip/GeoLiteCity.dat"
add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ]
}
mutate {
convert => [ "[geoip][coordinates]", "float" ]
}
}
}
}
}

output {
if [event_type] and [event_type] != 'stats' {
elasticsearch {
hosts => [MYAWS:443]
index => "logstash-%{event_type}-%{+YYYY.MM.dd}"
template => "/etc/logstash/elasticsearch5-template.json"
}
} else {
elasticsearch {
hosts => [MYAWS:443]
index => "logstash-%{+YYYY.MM.dd}"
template => "/etc/logstash/elasticsearch5-template.json"
}
}
}

• Sample Data:

{"timestamp":"2019-03-29T15:24:10.001031+0000","flow_id":1398236122980813,"event_type":"flow","src_ip":"2.2.2.2","src_port":54589,"dest_ip":"1.1.1.1","dest_port":53,"proto":"UDP","app_proto":"dns","flow":{"pkts_toserver":2,"pkts_toclient":1,"bytes_toserver":332,"bytes_toclient":226,"start":"2019-03-29T15:19:08.664013+0000","end":"2019-03-29T15:19:08.719484+0000","age":0,"state":"established","reason":"timeout","alerted":false}}

{"timestamp":"2019-03-29T15:24:12.068563+0000","flow_id":1597537657818067,"in_iface":"eth1","event_type":"dns","src_ip":"2.2.2.2","src_port":37639,"dest_ip":"1.1.1.1","dest_port":53,"proto":"UDP","dns":{"type":"query","id":1234,"rrname":"a.root-servers.net","rrtype":"A","tx_id":0}}

{"timestamp":"2019-03-29T15:24:12.081101+0000","flow_id":1597537657818067,"in_iface":"eth1","event_type":"dns","src_ip":"2.2.2.2","src_port":53,"dest_ip":"1.1.1.1","dest_port":37639,"proto":"UDP","dns":{"type":"answer","id":1234,"rcode":"NOERROR","rrname":"a.root-servers.net","rrtype":"A","ttl":59493,"rdata":"198.41.0.4"}}

• Steps to Reproduce:

  root@raspberrypi:~# LS_JAVA_OPTS="-Xms512m -Xmx1024m" /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash7.conf

  WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
  Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
  [ERROR] 2019-03-29 17:49:51.343 [LogStash::Runner] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (LoadError) load error: ffi/ffi -- java.lang.NullPointerException: null

  WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
  Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
  [ERROR] 2019-03-29 17:26:06.459 [LogStash::Runner] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (LoadError) load error: ffi/ffi -- java.lang.NullPointerException: null

IIRC it's important to have libjffi-1.2.so installed. I believe that's all fine.

I am not sure where the error is coming from - did I botch something or other?

I believe there's enough RAM/swap. I watched free mem go down but not fully consumed and released on the final error out.

Thanks in advance