Probleme with grock logstash 5.0

Elastic Stack Logstash
1

hey,

i have a problem with logstash, syntax of logstash.

my log is :
14>1 2016-03-02T14:26:10.620+01:00 AAAAAAA-AAAA AA_AAAA- AA_FLOW_SESSION_CLOSE
[blablabla@2222.2.1.1.2.222 reason="idle Timeout" source-address="8.8.8.8"
source-port="8" destination-address="8.8.8.8"
destination-port="8" service-name="udp"
nat-source-address="8.8.8.8" nat-source-port="8"
nat-destination-address="8.8.8.8"
nat-destination-port="8" src-nat-rule-type="N/A"
src-nat-rule-name="N/A" dst-nat-rule-type="N/A"
dst-nat-rule-name="N/A" protocol-id="8"
policy-name="test_1"
source-zone-name="test_2" destination-zone-name="test-3"
session-id-32="8" packets-from-client="1"
bytes-from-client="108" packets-from-server="0"
bytes-from-server="0" elapsed-time="60"
application="UNKNOWN" nested-application="UNKNOWN"
username="N/A" roles="N/A" packet-incoming-interface="eth0"
encrypted="UNKNOWN"] session closed idle

i want keep strong text

my grok is :

filter {
grok {
match => { "message" => " %{DATA} %{TIMESTAMP_ISO8601:Date}%{DATA}source-address="%{IP:ip_source_address}" source-port="%{NUMBER:source_port}" destination-address="%{IP:ip_destination_address}" destination-port="%{NUMBER:destination_port}" service-name="%{WORD:service_name}" %{DATA} protocol-id="%{NUMBER:protocol_id}" policy-name="%{DATA:policy_name}" source-zone-name="%{DATA:source_zone_name}" destination-zone-name="{DATA:destination_zone_name}" " }
}
}
if you test this on : https://grokdebug.herokuapp.com/ it's ok but in logstash after a configtest i have an error , plz help me

2

If you want double quotes inside the grok expression I suggest you make the expression single quoted, i.e. do

match => { "message" => ' %{DATA}...' }

instead of this:

match => { "message" => " %{DATA}..." }
3

my configuration , but in kibana nothing 0 match, 0 index pattern .. WHY ?

input {
file {
start_position => "beginning"
path => "/var/log/firewall/test.log"

}
}

filter {
grok {
break_on_match => "false"
match => { "message" => '%{DATA}%{TIMESTAMP_ISO8601:Date} %{DATA}source-address="%{IP:ip_source_address}"%{DATA}source-port="%{NUMBER:source_port}"%{DATA}destination-address="%{IP:ip_destination_address}"%{DATA}destination-port="%{NUMBER:destination_port}"%{DATA}service-name="%{WORD:service_name}" %{DATA} protocol-id="%{NUMBER:protocol_id}" policy-name="%{DATA:policy_name}"%{DATA}source-zone-name="%{DATA:source_zone_name}"%{DATA}'}

}
}

output {
elasticsearch {
hosts => ["localhost:9200"]
}
}

location of my log :

ls -ls /var/log/firewall/test.log
4 -rw-rw----. 1 logstash logstash 44 6 déc. 21:10 /var/log/firewall/test.log

4

Probably because Logstash is tailing test.log. start_position => "beginning" only matters for new files. Delete the sincedb file or set sincedb_path => "/dev/null". This exact problem is discussed here every week.

1 Like
5

with this nothing ...

input {
file {
start_position => "beginning"
path => "/var/log/firewall/test.log"
sincedb_path => "/dev/null"
}
}
with a new file nothing ...

input {
file {
start_position => "beginning"
path => "/var/log/firewall/test.log"
}
}
serious i don't understand

6

Does the logstash user have access to the /var/log/firewall directory? Check your Logstash log for clues. You probably have to turn up the log level at least one notch.

7

without sincedb_path => "/dev/null" it's good with no match

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.