Logstash custom pattern not working

Elastic Stack Logstash
1

Hello everyone,

New to logstash so please bare with me. My logstash config is not working as desired with a pattern file using regex to pull for very specific text from a log message.

From the specific log message, here is the string that I am trying to pull out as is exactly:

"dest_port":21,"

And here is my filter:

if "xxxx" in [tags] {
grok {
patterns_dir => ["./patterns"]
match => { "message" => "%{FTP:ftp}" }
add_tag => "FTP"
}
}

Here is the output from my pattern file:

/patterns # more extra
FTP << " >><< d >><< e >><< s >><< t >><< _ >><< p >><< o >><< r >><< t >><< " >><< : >><< 2 >><< 1 >><< , >><< " >>

All running in docker.

Any help would be very very much appreciated.

Thank you-

2

If you want to test whether the [message] field contains that string then why not test it directly

if '"dest_port":21,"' in [message] { ... }
3

Thank you for the reply, really appreciate it. Still no joy..

Here is my current config based on your feedback:

if '"dest_port": 21,' in [message] {
mutate {
add_tag => [ "FTP" ]
}
}

JSON shows an extra space before 21.

At a loss at the moment.

Cheers-

4

At this point, I will accept any alternative solution. My end state objective it to tag this specific data from a log file as input for further processing during output using a filter..

here is a sample raw log:

{"timestamp":"2019-09-16T18:02:49.589089+0000","flow_id":2139429747809040,"in_iface":"ens192","event_type":"alert","src_ip":"192.168.1.1","src_port":54994,"dest_ip":"192.168.1.210","dest_port":21,"proto":"TCP","alert":{"action":"allowed","gid":1,"signature_id":2010736,"rev":2,"signature":"ET FTP FTP RETR command attempt without login","category":"Attempted Information Leak","severity":2,"metadata":{"updated_at":["2010_07_30"],"created_at":["2010_07_30"]}},"app_proto":"failed","flow":{"pkts_toserver":16,"pkts_toclient":18,"bytes_toserver":1030,"bytes_toclient":1188,"start":"2019-09-16T18:02:11.711440+0000"},"payload":"T1BUUyBVVEY4IE9ODQpVU0VSIGtpdGthdA0KUEFTUyBraXRrYXQNClBPUlQgMTAsNTAsNTAsMSwyNDUsNTINClJFVFIgKi5leGUNClBPUlQgMTAsNTAsNTAsMSwyNDUsNTMNClFVSVQNCg==","payload_printable":"OPTS UTF8 ON\r\nUSER kitkat\r\nPASS kitkat\r\nPORT 192,168,1,1,245,52\r\nRETR *.exe\r\nPORT 192,168,1,1,245,53\r\nQUIT\r\n","stream":1}

Cheers-

5

If you have JSON data coming in I would parse it

filter { json { source => "message" remove_field => [ "message" ] } }

then you can use

if [dest_port] == 21 {

(or maybe use a translate filter to do a lookup).

6

Awesome! That worked. Thank you very much; your a lifesaver! :wink:

Appreciate the help.

Cheers-

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.