Hi,
Can you please help me in creating separate fields for every value like below:
Input file:
DATE: 2019-12-03
Input File/Folder:1.txt
SHA256 hash of 1.txt: 2f1d3f3343dbc60768a290565c9136f754015438e4ea25770b61e624e6443792
Initiator Name: req1@hpe.com
OS:Linux
Scanning Tools: McAfee & ClamAV
*********************!!! MCAFEE REPORT ON LINUX!!! **********************
McAfee VirusScan Command Line for Linux64 Version: 6.0.6.653
Copyright (C) 2015 McAfee, Inc.
(408) 988-3832 LICENSED COPY - March 14 2016
AV Engine version: 5800.7501 for Linux64.
Dat set version: 9458 created Dec 1 2019
Scanning for 668685 viruses, trojans and variants.
2019-Dec-03 23:21:47
Options:
--AFC= 512 --ANALYZE --IGNORE-LINKS --MIME --SUB --SUMMARY --VERBOSE --ATIME-PRESERVE --REPORT=/AVScan_Logs/Linux/McAfee_1.txt.log --UNZIP --THREADS=4 --CLEAN /data/mysftpuser/MalwareScan/1.txt
/data/mysftpuser/MalwareScan/1.txt ... is OK.
Summary Report on /data/mysftpuser/MalwareScan/1.txt
File(s)
Total files:................... 111
Clean:......................... 222
Not Scanned:................... 333
Possibly Infected:............. 444
Cleaned:....................... 555
Deleted:....................... 666
Time: 00:00.00
*********************!!! ClamAV(OpenSource) REPORT ON LINUX!!! **********************
-------------------------------------------------------------------------------
/data/mysftpuser/MalwareScan/1.txt: Empty file
----------- SCAN SUMMARY -----------
Known viruses: 6578161
Engine version: 0.101.4
Scanned directories: 777
Scanned files: 888
Infected files: 999
Data scanned: 0123.00 MB
Data read: 345.00 MB (ratio 0.00:1)
Time: 33.534 sec (0 m 33 s)
========================
My grok:
input {
file {
path => "/root/elk/data/mslog"
start_position => "beginning"
codec => multiline {
pattern => "(^\s*)"
negate => true
what => "previous"
}
}
}
filter {
grok {
match => { "message" => ".?: %{(20%{DATE}):scanday}\n\t\n\t.? .?/.?:%{INT:field}.?\n\t\n\t.? .? .? %{INT:field2}.?: %{BASE16NUM:shahid}\n\t\n\t.? .?: %{EMAILADDRESS:initiatorname}\n\n\t\n\t.?:.?\n\t\n\t.? .?: .? .? .?\n\t\n\t\n\t\n\t*********************!!! .? .? .? .?!!! **********************\n\t\n\t.? .? .? .? .? .? .?: .?\n\t.? (%{BASE16NUM:field5}) %{INT:field4} .?, .?\n\t(%{INT:field6}) .? .? .? .? .? %{INT:field7} %{INT:field8}\n\t\n\t.? .? .?: %{NUMBER:avengineversion} .? .?\n\t.? .? .?: %{INT:datsetversion} .? %{BASE16NUM:field12} %{INT:field11} %{INT:field13}\n\t.? .? %{INT:field14} .?, .? .? .?\n\t\n\t\n\t.? %{TIME:scantime}\n\t\n\t\n\t.?:\n\t.?= %{INT:field15} .? .? .? .? .? .? .? .?=%{PATH:path} .? .?\bTHREADS=%{USER:THREADS} .? %{PATH:path2} \n\t\n\t%{PATH:path3} .? .? .?\n\t\n\t\n\t.? .? .? %{PATH:path4}\n\t.?(.?)\n\t .? .?:.? %{INT:totalfiles}\n\t .?:.? %{INT:cleanfiles}\n\t .? .?:.? %{INT:notScanned}\n\t .? .?:.? %{INT:iossiblyinfected}\n\t .?:.? %{INT:cleaned}\n\t .?:.? %{INT:deleted}\n\t\n\t\n\t.?: %{INT:field22}:%{NUMBER:field23}\n\t\n\t\n\t\n\t*********************!!! .?(.?) .? .? .?!!! **********************\n\t\n\t\n\t.?\n\t\n\t%{PATH:path5} .? .?\n\t\n\t.? .? .? .?\n\t.? .?: %{INT:knownviruses}\n\t.? .?: .?\n\t.? .?: %{INT:scanneddirectories}\n\t.? .?: %{INT:scannedfiles}\n\t.? .?: %{INT:infectedfiles}\n\t.? .?: %{NUMBER:datascannedmb} .?\n\t.? .?: %{NUMBER:dataread} .? (.? %{NUMBER:field31}:%{INT:field28})\n\t.?: %{NUMBER:timetaken} .? (%{INT:field32} .? %{INT:field34} .*" }
}
mutate {
add_field => {
"scanday1" => "%{scanday}"
"shahid1" => "%{shahid}"
}
}
}
output {
elasticsearch {
hosts => ["localhost:8080"]
index => "bhtab12"
document_type => "bhtabd12"
}
stdout {
codec => "rubydebug"
}
}
============
Output:
Dec 23, 2019 @ 16:02:03.467 Data scanned: 0123.00 MB
Dec 23, 2019 @ 16:02:03.467 Data read: 345.00 MB (ratio 0.00:1)
Dec 23, 2019 @ 16:02:03.466 Scanned directories: 777
Dec 23, 2019 @ 16:02:03.466 Scanned files: 888
Dec 23, 2019 @ 16:02:03.466 Infected files: 999
Dec 23, 2019 @ 16:02:03.465 Known viruses: 6578161
Dec 23, 2019 @ 16:02:03.465 Engine version: 0.101.4