How to concatenate multiple fields into single timestamp

HI,

I am loading data from a text file to logstash using multiline grok, but every time stamp is creating for different fields, can you please help me here

Input file:

DATE: 2019-12-03

    Input File/Folder:1.txt

    SHA256 hash of 1.txt: 2f1d3f3343dbc60768a290565c9136f754015438e4ea25770b61e624e6443792

    Initiator Name: req1@hpe.com


    OS:Linux

    Scanning Tools: McAfee & ClamAV



    *********************!!! MCAFEE REPORT ON LINUX!!! **********************

    McAfee VirusScan Command Line for Linux64 Version: 6.0.6.653
    Copyright (C) 2015 McAfee, Inc.
    (408) 988-3832 LICENSED COPY - March 14 2016

    AV Engine version: 5800.7501 for Linux64.
    Dat set version: 9458 created Dec 1 2019
    Scanning for 668685 viruses, trojans and variants.


    2019-Dec-03 23:21:47


    Options:
    --AFC= 512 --ANALYZE --IGNORE-LINKS --MIME --SUB --SUMMARY --VERBOSE --ATIME-PRESERVE --REPORT=/AVScan_Logs/Linux/McAfee_1.txt.log --UNZIP --THREADS=4 --CLEAN /data/mysftpuser/MalwareScan/1.txt

    /data/mysftpuser/MalwareScan/1.txt ... is OK.


    Summary Report on /data/mysftpuser/MalwareScan/1.txt
    File(s)
           Total files:...................     111
           Clean:.........................     222
           Not Scanned:...................     333
           Possibly Infected:.............     444
           Cleaned:.......................     555
           Deleted:.......................     666


    Time: 00:00.00



    *********************!!! ClamAV(OpenSource) REPORT ON LINUX!!! **********************


    -------------------------------------------------------------------------------

    /data/mysftpuser/MalwareScan/1.txt: Empty file

    ----------- SCAN SUMMARY -----------
    Known viruses: 6578161
    Engine version: 0.101.4
    Scanned directories: 777
    Scanned files: 888
    Infected files: 999
    Data scanned: 0123.00 MB
    Data read: 345.00 MB (ratio 0.00:1)
    Time: 33.534 sec (0 m 33 s)

====================
we need seperate fields for most of the lines to use it in Kibana graphs, can some help me here pls
My Config file:
input {
file {
path => "/root/elk/data/mslog"
start_position => "beginning"
codec => multiline {
#pattern => "(^\s*)"
pattern => "((^\s*)|(\n))"
negate => true
what => "previous"
}
}
}
filter {
if [message] =~ "DATE:" {
grok {
match => { "message" => "(20%{DATE:startdate})" }
}
} else if [message] =~ "Input File/Folder:" {
grok {
match => { "message" => "%{GREEDYDATA:filefolder}" }
}
} else if [message] =~ "SHA256 hash of" {
grok {
match => { "message" => ":%{GREEDYDATA:shahid}" }
}
} else if [message] =~ "Initiator Name:" {
grok {

==================

totalfiles	scanneddirectories	scannedfiles	scanningtools	scantime	shahid	startdate	options	notscanedfiles	knownviruses	infectedfiles	filefolder	engineversion	deleted
Dec 23, 2019 @ 15:20:15.638	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 
Dec 23, 2019 @ 15:20:15.638	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 
Dec 23, 2019 @ 15:20:15.637	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	999	 - 	 - 	 - 
Dec 23, 2019 @ 15:20:15.637	 - 	 - 	888	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 
Dec 23, 2019 @ 15:20:15.636	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	6578161	 - 	 - 	 - 	 - 
Dec 23, 2019 @ 15:20:15.636	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	0.101	 - 
Dec 23, 2019 @ 15:20:15.636	 - 	777	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 	 - 
Dec 23, 2019 @ 15:20:15.635	 - 	 -

{
"@timestamp" => 2019-12-23T10:03:16.376Z,
"@version" => "1",
"path" => "/root/elk/data/mslog",
"message" => "\tData scanned: 0123.00 MB",
"host" => "securedev31.in.rdlabs.hpecorp.net"
}
{
"@version" => "1",
"host" => "securedev31.in.rdlabs.hpecorp.net",
"@timestamp" => 2019-12-23T10:03:16.346Z,
"path" => "/root/elk/data/mslog",
"message" => "\tData read: 0.00 MB (ratio 0.00:1)",
"dataread" => "\tData read: 0.00 MB (ratio 0.00:1)"
}
{
"@version" => "1",
"host" => "securedev31.in.rdlabs.hpecorp.net",
"@timestamp" => 2019-12-23T10:03:16.376Z,
"path" => "/root/elk/data/mslog",
"message" => "\tData read: 345.00 MB (ratio 0.00:1)",
"dataread" => "\tData read: 345.00 MB (ratio 0.00:1)"
}
{
"@version" => "1",
"host" => "securedev31.in.rdlabs.hpecorp.net",
"@timestamp" => 2019-12-23T10:03:16.348Z,
"path" => "/root/elk/data/mslog",
"message" => "\tTime: 33.534 sec (0 m 33 s)",
"scantime" => "\tTime: 33.534 sec (0 m 33 s)"
}
{
"@version" => "1",
"host" => "securedev31.in.rdlabs.hpecorp.net",
"@timestamp" => 2019-12-23T10:03:16.370Z,
"path" => "/root/elk/data/mslog",
"message" => "\tTime: 00:00.00",
"scantime" => "\tTime: 00:00.00"
}

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.