Filebeat & logstash Grok patterns

Elastic Stack Logstash
1

Hi I went through all the documents available in online have I'm sure I'm doing small mistake some were

I'm unable to see follow log in kibana dashboard

I'm transporting via filebeat to logstash

else if "drive.db" in [source] {
grok {
break_on_match => true
match => { "message" => ") %{GREEDYDATA:msg}" }
}

}

40401-01 plugins]# cat /var/log/drive.db

Drive Database - this file is auto-generated. Do not edit.

Slot, Serial , State , First Seen, Fail Time , Failure Source

boot24, 6XM4WYY2, Ok , 1477000831
boot25, 6XM136KD, Ok , 1477000831
slot00, WXC1E32JRYH9, Ok , 1477000831
slot01, WXK1E32AUNKN, Ok , 1477000831
slot02, WXK1E32AURFJ, Ok , 1477000831
slot03, WXK1E32AUNFW, Ok , 1477000831
slot04, WXF1E32TVYH2, Ok , 1477000831
slot05, WX41EA1UVZ38, Ok , 1477000831
slot06, WXF1E32TJFJ8, Ok , 1477000831
slot07, WX91EC1AAYR6, Ok , 1477000831
slot08, WXK1E32CETRD, Ok , 1477000831
slot09, WXC1E32DUJH7, Ok , 1477000831
slot10, WXC1E32JRXZ7, Ok , 1477000831
slot11, WXF1E32TWEV7, Ok , 1477000831
slot12, 2720A00NFRD2, Ok , 1490129238
slot13, WXF1E32TVRK5, Ok , 1477000831
slot14, WXC1E32KZUD6, Ok , 1477000831
slot15, WXF1E32TVZJ3, Ok , 1477000831
slot16, WXF1E32TWAN7, Ok , 1477000831
slot17, WXF1E32TWWD8, Ok , 1477000831
slot18, WXC1E32JRRK2, Ok , 1477000831
slot19, WXC1E32JRMY5, Ok , 1477000831
slot20, WXK1E32XVUR9, Ok , 1477000831
slot21, WXF1E32TWFH5, Ok , 1477000831
slot22, WXK1E32AJYUF, Ok , 1477000831
slot23, WXF1E32TVTM4, Ok , 1477000831

Historical Drive info

Slot, Serial , State , First Seen, Fail Time , Failure Source

-- , WXF1E32LYWR1, FAILED, 1477000831, 1483934430, Dell
[root@ccdn-ats-tk-40401-01 plugins]#

2

Hi,

Test this:

else if "drive.db" in [source] {
grok {
break_on_match => true
match => { "message" => "%{GREEDYDATA:msg}" }

    }

}

I just remove the ")" in the line match => { "message" => "%{GREEDYDATA:msg}" }

3

thanks for you reply its was by mistake wrong copy paste of mine I do not have ")"
however able to see logs in kibana but not in correct order I need complete messege

{code}
@timestamp March 28th 2017, 15:18:46.363
t @version 1
t _id AVsWj8ItElKBWXhcGycu
t _index logstash-2017.03.28

_score -

t _type logs
t beat.hostname ccdn-ats-tk-40401-01.hostname
t beat.name ccdn-ats-tk-40401-01.hostname
t beat.version 5.0.0
t host ccdn-ats-tk-40401-01.hostname
t input_type log
t message # Drive Database - this file is auto-generated. Do not edit.
t msg # Drive Database - this file is auto-generated. Do not edit.

offset 61

t source /var/log/drive.db

{code}

4

I think i may need to add more lines in pattrens

Drive Database - this file is auto-generated. Do not edit.

Slot, Serial , State , First Seen, Fail Time , Failure Source

boot24, 6XM4WYY2, Ok , 1477000831
boot25, 6XM136KD, Ok , 1477000831
slot00, WXC1E32JRYH9, Ok , 1477000831
slot01, WXK1E32AUNKN, Ok , 1477000831
slot02, WXK1E32AURFJ, Ok , 1477000831
slot03, WXK1E32AUNFW, Ok , 1477000831
slot04, WXF1E32TVYH2, Ok , 1477000831
slot05, WX41EA1UVZ38, Ok , 1477000831
slot06, WXF1E32TJFJ8, Ok , 1477000831
slot07, WX91EC1AAYR6, Ok , 1477000831
slot08, WXK1E32CETRD, Ok , 1477000831
slot09, WXC1E32DUJH7, Ok , 1477000831
slot10, WXC1E32JRXZ7, Ok , 1477000831
slot11, WXF1E32TWEV7, Ok , 1477000831
slot12, 2720A00NFRD2, Ok , 1490129238
slot13, WXF1E32TVRK5, Ok , 1477000831
slot14, WXC1E32KZUD6, Ok , 1477000831
slot15, WXF1E32TVZJ3, Ok , 1477000831
slot16, WXF1E32TWAN7, Ok , 1477000831
slot17, WXF1E32TWWD8, Ok , 1477000831
slot18, WXC1E32JRRK2, Ok , 1477000831
slot19, WXC1E32JRMY5, Ok , 1477000831
slot20, WXK1E32XVUR9, Ok , 1477000831
slot21, WXF1E32TWFH5, Ok , 1477000831
slot22, WXK1E32AJYUF, Ok , 1477000831
slot23, WXF1E32TVTM4, Ok , 1477000831

Historical Drive info

Slot, Serial , State , First Seen, Fail Time , Failure Source

-- , WXF1E32LYWR1, FAILED, 1477000831, 1483934430, Dell

5

Not sure what but I'm able to see all drive status in kibana but I would like to remove last 3 lines from ingesting reason it will count as failed metric

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.