Create table from jenkins logs

(Mohanraj B) #1

Hello Elasticsearch users,

Just starting with elastic search!

I am running a jenkins job in a server that access a network device and print various show commands as logs.

I had gone through the manual and installed filebeat in the server where jenkins installed to collect logs.
Elastic search, kibana , logstash are installed in another machine. I could able to send the log to elastic search/logstash in second machine.

My query - is it possible to create a table kind of visualization by reading below logs - filtering the network show commands and print as table in kibana

Given below the example jenkins log, Highlited part are the ones that needs to be filterd out and display in table format.

Print Router show commands

terminal length 0
Router-2#

show bgp sum
BGP router identifier 2.2.2.2, local AS number 1
BGP table version is 884, main routing table version 884
10 network entries using 2480 bytes of memory
13 path entries using 1664 bytes of memory
4/4 BGP path/bestpath attribute entries using 1056 bytes of memory
1 BGP AS-PATH entries using 24 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 5224 total bytes of memory
BGP activity 247/237 prefixes, 463/450 paths, scan interval 60 secs

some

Router-2#

show bgp
BGP table version is 884, local router ID is 2.2.2.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
t secondary path,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

table1

How this can be achieved in elastic search?

logstash config:

input {
beats {
port => "5044"
}
}

output {
elasticsearch {
hosts => ["localhost:9200"]
}
stdout { codec => rubydebug }
}

filebeat config:
filebeat.inputs:

  • type: log
    enabled: true
    paths:

    • /var/log/jenkins/jenkins.log
  • type: log
    paths:

    • /var/lib/jenkins/jobs/NETWORK/builds/*/log

output.logstash:
hosts: ["192.168.43.5:5044"]

Thanks and Regards,
Mohan

(Mohanraj B) #2

Hello,

Using below filebeat config and logstash config file, I could able to trim all other unwanted messages and print only the messages that has the keyword Hop in it.

filebeat config

filebeat.inputs:

  • type: log
    enabled: true
    paths:

    • /var/log/jenkins/jenkins.log
  • type: log
    paths:

    • /var/lib/jenkins/jobs/NETWORK/builds/*/log
      multiline.pattern: '^\s*?r?'
      multiline.negate: false
      multiline.match: after

output.logstash:
hosts: ["192.168.43.5:5044"]

logstash config
input {
beats {
port => 5044
}
}

filter {
if [message] !~ "Hop" {
drop { }
}
}

output {
elasticsearch {
hosts => ["http://localhost:9200"]
}
}

But it is still a formatted string.

Need to form a table with table headers as Network, Next Hop, Metric, LocPrf, Weight, Path and visualize it.

I am stuck in this part. How can it be achieved.

Thank,
Mohan