Logtrail : Kibana plugin to view, search and tail logs in realtime


#1

All,

Good day!

I have written a Kibana plugin to view, search and tail logs in developer/sysadmin friendly interface ( inspired by Papertrailapp).

Installation and configuration instructions is at [https://github.com/sivasamyk/logtrail] (https://github.com/sivasamyk/logtrail). Currently it is supported for Kibana 4.x version. Following are the features supported:

  • View, analyze and search log events from a centralized, developer and sysadmin friendly interface
  • Live tail
  • Filter aggregated logs by hosts and program
  • Quickly seek to logs based on specific time

Checkout the plugin and let me know your feedback.

You can install the plugin by executing following command (requires restart of kibana after installation and config changes)

./bin/kibana plugin -i logtrail -u https://github.com/sivasamyk/logtrail/releases/download/v4.x-0.1.0/logtrail-4.x-0.1.0.tar.gz

Screenshot:

Thanks


(Mark Walkom) #2

Thanks for sharing this!


(Ranvijay Jamwal) #3

Normal Kibana is working.
This doesn't work. Do I need to change the logtrail.json file?
I can go to the dark log trail console but can't see any logs there. Help ASAP.


#4

Yes you need to customize logtrail.json to specify the index to query and
map the fields accordingly.


(Ranvijay Jamwal) #5

This is the conf I am using

"es": {
"default_index": "logstash-*",
"allow_url_parameter": false,
"timezone": "UTC"
},
"tail_interval_in_seconds": 10,
"max_buckets": 500,
"nested_objects" : false,
"default_time_range_in_days" : 0,
"fields" : {
"mapping" : {
"timestamp" : "@timestamp",
"display_timestamp" : "syslog_timestamp",
"hostname" : "hostname",
"program": "program",
"message": "syslog_message"
}
}
}

What do I change? Elasticsearch is on the same host. Data is already coming into from Logstash Elasticsearch. I installed the plugin, I can switch between kibana and logtrail on the browser, but logtrail does not have data. Indexes are made like logstash-* only. Please help.

This is if the above ever works for me: Also, is it possible that I can search something specific like a particular log file and show only that on logtrail console?


#6

Your logtrail.json looks good. Do you have any errors in kibana logs? Are
you able to see the logs from kibana view?


(Ranvijay Jamwal) #7

Yes. All the logs. I can see everything properly on kibana. But logtrail says no events. I tried changing the time as well. It should trail all the logs as they are coming right?


(Ranvijay Jamwal) #8

I made these changes from above:

{
"es": {
"default_index": "logstash-",
"allow_url_parameter": false,
"timezone": "UTC"
},
"tail_interval_in_seconds": 10,
"max_buckets": 500,
"nested_objects" : false,
"default_time_range_in_days" : 0,
"fields" : {
"mapping" : {
"timestamp" : "@timestamp",
"display_timestamp" : "syslog_timestamp",
"hostname" : "localhost",
"program": "program",
"message": "
"
}
}
}

It says program.raw undefined.

PFA

Screen Shot


#9

Hi,

If you are using logstash for ingesting logs into ES, can you paste your
logstash config file? Looks like the fields specified in logtrail.json are
not present in ES.


(system) #10