Dashboard / Visualization to import

Elastic Stack Kibana
1

I'm feeding log data from eight apache hosts into Elasticsearch, then using Kibana 4 to show the end users graphs, totals, and so on.

I see examples on the net where others have designed some very nifty visualizations and dashboards. Is there a repository where one can grab the json and import?

It's not that I'm opposed to the hard work, but if someone has already done it, I don't want to re-invent the wheel.

Thanks in advance.

2

Hi Brian - There is currently no centralized repository for these types of examples and templates. For now, your best bet is to Google for what you want and see what is available in the community. Many folks expose sample configurations in their blogs or Github repositories.

A couple of things to watch out for, as you search:

  • At the moment, a lot of examples are Kibana3-centric, and that dashboard format is not compatible with Kibana 4.
  • Any dashboard you find with the community will be tied to specific field names and types indexed in Elasticsearch, so you'll need to obtain the Logstash config they used and also their Elasticsearch index template, if they changed it from the default.
  • Your log file format needs to be the same as anticipated by the Logstash config above - if you made changes to your log configuration to produce logs of different format than the default, you'll likely need to adjust your grok pattern in Logstash config.
  • Kibana 4 json export does not include the index pattern on top of which you defined the dashboard, so you'll need to add that manually.
  • There are probably some other gotchas I forgot :slight_smile:

I went ahead and exported my Logstash config and a sample Kibana 4 dashboard based on Apache2 logs for you to try. To use these files:

  1. Make sure ELK stack is installed per Getting Started docs online
  2. Make sure you are using the same or similar Logstash config in your setup, so that you have the same fields available in Elasticsearch
  3. Make sure your Apache log format is of the same format as this sample file
  4. Manually configure the index pattern in Kibana to be [logstash-]YYYY.MM.DD
  5. Import the sample Kibana dashboard in Kibana 4 Settings >> Object page

Let me know if this works for you!

3

I'll fool around with those this morning, and will report back.

Thank you, Tanya!

4

Interesting: my index pattern is logstash-* - I think this was setup by direction of the 'getting started' guide.

Is there a benefit to [logstash-]YYYY.MM.DD ?

5

Is there a benefit to [logstash-]YYYY.MM.DD ?

Yes, it allows Kibana to restrict queries to only the indexes that are covered by the date range you've chosen. No point in searching logstash-2015.05.01 if you're only interested in the past week's log, right?

6

Interestingly, some of these nuances will go away with enhancements enabled by Elasticsearch 2.0: https://github.com/elastic/kibana/issues/4342

7

That has improved performance 1000%. Subjectively speaking, of course, but man: kibana is doing everything way, way, faster. Thanks for the explanation, and thanks Tanya for the hint.

8

It did! Two of the visualizations error 'Could not locate that index-pattern-field' for useragent.device.raw and useragenent.os.raw but I can figure that out, I think.

Something like that could be super helpful. I've seen a few very slick looking dashboards/ visualizations with no idea how to duplicate the work, except pick through them step by step. That's great for learning, but it's hell on deadlines.