Using Kibana and Elasticsearch for non-log large datasets

Hello!

I am trying to use Kibana 4.1 (with Elasticsearch 1.7.2) for initial 100k+ dataset of LARGE objects (900+ fields)
for visualization and analysis of data.

Here is my experiences so far

  • [Discover] step in kibana right after click is horrendous slow as it tries to show 900+ fields of "first"? objects.
    Is there any way to disable such search right after click to [Discover]?

  • In [Discover] and [Visualiize] steps left side bar is too narrow. Can it be made resizeable and remember that setting?

  • We have a complex/nested JSON structure and looking through all fields in [Discover] is very inconvenient. Any form of field search, etc. would be usable.

  • Is there any way for all visualization on dashboard to export resulting (new) datasets, i.e. aggregate data into another elasticsearch indice? Perhaps in real time?

  • On data load Elasticsearch locks field type to first found. For example room number - if first incoming value is 1 it will become number. Afterwards all future non-numeric values are invalid for example - 1a (perfectly valid for room number)
    Is there any way to override this? This question is more specific to Elasticsearch I suppose.

1 Like

That's an ES thing, look at creating explicit mappings for your data.