Visualization between two indexes

Elastic Stack Kibana
1

Hello,

I have two indexes - "logstash-*" and let's call it "myindex".

The first one contains basic information I need. It has the following fields related to my question:

"timestamp"
"event" - string field
"myinfo" - string field

The second one contains some additional information that I want to use in couple of visualizations. It also contains one field that could be linked with fields from the "logstash-*" index:

"myinfo" - string field

What I want to do:

Pull special events from "logstash-*" index and build a pie chart based on information I have in "myindex" index for every unique occurence of "myinfo" field.

Is it possible?

Or it will be simplier/smarter to update some of my documents in "logstash-*" index with my additional information?

1 Like
2

Hi @fetch ,

wouldnt it be easier to put both indexes together?

im doing it with aws ses and postfix and throw them into "my-smtp-*"

3

Hey @lueneburger,

Thanks for the reply!

If the thing I want is too complicated to make, then I'll difinitely try to merge information from my second index into the main (logstash-*).

My initial idea was to distinguish information from different sources into different indexes, but I didn't expect it will bring me troubles with building visualisations.

The reason why I separated some information into the "myindex" is fact that "logstash-* " index could have a lot of events (up to 100k atm) with the same "myinfo" field, so updating "logstash-*" index documents will produce overhead.

4

Hi @fetch ,

did you try to create a new index in Kibana with the name "_all" ?

maybe that could help you :slight_smile:

image.png1343×450 21.3 KB

don't know how the performance impact is for searches

5

Thanks for the tip, but it doesn't work. None of the documents from the "myindex" satisfy the search query (I'm searching for the specific event).

As far as I understand, in order to make it work, my documents in "myindex" index must have two fields:

"timestamp" - in order to make it possible to visualize depending on the time interval choosen in Kibana
"event" - because I'm running a search for specific event type

But adding this two fields to the second index will result in is almost the same situation as if I just update my "logstash-*" index with the additional information from the "myindex".

6

Use something like logstash-* and logstash-myindex-*, then you can use logstash-* as the pattern,

7

Thanks for the reply @warkolm!

I understand the point how to make documents from both indices visible for Kibana.

What I don't understand is how to draw a pie chart using this data.

I want a pie chart, where:

  • query performs search for specific string in "event" field of main index
  • a slice size represents count of every unique occurence of "myinfo" field (Slice size -> Count)
  • a slice name represents a string field from my second index looked up by corresponding "myinfo" value.

So my problem with the slice naming, is it possible to perform such look up?

8

You can't do that.

9

Okey, thank you.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.