Multi Index Visualization

Elastic Stack Kibana
1

Trying to set up one visualization graph that has several indexes on it.
I have indexes like this:

agslx-blabla-varnish
agslx-blabla2-varnish
agslx-blabla3-varnish

I've configured Kibana to use: "agslx-*-varnish" index which didn't throw any errors. Now I want to graph X field from all the indexes in one graph. Example data:

{
       "@timestamp" => "2016-07-15T23:04:02.948Z",
             "type" => "vc_server",
         "hostname" => "agslx-hpclavc06",
    "MAIN_hit_rate" => 99.66,
             "tags" => [
        [0] "varnishstat"
    ]
}
{
       "@timestamp" => "2016-07-15T23:04:02.172Z",
             "type" => "vc_server",
         "hostname" => "agslx-hpclavc05",
    "MAIN_hit_rate" => 99.57,
             "tags" => [
        [0] "varnishstat"
    ]
}

I want to graph the MAX every 1 min of "MAIN_hit_rate". What I accomplished so far looks like this but only shows the data from one index: graph

2

I don't know if KB supports wildcards in the middle like that, try agslx-*?

3

Doesn't work either. Still only shows one index even If I have two indexes with the same field name.
Graph

4

That definitely works, it's why the default pattern is logstash-*.

What does the mappings for the index look like?

5

Here are the mappings for the two indexes:

{
  "agslx-hpclavc05-varnishstat" : {
    "mappings" : {
      "vc_server" : {
        "properties" : {
          "@timestamp" : {
            "type" : "date",
            "format" : "strict_date_optional_time||epoch_millis"
          },
          "MAIN_backend_fail" : {
            "type" : "double"
          },
          "MAIN_fetch_1xx" : {
            "type" : "double"
          },
          "MAIN_fetch_204" : {
            "type" : "double"
          },
          "MAIN_fetch_304" : {
            "type" : "double"
          },
          "MAIN_fetch_failed" : {
            "type" : "double"
          },
          "MAIN_hit_rate" : {
            "type" : "double"
          },
          "MAIN_n_lru_nuked" : {
            "type" : "double"
          },
          "MAIN_sess_drop" : {
            "type" : "double"
          },
          "hostname" : {
            "type" : "string"
          },
          "tags" : {
            "type" : "string"
          },
          "type" : {
            "type" : "string"
          }
        }
      }
    }
  }
}

{
  "agslx-hpclavc06-varnishstat" : {
    "mappings" : {
      "vc_server" : {
        "properties" : {
          "@timestamp" : {
            "type" : "date",
            "format" : "strict_date_optional_time||epoch_millis"
          },
          "MAIN_backend_fail" : {
            "type" : "double"
          },
          "MAIN_fetch_1xx" : {
            "type" : "double"
          },
          "MAIN_fetch_204" : {
            "type" : "double"
          },
          "MAIN_fetch_304" : {
            "type" : "double"
          },
          "MAIN_fetch_failed" : {
            "type" : "double"
          },
          "MAIN_hit_rate" : {
            "type" : "double"
          },
          "MAIN_n_lru_nuked" : {
            "type" : "double"
          },
          "MAIN_sess_drop" : {
            "type" : "double"
          },
          "hostname" : {
            "type" : "string"
          },
          "tags" : {
            "type" : "string"
          },
          "type" : {
            "type" : "string"
          }
        }
      }
    }
  }
}

I want to display the values of MAIN_hit_rate for both indexes on the same graph. Is it in any way possible?

6

Any ideas on how to accomplish this? Am I approaching storing the data in a wrong way? How would I go about graphing data from several different servers?

7

Did you alter the pattern to agslx-*?
There's nothing that should be stopping this from being graphed.

8

Yes I did alter the pattern to agslx-* but I can't find any options on how to graph different indexes. I can only graph the max value from one field but it shows that field from one index instead of all of them.

9

It'll show it from all applicable indices.

10

Look at this image: graph. It shows the index is agslx-*. Only graphs from one index in particular instead of both.

11

How do you know this?

12

Because there's only one line there and it's the value from one index only. I'm checking the values with logstash and there are two different ones yet there's only one line there.

13

Yeah but your graph is the MAX value from any value in that field, irrespective of the index.
Maybe you are after something else?

14

Oh I get it now. I graphed the unique count and it shows as "2" so that's working as intended. How would I go about graphing the values of both indexes at once?

Every 1 minute we receive this kind of data:

{
       "@timestamp" => "2016-07-20T19:35:06.627Z",
             "type" => "vc_server",
         "hostname" => "agslx-hpclavc05",
    "MAIN_hit_rate" => 99.63,
             "tags" => [
        [0] "varnishstat"
    ]
}

How can I graph the value of MAIN_hit_rate regardless of min/max/avg?
I think the easiest approach would be to edit the field and include the hostname to achieve what I want.

15

KB only shows aggregations of some sort, so it can't show the exact value. It's kinda counterintuitive.