Application wise dashboard in kibana

pankaj_ghadge · February 12, 2014, 2:08pm

Hello All,

I want to create a application wise dashboard in kibana like as

1) Dashboard for domain example1.com: It should show php, css, js, and
jpg/png hit count for domain example1.com
2) Dashboard for domain example2.com: It should show php, css, js, and
jpg/png hit count for domain example2.com
3) Dashboard for domain example3.com: It should show php, css, js, and
jpg/png hit count for domain example3.com

JSON Data for Apache access:

{
"_index": "logstash-2014.02.12",
"_type": "apache-access",
"_id": "BQDaP5GvR0OVb_EzXtU-Fg",
"_score": null,
"_source": {
"@timestamp": "2014-02-12T13:30:15+00:00",
"message": "172.23.35.10 - - [12/Feb/2014:13:30:14 +0000] "GET //img/logo.png HTTP/1.1" 200 7819 "-" "curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5" example1.com",
"host": "192.168.1.55",
"@source_host": "192.168.1.79",

"vhost": "example1.com",
"bytes": "7819",
"clientip": "192.168.1.23",
"method": "GET",
"request": "//img/logo.png",
"referrer": "\"-\"",
"useragent": "curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5",
"status": "200"

},
}

Mysql slow query log:

1) Dashboard for mysql slow query from host1 (like 192.168.1.85):

2) Dashboard for mysql slow query from host2 (like 192.168.1.86):

JSON Data for mysql slow query:

{
"_index": "logstash-2014.01.28",
"_type": "mysql-slowquery",
"_id": "mBmHejbmQvSs9BXV65DUpw",
"_score": null,
"_source": {
"message": "# User@Host: root[root] @ localhost []\n# Query_time: 3.475852 Lock_time: 0.000027 Rows_sent: 284635 Rows_examined: 284635\nSET timestamp=1390899647;\nSELECT /*!40001 SQL_NO_CACHE */ * FROM session;",
"@version": "1",
"@timestamp": "2014-01-28T09:00:47.000+00:00",
"type": "mysql-slowquery",
"host": "192.168.1.90",
"path": "/usr/local/mysql/data/mysql-slow.log",
"duration": 3.475852,
"lock_wait": 0.000027,
"results": 284635,
"scanned": 284635
},
}

Tomcat logs:

Dashboard for tomcat from host1 (like 192.168.1.25):
Dashboard for tomcat from host2 (like 192.168.1.26):

JSON data for tomcat:

{

"_index": "logstash-2014.02.12",
"_type": "tomcat",
"_id": "fMDNXBsfSOm-fAb7Qz0MkQ",
"_score": null,
"_source": {
"@timestamp": "2014-02-12T09:26:40.283+00:00",
"message": "Dec 13, 2012 5:41:06 PM org.apache.catalina.connector.Connector pause",
"@version": "1",
"type": "tomcat",
"host": "192.168.1.25",
"path": "/usr/local/tomcat/logs/catalina.out.0"
},
}

Suppose person1 is responsible for monitoring apache logs and will be able to see only those logs statistic. Person2 is responsible for monitoring "mysql slow query log" then , will be able to see "mysql slow query" log statistic only.

so, I want to know, how to create application wise dashboard in kibana?

Thanks,

--pankaj

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e963f673-d9fc-4958-a9d9-e5fa5370fe51%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Binh_Ly · February 12, 2014, 4:29pm

Pankaj:

You should be able to pin a query on each dasboard to filter down to only
the log events that you're interested in. So for example in your first
case, you can pin a filter (query) like:

vhost.raw:example1.com

And everything in your current dashboard will narrow down to example1.com
data.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b027143a-910a-4417-8d5c-0fe1503d7b85%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

pankaj_ghadge · February 13, 2014, 5:59am

Hi Binh,

Thanks for reply,

But, I want to search php, css ,js and jpg count from example1.com then I
think, I need to query like below:

"vhost:example1.com AND *.php" for php count.
"vhost:example1.com AND *.css" for css count.
"vhost:example1.com AND *.js" for js count.

I think it looks bit odd. is their any other way around?

On Wednesday, 12 February 2014 21:59:34 UTC+5:30, Binh Ly wrote:

Pankaj:

You should be able to pin a query on each dasboard to filter down to only
the log events that you're interested in. So for example in your first
case, you can pin a filter (query) like:

vhost.raw:example1.com

And everything in your current dashboard will narrow down to example1.comdata.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/53d3af4d-a2ed-4781-9c5b-fe6630f9a123%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

pankaj_ghadge · February 13, 2014, 10:34am

Hi all,

Is it possible by changing default.json file in kibana, so it will not
display additional query in kibana query box.

File default.json:

{ "title": "Logstash Search", "services": { "query": {
"idQueue": [ 1 ], "list": { "0": {
"query": ".php", "alias": "", "color": "#EAB839",
"pin": false, "id": 1, "type": "lucene" },
"1": { "query": ".css", "alias": "", "color":
"#6ED0E0", "pin": false, "id": 2, "type":
"lucene" }, "2": { "query": ".png",
"alias": "", "color": "#EF843C", "pin": false,
"id": 3, "type": "lucene" }, "3": {
"query": ".gif", "alias": "", "color": "#E24D42",
"pin": false, "id": 4, "type": "lucene" }
}, "ids": [ 0, 1, 2, 3 ] },
"filter": { "idQueue": [ 1 ], "list": { "0":
{ "type": "time", "field": "@timestamp", "from":
"now-{{ARGS.from || '24h'}}", "to": "now", "mandate":
"must", "active": true, "alias": "", "id": 0
} }, "ids": [ 0 ] } }, "rows": [ {
"title": "Graph", "height": "350px", "editable": true,
"collapse": false, "collapsable": true, "panels": [ {
"span": 12, "editable": true, "group": [
"default" ], "type": "histogram", "mode":
"count", "time_field": "@timestamp", "value_field":
null, "auto_int": true, "resolution": 100,
"interval": "10m", "fill": 3, "linewidth": 3,
"timezone": "browser", "spyable": true, "zoomlinks":
true, "bars": true, "stack": true, "points":
false, "lines": false, "legend": true, "x-axis":
true, "y-axis": true, "percentage": false,
"interactive": true, "queries": { "mode": "all",
"ids": [ 0 ] }, "title":
"Events over time", "intervals": [ "auto",
"1s", "1m", "5m", "10m",
"30m", "1h", "3h", "12h",
"1d", "1w", "1M", "1y" ],
"options": true, "tooltip": { "value_type":
"cumulative", "query_as_alias": false } }
], "notice": false }, { "title": "Events", "height":
"350px", "editable": true, "collapse": false, "collapsable":
true, "panels": [ { "title": "All events",
"error": false, "span": 12, "editable": true,
"group": [ "default" ], "type": "table",
"size": 100, "pages": 5, "offset": 0,
"sort": [ "@timestamp", "desc" ],
"style": { "font-size": "9pt" }, "overflow":
"min-height", "fields": [], "highlight": [],
"sortable": true, "header": true, "paging": true,
"spyable": true, "queries": { "mode": "all",
"ids": [ 0 ] }, "field_list":
true, "status": "Stable", "trimFactor": 300,
"normTimes": true } ], "notice": false } ],
"editable": true, "failover": false, "index": { "interval": "day",
"pattern": "[logstash-]YYYY.MM.DD", "default":
"NO_TIME_FILTER_OR_INDEX_PATTERN_NOT_MATCHED" }, "style": "dark",
"panel_hints": true, "pulldowns": [ { "type": "query",
"collapse": false, "notice": false, "query": "", "pinned":
true, "history": [], "remember": 10 }, { "type":
"filtering", "collapse": true, "notice": false } ], "nav":
[ { "type": "timepicker", "collapse": false, "notice":
false, "status": "Stable", "time_options": [ "5m",
"15m", "1h", "6h", "12h", "24h", "2d",
"7d", "30d" ], "refresh_intervals": [ "5s",
"10s", "30s", "1m", "5m", "15m",
"30m", "1h", "2h", "1d" ], "timefield":
"@timestamp", "now": true, "filter_id": 0 } ], "loader": {
"save_gist": false, "save_elasticsearch": true, "save_local":
true, "save_default": true, "save_temp": true,
"save_temp_ttl_enable": true, "save_temp_ttl": "30d", "load_gist":
true, "load_elasticsearch": true, "load_elasticsearch_size": 20,
"load_local": true, "hide": false }, "refresh": false}*

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/70b91910-bcf2-4e80-9fc8-704b2a9c636a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

pankaj_ghadge · February 18, 2014, 7:20am

Hi,

Is their any other way around for this or not? please let us know.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e0febda0-2ac6-44a9-ba38-e0cb02129ed4%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Penugonda_chenna_red · April 10, 2014, 4:11am

Pankaj.

were you able to solve this issue.. I am also stuck in similar need.

If you could able to solve it pl. share the details.

regards
On Tuesday, February 18, 2014 12:50:45 PM UTC+5:30, pankaj ghadge wrote:

Hi,

Is their any other way around for this or not? please let us know.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0ee7b826-76ae-4198-a3d3-69727b478b73%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.