Requirements(ElasticSearch+Kibana)

signin_vasanth · July 20, 2013, 5:51am

Hi to All,

I am working on ElasticSearch+Kibana combination it means i am trying to
feed my elastic search log data into kibana ....

I got the result i.e elastic search is working well and tested the same
with the URL* http://localhost:9200/_plugin/head/* (head front end
controller)...It fetching my database table every 1 minutes..

Now i have to implement the same with kibana....What are the things i
have to perform to act my search in kibana ?...

So for i tried in many ways i could in get clear vision in any of the
site..I hope this group will help me out..

Awaiting your response....

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

jprante · July 20, 2013, 4:47pm

Kibana is highly configurable but for best OOTB experience it is aligned
with Logstash.

There are many examples on Google... for example here

For your log data, you could try the logstash "timestamped metadata
mapping" which has the following fields

@source: The source of the event which includes the plugin that generated
it and the hostname that produced it.
@tags: An array of tags on the event.
@fields: A set of fields, for example "user": "james" for the event.
@timestamp: An ISO8601 timestamp.
@source_host: The source host of the event.
@source_path: The path, if any, of a source, for example /var/log/messages.
@message: The event's message. In our case it is what we put into STDIN.
@type: The value of the type configuration option we set.

A sample

{
"@source":"syslog",
"@tags":,"@fields":{},
"@timestamp":"2013-03-26T06:40:36.692Z",
"@source_host":"lxb009",
"@source_path":"/var/log/rsyslog.d/lxb007/messages",
"@message":"Mar 26 07:40:35 lxb007 chef: [2013-03-26T07:
40:32+01:00] INFO: Starting Chef Run for
lxb007.devops.test","@type":"linux-syslog"
}

A mapping template can be found at

gist.github.com

https://gist.github.com/deverton/2970285

logstash-template.json

{
    "template": "logstash-*",
    "settings" : {
        "number_of_shards" : 1,
        "number_of_replicas" : 0,
        "index" : {
            "query" : { "default_field" : "@message" },
            "store" : { "compress" : { "stored" : true, "tv": true } }
        }
    },

This file has been truncated. show original

set-template.sh

curl -XPUT 'http://localhost:9200/_template/template_logstash/' -d @logstash-template.json

and I used this

{
"default": {
"properties" : {
"@fields": { "type": "object", "dynamic": true,
"path": "full" },
"@message" : { "type" : "string", "index" : "analyzed" },
"@source" : { "type" : "string", "index" : "not_analyzed" },
"@source_host" : { "type" : "string", "index" :
"not_analyzed" },
"@source_path" : { "type" : "string", "index" :
"not_analyzed" },
"@tags": { "type": "string", "index" : "not_analyzed" },
"@timestamp" : { "type" : "date", "index" : "not_analyzed" },
"@type" : { "type" : "string", "index" : "not_analyzed" }
}
}
}

I switched recently to this mapping when indexing highly structured
bibliographic data (many hundred fields) in @fields to get them visualized
in Kibana. Works like a charm!

Jörg

On Sat, Jul 20, 2013 at 7:51 AM, Vasanthakumar Rajendran <
signin.vasanth@gmail.com> wrote:

Hi to All,

I am working on Elasticsearch+Kibana combination it means i am trying
to feed my Elasticsearch log data into kibana ....

I got the result i.e Elasticsearch is working well and tested the same
with the URL* http://localhost:9200/_plugin/head/* (head front end
controller)...It fetching my database table every 1 minutes..

Now i have to implement the same with kibana....What are the things i
have to perform to act my search in kibana ?...

So for i tried in many ways i could in get clear vision in any of the
site..I hope this group will help me out..

Awaiting your response....

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.