Differences between _source in Kibana and in elasticsearch

Elastic Stack Elasticsearch
1

Hi,

I use logstash's syslog plugin to collect logs, searching elastic search
and kibana for the same object gives different results in the _source
field...

Elasticsearch version 1.4.0, Kibana 4.0.0-BETA2

When querying elasticsearch with curl I get:

curl -XGET http://localhost:9200/logstash*/_search?pretty
stml@riakcs:~/work/java/elasticsearch/data/stml_elasticsearch/nodes/0/indices$
curl -XGET
'http://localhost:9200/logstash*/_search?pretty&q=_id:AUoVYl3Ayvv7Nc0uRA6X'
{
"took" : 7,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 1.0,
"hits" : [ {
"_index" : "logstash-2014.12.04",
"_type" : "syslog",
"_id" : "AUoVYl3Ayvv7Nc0uRA6X",
"_score" : 1.0,
"_source":{"message":"pam_authenticate: Authentication
failure","@version":"1","@timestamp":"2014-12-04T12:59:35.000Z","type":"syslog","host":"0:0:0:0:0:0:0:1","priority":83,"timestamp":"Dec
4
13:59:35","logsource":"riakcs","program":"su","pid":"15292","severity":3,"facility":10,"facility_label":"security/authorization","severity_label":"Error"}
} ]
}
}

But in Kibana I get:

@timestamp December 4th 2014, 13:59:35.000 @version 1 _id
AUoVYl3Ayvv7Nc0uRA6X _index logstash-2014.12.04 _source {"message":"pam_authenticate:
Authentication
failure","@version":"1","@timestamp":"2014-12-04T12:59:35.000Z","type":"syslog","host":"0:0:0:0:0:0:0:1"}
_type syslog host 0:0:0:0:0:0:0:1 message pam_authenticate:
Authentication failure type syslog

Missing a lot of fields in _source...

I would have expected these views of the same field to be alike...have I
misunderstood something

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/25a96d8d-6e51-4e48-8294-14bd9b52be34%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

I would guess that you need to refresh your field list in the Settings >
Indices > Index pattern section of Kibana4; this is a new thing in Kibana4
that's very different from v3. Drove me crazy trying to figure it out until
I filed an issue. See Rashid's answer to my Github issue here:

It would be nice to see this happen at least semi-automagically in the
future; it's going to bite a lot of people during migration.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fa95a091-752b-40de-b5f4-99dc3cc0c5aa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

3

Hi Jay,

Thank you!

That did the trick. I did go crazy over this problem too :wink: I simply can't
understand how Kibana can choose to fiddle with _source before showing
it...seems very strange to me...

Cheers,
Stefan

Den torsdag den 4. december 2014 22.47.49 UTC+1 skrev Jay Swan:

I would guess that you need to refresh your field list in the Settings >
Indices > Index pattern section of Kibana4; this is a new thing in Kibana4
that's very different from v3. Drove me crazy trying to figure it out until
I filed an issue. See Rashid's answer to my Github issue here:

Discover doesn't show all fields · Issue #1995 · elastic/kibana · GitHub

It would be nice to see this happen at least semi-automagically in the
future; it's going to bite a lot of people during migration.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/17cf6d2f-1bbb-400f-a0a6-043c4ae7d0d8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.