Oh, this is a good one! Not sure why I didn't catch it when I looked at
your mapping the first time. When you specify "@fields.size" it can point
to one of two things depending on the context: 1) the field with the name
"@fields.size" and 2) the field "size" inside the field "@fields". It looks
like your data look like this:
"fields" : {
"size": 12345
....
}
So, you need to setup mapping for the second case. Therefore, the mapping
in your template should look like this:
"properties" : {
"@fields": {
"type": "object",
"dynamic": true,
"path": "full",
"properties": {
"syslog_message" : { "type" : "string", "analyzer" :
"whitespace" },
"size" : { "type" : "long" , "index" : "not_analyzed"}
}
},
"@source" : { "type" : "string", "index" : "not_analyzed" },
"@source_host" : { "type" : "string", "index" : "not_analyzed" },
"@source_path" : { "type" : "string", "index" : "not_analyzed" },
"@timestamp" : { "type" : "date", "index" : "not_analyzed" },
"@type" : { "type" : "string", "index" : "not_analyzed" },
"@message" : { "type" : "string", "analyzer" : "standard" }
}
On Friday, April 19, 2013 5:52:07 AM UTC-4, Abhijeet Rastogi wrote:
Hi Igor,
Thanks for your reply. I've found the root cause of it and I would
really appreciate if you help me understanding what happened!
working_template.json: http://pb.abhijeetr.com/jeNV
working_mapping.json (of newly created index):
http://pb.abhijeetr.com/JLMH
Diff:
non_working_template.json: http://pb.abhijeetr.com/KSWd
non_working_mapping.json (of newly created index):
http://pb.abhijeetr.com/XXBF
Diffs for quick view:
$ diff working_template.json non_working_template.json
33,34c33
< "path_match": "@fields.*",
< "path_unmatch": "@fields.size"
"path_match": "@fields.*"
$ diff working_mapping.json non_working_mapping.json
9,10c9
< "path_match" : "@fields.*",
< "path_unmatch" : "@fields.size"
"path_match" : "@fields.*"
50c49,52
< "type" : "string"
"type" : "string",
"index" : "not_analyzed",
"omit_norms" : true,
"index_options" : "docs"
Doubts:
-
In your other thread, you mentioned that if I provide something in
properties, the dynamic template won't be applied to it. This doesn't
look like the case because I can see size type as string in
http://pb.abhijeetr.com/JLMH (working_mapping.json). But, I also see
@fields.size as long in the same mapping. I don't understand the
reason b/w the two. If I don't mention size in path_unmatch, I also
start getting Infinity/-Infinity in facet queries. I would really
appreciate the help here.
-
Why do we have "size" and "@fields.size" separately? One says as
string and the other as long, I am really lost here.
On Fri, Apr 19, 2013 at 1:31 AM, Igor Motov <imo...@gmail.com<javascript:>>
wrote:
If you don't see fields.size in the source, it means this field wasn't
there
when you sent the records for indexing. So, you need to go upstream and
figure out why application that is feeding elasticsearch with data
doesn't
populate this field.
On Thursday, April 18, 2013 2:38:46 PM UTC-4, Abhijeet Rastogi wrote:
I feel bad to bump this thread, but anyone? I'm really clueless here.
On Thu, Apr 18, 2013 at 12:08 AM, Abhijeet Rastogi abhije...@gmail.com
wrote:
Sorry for one more mail guys.. By writing " I can't execute a lucene
query", what I actually meant was I don't get any search results.
Silly me.
On Thu, Apr 18, 2013 at 12:02 AM, Abhijeet Rastogi <
abhije...@gmail.com>
wrote:
One thing I noticed is, I can't execute a lucene query (meaning
query_string type query) like
exists:"@fields.size"
But, I can do that for other fields. What can be the reason for that?
Also, when I do a simple matchall query with size as 10, I can see
all
fields being returned except "@fields.size", what exactly might have
happened to this field?
This is how my template looks like: http://pb.abhijeetr.com/jaVK
I would really appreciate a quick help here. This one is driving me
crazy.
On Wed, Apr 17, 2013 at 3:45 PM, Abhijeet Rastogi <
abhije...@gmail.com>
wrote:
Hi all,
I'm doing a simple query like:
curl -XPOST '
http://localhost:9200/logstash-2013.04.17/_search?pretty'
-d '{
"query": {
"match_all": { }
},
"facets" : {
"domain_size" : {
"terms_stats" : {
"key_field" : "@fields.from",
"value_field" : "@fields.size"
}
}
}
}'
In this, @fields.size is a long and @fields.from is a string which
is
non_analyzed.
I get results like:
"domain_size" : {
"_type" : "terms_stats",
"missing" : 622069,
"terms" : [ {
"term" : "",
"count" : 10689,
"total_count" : 0,
"min" : "Infinity",
"max" : "-Infinity",
"total" : 0.0,
"mean" : 0.0
}, {
"term" : "ema...@domain.com",
"count" : 6931,
"total_count" : 0,
"min" : "Infinity",
"max" : "-Infinity",
"total" : 0.0,
"mean" : 0.0
}
.. ..
..
..
I've truncated the output and have shown only the relevant parts.
I'm
not where do I look at to solve the problem.
--
Regards,
Abhijeet Rastogi (shadyabhi)
http://blog.abhijeetr.com
--
Regards,
Abhijeet Rastogi (shadyabhi)
http://blog.abhijeetr.com
--
Regards,
Abhijeet Rastogi (shadyabhi)
http://blog.abhijeetr.com
--
Regards,
Abhijeet Rastogi (shadyabhi)
http://blog.abhijeetr.com
--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.
--
Regards,
Abhijeet Rastogi (shadyabhi)
http://blog.abhijeetr.com
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
