Finding fields with a "-"

Elastic Stack Kibana
1

Running ELK 5.3.1

We are working on trying to track down an issue where our Apache responses sometimes have a content length of 0 bytes. Unfortunately 0 actually gets logged as a "-" by apache. I'm looking at changing this in apache but we need to analyze our current logs to track this down.

I tried this as my kibana search:

type:apache AND bytes:\-

but I get the following error from kibana:

Discover: failed to create query: { "bool" : { "must" : [ { "query_string" : { "query" : "type:apache AND bytes:-", "fields" : [ ], "use_dis_max" : true, "tie_breaker" : 0.0, "default_operator" : "or", "auto_generate_phrase_queries" : false, "max_determinized_states" : 10000, "enable_position_increments" : true, "fuzziness" : "AUTO", "fuzzy_prefix_length" : 0, "fuzzy_max_expansions" : 50, "phrase_slop" : 0, "analyze_wildcard" : true, "escape" : false, "split_on_whitespace" : true, "boost" : 1.0 } }, { "match" : { "bytes" : { "query" : "-", "type" : "phrase", "operator" : "OR", "prefix_length" : 0, "max_expansions" : 50, "fuzzy_transpositions" : true, "lenient" : false, "zero_terms_query" : "NONE", "boost" : 1.0 } } }, { "range" : { "@timestamp" : { "from" : 1509560579745, "to" : 1509561479745, "include_lower" : true, "include_upper" : true, "format" : "epoch_millis", "boost" : 1.0 } } } ], "disable_coord" : false, "adjust_pure_negative" : true, "boost" : 1.0 } }

Is there a way to filter on these existing log entries that have a bytes field of "-"?

Thanks,
Dan

2

Can you please let us know your mappings to debug further. if its a key word field you can wrap the keyword in quotes.

Thanks
Rashmi

3

I'd be happy to if you wouldn't mind helping me that. How can I get my field mappings?

4

you may use GET /_mapping or may be specifically for an index like : GET /my-index/_mapping
For future ref: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-get-mapping.html

Thanks
Rashmi

5

If I'm reading this right, it looks to be:

"bytes":{"type":"long"}

6

Can you try _exists_:bytes. Am guessing the field aren't hyphens but actually nulls

Thanks
Rashmi

7

I tried that. It returned lots of log entries. Every apache log that we get has the bytes field. It certainly exists but I'm trying to the find the ones that are either 0 (none return as 0 when searching bytes:0) or "-".

From the apache documentation on how it's currently logging:
"Size of response in bytes, excluding HTTP headers. In CLF format, i.e. a '-' rather than a 0 when no bytes are sent."

so 0 should be getting logged as "-" according to that.

I tried searching for bytes:null and bytes:"null" but it didn't like either of those.

8

Can we try taking a look at your raw elasticsearch data to see what is stored in the field? If the bytes field is mapped as a long, we'll get a number parsing exception if an '-' is indexed. If it's a hyphen I'm a little confused as to how it's being added.

To search over the index:

GET  index/_search?size=100

If the field is 0, then bytes:0 should work. If it's null or empty (""), NOT _exists_:bytes should work.

9

I'm trying to run that like so:

curl -XGET 'http://localhost:9200/index/_search?size=100'

but it doesn't like that. Am I missing something?

edit: I figured out what I was doing wrong and I've run it against a few indices. I'm still not getting any bytes field in anything that is being returned though. I'm now running it like so:

curl -XGET 'http://localhost:9200/logstash-2017.12.27/_search?size=100'

10

Not sure I'm totally doing this right but I tried some specific searches. This returned 0:

curl -XGET 'http://localhost:9200/logstash-2017.10.31/_search?q=bytes:0'

and every form of search where I tried -, \-, "-" or "\-" resulted in a "failed to create query" error.

11

Actually, maybe logstash is just dropping it if it's a "-". Here's the grok pattern that is being used:

(?:%{NUMBER:bytes}|-)

I'm not a grok expert but I think that says to grab it if it's a NUMBER otherwise do nothing?

If this is the case we may only be capturing from 1 and higher.

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.