Kibana: how to regex query for 'hostname'?

Elastic Stack Kibana
#1

Hello,

I'm using Kibana 4 with ElasticSearch 1.4.4 to store logs pushed by Logstash. I have a 'hostname' field which is not_analyzed and can't find how to query with a regex on this field.

Field content is a hostname with content like

machine.my-domain.tld
machine2.my-domain.tld
my1-machine2.my3-domain.tld

I want to search on the first part of this field, because in some events the hostname is short on other is FQDN. For example i want all my1-machine2.my3-domain.tld and my1-machine2

I tried queries like

hostname:"my1-machine2*"
hostname:/my1-machine2.*/

but that don't work.

Do you understand what I'm doing bad?

edit

When I search for hostname:"my1-machine2*" I got events with short hostname (a lot) and FQDN (a few). When I use a filter to have hostname: "my1-machine2.my3-domain.tld" I see events which were not visible with my query

How do I index regular expressions?
#2

Hi Eric,

I just tried this and your first example should work if you do it without the quotes.

hostname.raw: my1-machine2*

Selection_046.jpg517×549 38.4 KB

You can find details on Kibana queries here.

/Jakob

1 Like
#3

When I do the same (but with hostname not hostame.raw)

hostname: my1-machine2*

I find more events than wanted, I see events which hostname field got my1 or machine2 separated. I even see events wihere it matches on another field.

It could mean my hostname field is analyzed, but if I look in my Kibana settings I see hostname -> analyzed : false and my template mapping looks like

{
  "template" : "logs-*",
  "settings" : {
    "number_of_shards" : 3,
    "index.refresh_interval" : "5s"
  },
  "mappings" : {
    "_default_" : {
       "_all" : {"enabled" : true},
       "dynamic_templates" : [ {
         "not_analyzed_string_fields" : {
           "match" : "*",
           "match_mapping_type" : "string",
           "mapping" : {
             "type" : "string", "index" : "not_analyzed", "omit_norms" : true, "doc_values" : true
           }
         }
       } ],
       "properties" : {
         "@version": { "type": "string", "index": "not_analyzed" },
         "geoip"  : {
           "type" : "object",
             "dynamic": true,
             "path": "full",
             "properties" : {
               "location" : { "type" : "geo_point" }
             }
         },
        "message" : {
          "type" : "string",
          "index" : "analyzed"
        },
        "ipv4" : {
          "type" : "ip",
          "index" : "analyzed",
          "store" : true
        }
       }
    }
  }
}
#4

Hi Eric,

I chose hostname.raw because my hostname field is analyzed.

You are right, it sounds like your hostname field is still analyzed (at least in some indices). Did you change the mapping to not analyzed before indexing data for the first time or while there where already logstash indices created?
Mapping changes are not applied to already existing data.

You could try to choose a time range where you are sure that the hostname field was set to not analyzed.

/Jakob

#5

Hum... I just checked my configuration and even if I thought that mapping templates were applied, it seems that it's not the case.

curl -XGET localhost:9200/_template/

indicates that i got no template for these indices.

That's a problem and I need to find why they are not applied.
Thanks for your help @jakommo

1 Like
#6