Kibana: howto query for IPaddr:Port in String with regex/wildcard


(Mischa Diehm) #1

Hi,

I use logstash to receive our network device logs and pump them into
elasticsearch. For the moment this data is just parsed with the logstash
syslog filter.
I which to search within the default kibana logstash interface for various
formats of IPaddr Port combinations. These are stored in the "message:"
field.

As far as I understood it makes a difference if I do wildcard searches or
regular expression searches:

when I query for: "10.34.227.1:443" -> I get results
when I query for: "10.34.227.?:443" -> I get no results
when I query for: "10.34.227.1?443" -> I get results
=> Is this because ? doesn't replace numbers just characters?

What would be the way to search for flexible IPaddr/Port combinations where
parts of the search query would be regular expressions or ranges? Are all
searches possible for single words and phrases?

Do I in general misunderstand the query possibilities of kibana? I read all
the docs from kibana, elasticsearch and lucene but couldn't find an answer
to my problem. Is there sth. I'm missing or misinterpreting?

thanks in advance,
Mischa


(Mischa Diehm) #2

Sorry for posting this 3 times. When trying to mailpost I got the following reply for my mails. So I went and posted by the webinterface. Then I saw it was posted already.

Is it normal to get these Delivery Status Notivications?

Received: from mx1-pub.urz.unibas.ch (131.152.226.162) by exch.unibas.ch
(131.152.8.133) with Microsoft SMTP Server id 14.3.174.1; Wed, 25 Jun 2014
13:46:31 +0200
Received: from jinhua.switch.ch (jinhua.switch.ch [130.59.138.50]) by
mx1-pub.urz.unibas.ch (Postfix) with ESMTPS id 858B93E014B for
mischa.diehm@unibas.ch; Wed, 25 Jun 2014 13:46:31 +0200 (CEST)
Received: from mail-ob0-x24d.google.com (mail-ob0-x24d.google.com
[IPv6:2607:f8b0:4003:c01::24d]) by jinhua.switch.ch (8.14.4/8.14.4/Debian-4)
with ESMTP id s5PBkTSd018537 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128
verify=NOT) for mischa.diehm@unibas.ch; Wed, 25 Jun 2014 13:46:30 +0200
Received: by mail-ob0-f205.google.com with SMTP id uy5so24660obc.0 for
mischa.diehm@unibas.ch; Wed, 25 Jun 2014 04:46:25 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=google.com; s=20120113;
h=mime-version:from:to:subject:message-id:date:content-type;
bh=MQCJ4D5OzcL2NE3s3byyW2VIH8RNZ02I8QuWnBo3ycI=;
b=d9+mV/FDnwCpl4/7IZPTcsIpWCzQccNLjU9XN6RmqIy3dB7cEkMpPGRV2Sy5NDy6fZ
yZDFszrzLxiEhRrHXCS1/WEq8Sc0TxM3nMPF5WiraJQWU8F/eOziJFk8X4xMA+ew+mFG
5fbvafUZU7woChfC+rOivjS0xR7zaKULNCo5aKVPK9BRVwxp0yyQ/myEf4s10khbZEUT
VV6VbzlLZ+gBt5hHRBD2sWrbiW+wBDEjCkOitNiMVz5Sb9gPwVsnEfdmhkh1OTuJRPpe
HUDrbubvvlTQI8GVp98T/U8uAscBA6GWX8Yq/7RtiXcvGgEGF6JrIGDPgj1N9UFBLQif
YvZw==
X-Received: by 10.182.58.71 with SMTP id o7mr4232478obq.3.1403696785766;
Wed, 25 Jun 2014 04:46:25 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.182.58.71 with SMTP id o7mr6410979obq.3; Wed, 25 Jun 2014
04:46:25 -0700 (PDT)
From: Mail Delivery Subsystem mailer-daemon@google.com
To: mischa.diehm@unibas.ch
X-Failed-Recipients: elasticsearch@googlegroups.com
Subject: Delivery Status Notification (Failure)
Message-ID: e89a8f83a9fb49b79404fca7a0f0@google.com
Date: Wed, 25 Jun 2014 11:46:25 +0000
X-Bayes-Prob: 0.0001 (Score 0, tokens from: mischa.diehm@unibas.ch, unibas-ch:default, base:default, @@RPTN)
X-Spam-Score: 1.31 (*) [Tag at 9.00] URI_HEX:1.313
X-UNIBASEL: Spam detected for unibas.ch NO X
X-CanIt-Geo: ip=2607:f8b0:4003:c01::24d; country=US
X-CanItPRO-Stream: unibas-ch:mischa.diehm@unibas.ch (inherits from unibas-ch:default,base:default)
X-Canit-Stats-ID: 02MizKu4u - 2cb418524362 - 20140625
X-Antispam-Training-Forget: https://ham.switch.ch/canit/b.php?i=02MizKu4u&m=2cb418524362&t=20140625&c=f
X-Antispam-Training-Nonspam: https://ham.switch.ch/canit/b.php?i=02MizKu4u&m=2cb418524362&t=20140625&c=n
X-Antispam-Training-Spam: https://ham.switch.ch/canit/b.php?i=02MizKu4u&m=2cb418524362&t=20140625&c=s
X-Scanned-By: CanIt (www . roaringpenguin . com)
Return-Path: <>
X-MS-Exchange-Organization-AuthSource: URZ-HT-CAS-3.urz.unibas.ch
X-MS-Exchange-Organization-AuthAs: Anonymous
X-MS-Exchange-Organization-AVStamp-Mailbox: SYMANTEC;517472320;0;info
Content-type: text/plain;
charset="US-ASCII"
Content-transfer-encoding: 7bit

Hello mischa.diehm@unibas.ch,

We're writing to let you know that the group you tried to contact
(elasticsearch) may not exist, or you may not have permission to post
messages to the group. A few more details on why you weren't able to post:

  • You might have spelled or formatted the group name incorrectly.
  • The owner of the group may have removed this group.
  • You may need to join the group before receiving permission to post.
  • This group may not be open to posting.

If you have questions related to this or any other Google Group, visit the
Help Center at http://groups.google.com/support/.

Thanks,

Google Groups

----- Original message -----

X-Received: by 10.182.58.71 with SMTP id o7mr4232463obq.3.1403696785641;
Wed, 25 Jun 2014 04:46:25 -0700 (PDT)
Return-Path: mischa.diehm@unibas.ch
Received: from sam.nabble.com (sam.nabble.com. [216.139.236.26])
by gmr-mx.google.com with ESMTPS id
ha2si165778igb.1.2014.06.25.04.46.25
for elasticsearch@googlegroups.com
(version=TLSv1 cipher=RC4-SHA bits=128/128);
Wed, 25 Jun 2014 04:46:25 -0700 (PDT)
Received-SPF: none (google.com: mischa.diehm@unibas.ch does not designate
permitted sender hosts) client-ip=216.139.236.26;
Authentication-Results: gmr-mx.google.com;
spf=neutral (google.com: mischa.diehm@unibas.ch does not designate
permitted sender hosts) smtp.mail=mischa.diehm@unibas.ch
Received: from ben.nabble.com ([192.168.236.152])
by sam.nabble.com with esmtp (Exim 4.72)
(envelope-from mischa.diehm@unibas.ch)
id 1Wzlej-0003W7-7t
for elasticsearch@googlegroups.com; Wed, 25 Jun 2014 04:46:25 -0700
Date: Wed, 25 Jun 2014 04:46:25 -0700 (PDT)
From: Mischa Diehm mischa.diehm@unibas.ch
To: elasticsearch@googlegroups.com
Message-ID: 1403696785234-4058501.post@n3.nabble.com
Subject: Kibana: howto query for IPaddr:Port in String with regex/wildcard
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Hi,

I use logstash to receive our network device logs and pump them into
elasticsearch. For the moment this data is just parsed with the logstash
syslog filter.
I which to search within the default kibana logstash interface for various
formats of IPaddr Port combinations. These are stored in the "message:"
field.

As far as I understood it makes a difference if I do wildcard searches or
regular expression searches:

when I query for: "10.34.227.1:443" -> I get results
when I query for: "10.34.227.?:443" -> I get no results
when I query for: "10.34.227.1?443" -> I get results
=> Is this because ? doesn't replace numbers just characters?

What would be the way to search for flexible IPaddr/Port combinations where
parts of the search query would be regular expressions or ranges? Are all
searches possible for single words and phrases?

Do I in general misunderstand the query possibilities of kibana? I read all
the docs from kibana, elasticsearch and lucene but couldn't find an answer
to my problem. Is there sth. I'm missing or misinterpreting?

thanks in advance,
Mischa

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Kibana-howto-query-for-IPadd
r-Port-in-String-with-regex-wildcard-tp4058501.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.


(system) #3