How to change Data-Type in a Field to only get an Ip-Adr?

#1

Hey Community,
so basically i'm totally new to ES. I set up an Elk-Cluster with 2 Worker Node's, and a Master Node running Kibana etc.

I attached a security system to it via logstash, so now i get the syslogs and it all works out just fine.
I wanted to use the GeoIP Module because bosses love that stuff.
My Problem is, that the Firewall only attaches the IP address in a field call 'ui' but painfully its written like "ssh (x.x.x.x)" like 60% of the time. Instead of reconfiguring the Firewall, is there a way for me to convert this field into a type which fits my needs?
Sorry for asking something this low -level, but as i said, totally new in this field.

I know that it has something to do with the Mapping Configuration.

    *"ui": {*

  •      "type": "text",*

  •      "norms": false,*

  •      "fields": {*

  •        "keyword": {*

  •          "type": "keyword",*

  •          "ignore_above": 256*

But yet i'm too new to this topic to know where to find this setting.

Greetz
Moritz

#2

What does that field look like the other 40% of the time?

#3

@Badger
Plain RemoteIP's 'x.x.x.x' , just the format i'd need.
It depends on which Access Control is being tested.

#4

OK, so you can use grok to extract the ip address

grok { match => { "someField" => "%{IPV4:ip}" } }

That will work for both "ssh (10.2.3.4)" and "10.1.2.3".

#5

So simply put it in my /etc/logstash/conf.d/XX-syslog-input.conf file?
But will this not trigger a reindex of all the data stored in there?

#6

Yes, add that to your configuration and new events will have an [ip] field added to them. It does not do anything to documents that have already been indexed.

#7

Great, i'll check it out today evening !
Thanks in advance :slight_smile:

#8

Ok i just noticed, will this be a Problem for my Logs if there are around 30000/90000 per Day which do not even have this "ui" Field? Since i changed up the Config, there are no logs comming in at all. Sorry for asking that much.

#9

So to give you a better view of how it's done in my file here some grok match => line which i'll propably have to extend, but i'm not sure about the syntax.

grok {
match => [ "message", "\S+ \S+ (?<syslog_hostname>\S+) (?\S+): (?\S+) (?\S+) [Originator@\d+ (?[^]]+)] (?.*)" ]

#10

If the field that it tries to match does not exist then the grok is a no-op. If no logs are being ingested you may have a syntax error in the configuration.