Logstash output

Elastic Stack Logstash
1

I wrote a conf file and when it reads my ip addresses from the DB it outputs them as an integer instead of the ip address.
I tried to do a mutate like the following:
filter {
mutate {
convert => { "sourceipv4" => "string" }
}
}

But it will still just output it to a string when I run it just in my console to see what the data going into elastic will be.

2

What you are trying to achieve? It is not clear.

Can you share the output you are getting?

3

Read from which database? (Oracle, MSQL, mysql, .... )

What type is that column in the database?

4

For instance I am trying to get the sourceipv4 from a ms sql DB and when logstash parses it it comes out like -1979695601 instead of an ip address.

This is the filter I am trying to use to get them back to IP addresses.

filter {
mutate {
convert => {
"targetipv4" => "string"
"sourceipv4" => "string"
"analyzeripv4" => "string"
}
}

grok {
match => {
"targetipv4" => "%{IPV4:targetipv4}"
"sourceipv4" => "%{IPV4:sourceipv4}"
"analyzeripv4" => "%{IPV4:analyzeripv4}"
}
}
}

5

You would need to provide more context on how your pipeline looks like, what filters do you have, how the source field looks like.

The filters you shared will not work because -1979695601 is not a valid IP address, so the grok filter will fail.

6

It originally was an IP address but when logstash brings the data over it is converting it to an integer. I wish I could share the outputy let me try to sanatize it and I will.

7

This does not sound right, as far as I know Logstash does not converts anything unless told to do so, but I'm not sure what input and filters you are using.

You will need to provide how the full configuration looks like.

Also, can you show some examples on how the value for those columns looks like in the MSSQL database?

8

Funnily enough, positive 1979695601 has a 31 bit representation, with leading 0.

So -1979695601 is 32 bits, leading bit one.

An IPv4 address is 32 bits.

-->

I dont think an IPaddress is being output by MSSQL. I don't believe there is a native IPaddress type in MSSQL, it can be represented lots of ways, one of which is a 32 bit integer.

9

It looks like that in the source the IP is stored as the integer representation.

So it needs to be converted back to the dotted representation in Logstash.

This probably can be done with a ruby filter a this is a one liner in ruby.

irb(main):001:0> require 'ipaddr'
=> true
irb(main):002:0> result = IPAddr.new(1979695601, Socket::AF_INET).to_s
=> "117.255.193.241"
irb(main):003:0>

This works:

filter {  
  mutate {
    gsub => ["sourceipv4","-",""]
  }
  ruby {
    code => '
      ipv4 = event.get("sourceipv4").to_i
      ipv4_converted = IPAddr.new(ipv4, Socket::AF_INET).to_s
      event.set("sourceipv4", ipv4_converted)
    '
  }
}
2 Likes
10

Thank you this worked. What would I need to do for it to grab the other 2 that I listed as well?

11

actually I used this and it seems to be working from your starter that you gave me.

filter {
  mutate {
    gsub => [
      "sourceipv4", "-", "",
      "targetipv4", "-", "",
      "analyzeripv4", "-", ""
    ]
  }

  ruby {
    code => '
      if event.get("sourceipv4")
        ipv4_source = event.get("sourceipv4").to_i
        ipv4_converted_source = IPAddr.new(ipv4_source, Socket::AF_INET).to_s
        event.set("sourceipv4", ipv4_converted_source)
      end

      if event.get("targetipv4")
        ipv4_target = event.get("targetipv4").to_i
        ipv4_converted_target = IPAddr.new(ipv4_target, Socket::AF_INET).to_s
        event.set("targetipv4", ipv4_converted_target)
      end

      if event.get("analyzeripv4")
        ipv4_analyzer = event.get("analyzeripv4").to_i
        ipv4_converted_analyzer = IPAddr.new(ipv4_analyzer, Socket::AF_INET).to_s
        event.set("analyzeripv4", ipv4_converted_analyzer)
      end
    '
  }
1 Like