Read/Match Data

Elastic Stack Logstash
1

Hi Elastic,

I want to match IP address from below type of data with field, i tried Translate filter but it is not supporting for .txt format,
Kindly help me to fix this.

I am downloading this from URL, using http_poller plugin.

185.93.185.235
219.151.8.160
93.174.93.136 no-reverse-dns-configured.com
116.31.116.5
209.58.129.109
113.108.10.31
207.244.70.169
208.100.26.228 ip228.208-100-26.static.steadfastdns.net
186.119.94.213
220.179.124.181
171.233.202.181
27.255.66.254
115.239.248.72
118.33.99.123
82.78.95.165 82.78.95.165-static.cluj.rdsnet.ro
192.99.236.73 ip73.ip-192-99-236.net
58.218.204.245
181.138.78.97 hfc-181-138-78-97.une.net.co
177.98.247.12 177.98.247.dynamic.adsl.gvt.net.br
104.223.17.24 104.223.17.24.static.quadranet.com
109.237.111.108
216.243.31.2
221.194.44.218
121.18.238.19
80.82.70.198 icsresearch1.plcscan.org
179.181.177.125 179.181.177.125.dynamic.adsl.gvt.net.br
222.186.51.181
94.102.49.174 mail.picdown.me
104.193.252.230 edwardmurphy.clientshostname.com
169.229.3.91 researchscan1.EECS.Berkeley.EDU

2

You're downloading that data with http_poller and you want to look up the IP addresses using the translate filter? Or you want to use the data downloaded with http_poller as the lookup table for the translate filter?

3

Either ways will work for me, Suggest me the best approach.

If my log contains this IP it should match

4

Transform the data into something the translate filter understands. You have a space-separated text file so it's not like YAML, JSON, or CSV is vastly different.

5

I tried to match the data with below mentioned script, but not getting correct output, at the same time i am able to achieve at grok debugger.

input {

stdin {codec => json_line}
}
filter { grok {match => { "message" => "(?<IP1>.*)\t(?<msg1>.*)\n(?<IP2>.*)\t(?<msg2>.*)\n(?<IP3>.*)\t(?<msg3>.*)" }    }

}

output

{
stdout { codec => rubydebug}
}