GeoIP Lite EOL pipeline filter


(bruno.dev) #1

Hi it seems Geoip Lite DB is not free anymore for home user and will be not be updated anymore as before. What services do you plan to offer to replace this one ? Does IP2Location LITE is in the road map ?
I am using geopip lite in pipeline filter . Thanks to let us know.


IP2location In place of GeoIP
(Robert Cowart) #2

You can still use GeoLite2 DBs. The notice from MaxMind is...

We will be discontinuing updates to the GeoLite Legacy databases as of April 1, 2018. You will still be able to download the April 2018 release until January 2, 2019. GeoLite Legacy users will need to update their integrations in order to switch to the free GeoLite2 or commercial GeoIP databases by April 2018.
In addition, in 2019, latitude and longitude coordinates in the GeoLite2 databases will be removed.* Latitude and longitude coordinates will continue to be provided in GeoIP2 databases. Please check back for updates.

Since around v5 Logstash supports the GeoLite2 DBs.


(Michael) #3

The latitude and longitude in GeoLite2 will be removed in 2019.

Can we support IP2Location LITE with more information such as latitude, longitude, ZIP code and time zone soon? The development should be minimal because they have an external filter ready.


(bruno.dev) #4

Hi all,

I read that we all agree that geoip is moving away.

It seems great some can move also toward with ip2location. But, I am using the pipeline filter and can only used this as i extract the geoIP from parsing syslog message. the GeopIP is then pipeline to fill the geoip struct.

  1. can we plan to have IP2location in pipeline mode ? is it already done ?
  2. can we plan to have also the information from ip2Proxy ? (i found very usefull the classification VPN, TOR, etc...)

Thanks, regards


#5

Hi Bruno-d,

We have found two good articles about using IP2Location and IP2Proxy filters in Elastic Stack. Please review it and let us know if it is useful.


Meanwhile, I strongly agree that we should integrate IP2Location LITE into road map this year.


(bruno.dev) #6

Hi thanks to all.
It's seems the usecase is with filebeats and it is not clear to me it will be processed through elastisearch with pipeline filter. My conf is pretty full and is Syslog-Logstach-Elastisearch-Kibana on a RPI3. ELK 5.6. I computed the IP to be parsed with querys and painless script to fill a specific variable field every 10 minutes. Querys are CURL in shell execute in Crontab. The reason is IP comes in the middle of syslog message alerts from the f/w of the router. The painless script do string search, extraction and set to a variable. I then pipeline the geoip filter to this variable every 10 minutes to get the GEOIP results. The PI is at the maximum of capacity in terms of memory (700Mega no swap) and averageLoad to 2-2.5. My usage of the geoip pipeline was in a shell as

curl -XPOST "http://localhost:9200/<logstash-{now%2Fd}>/syslog/_update_by_query?pipeline=geoip" -H 'Content-Type: application/json' -d' { "query": { "range" : { "@timestamp" : { "gte" : "now-14m" } } } }'

So, I am not sure i can manage this usecase with IP2Location.
I will use in the next weeks a new PI to test the same configuration using IP2location and will come back to you.
Thanks if you have in a meanwhile some tips (like you are pretty sure i will failed :joy: because I am not an ELK expert )

regards


(system) #7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.