I'm working on a system to record usage data for an application that
submits its data to an ES cluster. I would like to record the location of
each data point based on IP geolocation. I found the Logstash plugin that
uses the GeoIP databases, but I was unable to find any solutions built for
just Elasticsearch. Has anybody done something like this before?
In addition, it would be convenient to extract the IP of the point itself
from the "X-Forwarded-For" header of the incoming data point. Is there a
way to access these headers when the point is received by Elasticsearch?
On Thursday, July 17, 2014 10:20:55 AM UTC-4, Justin Koehler wrote:
I'm working on a system to record usage data for an application that
submits its data to an ES cluster. I would like to record the location of
each data point based on IP geolocation. I found the Logstash plugin that
uses the GeoIP databases, but I was unable to find any solutions built for
just Elasticsearch. Has anybody done something like this before?
This is something that's typically done outside ES, in a document
processing pipeline or indexer.
In addition, it would be convenient to extract the IP of the point itself
from the "X-Forwarded-For" header of the incoming data point. Is there a
way to access these headers when the point is received by Elasticsearch?
this is exactly what logstash is for, so you may want to give it a try, as
it is already there.
Also you can use the geoip filter to extract the ip address from the header
as well, granted you log that one.
On Thursday, July 17, 2014 10:20:55 AM UTC-4, Justin Koehler wrote:
I'm working on a system to record usage data for an application that
submits its data to an ES cluster. I would like to record the location of
each data point based on IP geolocation. I found the Logstash plugin that
uses the GeoIP databases, but I was unable to find any solutions built for
just Elasticsearch. Has anybody done something like this before?
This is something that's typically done outside ES, in a document
processing pipeline or indexer.
In addition, it would be convenient to extract the IP of the point itself
from the "X-Forwarded-For" header of the incoming data point. Is there a
way to access these headers when the point is received by Elasticsearch?
how exactly could this work?
For example we are using the pattern "Quotedstring" to extract the up to 4
IPs in the X-Forwared-For header of our Apache Logs.
When we then try using this one in the geoip filter the filter seems to
miss the IP.
Am Montag, 4. August 2014 09:56:53 UTC+2 schrieb Alexander Reelsen:
Hey,
this is exactly what logstash is for, so you may want to give it a try, as
it is already there.
Also you can use the geoip filter to extract the ip address from the
header as well, granted you log that one.
--Alex
On Sat, Jul 19, 2014 at 6:26 AM, Otis Gospodnetic <otis.gos...@gmail.com
<javascript:>> wrote:
Hi,
On Thursday, July 17, 2014 10:20:55 AM UTC-4, Justin Koehler wrote:
I'm working on a system to record usage data for an application that
submits its data to an ES cluster. I would like to record the location of
each data point based on IP geolocation. I found the Logstash plugin that
uses the GeoIP databases, but I was unable to find any solutions built for
just Elasticsearch. Has anybody done something like this before?
This is something that's typically done outside ES, in a document
processing pipeline or indexer.
In addition, it would be convenient to extract the IP of the point itself
from the "X-Forwarded-For" header of the incoming data point. Is there a
way to access these headers when the point is received by Elasticsearch?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
