I am working on adding logging functionality to an internal tool used as a build environment manager/build orchestrator. I am currently sending data to ES by formatting a JSON with relevant data and using a POST request directly to the ES index containing the JSON. We would like to correlate build times with geographic locations to understand how distance from our servers affects these times. Is there anyway to leverage the GeoIP plugin without using Filebeat to ship logs?
If you use an ingest pipeline that has it defined you can, the data doesn't have to be sent via beats, it just has to go via the API 
So I made an ingest pipeline like this:
{
"description": "Pipeline for parsing IP for location info",
"processors": [
{
"geoip": {
"field": "ip"
}
}
]
}
I attempt a POST like this:
curl -XPOST 'ES_HOST:ES_PORT/test_index/test_type/test_id?pipeline=location-pipeline' -H 'Content-Type: application/json' -d '{
"ip": "8.8.0.0"
}'
My data gets through to ES and I am able to see it in Kibana, but my _id field is static and new entries just replace the previous one. Is there a different way of going about this?
Are all the entries the same?
Not sure what you mean by that. When I XPOST the json containing the IP, I switched the IP so I can tell what happens. If I send 8.8.8.8 followed by 9.9.9.9, the 8.8.8.8 entry in Kibana gets replaced with the 9.9.9.9 because the _id is test_id. I'm not sure how to get around this, considering that Filebeat usually creates a unique ID when sending directly to an index. Is there any other way to post directly to a pipeline without specifying the _id?
Right, but did you change any of
?
If not, then it will overwrite as you are specifying the same document _id.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.