Create a new field using two separate fields to produce geo-point

Elastic Stack Elasticsearch
1

I am using Elasticsearch and Kibana to visualize network traffic which includes Latitude and Longitude coordinates for IP addresses in a separate filed both of type "number"

As far as I could find in the documentations I needed to have a unique field for coordinates which should include both Latitude and Longitude and its type should be "geo-point".

I do not want Logstash to do this for me, as the package I use to create traffic flows directly writes to Elastic search.
So I thought maybe a possible solution is to create a new scripted field of string type, concatenate them with a "," and change the mapping as "geo-point" something like this:

If Longitude= 100 and Latitude=50 then my new scripted field that I named "Geo_Loc" would be:
Geo_Loc=100,50

kibana
kibana.png1680×1050 201 KB

This is exactly what I see when I explore my data in Discover tab in Kibana. I have also managed to create a new visualization using tile maps by applying Geohash aggregation on my scripted field "Geo_Loc".

However, what I see is an empty world map like this

kibana1
kibana1.png1680×1050 380 KB

(Note when I change the map type in the option tab, from scaled circled markers to other types such as heat map or etc I cannot see any changes, I am connected to internet and can browse without any problem)
I appreciate if someone can tell what's wrong and which step I am doing incorrectly.
Siamak

2

Query-time patching of omissions to your indexing logic will only get you so far.

You can't expect client-side javascript to make up for a lack of spatial indexing in your server.

Have a look at using ingest pipelines to massage your JSON immediately before insertion into elasticsearch.

1 Like
3

Thanks.
I will have a look.

Cheers,
Siamak

4

Hi Mark,

I created a pipeline like this:

PUT _ingest/pipeline/1
{
  "description" : "combines two fields",
  "processors" : [
    {
      "append" : {
        "field": "Geo2",
        "value": "doc['DST_IP_LAT'].value , doc['DST_IP_LONG'].value"
      }
    }
  ]
}

where the two fields, DST_IP_LAT and DST_IP_LONG include latitude and longitude respectively. I tried these as well

PUT _ingest/pipeline/1
{
  "description" : "combines two fields",
  "processors" : [
    {
      "append" : {
        "field": "Geo2",
        "value": "DST_IP_LAT , DST_IP_LONG"
      }
    }
  ]
}

and I applied the pipeline in the input. However, what I get in the output is exactly"

"doc['DST_IP_LAT'].value , doc['DST_IP_LONG'].value"

or

"DST_IP_LAT , DST_IP_LONG"

and not their values :frowning_face:

I am sorry for asking trivial question, but I could not find it anywhere in Elasticsearch documentation explained clearly (for a beginner like me).

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.