Convert string LLA to Geo-Point

Elastic Stack Logstash
1

I have a string field that is in Latitude, Longitude, Altitude. In the pipeline, I am taking the location in x, y, z in ECEF coordinates and converting to LLA:

    - pipeline.id: entity-state-processing
      config.string: |
        input { pipeline { address => entitystatelogs } }
        filter {
          # Checks if the log has locational data before processing it into WGS 84 lat/long
          if [attributes][entityLocation] {
            mutate {
              add_field => {
              "[location]" => "null"
            }
          }

          # Ruby script to convert the cartesian x,y,z coordinates into lat/long
          ruby {
            init => "
              R = 6360000
              "
            code => "
              x = event.get('[attributes][entityLocation][x]').to_f
              y = event.get('[attributes][entityLocation][y]').to_f
              z = event.get('[attributes][entityLocation][z]').to_f

              range = Math.sqrt(x*x + y*y + z*z)

              lat = 180*Math.asin(z/range)/Math::PI
              long = 180*Math.atan2(y,x)/Math::PI
              alt  = range - 6369783.457722581

              event.set('[location]',  lat.to_s + ',' + long.to_s, + ',' + alt.to_s)
              "
              }
            }
          }
        output {
          # Sends parsed logs to elasticsearch
          elasticsearch {
            hosts => ["${OUTPUT_HOST}"]
            user => "${ELASTIC_USER}"
            password => "${ELASTIC_PASS}"
            index => "{{ .Release.Namespace }}-entity-state-%{+yyyy.MM.dd}"
            }
        }

So, in ELK, location is a string that looks like: (12.5, 50, 1000). Is there a way to convert this to a geo-point inside the pipeline? Do I have to use a template. If so, how is that done?

2

logstash has no concept of a geo_point. If you have created a field in elasticsearch that is mapped as a geo_point (using a template) then elasticsearch will parse several formats, including a string "12.5, 50". This is documented here. It includes an example of how to set the template.

3

I don't see how this explains how to do so using a template.

It also doesn't show that the geo-point converts strings of the format (12, 50, 100).

4

Geo-point fields in Elasticsearch are latitude and longitude only, there is no altitude, so you need to transform your field to remove the altitude value before sending the data to Elastisearch.

The documentation linked has some examples on how the value of a location field should be.

So you would need to edit your ruby script to remove the altitude value and use the latitude and longitude value on one of the supported formats, like for example location: "lat, lon".

Also, for a field to work as a geo_point, you need to map it first, before sending any data, this can be done as the example in the documentation linked or using an index template with this mapping.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.