Add Field In Alter Plugin Not Showing values inside field instead showing field name

Elastic Stack Logstash
1

Hi have created the following code using Alter plugin because the mutate plugin won't work in my logstash server.

filter {
 date{match => ["REQUEST_TIMESTAMP","yyyy-MM-dd HH:mm:ss"]}
  alter {
      add_field => { "[location1][lat]" => "%%{[A_LATITUDE]}" }
      add_field => { "[location1][lon]" => "%%{[A_LONGITUDE]}" }
        }
        }

Using the above code, instead of getting the the Lat and Lon values in the field location1, I am getting the following in the logstash output,

"location1" => {
    "lat" => "%%{[A_LATITUDE]}",
    "lon" => "%%{[A_LONGITUDE]}"
}

What am I doing wrong in the filter plugin?

2

A sprintf reference should use a single %. So that should be "%{[A_LATITUDE]}"

3

Yes @Badger, I tried that too, still the same.

4

Also getting this error,

"error"=>{"type"=>"mapper_parsing_exception", 
"reason"=>"failed to parse field [location1] of type 
[geo_point]", "caused_by"=>{"type"=>"parse_exception", 
"reason"=>"[lat] and [lon] must be valid double values", 
"caused_by"=>{"type"=>"number_format_exception", 
"reason"=>"For input string: \"%{[A LONGITUDE]}\""}}}}}}
5

That would suggest the source field does not exist.

6

@Badger what does that mean?

A view of Logstash:
Capture

7

You are trying to reference the value of a field called A_LONGITUDE, but that field does not exist.

8

@Badger, But it is present :worried:

issue

9

Is it possible you are referencing it before it is created?

10

@Badger I am creating a PUT mapping in Kibana dev tools as follows,

PUT test-connectivity-ebond
{
"mappings": {
"properties": {
  "location1":{
    "type": "geo_point"
  },
  "location2":{
    "type": "geo_point"
  }
}
}
}

Then coming to Logstash,

filter {

   alter {
      add_field => { "[location1][lat]" => "%{[A_LATITUDE]}" }
      add_field => { "[location1][lon]" => "%{[A_LONGITUDE]}" }
  }

  alter{
     add_field => { "[location2][lat]" => "%{[B_LATITUDE]}" }
     add_field => { "[location2][lon]" => "%{[B_LONGITUDE]}" }
 }

   date {
   match => [ "REQUEST_TIMESTAMP", "yyyy-MM-dd HH:mm:ss" ]
  }

}
11

How does the [A_LONGITUDE] field get created?

12

@Badger, it is a column from the database.

13

@Badger, so I have Lat and Lon values in two separate columns called A_LATITUDE and A_LONGITUDE similarly two other columns called B_LATITUDE and B_LONGITUDE.

14

I cannot explain that.

15

@Badger, got it solved using the Mutate Filter Plugin! Thank you :slight_smile:

16

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.