Translate Config works from STDIN but not Beats input

The following code works if I paste the data into STDIN, but when the data comes from BEATS it fails to create the geoip fields. I just get a translation field with the result in it.

#This is from the logstash config

	#Private IP Geo Lookup
	if "_geoip_lookup_failure" in [tags] {
		translate {
			regex => true
			dictionary_path => "./config/priv_ip_trans.yml"
			field => "CallingStation_IP"
			remove_tag =>"_geoip_lookup_failure"

#This is a sample line from the priv_ip_trans.yml
'10.90.3.*': '{"geoip":{"Latitude": 37.7576948, "Longitude": -122.4726193, "Location": [-122.4726193, 37.7576948],"country_name": "United States", "country_code2": "US", "country_code3": "US", "continent_code":" NA", "city_name": "San Francisco", "region_code": "CA", "region_name": "California", "postal_code": "94114", "timezone": "America/Los_Angeles"}}'

#This is what gets output when using beats:

              "translation" => "{\"geoip\":{\"Latitude\": 37.7576948, \"Longitude\": -122.4726193, \"Location\": [-122.4726193, 37.7576948],\"country_name\": \"United States\", \"country_code2\": \"US\", \"country_code3\": \"US\", \"continent_code\":\" NA\", \"city_name\": \"San Francisco\", \"region_code\": \"CA\", \"region_name\": \"California\", \"postal_code\": \"94114\", \"timezone\": \"America/Los_Angeles\"}}",

This is what I get when using STDIN:

      "geoip" => {
         "city_name" => "San Francisco",
          "timezone" => "America/Los_Angeles",
      "country_name" => "United States",
     "country_code2" => "US",
     "country_code3" => "US",
    "continent_code" => " CA",
          "Latitude" => 36.0768949,
       "region_name" => "California",
       "postal_code" => "94114",
         "Longitude" => -122.4726193,
          "Location" => [
        [0] 36.0768949,
        [1] -122.4726193
       "region_code" => "CA"

To me it sounds very unlikely that the translate filter would behave differently depending on the input. You can always use a json filter to deserialize the JSON string.

Your stdin example doesn't match your sample line from priv_ip_trans.yml. Had you shown the entire event from your stdout output it would've been easier to see if that was expected.

You were right, I was missing the JSON Filter. Don't know how I missed that. Thanks for the kick in the head to straighten my thoughts out.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.