# Logstash plugin unrecoverable error when using a mapping

**URL:** <https://discuss.elastic.co/t/logstash-plugin-unrecoverable-error-when-using-a-mapping/33815>\
**Category:** Logstash\
**Created:** [November 4, 2015, 10:18pm UTC](https://discuss.elastic.co/t/logstash-plugin-unrecoverable-error-when-using-a-mapping/33815 "2015-11-04T22:18:52Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Iain\_Woolf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iain_woolf/32/981_2.png) [@Iain\_Woolf](https://discuss.elastic.co/u/Iain_Woolf)\
**Post date:** [November 4, 2015, 10:18pm UTC](https://discuss.elastic.co/t/logstash-plugin-unrecoverable-error-when-using-a-mapping/33815/1 "2015-11-04T22:18:52Z")

</div>

I'm trying to pull in UK Crime data to Elasticsearch so that I can experiment with the map visualizations in Kibana 4. When I create a mapping to show that the location field is a geo\_point, I then get errors when trying to import data.

Here are the steps to reproduce:

Create an empty index to receive crime data:

```
curl -XPUT 'http://localhost:9200/crime' -d '
index :
  number_of_shards : 1
  number_of_replicas : 1
'

```

Create a mapping to show that the json data already has a location field which gives latitude and longitude:

```
curl -XPUT 'http://localhost:9200/crime/_mapping/crimestats' -d '
{
  "crimestats" : {
     "properties" : {
        "location" : {"type" : "geo_point"}
     }
  }
}
'

```

Using data from [https://data.gov.uk/dataset/england-national-crime-mapping](https://data.gov.uk/dataset/england-national-crime-mapping), paste a single line of JSON using the stdin input plugin. Get an error from Elasticsearch.

```
/opt/logstash/bin/logstash -e '
input {
  stdin {

    type => "crimestats"
    codec => "json_lines"

  }
}
output {
  elasticsearch {
    index => "crime"
    host => "127.0.0.1"
    protocol => "http"
    manage_template => "false"
  }
  stdout {
  codec => "rubydebug"
  }
}
'
[2015-11-04 16:39:30.927] WARN -- Concurrent: [DEPRECATED] Java 7 is deprecated, please use Java 8.
Java 7 support is only best effort, it may not work. It will be removed in next release (1.0).
Logstash startup completed

```

Paste the following:

```
{"category":"anti-social-behaviour","location_type":"Force","location":{"latitude":"52.634845","street":

```

Here's the error returned from Elasticsearch:

```
{"id":332249,"name":"On or near Queen Street"},"longitude":"-1.124838"},"context":"","outcome_status":null,"persistent_id":"","id":3350469,"location_subtype":"","month":"2011-08"}
failed action with response of 400, dropping action: ["index", {:_id=>nil, :_index=>"crime", :_type=>"crimestats", :_routing=>nil}, #<LogStash::Event:0x7ac5cba7 @metadata={"retry_count"=>0}, @accessors=#<LogStash::Util::Accessors:0x2218c2e5 @store={"category"=>"anti-social-behaviour", "location_type"=>"Force", "location"=>{"latitude"=>"52.634845", "street"=>{"id"=>332249, "name"=>"On or near Queen Street"}, "longitude"=>"-1.124838"}, "context"=>"", "outcome_status"=>nil, "persistent_id"=>"", "id"=>3350469, "location_subtype"=>"", "month"=>"2011-08", "@version"=>"1", "@timestamp"=>"2015-11-04T22:21:56.913Z", "type"=>"crimestats", "host"=>"uottvr17ctrl01"}, @lut={"type"=>[{"category"=>"anti-social-behaviour", "location_type"=>"Force", "location"=>{"latitude"=>"52.634845", "street"=>{"id"=>332249, "name"=>"On or near Queen Street"}, "longitude"=>"-1.124838"}, "context"=>"", "outcome_status"=>nil, "persistent_id"=>"", "id"=>3350469, "location_subtype"=>"", "month"=>"2011-08", "@version"=>"1", "@timestamp"=>"2015-11-04T22:21:56.913Z", "type"=>"crimestats", "host"=>"uottvr17ctrl01"}, "type"], "host"=>[{"category"=>"anti-social-behaviour", "location_type"=>"Force", "location"=>{"latitude"=>"52.634845", "street"=>{"id"=>332249, "name"=>"On or near Queen Street"}, "longitude"=>"-1.124838"}, "context"=>"", "outcome_status"=>nil, "persistent_id"=>"", "id"=>3350469, "location_subtype"=>"", "month"=>"2011-08", "@version"=>"1", "@timestamp"=>"2015-11-04T22:21:56.913Z", "type"=>"crimestats", "host"=>"uottvr17ctrl01"}, "host"]}>, @data={"category"=>"anti-social-behaviour", "location_type"=>"Force", "location"=>{"latitude"=>"52.634845", "street"=>{"id"=>332249, "name"=>"On or near Queen Street"}, "longitude"=>"-1.124838"}, "context"=>"", "outcome_status"=>nil, "persistent_id"=>"", "id"=>3350469, "location_subtype"=>"", "month"=>"2011-08", "@version"=>"1", "@timestamp"=>"2015-11-04T22:21:56.913Z", "type"=>"crimestats", "host"=>"uottvr17ctrl01"}, @metadata_accessors=#<LogStash::Util::Accessors:0x7ccb7779 @store={"retry_count"=>0}, @lut={}>, @cancelled=false>] {:level=>:warn}

```

And rubydebug output

```
{
            "category" => "anti-social-behaviour",
       "location_type" => "Force",
            "location" => {
         "latitude" => "52.634845",
           "street" => {
              "id" => 332249,
            "name" => "On or near Queen Street"
        },
        "longitude" => "-1.124838"
    },
             "context" => "",
      "outcome_status" => nil,
       "persistent_id" => "",
                  "id" => 3350469,
    "location_subtype" => "",
               "month" => "2011-08",
            "@version" => "1",
          "@timestamp" => "2015-11-04T22:21:56.913Z",
                "type" => "crimestats",
                "host" => "uottvr17ctrl01"
}

```

Note: no error if the index was created without a mapping.

Versions I'm using:

```
rpm -q elasticsearch kibana logstash
 elasticsearch-1.7.1-1.noarch
 kibana-4.1.1-1.x86_64
 logstash-1.5.4-1.noarch
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 5, 2015, 7:50am UTC](https://discuss.elastic.co/t/logstash-plugin-unrecoverable-error-when-using-a-mapping/33815/2 "2015-11-05T07:50:05Z")

</div>

You're telling ES that the `location` field is a geo\_point when the data you're passing looks like this:

```
"location": {
  "latitude": "52.634845",
  "street": {
    "id": 332249,
    "name": "On or near Queen Street"
  },
  "longitude": "-1.124838"
}

```

I don't recall exactly what a field must look like to fit in a geo\_point, but you definitely have to remove the `street` subfield from there.

---

<div class="post-metadata">

**Author:** ![Iain\_Woolf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iain_woolf/32/981_2.png) [@Iain\_Woolf](https://discuss.elastic.co/u/Iain_Woolf)\
**Post date:** [November 5, 2015, 1:40pm UTC](https://discuss.elastic.co/t/logstash-plugin-unrecoverable-error-when-using-a-mapping/33815/3 "2015-11-05T13:40:40Z")

</div>

I thought it would be OK with an extra field in there, but I've taken it out on your recommendation. The result is the same.

(BTW, the input type is documented here [https://www.elastic.co/guide/en/elasticsearch/reference/1.7/mapping-geo-point-type.html](https://www.elastic.co/guide/en/elasticsearch/reference/1.7/mapping-geo-point-type.html))

Input:

```
{"category":"anti-social-behaviour","location_type":"Force","location":{"latitude":"52.634845","longitude":"-1.124838"},"context":"","outcome_status":null,"persistent_id":"","id":3350469,"location_subtype":"","month":"2011-08"}

```

Elasticsearch plugin output:

```
failed action with response of 400, dropping action: ["index", {:_id=>nil, :_index=>"crime", :_type=>"crimestats", :_routing=>nil}, #<LogStash::Event:0x79f7411 @metadata={"retry_count"=>0}, @accessors=#<LogStash::Util::Accessors:0x649ece9b @store={"category"=>"anti-social-behaviour", "location_type"=>"Force", "location"=>{"latitude"=>"52.634845", "longitude"=>"-1.124838"}, "context"=>"", "outcome_status"=>nil, "persistent_id"=>"", "id"=>3350469, "location_subtype"=>"", "month"=>"2011-08", "@version"=>"1", "@timestamp"=>"2015-11-05T13:36:59.098Z", "type"=>"crimestats", "host"=>"uottvr17ctrl01"}, @lut={"type"=>[{"category"=>"anti-social-behaviour", "location_type"=>"Force", "location"=>{"latitude"=>"52.634845", "longitude"=>"-1.124838"}, "context"=>"", "outcome_status"=>nil, "persistent_id"=>"", "id"=>3350469, "location_subtype"=>"", "month"=>"2011-08", "@version"=>"1", "@timestamp"=>"2015-11-05T13:36:59.098Z", "type"=>"crimestats", "host"=>"uottvr17ctrl01"}, "type"], "host"=>[{"category"=>"anti-social-behaviour", "location_type"=>"Force", "location"=>{"latitude"=>"52.634845", "longitude"=>"-1.124838"}, "context"=>"", "outcome_status"=>nil, "persistent_id"=>"", "id"=>3350469, "location_subtype"=>"", "month"=>"2011-08", "@version"=>"1", "@timestamp"=>"2015-11-05T13:36:59.098Z", "type"=>"crimestats", "host"=>"uottvr17ctrl01"}, "host"]}>, @data={"category"=>"anti-social-behaviour", "location_type"=>"Force", "location"=>{"latitude"=>"52.634845", "longitude"=>"-1.124838"}, "context"=>"", "outcome_status"=>nil, "persistent_id"=>"", "id"=>3350469, "location_subtype"=>"", "month"=>"2011-08", "@version"=>"1", "@timestamp"=>"2015-11-05T13:36:59.098Z", "type"=>"crimestats", "host"=>"uottvr17ctrl01"}, @metadata_accessors=#<LogStash::Util::Accessors:0x2c5b1cf5 @store={"retry_count"=>0}, @lut={}>, @cancelled=false>] {:level=>:warn}

```

Stdout rubydebug output:

```
{
            "category" => "anti-social-behaviour",
       "location_type" => "Force",
            "location" => {
         "latitude" => "52.634845",
        "longitude" => "-1.124838"
    },
             "context" => "",
      "outcome_status" => nil,
       "persistent_id" => "",
                  "id" => 3350469,
    "location_subtype" => "",
               "month" => "2011-08",
            "@version" => "1",
          "@timestamp" => "2015-11-05T13:36:59.098Z",
                "type" => "crimestats",
                "host" => "uottvr17ctrl01"
}
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 5, 2015, 2:24pm UTC](https://discuss.elastic.co/t/logstash-plugin-unrecoverable-error-when-using-a-mapping/33815/4 "2015-11-05T14:24:49Z")

</div>

See the Input Structure section in the documentation page you linked to. The location value you're passing doesn't match any of the supported formats, but if you convert the string values to floats (using the mutate filter's convert option) things should be fine.

---

<div class="post-metadata">

**Author:** ![Iain\_Woolf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iain_woolf/32/981_2.png) [@Iain\_Woolf](https://discuss.elastic.co/u/Iain_Woolf)\
**Post date:** [November 5, 2015, 2:39pm UTC](https://discuss.elastic.co/t/logstash-plugin-unrecoverable-error-when-using-a-mapping/33815/5 "2015-11-05T14:39:38Z")

</div>

Doh! Thanks for pointing that out. My brain was equating "lat" with "latitude" and I'd overlooked it and the string/float.

This works:

Input:

```
{"category":"anti-social-behaviour","location_type":"Force","location":{"lat":52.634845,"lon":-1.124838},"context":"","outcome_status":null,"tent_id":"","id":3350469,"location_subtype":"","month":"2011-08"}

```

Stdout/rubydebug:

```
{
            "category" => "anti-social-behaviour",
       "location_type" => "Force",
            "location" => {
        "lat" => 52.634845,
        "lon" => -1.124838
    },
             "context" => "",
      "outcome_status" => nil,
       "persistent_id" => "",
                  "id" => 3350469,
    "location_subtype" => "",
               "month" => "2011-08",
            "@version" => "1",
          "@timestamp" => "2015-11-05T14:35:31.505Z",
                "type" => "crimestats",
                "host" => "uottvr17ctrl01"
}

```

More importantly I can also create a tilemap visualization from this data!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:23am UTC](https://discuss.elastic.co/t/logstash-plugin-unrecoverable-error-when-using-a-mapping/33815/6 "2017-07-06T05:23:51Z")

</div>