ES 7.0.1: GeoIp processor and geo_point mapping troubles

Elastic Stack Elasticsearch
1

hi all, I am using fluent-bit to feed data into ES. I have the GeoIP processor "activated". My index mapping is here. The Geo IP processor is doing it's job correctly, but on the ES side, I am getting this error:

{"type": "server", "timestamp": "2019-05-14T04:34:01,014+0000", "level": "DEBUG", "component": "o.e.a.b.TransportShardBulkAction", "cluster.name": "docker-cluster", "node.name": "ip-192-168-12-243.ap-southeast-2.compute.internal", "cluster.uuid": "H4SUXuGIQhWcMqL3hYBoKA", "node.id": "mhSlcK1gRNaKEnxM74agMg",  "message": "[nginx_geoip][0] failed to execute bulk item (index) index {[nginx_geoip][flb_type][5XietGoBOBIN2Y3DZtOt], source[{\"remote_addr\":\"127.0.0.1\",\"request\":\"GET / HTTP/1.1\",\"real_ip\":\"8.8.8.8\",\"geoip\":{\"continent_name\":\"North America\",\"country_iso_code\":\"US\",\"location\":{\"lon\":-97.822,\"lat\":37.751}},\"response_status\":\"200\",\"body_bytes_sent\":\"12\",\"time_local\":\"14/May/2019:04:33:45 +0000\",\"request_method\":\"GET\",\"http_user_agent\":\"curl/7.29.0\",\"remote_user\":\"\",\"@timestamp\":\"2019-05-14T04:33:45.455Z\",\"request_time\":\"0.000\",\"http_referrer\":\"\"}]}" , 
"stacktrace": ["java.lang.IllegalArgumentException: mapper [geoip.location] of different type, current_type [geo_point], merged_type [ObjectMapper]",

To reproduce:

Create ES docker container:

sudo docker run --network=host -d -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.0.1

Setup processor:

curl  -X PUT "localhost:9200/_ingest/pipeline/geoip" -H 'Content-Type: application/json' -d' 
{
  "description" : "Add geoip info",
  "processors" : [
    {
      "geoip" : {
        "field" : "real_ip"
      }
    }
  ]
}

Create the index with settings above:

curl -H 'Content-Type: application/json'  -XPUT "http://elasticsearch:9200/nginx_geoip" -d '
{mapping}
'

Try to ingest a document:

curl -H 'Content-Type: application/json' -XPOST 'http://localhost:9200/nginx_geoip/metrics?pipeline=geoip' -d '
{
  "real_ip": "192.30.253.113",
   "@timestamp":"2019-05-14T04:33:45.455Z"
}
'
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"mapper [geoip.location] of different type, current_type [geo_point], merged_type [ObjectMapper]"}],"type":"illegal_argument_exception","reason":"mapper [geoip.location] of different type, current_type [geo_point], merged_type [ObjectMapper]"},"status":400}

Any thoughts on where I am going wrong?

Thanks for any help!

2

I think you need to change the target field to something else than geoip which is the default IIRC.

3

Thanks @dadoonet. I tried that, but no luck.

Example document and error:

{"type": "server", "timestamp": "2019-05-14T23:49:17,177+0000", "level": "DEBUG", "component": "o.e.a.b.TransportShardBulkAction", "cluster.name": "docker-cluster", "node.name": "ip-192-168-12-243.ap-southeast-2.compute.internal", "cluster.uuid": "3b3CwePfR2KOqyHadEG3uA", "node.id": "9utroxCiTcqxFse5KZyHTQ",  "message": "[nginx_geoip][0] failed to execute bulk item (index) index {[nginx_geoip][flb_type][FyDAuGoB1J1VAp3oFCtY], source[{\"remote_addr\":\"127.0.0.1\",\"request\":\"GET / HTTP/1.1\",\"real_ip\":\"8.8.8.8\",\"iplocation\":{\"continent_name\":\"North America\",\"country_iso_code\":\"US\",\"location\":{\"lon\":-97.822,\"lat\":37.751}},\"response_status\":\"200\",\"body_bytes_sent\":\"12\",\"time_local\":\"14/May/2019:23:46:58 +0000\",\"request_method\":\"GET\",\"http_user_agent\":\"curl/7.29.0\",\"remote_user\":\"\",\"@timestamp\":\"2019-05-14T23:46:58.282Z\",\"request_time\":\"0.000\",\"http_referrer\":\"\"}]}" , 
"stacktrace": ["java.lang.IllegalArgumentException: mapper [iplocation.location] of different type, current_type [geo_point], merged_type [ObjectMapper]",
4

I think (because the sample document is hard to read) that iplocation.location already exists so you can't choose it as a target field which I suppose you did.

Could you define foo as the target?

5

Thanks again. I tried your suggestion with the same result. I have scripts here that I am using to test this.

To reproduce:

  • ./setup-7.0.sh
  • ./create_es_index.sh
  • ./test_event.sh

I get the following:

{
    "error": {
        "reason": "mapper [foo.location] of different type, current_type [geo_point], merged_type [ObjectMapper]",
        "root_cause": [
            {
                "reason": "mapper [foo.location] of different type, current_type [geo_point], merged_type [ObjectMapper]",
                "type": "illegal_argument_exception"
            }
        ],
        "type": "illegal_argument_exception"
    },
    "status": 400
}

Hopefully that's a bit clearer. Any idea as to what I am doing wrong?

6

I'd remove this part from the mapping:

And let Elasticsearch at the first run, create the mapping for you.
Then get the generated mapping to see how it has been modified.
From this, edit the mapping, delete the index and put the mapping again.

And test again.

7

Thanks. So i removed the foo mapping, create the index. Then I tried to post the document, I get:

amit@localhost ~/work/github.com/amitsaha/systemd-fluent-bit-es/es (master)$ ./test_event.sh 
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Rejecting mapping update to [nginx_geoip] as the final mapping would have more than 1 type: [_doc, metrics]"}],"type":"illegal_argument_exception","reason":"Rejecting mapping update to [nginx_geoip] as the final mapping would have more than 1 type: [_doc, metrics]"},"status":400}

Further, i deleted the index entirely and created the document so that ES would create the index and mapping entirely. This is what I get (relevant part):

 "foo": {
        "properties": {
          "continent_name": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "country_iso_code": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "location": {
            "properties": {
              "lat": {
                "type": "float"
              },
              "lon": {
                "type": "float"
              }
            }
          }
        }
      },

Which is of course expected, but geo_point is what i am after.

8

Can you now modify this mapping and change location to become a geopoint and create the index again?

9

You mean:

..
"location": {
           "type": "geo_point",
            "properties": {
              "lat": {
                "type": "float"
              },
              "lon": {
                "type": "float"
              }
            }
          }

Assuming that's what you meant, i get this:

{
    "error": {
        "caused_by": {
            "reason": "Mapping definition for [location] has unsupported parameters:  [properties : {lon={type=float}, lat={type=float}}]",
            "type": "mapper_parsing_exception"
        },
        "reason": "Failed to parse mapping [_doc]: Mapping definition for [location] has unsupported parameters:  [properties : {lon={type=float}, lat={type=float}}]",
        "root_cause": [
            {
                "reason": "Mapping definition for [location] has unsupported parameters:  [properties : {lon={type=float}, lat={type=float}}]",
                "type": "mapper_parsing_exception"
            }
        ],
        "type": "mapper_parsing_exception"
    },
    "status": 400
}
10

I meant:

     "location": {
       "type": "geo_point"
        }
      }
11

Right ok, i tried that and get back this error when posting a document:

{
    "error": {
        "reason": "mapper [foo.location] of different type, current_type [geo_point], merged_type [ObjectMapper]",
        "root_cause": [
            {
                "reason": "mapper [foo.location] of different type, current_type [geo_point], merged_type [ObjectMapper]",
                "type": "illegal_argument_exception"
            }
        ],
        "type": "illegal_argument_exception"
    },
    "status": 400
}
12

Here is what I tried on my side:

PUT /_ingest/pipeline/geoip
{
  "description" : "Add geoip info",
  "processors" : [
    {
      "geoip" : {
        "field" : "real_ip"
      }
    }
  ]
}

DELETE nginx_geoip
PUT nginx_geoip
{
  "mappings": {
    "properties": {
      "@timestamp": {
        "type": "date"
      },
      "geoip": {
        "properties": {
          "continent_name": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "country_iso_code": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "location": {
            "type": "geo_point"
          }
        }
      },
      "real_ip": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      }
    }
  }
}

PUT /nginx_geoip/_doc/1?pipeline=geoip
{
  "real_ip": "192.30.253.113",
  "@timestamp":"2019-05-14T04:33:45.455Z"
}
GET /nginx_geoip/_doc/1

This works well. Could you use that as a base?

13

Thanks - same issue with 7.0.1. Were you using the same version?

However, the interesting thing is, i got it to work with 6.4. My scripts are here. Should I file a bug?

14

I tested it on 7.0.1.

Did you try the exact same script or did you modify it?
If you did not change anything, I suspect that you have a template somewhere.

15

the scripts i used for testing are in my above link (I used the 7.0 versions). Would you be able to use my scripts and verify that what you are suggesting is working for you?

16

Your script are failing. That's why I suggest you test mine and start from there.

17

HI @dadoonet - i should have been clearer. My scripts do exactly what you suggested. Please take a look at them.

18

If your scripts are exactly the same I pasted, then I don't need to look at them as I tested my scripts and they were working well.

Did you try the script I shared? I mean just by copying pasting it in Kibana console and running it?

19

Anyway look at:

POST nginx_geoip/metrics

vs

PUT nginx_geoip/_doc/1

This makes at least 2 differences between our scripts.

20

I'm having a similar issue: 7.0.1: template with geo_point + geoip processor in pipeline + bulk = crash