Geoip pipeline creation issue

Elastic Stack Elasticsearch
1

I have created ingest pipeline as follow:

PUT _ingest/pipeline/geoip
{
  "description": "Add geoip info",
  "processors": [
    {
      
      "geoip": {
        "field": "dest.ip",
        "ignore_failure": true
      }
    }
  ]
}

Next I have indexed a single document:

POST packetbeat-dm-6.6.1-2019.02/doc/?pipeline=geoip
{
  "dest.ip":"80.34.121.50"
}

However, when I have requested this document,

GET packetbeat-dm-6.6.1-2019.02/_search
{
  "query": {
    "match": {
      "dest.ip":"80.34.121.50"
    }
  }
}

I couldnt see any of the processing done. What am I doing wrong here?

"hits" : [
      {
        "_index" : "packetbeat-dm-6.6.1-2019.02",
        "_type" : "doc",
        "_id" : "HgQnNGkB6DzpB6JS5TVw",
        "_score" : 8.552432,
        "_source" : {
          "dest.ip" : "80.34.121.50"
        }
      }
    ]
2

I'm not able to reproduce the problem with:

POST _ingest/pipeline/_simulate
{
  "pipeline" :{
  "description": "date pipeline ",
  "processors": [
    {
      
      "geoip": {
        "field": "dest.ip",
        "ignore_failure": true
      }
    }
  ]},
  "docs": [
    {
      "_index": "index",
      "_type": "_doc",
      "_id": "id",
       "_source": {
          "dest": {
            "ip": "80.34.121.50"
          }
       }
    }
  ]
}

It gives:

{
  "docs" : [
    {
      "doc" : {
        "_index" : "index",
        "_type" : "_doc",
        "_id" : "id",
        "_source" : {
          "geoip" : {
            "continent_name" : "Europe",
            "region_iso_code" : "ES-M",
            "city_name" : "Pozuelo de Alarcón",
            "region_name" : "Madrid",
            "location" : {
              "lon" : -3.8134,
              "lat" : 40.4329
            },
            "country_iso_code" : "ES"
          },
          "dest" : {
            "ip" : "80.34.121.50"
          }
        },
        "_ingest" : {
          "timestamp" : "2019-02-28T14:02:29.878046Z"
        }
      }
    }
  ]
}
3

Hi, on _simulate API it works for me as well - no issue. However not when I follow the steps in my post. Any idea what I should be looking at. What I have put in the post is almost 1:1 copy from the documentation. I would appreciate any suggestion

4

That's because your document is:

{
  "dest.ip":"80.34.121.50"
}

Where mine is:

{
  "dest": {
    "ip":"80.34.121.50" 
  }
}

Not sure why this is not working though and if it is supposed to work with the dot notation.

5

Hey - bingo - dot notation is not working - once changed it is ok - thank you

6

When you remove the "ignore_failure": true, then you are getting a proper message:

{
  "docs" : [
    {
      "error" : {
        "root_cause" : [
          {
            "type" : "exception",
            "reason" : "java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [dest] not present as part of path [dest.ip]",
            "header" : {
              "processor_type" : "geoip"
            }
          }
        ],
        "type" : "exception",
        "reason" : "java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [dest] not present as part of path [dest.ip]",
        "caused_by" : {
          "type" : "illegal_argument_exception",
          "reason" : "java.lang.IllegalArgumentException: field [dest] not present as part of path [dest.ip]",
          "caused_by" : {
            "type" : "illegal_argument_exception",
            "reason" : "field [dest] not present as part of path [dest.ip]"
          }
        },
        "header" : {
          "processor_type" : "geoip"
        }
      }
    }
  ]
}
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.