# Filebeat json to Elasticsearch, error processing pipeline

**URL:** https://discuss.elastic.co/t/filebeat-json-to-elasticsearch-error-processing-pipeline/245767
**Category:** Elasticsearch
**Created:** [August 20, 2020, 12:39pm UTC](https://discuss.elastic.co/t/filebeat-json-to-elasticsearch-error-processing-pipeline/245767 "2020-08-20T12:39:31Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![felixbarbeira](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/felixbarbeira/32/63637_2.png) [@felixbarbeira](https://discuss.elastic.co/u/felixbarbeira)
#### Post date: [August 20, 2020, 12:39pm UTC](https://discuss.elastic.co/t/filebeat-json-to-elasticsearch-error-processing-pipeline/245767/1 "2020-08-20T12:39:31Z")

</div>

Hi,

I'm sending a json log file with filebeat to an Elasticsearch cluster. When I try to parse the ip field with geoip filter it shows the following error in the **ingestError** field:

`field [client] not present as part of path [client.ip]`

This is an example log line:

`{"client.ip":"8.8.8.8","email.from":"user@email.com"}`

This is the input defined on filebeat:

```auto
- type: log
  paths:
  - /var/log/myservice.log
  encoding: plain
  ignore_older: 24h
  pipeline: mypipeline
  index: myindex-write
  json:
    add_error_key: true
    keys_under_root: true

```

And finally this is the processor on the pipeline:

```auto
{
      "geoip": {
        "field": "client.ip",
        "properties": [
          "country_iso_code"
        ],
        "ignore_failure": true
      }
    }

```

It's strange because the same configuration it works with regular logs, but not with raw json logs. With a regular log If I grok the field "client.ip" in the same pipeline, then it worked fine. Maybe it's something related with the way that filebeat sends the json message?

Somebody knows a way to parse **client.ip** field on a ingest pipeline coming from a json event?

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [August 20, 2020, 2:00pm UTC](https://discuss.elastic.co/t/filebeat-json-to-elasticsearch-error-processing-pipeline/245767/2 "2020-08-20T14:00:25Z")

</div>

Perhaps take a look at this

[https://www.elastic.co/guide/en/elasticsearch/reference/current/dot-expand-processor.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/dot-expand-processor.html)

"client.ip" is not a "valid" name / json construction with respect to elasticsearch.

`{"client.ip":"8.8.8.8","email.from":"user@email.com"}`

valid json should look like this

`{"client" : {"ip":"8.8.8.8"} ,"email" : {"from":"user@email.com"}}`

the grok in regular logs is creating the correct json.

So you might need to use the dot expander I referenced above

EDIT : This can seem a bit confusing because after you create valid json you can reference a field like `client.ip` but that is not the correct way to create it from a json document

---

<div class="post-metadata">

### Author: ![felixbarbeira](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/felixbarbeira/32/63637_2.png) [@felixbarbeira](https://discuss.elastic.co/u/felixbarbeira)
#### Post date: [August 21, 2020, 6:37am UTC](https://discuss.elastic.co/t/filebeat-json-to-elasticsearch-error-processing-pipeline/245767/3 "2020-08-21T06:37:17Z")

</div>

That's exactly what I need. Thanks a lot for the explanation 😃

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 18, 2020, 6:37am UTC](https://discuss.elastic.co/t/filebeat-json-to-elasticsearch-error-processing-pipeline/245767/4 "2020-09-18T06:37:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
