# Docker logs not parsed

**URL:** <https://discuss.elastic.co/t/docker-logs-not-parsed/229075>\
**Category:** Beats\
**Tags:** docker, filebeat\
**Created:** [April 21, 2020, 2:59pm UTC](https://discuss.elastic.co/t/docker-logs-not-parsed/229075 "2020-04-21T14:59:44Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![cjack03](https://avatars.discourse-cdn.com/v4/letter/c/c0e974/32.png) [@cjack03](https://discuss.elastic.co/u/cjack03)\
**Post date:** [April 21, 2020, 2:59pm UTC](https://discuss.elastic.co/t/docker-logs-not-parsed/229075/1 "2020-04-21T14:59:45Z")

</div>

I'm pulling my hair out, please help!! I have a very basic setup:

Filebeat (7.6.2) sending docker logs to elasticsearch (7.6.2) everything is working exactly as expected EXCEPT the actual docker logs are not being decoded to provide the different pieces of information such as IP etc.

A snippet of my config from filebeat.yml:

```auto
filebeat.inputs:
- type: container
    paths: 
    - "/var/lib/docker/containers/*/*.log

```

in kibana i have the complete log message in the `message` field. An extract from the JSON is here:

```auto
"stream": "stdout",
    "message": "49.38.129.6:56661 - - [21/Apr/2020:14:33:23 +0000] \"POST /xxxxx HTTP/1.1\" 200 4271 \"-\" \"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)\" 1.2.3.4",
    "log": {
      "file": {
        "path": "/var/lib/docker/containers/c4948ade03f1f76735cd3a8768ff25754935332f0f021b65118f72e583f1c426/c4948ade03f1f76735cd3a8768ff25754935332f0f021b65118f72e583f1c426-json.log"
      },
      "offset": 6293394045
    },

```

now i have tried various combination of the json\_\* options including `json.keys_under_root: true` but when this is enabled i receive tons of error meesages in the logs such as:

`Error decoding JSON: json: cannot unmarshal number into Go value of type map[string]interface {}`

All i'm looking for is to decode the log mesage and enrich the data with geoip information to build some nice dashboards.

I'm obviously missing something incredible simple but just cant see it.

please help!!

 ![Screenshot 2020-04-21 at 16.58.36](https://us1.discourse-cdn.com/elastic/original/3X/e/5/e59801396d8222560cf7278a74b56b18c889e846.png)

---

<div class="post-metadata">

**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\
**Post date:** [April 21, 2020, 5:00pm UTC](https://discuss.elastic.co/t/docker-logs-not-parsed/229075/2 "2020-04-21T17:00:24Z")

</div>

I assume what you're looking for is parsing out the parts like `49.38.129.6`, etc. into their own fields. For this you'll need to add your own parsing rules, either using [Beats processors](https://www.elastic.co/guide/en/beats/filebeat/current/defining-processors.html) or an [Elasticsearch Ingest Node Pipeline](https://www.elastic.co/guide/en/elasticsearch/reference/master/ingest.html).

Alternatively, if your logs are using a well-known format, e.g. Apache logs or Nginx logs, there's probably a [Filebeat module](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-modules.html) that can do all the parsing for you. To use modules in a container environment see [this guide](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-autodiscover.html).

---

<div class="post-metadata">

**Author:** ![cjack03](https://avatars.discourse-cdn.com/v4/letter/c/c0e974/32.png) [@cjack03](https://discuss.elastic.co/u/cjack03)\
**Post date:** [April 21, 2020, 5:44pm UTC](https://discuss.elastic.co/t/docker-logs-not-parsed/229075/3 "2020-04-21T17:44:33Z")

</div>

Thanks for your reply Shaunak!

That’s exactly what I want to do, I had naively assumed that the beat would take care of this for me.

I’ll have a look at those links you’ve provided and see how I get on.

Again, many thanks.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 19, 2020, 5:44pm UTC](https://discuss.elastic.co/t/docker-logs-not-parsed/229075/4 "2020-05-19T17:44:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.