# Apache Docker containers autodiscovery ends up with grok error for error messages

**URL:** <https://discuss.elastic.co/t/apache-docker-containers-autodiscovery-ends-up-with-grok-error-for-error-messages/265746>\
**Category:** Beats\
**Tags:** docker, filebeat\
**Created:** [March 1, 2021, 3:41am UTC](https://discuss.elastic.co/t/apache-docker-containers-autodiscovery-ends-up-with-grok-error-for-error-messages/265746 "2021-03-01T03:41:57Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![jerrac](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerrac/32/52980_2.png) [@jerrac](https://discuss.elastic.co/u/jerrac)\
**Post date:** [March 1, 2021, 3:41am UTC](https://discuss.elastic.co/t/apache-docker-containers-autodiscovery-ends-up-with-grok-error-for-error-messages/265746/1 "2021-03-01T03:41:57Z")

</div>

I set up a new instance of Elasticsearch, Kibana, and various beats today. Since my sites are all in containers, I eventually found my way to the autodiscovery settings.

I've been able to configure access logs so that Filebeat processes them just fine.

But since docker puts stderr into the same file as stdout, Filebeat tries to treat access logs as error logs if the error log config is last, or error logs as access logs if the access log config is last (or the only config.)

You end up with error.message fields like this:

```auto
Provided Grok expressions do not match field value: [[Mon Mar 01 03:28:30.532028 2021] [core:crit] [pid 29] (13)Permission denied: [client 10.5.1.1:47622] AH00529: /var/www/html/.htaccess pcfg_openfile: unable to check htaccess file, ensure it is readable and that '/var/www/html/' is executable, referer: https://davidreagan.net/biography]

```

(I manually messed up permissions on the site.)

Or the normal restart message:

```auto
Provided Grok expressions do not match field value: [[Mon Mar 01 03:30:14.749871 2021] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND']

```

In the docker log file it looks like:

```json
{"log":"[Mon Mar 01 03:30:14.749871 2021] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'\n","stream":"stderr","time":"2021-03-01T03:30:14.750006026Z"}

```

My autodiscover settings look something like:

```auto
autodiscover:
  providers:
    - type: docker
      containers.stream: stdout
      templates:
        - condition:
            or:
              - contains:
                  docker.container.image: "alpha"
              - contains:
                  docker.container.image: "beta"
          config:
            - module: apache
              access:
                enabled: true
                var.paths: []
                input:
                  type: container
                  paths:
                    - /var/lib/docker/containers/${data.docker.container.id}/*.log
# - type: docker
# containers.stream: stderr
# templates:
# - condition:
# or:
# - contains:
# docker.container.image: "alpha"
# - contains:
# docker.container.image: "beta"
# config:
# - module: apache
# error:
# enabled: true
# var.paths: []
# input:
# type: container
# paths:
# - /var/lib/docker/containers/${data.docker.container.id}/*.log

```

As you can see I tried to split things with the `containers.stream: stderr|stdout`, but that didn't seem to do anything, and I didn't see it mentioned as usable like that in the docs.

I have stderr section commented out for now, since you get more access logs than error logs...

I did spend a good while searching for answers and didn't find anything helpful with the search terms I used.

So, anyone want to help?

How can I configure Filebeat so that logs from stdout are treated as access logs, and logs from stderr are treated as error logs?

Thanks in advance!

FYI, I'm on 7.11.1 for all ELK Stack apps.

---

<div class="post-metadata">

**Author:** ![jerrac](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerrac/32/52980_2.png) [@jerrac](https://discuss.elastic.co/u/jerrac)\
**Post date:** [March 13, 2021, 7:32pm UTC](https://discuss.elastic.co/t/apache-docker-containers-autodiscovery-ends-up-with-grok-error-for-error-messages/265746/2 "2021-03-13T19:32:45Z")

</div>

Anyone? I'm still stuck on this.

---

<div class="post-metadata">

**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\
**Post date:** [March 15, 2021, 6:58pm UTC](https://discuss.elastic.co/t/apache-docker-containers-autodiscovery-ends-up-with-grok-error-for-error-messages/265746/3 "2021-03-15T18:58:34Z")

</div>

Hi, could you remove this line from under `autodiscover.providers[0]`:

```auto
containers.stream: stdout

```

And add this line under `autodiscover.providers[0].templates.config[0].access.input`:

```auto
stream: stdout

```

Thanks,

Shaunak

---

<div class="post-metadata">

**Author:** ![jerrac](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerrac/32/52980_2.png) [@jerrac](https://discuss.elastic.co/u/jerrac)\
**Post date:** [March 17, 2021, 3:55pm UTC](https://discuss.elastic.co/t/apache-docker-containers-autodiscovery-ends-up-with-grok-error-for-error-messages/265746/4 "2021-03-17T15:55:10Z")

</div>

Ok, that helped me limit it to just the stdout logs, and I now have a "stream" field.

But I can't seem to get the stderr logs to work. They aren't showing up at all. The only value for stream is "stdout"

Here are the three things I've tried, each time making sure to generate error logs before checking Kibana.

#### Adding error config to the same section as the access config.

```auto
        autodiscover:
          providers:
            - type: docker
              templates:
                - condition:
                    or:
                      - contains:
                          docker.container.image: "imagea"
                      - contains:
                          docker.container.image: "imageb"
                  config:
                    - module: apache
                      access:
                        enabled: true
                        var.paths: []
                        input:
                          stream: stdout
                          type: container
                          paths:
                            - /var/lib/docker/containers/${data.docker.container.id}/*.log
                      error:
                        enabled: true
                        var.paths: []
                        input:
                          stream: stderr
                          type: container
                          paths:
                            - /var/lib/docker/containers/${data.docker.container.id}/*.log

```

#### Adding a second "module" config just for error.

```auto
        autodiscover:
          providers:
            - type: docker
              templates:
                - condition:
                    or:
                      - contains:
                          docker.container.image: "imagea"
                      - contains:
                          docker.container.image: "imageb"
                  config:
                    - module: apache
                      access:
                        enabled: true
                        var.paths: []
                        input:
                          stream: stdout
                          type: container
                          paths:
                            - /var/lib/docker/containers/${data.docker.container.id}/*.log
                    - module: apache
                      error:
                        enabled: true
                        var.paths: []
                        input:
                          stream: stderr
                          type: container
                          paths:
                            - /var/lib/docker/containers/${data.docker.container.id}/*.log

```

#### Adding a completely seperate providers config just for error.

```auto
        autodiscover:
          providers:
            - type: docker
              templates:
                - condition:
                    or:
                      - contains:
                          docker.container.image: "imagea"
                      - contains:
                          docker.container.image: "imageb"
                  config:
                    - module: apache
                      access:
                        enabled: true
                        var.paths: []
                        input:
                          stream: stdout
                          type: container
                          paths:
                            - /var/lib/docker/containers/${data.docker.container.id}/*.log
            - type: docker
              templates:
                - condition:
                    or:
                      - contains:
                          docker.container.image: "imagea"
                      - contains:
                          docker.container.image: "imageb"
                  config:
                    - module: apache
                      error:
                        enabled: true
                        var.paths: []
                        input:
                          stream: stderr
                          type: container
                          paths:
                            - /var/lib/docker/containers/${data.docker.container.id}/*.log

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 14, 2021, 5:56pm UTC](https://discuss.elastic.co/t/apache-docker-containers-autodiscovery-ends-up-with-grok-error-for-error-messages/265746/5 "2021-04-14T17:56:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.