# Logs are not visible in Kibana

**URL:** https://discuss.elastic.co/t/logs-are-not-visible-in-kibana/353789
**Category:** Elasticsearch
**Tags:** docker
**Created:** [February 21, 2024, 12:23pm UTC](https://discuss.elastic.co/t/logs-are-not-visible-in-kibana/353789 "2024-02-21T12:23:57Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![moep](https://avatars.discourse-cdn.com/v4/letter/m/ad7895/32.png) [@moep](https://discuss.elastic.co/u/moep)
#### Post date: [February 21, 2024, 12:23pm UTC](https://discuss.elastic.co/t/logs-are-not-visible-in-kibana/353789/1 "2024-02-21T12:23:57Z")

</div>

Hello,

do test some logs Logstash configurations I want to use this [docker-compose.yml](https://privatebin.net/?0ad7bb5ed4431fe7#9D5ekqqGZ5TyoJLUnqqWwHhymHFmWtRAsRHr5Wxto5aU) and I'm running Debian 12 .

- Docker version 25.0.3, build 4debf41
- docker-compose version 1.29.2
- my user is in the docker group, to run docker

My usecase:

In my usecase I want to place some logs `mainlog` into the logstash directory. See tree below:

```auto
~/local-ELK/logstash$ tree -d
.
├── config
├── mainlog.log
└── pipeline

~/local-ELK/logstash/mainlog.log$ ls
mainlog

# filter.conf is my custom logstash filter
~/local-ELK/logstash/pipeline$ ls
filter.conf logstash.conf

```

Now I start my composition with `docker compose up`. I can login to Kibana on localhost, but there are no field names from my `filter.conf` and no data from my `mainlog`.  
I also tried after login into Kibana to restart the logstash container, but the result was the same.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/7/6725e6d8341b844e40bedfe1dad78a74d5d34d9d.png)

I checked with `docker exec -it local-elk-logstash /bin/bash` if the `filter.conf` and the `mainlog` exists and if its possible to read it and it was.

Do you have any suggestions?

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [February 21, 2024, 12:28pm UTC](https://discuss.elastic.co/t/logs-are-not-visible-in-kibana/353789/2 "2024-02-21T12:28:12Z")

</div>

You need to share both your logstash configurations and your docker compose, without it is not possible to know what may be the issue.

---

<div class="post-metadata">

### Author: ![moep](https://avatars.discourse-cdn.com/v4/letter/m/ad7895/32.png) [@moep](https://discuss.elastic.co/u/moep)
#### Post date: [February 21, 2024, 2:05pm UTC](https://discuss.elastic.co/t/logs-are-not-visible-in-kibana/353789/3 "2024-02-21T14:05:54Z")

</div>

My `docker-compose.yml` is linked see below.  
My `filter.conf` looks like:

```auto
input { 
        file {
                id => "main"
                path => "/usr/share/logstash/mainlog.log"
                sincedb_path => "/dev/null"
                start_position => "beginning"
        }
        stdin { } 
}

filter {
        # here starts everything which contains in the mainlog
        if [log][file][path] =~ "mainlog" {
                mutate {
                        add_field => {
                                "[@metadata][sourcetype]" => "exim-mainlog"
                                "state" => "mainlog"
                                "exim_msg_state" => "not-processed"
                                "action_id" => "not-processed"
                        }
                }
       # alot of groks more

   # here starts everything which contains in the rejectlog
        if [log][file][path] =~ "rejectlog" {
          mutate {
            add_field => {
              "[@metadata][sourcetype]" => "exim-reject"
              "state" => "rejectlog"
            }
          }

          mutate {
            add_field => { "exim_msg_state" => "not-processed" }
            add_field => { "action_id" => "not-processed" }
          }

         # more groks

output {
        elasticsearch { 
                hosts => ["elasticsearch:9200"]
        }
         
       stdout { codec => rubydebug }
}

```

The `filter.conf` is running in production, just without the rubydebug part. But I don't want to do testing on a productive system.

---

<div class="post-metadata">

### Author: ![moep](https://avatars.discourse-cdn.com/v4/letter/m/ad7895/32.png) [@moep](https://discuss.elastic.co/u/moep)
#### Post date: [February 24, 2024, 8:35pm UTC](https://discuss.elastic.co/t/logs-are-not-visible-in-kibana/353789/4 "2024-02-24T20:35:42Z")

</div>

> [@moep](#):
>
> `path => "/usr/share/logstash/mainlog.log"`

I found a solution. I was assuming everything in this path will be mappend and I can see it.  
After changing it to:

> path =\> "/usr/share/logstash/mainlog.log/mainlog.log"

its working now.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 23, 2024, 8:36pm UTC](https://discuss.elastic.co/t/logs-are-not-visible-in-kibana/353789/5 "2024-03-23T20:36:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
