# Multiple filebeat log with logstash not working

**URL:** https://discuss.elastic.co/t/multiple-filebeat-log-with-logstash-not-working/310472
**Category:** Logstash
**Created:** [July 24, 2022, 7:24am UTC](https://discuss.elastic.co/t/multiple-filebeat-log-with-logstash-not-working/310472 "2022-07-24T07:24:52Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![sazzad114](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sazzad114/32/108760_2.png) [@sazzad114](https://discuss.elastic.co/u/sazzad114)
#### Post date: [July 24, 2022, 7:24am UTC](https://discuss.elastic.co/t/multiple-filebeat-log-with-logstash-not-working/310472/1 "2022-07-24T07:24:52Z")

</div>

```auto
filebeat.input:
- type: log
  enabled: true
  paths:
    - /var/log/nginx/*log
  tags: ["server-log"]
  fields: {log_type: Server-log}
- type: log
  enabled: true
  paths:
    - /var/www/laravel/storage/logs/*.log
  tags: ["laravel-log"]
  feilds: {log_type: laravel-log}

setup.kibana:
host: " **** :5601"

output.logstash:
  # The Logstash hosts
  hosts: [" ***** :5044"]

```

```auto
input {
  beats {
    port => 5044
   }
}

filter {
 if ([fields][log_type] == "server-log") {
    grok {
    match => { message => "%{TIMESTAMP_ISO8601:timestamp} \[%{DATA:thread}] %{LOGLEVEL:log-level}%{DATA:class}- %{GREEDYDATA:message}"}
     }

   kv {
        source => "message"
        remove_field => "kv"
        field_split => " "
        value_split => ":"
        include_brackets => "false"
        remove_char_key => "{,"
        recursive => "true"
      }
                        }
 else if ([fields][log_type] == "laravel-log") {
    grok {
    match => { message => "%{TIMESTAMP_ISO8601:timestamp} \[%{DATA:thread}] %{LOGLEVEL:log-level}%{DATA:class}- %{GREEDYDATA:message}"}
     }

   kv {
        source => "message"
        remove_field => "kv"
        field_split => " "
        value_split => ":"
        include_brackets => "false"
        remove_char_key => "{,"
        recursive => "true"
      }
                        }
           }

output {
  elasticsearch {
    hosts => [" ***** :9200"]
    manage_template => false
    index => "%{[fields][log_type]}-index"
  }
  

```

Logstash Log \>\>\>\>

```auto
[2022-07-24T05:35:17,242][INFO][logstash.filters.kv][main] ECS compatibility is enabled but `target` option was not specified. This may cause fields to be set at the top-level of the event where they are likely to clash with the Elastic Common Schema. It is recommended to set the `target` option to avoid potential schema conflicts (if your data is ECS compliant or non-conflicting, feel free to ignore this message)
[2022-07-24T05:35:17,409][INFO][logstash.javapipeline][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, "pipeline.sources"=>["/etc/logstash/conf.d/app-server-1.conf"], :thread=>"#<Thread:0x3975ef97 run>"}
[2022-07-24T05:35:19,026][INFO][logstash.javapipeline][main] Pipeline Java execution initialization time {"seconds"=>1.61}
[2022-07-24T05:35:19,082][INFO][logstash.inputs.beats][main] Starting input listener {:address=>"0.0.0.0:5044"}
[2022-07-24T05:35:19,166][INFO][logstash.javapipeline][main] Pipeline started {"pipeline.id"=>"main"}
[2022-07-24T05:35:19,328][INFO][logstash.agent] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2022-07-24T05:35:19,470][INFO][org.logstash.beats.Server][main][91e605f9a7d96973e54e5cafaf46c8b7e263dfc362a8c48083b4ef0acbaab0ea] Starting server on port: 5044

```

filebeat log \>\>\>\>

```auto
{"log.level":"info","@timestamp":"2022-07-24T05:58:17.026Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":185},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":20,"time":{"ms":20}},"total":{"ticks":130,"time":{"ms":130},"value":0},"user":{"ticks":110,"time":{"ms":110}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":10},"info":{"ephemeral_id":"7eb60458-bc50-4556-b157-6f50e63c85e0","name":"filebeat","uptime":{"ms":30088},"version":"8.3.2"},"memstats":{"gc_next":20439800,"memory_alloc":13086720,"memory_sys":32850952,"memory_total":51808360,"rss":83009536},"runtime":{"goroutines":15}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0},"type":"logstash"},"pipeline":{"clients":0,"events":{"active":0},"queue":{"max_events":4096}}},"registrar":{"states":{"current":0}},"system":{"cpu":{"cores":1},"load":{"1":0,"15":0.05,"5":0.01,"norm":{"1":0,"15":0.05,"5":0.01}}}},"ecs.version":"1.6.0"}}

```

EDITED by Moderator to be readable, please properly format your text going forward

---

<div class="post-metadata">

### Author: ![sazzad114](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sazzad114/32/108760_2.png) [@sazzad114](https://discuss.elastic.co/u/sazzad114)
#### Post date: [July 24, 2022, 7:28am UTC](https://discuss.elastic.co/t/multiple-filebeat-log-with-logstash-not-working/310472/2 "2022-07-24T07:28:18Z")

</div>

please help me on that issue  
am new in ELK

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [July 24, 2022, 1:21pm UTC](https://discuss.elastic.co/t/multiple-filebeat-log-with-logstash-not-working/310472/3 "2022-07-24T13:21:52Z")

</div>

You need to provide more context about your issue, it is impossible to know what it is from what you shared.

What is not working? What is your expected output and what is your current output?

Also, edit your post and use the preformatted text option, `</>`, in your configuration and logs, it helps with the formatting.

---

<div class="post-metadata">

### Author: ![sazzad114](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sazzad114/32/108760_2.png) [@sazzad114](https://discuss.elastic.co/u/sazzad114)
#### Post date: [July 26, 2022, 5:30am UTC](https://discuss.elastic.co/t/multiple-filebeat-log-with-logstash-not-working/310472/4 "2022-07-26T05:30:50Z")

</div>

i have tried configure my nginx and laravel log with filebeats send log data to logtsash to elasticsearch in cloud , where cloud 3 node sperate for filebeat logstash and elactic .

i have share the configuration of  
filebeats logstash. is there any issue ??

later i have share these two filebeat and logstash log , didn't find any error !!!

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [July 26, 2022, 12:15pm UTC](https://discuss.elastic.co/t/multiple-filebeat-log-with-logstash-not-working/310472/5 "2022-07-26T12:15:55Z")

</div>

I do not see any issue in your configuration, but as I said, you still didn't say what is not working.

You said what you want to do, but not what is happening. Are the logs not arriving? Are they arriving without parse? You need to specify what is the issue, what is not working.

There is no error in the logs you shared and your configuration looks ok, so it is not possible to know until you say what is not working and share sample log messages from both `nginx` and `laravel`.

But, there is some things that you need to change in your filebeat, but I'm not sure if this is how it is in the file or if it is just a typo.

First is this, `fields: {log_type: Server-log}`, since you are using this field in the index name, it needs to be in lowercase, change to `server-log`, second is this `feilds: {log_type: laravel-log}`, should be `fields`, but it looks like a typo or else filebeat wouldn't start.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 23, 2022, 12:16pm UTC](https://discuss.elastic.co/t/multiple-filebeat-log-with-logstash-not-working/310472/6 "2022-08-23T12:16:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
