# Problems in Elasticsearch combining two types of logs using two pipelines (Logstash)

**URL:** https://discuss.elastic.co/t/problems-in-elasticsearch-combining-two-types-of-logs-using-two-pipelines-logstash/278048
**Category:** Logstash
**Tags:** elastic-stack-security, docker
**Created:** [July 7, 2021, 9:06am UTC](https://discuss.elastic.co/t/problems-in-elasticsearch-combining-two-types-of-logs-using-two-pipelines-logstash/278048 "2021-07-07T09:06:57Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![francesco96](https://avatars.discourse-cdn.com/v4/letter/f/c68b51/32.png) [@francesco96](https://discuss.elastic.co/u/francesco96)
#### Post date: [July 7, 2021, 9:06am UTC](https://discuss.elastic.co/t/problems-in-elasticsearch-combining-two-types-of-logs-using-two-pipelines-logstash/278048/1 "2021-07-07T09:06:57Z")

</div>

Hi everyone, I need some help.  
I have configured my ES stack inside a docker. I need to show two types of logs (syslog, log), in an Elastic index. I have created two configuration files

- logstash.cong
- logstash-syslog.conf

I also created two pipelines in logstash to manage the two files. Despite this Elasticsearch always shows only the logs of one of the two and never both together.

Configuration files below

logstash.cong

```auto
input {
  beats {
    port => 5044
  }
  tcp {
    port => 5000
  }

}

filter {
    mutate { replace => { "[host]" => "%{[host][name]}" } }
}

output {
  elasticsearch {

    hosts => "https://elasticsearch:9200"
    user => "user"
    password => "password"

    ecs_compatibility => disabled
    cacert => "config/elasticsearch-ca.pem"

    #index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
    index => "my-index"

  }
}

```

logstash-syslog.conf

```auto
input {
  tcp {
    port => 5044
    type => syslog
  }
  udp {
    port => 5044
    type => syslog
  }
}

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => ["received_at", "%{@timestamp}"]
      add_field => ["received_from", "%{host}"]
    }
    date {
      match => ["syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]
    }
  }
}

output {
  elasticsearch {

    hosts => "https://elasticsearch:9200"
    user => "user"
    password => "password"

    ecs_compatibility => disabled
    cacert => "config/elasticsearch-ca.pem"

    #index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
    index => "my-index"

  }
}

```

pipelines.yml

```auto
- pipeline.id: pipeline_1
  path.config: "/usr/share/logstash/pipeline/logstash.conf"
  pipeline.workers: 3
- pipeline.id: pipeline_2
  path.config: "/usr/share/logstash/pipeline/logstash-syslog.conf"
  queue.type: persisted

```

How can i report the two types of logs in the elastic index?

Thanks in advance

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 4, 2021, 9:07am UTC](https://discuss.elastic.co/t/problems-in-elasticsearch-combining-two-types-of-logs-using-two-pipelines-logstash/278048/2 "2021-08-04T09:07:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.