# Problems in Elasticsearch combining two types of logs using two pipelines (Logstash)

**URL:** https://discuss.elastic.co/t/problems-in-elasticsearch-combining-two-types-of-logs-using-two-pipelines-logstash/273900
**Category:** Logstash
**Created:** [May 25, 2021, 7:26am UTC](https://discuss.elastic.co/t/problems-in-elasticsearch-combining-two-types-of-logs-using-two-pipelines-logstash/273900 "2021-05-25T07:26:28Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![francesco96](https://avatars.discourse-cdn.com/v4/letter/f/c68b51/32.png) [@francesco96](https://discuss.elastic.co/u/francesco96)
#### Post date: [May 25, 2021, 7:26am UTC](https://discuss.elastic.co/t/problems-in-elasticsearch-combining-two-types-of-logs-using-two-pipelines-logstash/273900/1 "2021-05-25T07:26:28Z")

</div>

Hi everyone, I need some help.  
I have configured my ES stack inside a docker. I need to show two types of logs (syslog, log), in an Elastic index. I have created two configuration files

- logstash.cong
- logstash-syslog.conf

I also created two pipelines in logstash to manage the two files. Despite this Elasticsearch always shows only the logs of one of the two and never both together.

Configuration files below

logstash.cong

```
input {
  beats {
    port => 5044
  }
  tcp {
    port => 5000
  }

}

filter {
    mutate { replace => { "[host]" => "%{[host][name]}" } }
}

output {
  elasticsearch {

    hosts => "https://elasticsearch:9200"
    user => "user"
    password => "password"

    ecs_compatibility => disabled
    cacert => "config/elasticsearch-ca.pem"

    #index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
    index => "my-index"

  }
}

```

logstash-syslog.conf

```
input {
  tcp {
    port => 5044
    type => syslog
  }
  udp {
    port => 5044
    type => syslog
  }
}

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => ["received_at", "%{@timestamp}"]
      add_field => ["received_from", "%{host}"]
    }
    date {
      match => ["syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]
    }
  }
}

output {
  elasticsearch {

    hosts => "https://elasticsearch:9200"
    user => "user"
    password => "password"

    ecs_compatibility => disabled
    cacert => "config/elasticsearch-ca.pem"

    #index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
    index => "my-index"

  }
}

```

pipelines.yml

```
- pipeline.id: pipeline_1
  path.config: "/usr/share/logstash/pipeline/logstash.conf"
  pipeline.workers: 3
- pipeline.id: pipeline_2
  path.config: "/usr/share/logstash/pipeline/logstash-syslog.conf"
  queue.type: persisted

```

How can i report the two types of logs in the elastic index?

Thanks in advance

---

<div class="post-metadata">

### Author: ![francesco96](https://avatars.discourse-cdn.com/v4/letter/f/c68b51/32.png) [@francesco96](https://discuss.elastic.co/u/francesco96)
#### Post date: [June 9, 2021, 6:51am UTC](https://discuss.elastic.co/t/problems-in-elasticsearch-combining-two-types-of-logs-using-two-pipelines-logstash/273900/2 "2021-06-09T06:51:43Z")

</div>

Can anybody help me?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 7, 2021, 6:51am UTC](https://discuss.elastic.co/t/problems-in-elasticsearch-combining-two-types-of-logs-using-two-pipelines-logstash/273900/3 "2021-07-07T06:51:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.