# Send logs on a specify output depending on the paths of the prospectors

**URL:** <https://discuss.elastic.co/t/send-logs-on-a-specify-output-depending-on-the-paths-of-the-prospectors/69911>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [December 23, 2016, 1:06pm UTC](https://discuss.elastic.co/t/send-logs-on-a-specify-output-depending-on-the-paths-of-the-prospectors/69911 "2016-12-23T13:06:52Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![jstar](https://avatars.discourse-cdn.com/v4/letter/j/ba8739/32.png) [@jstar](https://discuss.elastic.co/u/jstar)\
**Post date:** [December 23, 2016, 1:06pm UTC](https://discuss.elastic.co/t/send-logs-on-a-specify-output-depending-on-the-paths-of-the-prospectors/69911/1 "2016-12-23T13:06:52Z")

</div>

Hi Everyone,

I'm playing around with filebeat and I wanted to sends logs to certain outputs in an intelligent way. That is I have a file in the /path/to/file2 which I want its logs to go to elasticsearch and another file in the /directory/to/file2 which I want to ship its logs to kafka.

I have given a try with [config\_dir](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-global-options.html#_config_dir) in my main filebeat configuration file. Please find enclosed the config that I did.

filebeat.yml

```
filebeat.config_dir: /etc/filebeat/conf.d
filebeat.prospectors:
- paths:
    - "/path/to/file1"
document_type: file1logs
output.kafka:
  enabled: true
  hosts: ["ip-address-kafka:9200"]
  topic: topic-for-file1

```

filebeatFile2.yml: which is found in the directory /etc/filebeat/conf.d

```
filebeat.prospectors:
- paths:
    - "/directory/to/file2"
  document_type: file2logs
output.elasticsearch:
  enabled: true
  hosts: ["ip-address-elasticsearch:9200"]
  index: "file2-index"

```

The issue with the configuration is that output.elasticsearch is not considered. In fact it is completely ignored. That is both logs of file1 and file2 are sent to kafka.

So here is my question: can somebody now help me to achieve that please?  
Thanks.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [December 23, 2016, 1:30pm UTC](https://discuss.elastic.co/t/send-logs-on-a-specify-output-depending-on-the-paths-of-the-prospectors/69911/2 "2016-12-23T13:30:34Z")

</div>

The output configuration in beats is global. As of now, filebeat does not support any enhanced event-routing. This is normally handled by logstash. Alternatively you can start 2 filebeat instances (make sure the `filebeat.registry_file` is different for both beats).

---

<div class="post-metadata">

**Author:** ![jstar](https://avatars.discourse-cdn.com/v4/letter/j/ba8739/32.png) [@jstar](https://discuss.elastic.co/u/jstar)\
**Post date:** [December 23, 2016, 1:55pm UTC](https://discuss.elastic.co/t/send-logs-on-a-specify-output-depending-on-the-paths-of-the-prospectors/69911/3 "2016-12-23T13:55:48Z")

</div>

@steffens  
Many thanks. I will give that a try. But I'm already sure this will work very well.  
Merry X-mas.

---

<div class="post-metadata">

**Author:** ![laibon](https://avatars.discourse-cdn.com/v4/letter/l/7cd45c/32.png) [@laibon](https://discuss.elastic.co/u/laibon)\
**Post date:** [December 29, 2016, 5:18am UTC](https://discuss.elastic.co/t/send-logs-on-a-specify-output-depending-on-the-paths-of-the-prospectors/69911/4 "2016-12-29T05:18:22Z")

</div>

Hi Steffens,  
Pardon me if i posted in the wrong area, you may direct me where to post this, but maybe my problem is related to your response.

I have been trying to route our logs to a central Syslog-ng server but found only the options (Elastic,Logstash,Kafika,Console) i tried to improvise by entering my syslog-ng server address under logstash hosts  
and commenting

#enabled: true

# hosts: ["localhost:9200"]

in the Elasticsearch output

# The Logstash hosts

hosts: ["192.168._._:514"]

I keep getting this error from my logs

Connecting error publishing events (retrying): Get [http://localhost:9200](http://localhost:9200): dial tcp [::1]:9200: getsockopt: connection refused

Please share ideas on how i can i route to my syslog-ng server?

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [December 29, 2016, 2:44pm UTC](https://discuss.elastic.co/t/send-logs-on-a-specify-output-depending-on-the-paths-of-the-prospectors/69911/5 "2016-12-29T14:44:20Z")

</div>

@laibon your question is totally unrelated. Please create your own topics. Filebeat does have no syslog output.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 26, 2017, 2:44pm UTC](https://discuss.elastic.co/t/send-logs-on-a-specify-output-depending-on-the-paths-of-the-prospectors/69911/6 "2017-01-26T14:44:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.