# Loadbalance: false - filebeat doesn't take random host to connect to

**URL:** <https://discuss.elastic.co/t/loadbalance-false-filebeat-doesnt-take-random-host-to-connect-to/43451>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [March 3, 2016, 10:13pm UTC](https://discuss.elastic.co/t/loadbalance-false-filebeat-doesnt-take-random-host-to-connect-to/43451 "2016-03-03T22:13:03Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Karol\_Stojek](https://avatars.discourse-cdn.com/v4/letter/k/ee7513/32.png) [@Karol\_Stojek](https://discuss.elastic.co/u/Karol_Stojek)\
**Post date:** [March 3, 2016, 10:13pm UTC](https://discuss.elastic.co/t/loadbalance-false-filebeat-doesnt-take-random-host-to-connect-to/43451/1 "2016-03-03T22:13:03Z")

</div>

the `loadbalance: false` option doesn't take random endpoint to connect to.  
I have a list in hosts that consists of 30 endpoints.  
I run 18 instances of filebeat with this configuration.

They all try to connect to exactly the same enpoint (it's a 3rd one from the hosts list).

Should I create an issue for it? Or is it a known bug?

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [March 4, 2016, 12:03am UTC](https://discuss.elastic.co/t/loadbalance-false-filebeat-doesnt-take-random-host-to-connect-to/43451/2 "2016-03-04T00:03:25Z")

</div>

sounds weird, could be a bug. Can you share your complete config (without comments)? I'd like to test to confirm bug or not.

Thanks.

---

<div class="post-metadata">

**Author:** ![Karol\_Stojek](https://avatars.discourse-cdn.com/v4/letter/k/ee7513/32.png) [@Karol\_Stojek](https://discuss.elastic.co/u/Karol_Stojek)\
**Post date:** [March 4, 2016, 1:14pm UTC](https://discuss.elastic.co/t/loadbalance-false-filebeat-doesnt-take-random-host-to-connect-to/43451/3 "2016-03-04T13:14:52Z")

</div>

Sure, here it is:

```
filebeat:
  prospectors:
       -
        paths:
        - /opt/idm/logs/essential-01.log
        - /opt/idm/logs/essential-01a.log
        - /opt/idm/logs/essential-01b.log
        - /opt/idm/logs/essential-01c.log
        - /opt/idm/logs/ident-01.log
        - /opt/idm/logs/extended-01.log
        - /opt/idm/logs/administrative-01.log

        input_type: log

        fields:
          environment: production
 
        fields_under_root: true

        document_type: dip

        publish_async: true

        multiline:
          pattern: ^[0-9]{4,8}
          negate: true
          match: after

       -
        paths:
        - /opt/idm/logs/essential-txn-audit-01.log
        - /opt/idm/logs/essential-txn-audit-01a.log
        - /opt/idm/logs/essential-txn-audit-01b.log
        - /opt/idm/logs/essential-txn-audit-01c.log
        - /opt/idm/logs/extended-txn-audit-01.log
        - /opt/idm/logs/ident-txn-audit-01.log
        - /opt/idm/logs/administrative-txn-audit-01.log

        input_type: log

        fields:
          environment: production

        fields_under_root: true

        document_type: dip-txn

        publish_async: true

        multiline:
          pattern: ^[0-9]{8,8}
          negate: true
          match: after

       -
        paths:
        - /opt/idm/essential*/current/logs/localhost_access_log.*.txt
        - /opt/idm/ident/current/logs/localhost_access_log.*.txt
        - /opt/idm/extended/current/logs/localhost_access_log.*.txt
        - /opt/idm/administrative/current/logs/localhost_access_log.*.txt
        
        input_type: log

        fields:
          environment: production

        fields_under_root: true

        document_type: dip-access-log

        publish_async: true

       -
        paths:
        - /opt/idm/logs/server-status/networkStatus.log

        input_type: log

        fields:
          environment: production
 
        fields_under_root: true

        document_type: network

        publish_async: true

       - 
        paths:
        - /opt/idm/logs/server-status/systemUsage.log
      
        input_type: log
     
        fields:
          environment: production
        
        fields_under_root: true
        
        document_type: usage

        publish_async: true
    

       -
        paths:
        - /opt/idm/logs/server-status/aliveLog.log

        input_type: log

        fields:
          environment: production
             
        fields_under_root: true

        document_type: alive

        publish_async: true

output:

  logstash:
    hosts: ["10.141.51.18:5002","10.141.51.18:5004","10.141.51.18:5006","10.141.51.18:5008","10.141.51.18:5010","10.141.51.19:5000","10.141.51.19:5001","10.141.51.19:5002","10.141.51.19:5003","10.141.51.19:5004","10.141.51.19:5005","10.141.51.19:5006","10.141.51.19:5007","10.141.51.19:5008","10.141.51.19:5009","10.141.51.19:5010"]

    loadbalance: false

logging:

  to_syslog: false

  to_files: true

  files:
    path: /opt/idm/logs/monitoring

     
    name: filebeat-dip
    rotateeverybytes: 10485760 # = 10MB
```

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [March 4, 2016, 2:44pm UTC](https://discuss.elastic.co/t/loadbalance-false-filebeat-doesnt-take-random-host-to-connect-to/43451/4 "2016-03-04T14:44:14Z")

</div>

Thanks, will check.

The publish\_async option is no prospector option, but a filebeat option. Actually publish\_async is mostly useful if `loadbalance: true`.

---

<div class="post-metadata">

**Author:** ![Karol\_Stojek](https://avatars.discourse-cdn.com/v4/letter/k/ee7513/32.png) [@Karol\_Stojek](https://discuss.elastic.co/u/Karol_Stojek)\
**Post date:** [March 8, 2016, 12:09am UTC](https://discuss.elastic.co/t/loadbalance-false-filebeat-doesnt-take-random-host-to-connect-to/43451/5 "2016-03-08T00:09:28Z")

</div>

Hello Stefen,  
Were you able to check if it's a filebeat bug or just my issue?

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [March 8, 2016, 10:13am UTC](https://discuss.elastic.co/t/loadbalance-false-filebeat-doesnt-take-random-host-to-connect-to/43451/6 "2016-03-08T10:13:55Z")

</div>

@Karol_Stojek As Steffen is currently on vacation, I plan to have a look at this. Which filebeat version are you using?

---

<div class="post-metadata">

**Author:** ![Karol\_Stojek](https://avatars.discourse-cdn.com/v4/letter/k/ee7513/32.png) [@Karol\_Stojek](https://discuss.elastic.co/u/Karol_Stojek)\
**Post date:** [March 8, 2016, 10:18am UTC](https://discuss.elastic.co/t/loadbalance-false-filebeat-doesnt-take-random-host-to-connect-to/43451/7 "2016-03-08T10:18:58Z")

</div>

I use filebeat 1.1.1

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [March 14, 2016, 9:57am UTC](https://discuss.elastic.co/t/loadbalance-false-filebeat-doesnt-take-random-host-to-connect-to/43451/8 "2016-03-14T09:57:54Z")

</div>

@Karol_Stojek I didn't have found time yet to try to reproduce this. Did you find in the meantime something on your side?

---

<div class="post-metadata">

**Author:** ![Karol\_Stojek](https://avatars.discourse-cdn.com/v4/letter/k/ee7513/32.png) [@Karol\_Stojek](https://discuss.elastic.co/u/Karol_Stojek)\
**Post date:** [March 14, 2016, 10:09am UTC](https://discuss.elastic.co/t/loadbalance-false-filebeat-doesnt-take-random-host-to-connect-to/43451/9 "2016-03-14T10:09:41Z")

</div>

@ruflin no, switched to loadbalance: true (and publish\_async: true) for now.  
But I had this issue 2 times with filebeats.  
It was working fine with logstash-forwarder as I remember - random endpoint was selected.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [March 24, 2016, 9:03am UTC](https://discuss.elastic.co/t/loadbalance-false-filebeat-doesnt-take-random-host-to-connect-to/43451/10 "2016-03-24T09:03:35Z")

</div>

I finally found the time to test this. It is indeed that it always tries to connect to the same host. I tried different host options and it is not necessarly always the third. Once it was also the first one but with the same config, it stays the same host. There seems to be some sorting involved.

@steffens Can you have a look into this?

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [March 24, 2016, 9:03am UTC](https://discuss.elastic.co/t/loadbalance-false-filebeat-doesnt-take-random-host-to-connect-to/43451/11 "2016-03-24T09:03:55Z")

</div>

@Karol_Stojek Can you open a Github issue based on this? And sorry for the long waiting.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 9:54pm UTC](https://discuss.elastic.co/t/loadbalance-false-filebeat-doesnt-take-random-host-to-connect-to/43451/12 "2017-07-05T21:54:14Z")

</div>


