# Should the Beat modules only be installed on client?

**URL:** <https://discuss.elastic.co/t/should-the-beat-modules-only-be-installed-on-client/187483>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [June 26, 2019, 6:30am UTC](https://discuss.elastic.co/t/should-the-beat-modules-only-be-installed-on-client/187483 "2019-06-26T06:30:51Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mhn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mhn/32/48793_2.png) [@Mhn](https://discuss.elastic.co/u/Mhn)\
**Post date:** [June 26, 2019, 6:30am UTC](https://discuss.elastic.co/t/should-the-beat-modules-only-be-installed-on-client/187483/1 "2019-06-26T06:30:51Z")

</div>

Hi everyone,

i'm newbi and i am using elasticsearch , logstash, kibana. I want to send logs from an client to my elastic server (logstash) via FileBeat. Should i install FileBeat modules and activate it only on client or each Beat need to be installed and activate both on client and server ?

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [June 26, 2019, 10:42am UTC](https://discuss.elastic.co/t/should-the-beat-modules-only-be-installed-on-client/187483/2 "2019-06-26T10:42:42Z")

</div>

@Mhn,

I am assuming that your architecture is like below:

Filebeat-\>Logstash-\>Elasticsaerch-\>Kibana

`Filebeat:` Filebeat need to install over all those servers for which you want to send logs over elasticsaerch and can view over kibana dashboard.

`Logstash:` Logstash is used for extra level parsing of logs.

`Elasticsearch:` Used to store all the logs for your all servers.

`Kibana:` Kibana is used to view the logs over GUI dashboard.

So, you need to install Filebeat over only those server for you want send logs over elasticsaerch.

Please let me know if you have any other query.

Thanks.

---

<div class="post-metadata">

**Author:** ![Mhn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mhn/32/48793_2.png) [@Mhn](https://discuss.elastic.co/u/Mhn)\
**Post date:** [June 26, 2019, 11:54am UTC](https://discuss.elastic.co/t/should-the-beat-modules-only-be-installed-on-client/187483/3 "2019-06-26T11:54:48Z")

</div>

@Tek_Chand

thank you for your quick answer. Yes i'm using this method =\>  
Filebeat-\>Logstash-\>Elasticsaerch-\>Kibana

my question is regards to an issue which I’ve with different OS. I’ve followed this tutorial `https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elastic-stack-on-ubuntu-18-04` and it works everything. Even though after update elastic from  
`6.4.3` to `7.0.2` was everything OK. But I’ve tried to install on `Debian9` and all works but I can see in Dashboards only data from `Syslog` and not more from `Sudo Command` and `SSH Logins` . My config is same as last elastic Installation (Ubuntu 18.04). Do you have any Idea why Dashboard is not working well ?

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [June 26, 2019, 12:08pm UTC](https://discuss.elastic.co/t/should-the-beat-modules-only-be-installed-on-client/187483/4 "2019-06-26T12:08:49Z")

</div>

@Mhn,

> [@Mhn](#):
>
> I can see in Dashboards only data from `Syslog` and not more from `Sudo Command` and `SSH Logins` .

As per your response you are getting the data from `syslog` on kibana dashboard not from Sudo and SSH logins.

The SSH logs store in different file. Have you specify that file path in your `filebeat.yml` file?  
Have you define grok filter for auth logs?

Please verify above things.

Thanks.

---

<div class="post-metadata">

**Author:** ![Mhn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mhn/32/48793_2.png) [@Mhn](https://discuss.elastic.co/u/Mhn)\
**Post date:** [June 26, 2019, 12:21pm UTC](https://discuss.elastic.co/t/should-the-beat-modules-only-be-installed-on-client/187483/5 "2019-06-26T12:21:13Z")

</div>

@Tek_Chand

> The SSH logs store in different file. Have you specify that file path in your `filebeat.yml` file?  
> Have you define grok filter for auth logs?

yes. here is my filebeat.yml =\>

```
 filebeat.inputs:
- type: log
 paths:
    - /var/log/*.log

```

and here is my filter

```auto
filter {
  if [fileset][module] == "system" {
    if [fileset][name] == "auth" {
      grok {
        match => { "message" => ["%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sshd(?:\[%{POSINT:[system][auth][pid]}\])?: %{DATA:[system][auth][ssh][event]} %{DATA:[system][auth][ssh][method]} for (invalid user )?%{DATA:[system][auth][user]} from %{IPORHOST:[system][auth][ssh][ip]} port %{NUMBER:[system][auth][ssh][port]} ssh2(: %{GREEDYDATA:[system][auth][ssh][signature]})?",
                  "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sshd(?:\[%{POSINT:[system][auth][pid]}\])?: %{DATA:[system][auth][ssh][event]} user %{DATA:[system][auth][user]} from %{IPORHOST:[system][auth][ssh][ip]}",
                  "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sshd(?:\[%{POSINT:[system][auth][pid]}\])?: Did not receive identification string from %{IPORHOST:[system][auth][ssh][dropped_ip]}",
                  "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sudo(?:\[%{POSINT:[system][auth][pid]}\])?: \s*%{DATA:[system][auth][user]} :( %{DATA:[system][auth][sudo][error]} ;)? TTY=%{DATA:[system][auth][sudo][tty]} ; PWD=%{DATA:[system][auth][sudo][pwd]} ; USER=%{DATA:[system][auth][sudo][user]} ; COMMAND=%{GREEDYDATA:[system][auth][sudo][command]}",
                  "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} groupadd(?:\[%{POSINT:[system][auth][pid]}\])?: new group: name=%{DATA:system.auth.groupadd.name}, GID=%{NUMBER:system.auth.groupadd.gid}",
                  "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} useradd(?:\[%{POSINT:[system][auth][pid]}\])?: new user: name=%{DATA:[system][auth][user][add][name]}, UID=%{NUMBER:[system][auth][user][add][uid]}, GID=%{NUMBER:[system][auth][user][add][gid]}, home=%{DATA:[system][auth][user][add][home]}, shell=%{DATA:[system][auth][user][add][shell]}$",
                  "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} %{DATA:[system][auth][program]}(?:\[%{POSINT:[system][auth][pid]}\])?: %{GREEDYMULTILINE:[system][auth][message]}"] }
        pattern_definitions => {
          "GREEDYMULTILINE"=> "(.|\n)*"
        }
        remove_field => "message"
      }
      date {
        match => ["[system][auth][timestamp]", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
      }
      geoip {
        source => "[system][auth][ssh][ip]"
        target => "[system][auth][ssh][geoip]"
      }
    }
    else if [fileset][name] == "syslog" {
      grok {
        match => { "message" => ["%{SYSLOGTIMESTAMP:[system][syslog][timestamp]} %{SYSLOGHOST:[system][syslog][hostname]} %{DATA:[system][syslog][program]}(?:\[%{POSINT:[system][syslog][pid]}\])?: %{GREEDYMULTILINE:[system][syslog][message]}"] }
        pattern_definitions => { "GREEDYMULTILINE" => "(.|\n)*" }
        remove_field => "message"
      }
      date {
        match => ["[system][syslog][timestamp]", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
      }
    }
  }
}

```

and what i mean with Dashboard you can see in this screenshot :

 ![ELK](https://us1.discourse-cdn.com/elastic/original/3X/8/4/84ea9525cacf55cc327ed52e38d4df4bca082ef5.jpeg)

As i mentioned if I click on another option like `Sudo commands` or `SSH logins`, haven't any data.

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [June 26, 2019, 12:30pm UTC](https://discuss.elastic.co/t/should-the-beat-modules-only-be-installed-on-client/187483/6 "2019-06-26T12:30:16Z")

</div>

@Mhn,

Are you getting auth log in Index? Are you using filebeat module for auth.log?  
Are you storing authlog into `filebeat` index?  
Thanks.

---

<div class="post-metadata">

**Author:** ![Mhn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mhn/32/48793_2.png) [@Mhn](https://discuss.elastic.co/u/Mhn)\
**Post date:** [June 26, 2019, 12:41pm UTC](https://discuss.elastic.co/t/should-the-beat-modules-only-be-installed-on-client/187483/7 "2019-06-26T12:41:17Z")

</div>

@Tek_Chand

> Are you getting auth log in Index?

yes =\>

 ![index](https://us1.discourse-cdn.com/elastic/original/3X/7/3/73bcf5bb7b7f62033b8ff4ba37598db389a1afda.png)

> Are you using filebeat module for auth.log?  
> Are you storing authlog into `filebeat` index?

i'm not sure what do you mean exactly but i think it's so  
/etc/filebeat/modules.d/system.yml =\>

```
    - module: system
      # Syslog
      syslog:
        enabled: true
# Authorization logs
  auth:
    enabled: true

```

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [June 26, 2019, 1:03pm UTC](https://discuss.elastic.co/t/should-the-beat-modules-only-be-installed-on-client/187483/8 "2019-06-26T13:03:32Z")

</div>

@Mhn,

> [@Mhn](#):
>
> [How To Install Elasticsearch, Logstash, and Kibana (Elastic Stack) on Ubuntu 18.04 | DigitalOcean](https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elastic-stack-on-ubuntu-18-04)

You have followed above link and its using filebeat module for system which enable logs for system and auth log.

As per my knowledge when we use logstash we can't use dashboard for auth log due to some pipeline issue. Can you please disable filebeat module once and specify the log file path in `filebeat.yml` file.

Thanks.

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [June 27, 2019, 5:19am UTC](https://discuss.elastic.co/t/should-the-beat-modules-only-be-installed-on-client/187483/9 "2019-06-27T05:19:29Z")

</div>

@Mhn,

Your configuration seems fine. Can you please disable filebeat system module and define in `filebeat.yml` file like below:

```auto
filebeat.inputs:
- type: log
 paths:
    - /var/log/auth.log

```

as well as replace the grok pattren for auth with below pattern:

```auto
["%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sshd(?:\[%{POSINT:[system][auth][pid]}\])?: %{DATA:[system][auth][ssh][event]} %{DATA:[system][auth][ssh][method]} for (invalid user )?%{DATA:[system][auth][user]} from %{IPORHOST:[system][auth][ssh][ip]} port %{NUMBER:[system][auth][ssh][port]} ssh2(: %{GREEDYDATA:[system][auth][ssh][signature]})?", "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sshd(?:\[%{POSINT:[system][auth][pid]}\])?: %{DATA:[system][auth][ssh][event]} user %{DATA:[system][auth][user]} from %{IPORHOST:[system][auth][ssh][ip]}", "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sshd(?:\[%{POSINT:[system][auth][pid]}\])?: Did not receive identification string from %{IPORHOST:[system][auth][ssh][dropped_ip]}", "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sudo(?:\[%{POSINT:[system][auth][pid]}\])?: \s*%{DATA:[system][auth][user]} :( %{DATA:[system][auth][sudo][error]} ;)? TTY=%{DATA:[system][auth][sudo][tty]} ; PWD=%{DATA:[system][auth][sudo][pwd]} ; USER=%{DATA:[system][auth][sudo][user]} ; COMMAND=%{GREEDYDATA:[system][auth][sudo][command]}", "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} groupadd(?:\[%{POSINT:[system][auth][pid]}\])?: new group: name=%{DATA:system.auth.groupadd.name}, GID=%{NUMBER:system.auth.groupadd.gid}", "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} useradd(?:\[%{POSINT:[system][auth][pid]}\])?: new user: name=%{DATA:[system][auth][user][add][name]}, UID=%{NUMBER:[system][auth][user][add][uid]}, GID=%{NUMBER:[system][auth][user][add][gid]}, home=%{DATA:[system][auth][user][add][home]}, shell=%{DATA:[system][auth][user][add][shell]}$", "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} %{DATA:[system][auth][program]}(?:\[%{POSINT:[system][auth][pid]}\])?: (?<system.auth.ssh.event>Accepted) (?<system.auth.ssh.method>publickey) \w+ (?<username>.*)", "%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} %{DATA:[system][auth][program]}(?:\[%{POSINT:[system][auth][pid]}\])?: %{GREEDYMULTILINE:[system][auth][message]}"]

```

Its working fine at my end.

Thanks.

---

<div class="post-metadata">

**Author:** ![Mhn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mhn/32/48793_2.png) [@Mhn](https://discuss.elastic.co/u/Mhn)\
**Post date:** [June 27, 2019, 10:54am UTC](https://discuss.elastic.co/t/should-the-beat-modules-only-be-installed-on-client/187483/10 "2019-06-27T10:54:36Z")

</div>

> [@Tek\_Chand](#):
>
> Your configuration seems fine. Can you please disable filebeat system module and define in `filebeat.yml` file like below:

it doesn't work if i disable filebeat system module.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 25, 2019, 10:54am UTC](https://discuss.elastic.co/t/should-the-beat-modules-only-be-installed-on-client/187483/11 "2019-07-25T10:54:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.