# Using Filebeat for logging ssh log in

**URL:** <https://discuss.elastic.co/t/using-filebeat-for-logging-ssh-log-in/133822>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [May 30, 2018, 8:28am UTC](https://discuss.elastic.co/t/using-filebeat-for-logging-ssh-log-in/133822 "2018-05-30T08:28:47Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![vishnuduttpv](https://avatars.discourse-cdn.com/v4/letter/v/5daacb/32.png) [@vishnuduttpv](https://discuss.elastic.co/u/vishnuduttpv)\
**Post date:** [May 30, 2018, 8:28am UTC](https://discuss.elastic.co/t/using-filebeat-for-logging-ssh-log-in/133822/1 "2018-05-30T08:28:47Z")

</div>

Hi,  
I am using the following configuration,  
OS - CentOS Linux release 7.4.1708 (Core)  
ES - 6.2  
LogStash - 6.2  
Kibana - 6.2  
FileBeat - 6.2

I was using the filebeat for pushing the logs from the server to the kibana. The enabled modules in filebeat are  
system  
apache2  
auditd

## The logs are being shown in the kibana except some logs such as 'ssh logins', 'new user and groups' and 'sudo commands'. The system module configuration is as follows,

```auto
- module: system
  # Syslog
  syslog:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths: ["/var/log/messages"]

    # Convert the timestamp to UTC. Requires Elasticsearch >= 6.1.
    var.convert_timezone: true

  # Authorization logs
  auth:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths: ["/var/log/secure"]

    # Convert the timestamp to UTC. Requires Elasticsearch >= 6.1.
    var.convert_timezone: true

```

* * *

I am not able to view the respective logins and other details as the /var/log/secure is having the data.  
Request to advise and thanks in advance.

Regards  
Vishnu

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [May 30, 2018, 6:18pm UTC](https://discuss.elastic.co/t/using-filebeat-for-logging-ssh-log-in/133822/2 "2018-05-30T18:18:54Z")

</div>

Are you using Logstash as the output destination for Filebeat? Filebeat modules don't work with Logstash because they require an Elasticsearch Ingest Node pipeline to do the parsing.

- [https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-modules-overview.html](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-modules-overview.html)
- [https://www.elastic.co/guide/en/logstash/6.2/filebeat-modules.html](https://www.elastic.co/guide/en/logstash/6.2/filebeat-modules.html)

---

<div class="post-metadata">

**Author:** ![vishnuduttpv](https://avatars.discourse-cdn.com/v4/letter/v/5daacb/32.png) [@vishnuduttpv](https://discuss.elastic.co/u/vishnuduttpv)\
**Post date:** [May 31, 2018, 7:40am UTC](https://discuss.elastic.co/t/using-filebeat-for-logging-ssh-log-in/133822/3 "2018-05-31T07:40:04Z")

</div>

Hi Andrew,  
Thanks for the reply  
Yeah... I have done that. I am pushing my logs to the ES Only and I have installed the Required plugins.

Please find the conf for the Filebeat and my ES plugin out.

```auto

output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["http://10.10.114.175:9200"]

```

```auto
 ./elasticsearch-plugin list
ingest-geoip
ingest-user-agent

```

Thanks in advance,  
Regards  
Vishnu

---

<div class="post-metadata">

**Author:** ![vishnuduttpv](https://avatars.discourse-cdn.com/v4/letter/v/5daacb/32.png) [@vishnuduttpv](https://discuss.elastic.co/u/vishnuduttpv)\
**Post date:** [May 31, 2018, 12:41pm UTC](https://discuss.elastic.co/t/using-filebeat-for-logging-ssh-log-in/133822/4 "2018-05-31T12:41:48Z")

</div>

Hi ,  
Whether the x-pack installation is required for the same?

Thanks  
Vishnu

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [May 31, 2018, 3:07pm UTC](https://discuss.elastic.co/t/using-filebeat-for-logging-ssh-log-in/133822/5 "2018-05-31T15:07:35Z")

</div>

X-Pack is not required by Filebeat modules.

> [@vishnuduttpv](#):
>
> The logs are being shown in the kibana except some logs such as 'ssh logins', 'new user and groups' and 'sudo commands'.

Am I correct in understanding that the logs not being shown are from `/var/log/secure`?

Can you check to see if you have any logs from that file? In Kibana under Dev Tools -\> Console you can run a query like this to see if it returns any hits.

```auto
POST /filebeat-*/_search
{
  "size": 10,
  "query": {
    "match": {
      "source": "/var/log/messages"
    }
  }
}

```

Next make this request in the dev console which will show stats from each ingest pipeline. Look for the system auth pipeline to see how many total events (`count`) and how many errors (`failed`).

`GET _nodes/stats/ingest`

If there's nothing in Elasticsearch then please share your Filebeat logs. Perhaps there some kind of issue on the sending side.

---

<div class="post-metadata">

**Author:** ![vishnuduttpv](https://avatars.discourse-cdn.com/v4/letter/v/5daacb/32.png) [@vishnuduttpv](https://discuss.elastic.co/u/vishnuduttpv)\
**Post date:** [June 1, 2018, 5:15am UTC](https://discuss.elastic.co/t/using-filebeat-for-logging-ssh-log-in/133822/6 "2018-06-01T05:15:30Z")

</div>

Hi Andew,  
Thanks for the reply! 🙂

I have gone through the commands. I am getting the results. As I told you before Iam getting the messages log in the kibana except some logs like ssh logins,sudo commands etc. Syslog Dashboard Iam getting and its working fine. The following are the OP.

> POST /filebeat-\*/\_search  
> {  
> "size": 10,  
> "query": {  
> "match": {  
> "source": "/var/log/messages"  
> }  
> }  
> }

OutPut:

> ```
> {
> "took": 6,
> "timed_out": false,
> "_shards": {
> "total": 9,
> "successful": 9,
> "skipped": 0,
> "failed": 0
> },
> 
> ```

Ouput for ,

> GET \_nodes/stats/ingest

> {  
> "\_nodes": {  
> "total": 1,  
> "successful": 1,  
> "failed": 0  
> },

Thanks  
Vishnu

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [June 1, 2018, 5:30pm UTC](https://discuss.elastic.co/t/using-filebeat-for-logging-ssh-log-in/133822/7 "2018-06-01T17:30:52Z")

</div>

The JSON output you posted appears to be truncated. The important parts are missing. Can you please share the full responses.

---

<div class="post-metadata">

**Author:** ![vishnuduttpv](https://avatars.discourse-cdn.com/v4/letter/v/5daacb/32.png) [@vishnuduttpv](https://discuss.elastic.co/u/vishnuduttpv)\
**Post date:** [June 4, 2018, 4:46am UTC](https://discuss.elastic.co/t/using-filebeat-for-logging-ssh-log-in/133822/8 "2018-06-04T04:46:28Z")

</div>

Hi,  
Please find the OP ,

> {  
> "\_nodes": {  
> "total": 1,  
> "successful": 1,  
> "failed": 0  
> },  
> "cluster\_name": "TEST\_GRP",  
> "nodes": {  
> "w7vAHqoVSgWDjPa\_l1lnZA": {  
> "timestamp": 1528087416330,  
> "name": "node-1\_175",  
> "transport\_address": "10.10.114.175:9300",  
> "host": "10.10.114.175",  
> "ip": "10.10.114.175:9300",  
> "roles": [  
> "master",  
> "data",  
> "ingest"  
> ],  
> "ingest": {  
> "total": {  
> "count": 811240,  
> "time\_in\_millis": 14337,  
> "current": 0,  
> "failed": 0  
> },  
> "pipelines": {  
> "filebeat-6.2.4-auditd-log-pipeline": {  
> "count": 155691,  
> "time\_in\_millis": 6781,  
> "current": 0,  
> "failed": 0  
> },  
> "filebeat-6.2.4-system-syslog-pipeline": {  
> "count": 446040,  
> "time\_in\_millis": 1703,  
> "current": 0,  
> "failed": 0  
> },  
> "filebeat-6.2.4-apache2-error-pipeline": {  
> "count": 9,  
> "time\_in\_millis": 13,  
> "current": 0,  
> "failed": 0  
> },  
> "filebeat-6.2.4-system-auth-pipeline": {  
> "count": 57561,  
> "time\_in\_millis": 1263,  
> "current": 0,  
> "failed": 0  
> },  
> "filebeat-6.2.4-apache2-access-default": {  
> "count": 62815,  
> "time\_in\_millis": 1482,  
> "current": 0,  
> "failed": 0  
> },  
> "filebeat-6.2.4-postgresql-log-pipeline": {  
> "count": 89124,  
> "time\_in\_millis": 3095,  
> "current": 0,  
> "failed": 0  
> }  
> }  
> }  
> }  
> }  
> }

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [June 4, 2018, 12:21pm UTC](https://discuss.elastic.co/t/using-filebeat-for-logging-ssh-log-in/133822/9 "2018-06-04T12:21:05Z")

</div>

> [@vishnuduttpv](#):
>
> "filebeat-6.2.4-system-auth-pipeline": {  
> "count": 57561,  
> "time\_in\_millis": 1263,  
> "current": 0,  
> "failed": 0  
> }

It looks like there are no errors in the ingest pipelines for processing the events. And there should be 57k events from the system auth Filebeat module that sends the `/var/log/secure` data generated on CentOS 7.

> [@vishnuduttpv](#):
>
> except some logs like ssh logins,sudo commands etc.

Can you confirm that this data is in `/var/log/secure` on the host?

These dev console queries should return data associated with ssh, sudo, useradd, and groupadd, respectively.

```auto
GET filebeat-*/_search?q=_exists_:system.auth.ssh

GET filebeat-*/_search?q=_exists_:system.auth.sudo

GET filebeat-*/_search?q=_exists_:system.auth.useradd

GET filebeat-*/_search?q=_exists_:system.auth.groupadd

```

---

<div class="post-metadata">

**Author:** ![luky83](https://avatars.discourse-cdn.com/v4/letter/l/13edae/32.png) [@luky83](https://discuss.elastic.co/u/luky83)\
**Post date:** [June 20, 2018, 4:42pm UTC](https://discuss.elastic.co/t/using-filebeat-for-logging-ssh-log-in/133822/10 "2018-06-20T16:42:45Z")

</div>

I'm experiencing exactly the same in Centos7.

> GET filebeat-\*/\_search?q=_exists_:system.auth.sudo

Output correctly parsed hits but the dashboard is empty ( 😐 no results found)

> {  
> "took": 57,  
> "timed\_out": false,  
> "\_shards": {  
> "total": 6,  
> "successful": 6,  
> "skipped": 0,  
> "failed": 0  
> },  
> "hits": {  
> "total": 113,  
> "max\_score": 1,  
> "hits": [  
> {  
> "\_index": "filebeat-6.3.0-2018.06.19",  
> "\_type": "doc",  
> "\_id": "hv3EGGQBEmS5frGURPIU",  
> "\_score": 1,  
> "\_source": {  
> "offset": 9543,  
> "prospector": {  
> "type": "log"  
> },  
> "source": "/var/log/secure",  
> "fileset": {  
> "module": "system",  
> "name": "auth"  
> },  
> "input": {  
> "type": "log"  
> },  
> "@timestamp": "2018-06-19T17:57:33.000Z",  
> "system": {  
> "auth": {  
> "hostname": "experimental",  
> "sudo": {  
> "tty": "pts/1",  
> "pwd": "/home/luca",  
> "user": "root",  
> "command": "/bin/filebeat -e -d publish"  
> },  
> "user": "luca",  
> "timestamp": "Jun 19 17:57:33"  
> }  
> },  
> "beat": {  
> "hostname": "experimental.box",  
> "name": "experimental.box",  
> "version": "6.3.0"  
> },  
> "host": {  
> "name": "experimental.box"  
> }  
> }  
> },  
> ...

---

<div class="post-metadata">

**Author:** ![vishnuduttpv](https://avatars.discourse-cdn.com/v4/letter/v/5daacb/32.png) [@vishnuduttpv](https://discuss.elastic.co/u/vishnuduttpv)\
**Post date:** [July 4, 2018, 9:52am UTC](https://discuss.elastic.co/t/using-filebeat-for-logging-ssh-log-in/133822/11 "2018-07-04T09:52:49Z")

</div>

```
{
  "took": 21,
  "timed_out": false,
  "_shards": {
    "total": 102,
    "successful": 102,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 276,
    "max_score": 1,
    "hits": [
      {
        "_index": "filebeat-6.2.4-2018.06.01",
        "_type": "doc",
        "_id": "KKoGumMBctxyZyiWQ7pA",
        "_score": 1,
        "_source": {
          "@timestamp": "2018-05-28T10:25:05.000Z",
          "system": {
            "auth": {
              "hostname": "server01",
              "ssh": {
                "method": "password",
                "port": "49285",
                "ip": "10.212.134.20",
                "event": "Accepted"
              },
              "pid": "13771",
              "user": "root",
              "timestamp": "May 28 10:25:05"
            }
          },
          "offset": 17867,
          "beat": {
            "hostname": "server01",
            "name": "server01",
            "version": "6.2.4"
          },
          "prospector": {
            "type": "log"
          },
          "source": "/var/log/secure",
          "fileset": {
            "module": "system",
            "name": "auth"
          }
        }
      },
      {
        "_index": "filebeat-6.2.4-2018.06.01",
        "_type": "doc",
        "_id": "WKoGumMBctxyZyiWQ7pr",
        "_score": 1,
        "_source": {
          "@timestamp": "2018-05-28T10:40:55.000Z",
          "system": {
            "auth": {
              "hostname": "server01",
              "ssh": {
                "method": "password",
                "port": "63885",
                "ip": "10.212.134.22",
                "event": "Accepted"
              },
              "pid": "13990",
              "user": "root",
              "timestamp": "May 28 10:40:55"
            }
          },
          "offset": 18903,
          "beat": {
            "hostname": "server01",
            "name": "server01",
            "version": "6.2.4"
          },
          "prospector": {
            "type": "log"
          },
          "source": "/var/log/secure",
          "fileset": {
            "module": "system",
            "name": "auth"
          }
        }
      },
      {
        "_index": "filebeat-6.2.4-2018.06.01",
        "_type": "doc",
        "_id": "3aoGumMBctxyZyiWRLu2",
        "_score": 1,
        "_source": {
          "@timestamp": "2018-05-28T12:23:48.000Z",
          "system": {
            "auth": {
              "hostname": "server01",
              "ssh": {
                "method": "password",
                "port": "65438",
                "ip": "10.212.134.20",
                "event": "Accepted"
              },
              "pid": "15072",
              "user": "root",
              "timestamp": "May 28 12:23:48"
            }
          },
          "offset": 23859,
          "beat": {
            "hostname": "server01",
            "name": "server01",
            "version": "6.2.4"
          },
          "prospector": {
            "type": "log"
          },
          "source": "/var/log/secure",
          "fileset": {
            "module": "system",
            "name": "auth"
          }
        }
      },
      {
        "_index": "filebeat-6.2.4-2018.06.01",
        "_type": "doc",
        "_id": "A6oGumMBctxyZyiWRLzk",
        "_score": 1,
        "_source": {
          "@timestamp": "2018-05-28T12:24:26.000Z",
          "system": {
            "auth": {
              "hostname": "server01",
              "ssh": {
                "method": "password",
                "port": "65447",
                "ip": "10.212.134.20",
                "event": "Accepted"
              },
              "pid": "15123",
              "user": "root",
              "timestamp": "May 28 12:24:26"
            }
          },
          "offset": 24493,
          "beat": {
            "hostname": "server01",
            "name": "server01",
            "version": "6.2.4"
          },
          "prospector": {
            "type": "log"
          },
          "source": "/var/log/secure",
          "fileset": {
            "module": "system",
            "name": "auth"
          }
        }
      },
      {
        "_index": "filebeat-6.2.4-2018.06.01",
        "_type": "doc",
        "_id": "OKoGumMBctxyZyiWU8zG",
        "_score": 1,
        "_source": {
          "@timestamp": "2018-05-29T15:27:24.000Z",
          "system": {
            "auth": {
              "hostname": "server01",
              "ssh": {
                "ip": "10.212.134.22",
                "event": "Invalid"
              },
              "pid": "10612",
              "user": "ROOT",
              "timestamp": "May 29 15:27:24"
            }
          },
          "offset": 97697,
          "beat": {
            "hostname": "server01",
            "name": "server01",
            "version": "6.2.4"
          },
          "prospector": {
            "type": "log"
          },
          "source": "/var/log/secure",
          "fileset": {
            "module": "system",
            "name": "auth"
          }
        }
      },
      {
        "_index": "filebeat-6.2.4-2018.06.01",
        "_type": "doc",
        "_id": "SaoGumMBctxyZyiWU8zs",
        "_score": 1,
        "_source": {
          "@timestamp": "2018-05-29T15:27:46.000Z",
          "system": {
            "auth": {
              "hostname": "server01",
              "ssh": {
                "method": "password",
                "port": "61032",
                "ip": "10.212.134.22",
                "event": "Accepted"
              },
              "pid": "10624",
              "user": "root",
              "timestamp": "May 29 15:27:46"
            }
```

---

<div class="post-metadata">

**Author:** ![vishnuduttpv](https://avatars.discourse-cdn.com/v4/letter/v/5daacb/32.png) [@vishnuduttpv](https://discuss.elastic.co/u/vishnuduttpv)\
**Post date:** [July 4, 2018, 9:54am UTC](https://discuss.elastic.co/t/using-filebeat-for-logging-ssh-log-in/133822/12 "2018-07-04T09:54:44Z")

</div>

Hi @andrewkroh,  
I have attached the output for  
`GET filebeat-*/_search?q=_exists_:system.auth.ssh`

Now also Iam not getting any user login details in the dashboard.  
Request to advise!  
Thanks  
Vishnu

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [July 8, 2018, 10:57am UTC](https://discuss.elastic.co/t/using-filebeat-for-logging-ssh-log-in/133822/13 "2018-07-08T10:57:01Z")

</div>

@vishnuduttpv, I ingested a few of the system.auth.ssh events you posted above to test the `[Filebeat System] SSH login attempts` dashboard. The board was working for me. The only thing I had to change was to extend the [time filter](https://www.elastic.co/guide/en/kibana/current/set-time-filter.html) to cover a larger range that included the times when the events occurred. Did you choose a range that included your events?

 ![filebeat-system-auth-ssh-dashboard](https://us1.discourse-cdn.com/elastic/original/3X/e/7/e79a850c56926ef0dde0485d744e127e29d83d22.png)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 5, 2018, 11:08am UTC](https://discuss.elastic.co/t/using-filebeat-for-logging-ssh-log-in/133822/14 "2018-08-05T11:08:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.