Cannot get Cisco Filebeat module to work

Elastic Stack Kibana
1

Hello Team,
I'm running on ELK 7.14.0 and I'm facing an issue with kibana dashboard and need your help.

I have an ELK stack which gets logs from filebeat (cisco module) and sends them directly to Elasticsearch. It works fine and data can be found in "discovery". Hovever, the data can't be visualized in Kibana dashboard.

Filebeat is installed on other linux machine which gets syslogs from cisco asa and ios and then sends the data to Elasticsearch. After command "filebeat setup -e" new dashboards were successfully added to Kibana but nothing can be found on graphs - "No results found".

Filebeat config and screenshots can be found below:

filebeat.inputs:
- type: log
paths:
    - /var/log/*.log

- type: filestream
 enabled: false
paths:
    - /var/log/*.log

filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1

setup.kibana:
host: "192.168.2.6:5601"

output.elasticsearch:
hosts: "192.168.2.6:9200"

processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

Cisco module config:

- module: cisco
  asa:
    enabled: true
var.input: syslog
var.syslog_host: 0.0.0.0
var.syslog_port: 5514

ftd:
    enabled: false

ios:
    enabled: true
 var.input: syslog
 var.syslog_host: 0.0.0.0
 var.syslog_port: 5514

 nexus:
    enabled: false

 meraki:
    enabled: false

 umbrella:
    enabled: false

 amp:
    enabled: false

And couple of pics:

1
11920×970 56.3 KB

2
21920×970 37.1 KB

3
31920×971 193 KB

2

Hi @titan_tm

Ok Lets Take Look... I am not a FW Expert so I can help from the mechanics but not the FW expertise perspective.

First we will assume that those formatting issues in the cisco module are just cut-n-paste error and that your yml is actually formatted correct.

There are a couple things we can check First I want you to go to Discover

In the KQL bar try these filters and let me know which one do and do not work.

cisco.asa.message_id:* and event.action:"firewall-rule"

cisco.asa.message_id*

cisco.asa.message_id.keyword:*

cisco.ios.facility:*

cisco.ios.facility.keyword:*

Let me know what works and does not work

From you images none of these may work as I do not see many cisco fields in the list on the left which may mean that your logs are a custom format / non standard format and are not being parsed correctly.

Also are these only ios Logs? I am not sure there is a Dashboard for those I think there maybe only the ASA dashboard at this time.

1 Like
3

Filebeat gets syslogs from cisco switch and asa, now. In future I plan send logs from router also. First, I need to be sure that stack is stable and logs are parsed correctly.
I've checked the filters in KQL but only one works -

cisco.ios.facility:*
4

And if that is the only field that starts with cisco. then I suspect your logs are not being parsed correctly because they are not the expected format.

5

Logs are parsed by filebeat, how can I change the format? Asa sends typical syslog messages I've just changed the path.

6

Here is a part of filebeat logs

Nov 23 10:37:14 asd filebeat[1448]: 2021-11-23T10:37:14.757+0500        INFO        [monitoring]        log/log.go:145        Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cgroup":{"cpuacct":{"total":{"ns":32979998}}},"cpu":{"system":{"ticks":43270},"total":{"ticks":136480,"time":{"ms":31},"value":136480},"user":{"ticks":93210,"time":{"ms":31}}},"handles":{"limit":{"hard":524288,"soft":1024},"open":13},"info":{"ephemeral_id":"2141e610-9e32-4f98-852f-c76217b2dde7","uptime":{"ms":253140066},"version":"7.14.0"},"memstats":{"gc_next":20063312,"memory_alloc":19187328,"memory_total":9313768216,"rss":111796224},"runtime":{"goroutines":39}},"filebeat":{"events":{"active":-1,"added":7,"done":8},"harvester":{"open_files":1,"running":1}},"libbeat":{"config":{"module":{"running":1}},"output":{"events":{"acked":8,"active":0,"batches":7,"total":8},"read":{"bytes":4892},"write":{"bytes":8384}},"pipeline":{"clients":3,"events":{"active":0,"published":7,"total":7},"queue":{"acked":8}}},"registrar":{"states":{"current":16}},"system":{"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}}}}
Nov 23 10:37:15 asd filebeat[1448]: 2021-11-23T10:37:15.203+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:36:42: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.117 to 17.253.39.207 on interface inside\n"}
Nov 23 10:37:22 asd filebeat[1448]: 2021-11-23T10:37:22.753+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:36:49: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.117 to 17.253.39.207 on interface inside\n"}
Nov 23 10:37:30 asd filebeat[1448]: 2021-11-23T10:37:30.367+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:36:57: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.117 to 17.253.39.207 on interface inside\n"}
Nov 23 10:37:34 asd filebeat[1448]: 2021-11-23T10:37:34.024+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:37:00: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.117 to 17.253.123.201 on interface inside\n"}
Nov 23 10:37:35 asd filebeat[1448]: 2021-11-23T10:37:35.015+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:37:01: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.117 to 17.242.150.14 on interface inside\n"}
Nov 23 10:37:37 asd filebeat[1448]: 2021-11-23T10:37:37.949+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:37:04: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.117 to 17.253.39.207 on interface inside\n"}
Nov 23 10:37:44 asd filebeat[1448]: 2021-11-23T10:37:44.751+0500        INFO        [monitoring]        log/log.go:145        Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cgroup":{"cpuacct":{"total":{"ns":26536765}},"memory":{"mem":{"usage":{"bytes":16384}}}},"cpu":{"system":{"ticks":43270,"time":{"ms":4}},"total":{"ticks":136500,"time":{"ms":29},"value":136500},"user":{"ticks":93230,"time":{"ms":25}}},"handles":{"limit":{"hard":524288,"soft":1024},"open":13},"info":{"ephemeral_id":"2141e610-9e32-4f98-852f-c76217b2dde7","uptime":{"ms":253170066},"version":"7.14.0"},"memstats":{"gc_next":20539424,"memory_alloc":12058744,"memory_total":9316155432,"rss":111796224},"runtime":{"goroutines":39}},"filebeat":{"events":{"added":6,"done":6},"harvester":{"open_files":1,"running":1}},"libbeat":{"config":{"module":{"running":1}},"output":{"events":{"acked":6,"active":0,"batches":5,"total":6},"read":{"bytes":3497},"write":{"bytes":6250}},"pipeline":{"clients":3,"events":{"active":0,"published":6,"total":6},"queue":{"acked":6}}},"registrar":{"states":{"current":16}},"system":{"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}}}}
Nov 23 10:38:07 asd filebeat[1448]: 2021-11-23T10:38:07.692+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:37:34: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.126 to 142.251.1.95 on interface inside\n"}
Nov 23 10:38:07 asd filebeat[1448]: 2021-11-23T10:38:07.984+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:37:34: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.126 to 173.194.221.94 on interface inside\n"}
Nov 23 10:38:08 asd filebeat[1448]: 2021-11-23T10:38:08.519+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:37:35: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.126 to 173.194.221.94 on interface inside\n"}
Nov 23 10:38:09 asd filebeat[1448]: 2021-11-23T10:38:09.017+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:37:35: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.126 to 173.194.221.94 on interface inside\n"}
Nov 23 10:38:10 asd filebeat[1448]: 2021-11-23T10:38:10.013+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:37:36: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.126 to 173.194.221.94 on interface inside\n"}
Nov 23 10:38:11 asd filebeat[1448]: 2021-11-23T10:38:11.994+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:37:38: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.126 to 173.194.221.94 on interface inside\n"}
Nov 23 10:38:12 asd filebeat[1448]: 2021-11-23T10:38:12.243+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:37:39: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.126 to 128.1.78.67 on interface inside\n"}
Nov 23 10:38:12 asd filebeat[1448]: 2021-11-23T10:38:12.922+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<162>Nov 23 2021 10:37:39: %ASA-2-106001: Inbound TCP connection denied from 169.136.114.97/443 to 192.168.8.251/47690 flags RST  on interface outside\n"}
Nov 23 10:38:12 asd filebeat[1448]: 2021-11-23T10:38:12.973+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:37:39: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.126 to 128.1.78.67 on interface inside\n"}
Nov 23 10:38:13 asd filebeat[1448]: 2021-11-23T10:38:13.786+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:37:40: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.126 to 128.1.78.67 on interface inside\n"}
Nov 23 10:38:14 asd filebeat[1448]: 2021-11-23T10:38:14.751+0500        INFO        [monitoring]        log/log.go:145        Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cgroup":{"cpuacct":{"total":{"ns":21845152}}},"cpu":{"system":{"ticks":43280,"time":{"ms":9}},"total":{"ticks":136530,"time":{"ms":26},"value":136530},"user":{"ticks":93250,"time":{"ms":17}}},"handles":{"limit":{"hard":524288,"soft":1024},"open":13},"info":{"ephemeral_id":"2141e610-9e32-4f98-852f-c76217b2dde7","uptime":{"ms":253200068},"version":"7.14.0"},"memstats":{"gc_next":20539424,"memory_alloc":14315040,"memory_total":9318411728,"rss":111796224},"runtime":{"goroutines":39}},"filebeat":{"events":{"active":1,"added":10,"done":9},"harvester":{"open_files":1,"running":1}},"libbeat":{"config":{"module":{"running":1}},"output":{"events":{"acked":9,"active":0,"batches":3,"total":9},"read":{"bytes":2162},"write":{"bytes":8541}},"pipeline":{"clients":3,"events":{"active":1,"published":10,"total":10},"queue":{"acked":9}}},"registrar":{"states":{"current":16}},"system":{"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}}}}
Nov 23 10:38:15 asd filebeat[1448]: 2021-11-23T10:38:15.405+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:37:42: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.126 to 128.1.78.67 on interface inside\n"}
Nov 23 10:38:16 asd filebeat[1448]: 2021-11-23T10:38:16.122+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:37:43: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.126 to 173.194.221.94 on interface inside\n"}
Nov 23 10:38:18 asd filebeat[1448]: 2021-11-23T10:38:18.692+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:37:45: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.126 to 128.1.78.67 on interface inside\n"}
Nov 23 10:38:24 asd filebeat[1448]: 2021-11-23T10:38:24.066+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:37:51: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.126 to 173.194.221.94 on interface inside\n"}
Nov 23 10:38:25 asd filebeat[1448]: 2021-11-23T10:38:25.093+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:37:52: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.126 to 128.1.78.67 on interface inside\n"}
Nov 23 10:38:28 asd filebeat[1448]: 2021-11-23T10:38:28.301+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:37:55: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.106 to 91.202.232.118 on interface inside\n"}
Nov 23 10:38:37 asd filebeat[1448]: 2021-11-23T10:38:37.892+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:38:04: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.126 to 128.1.78.67 on interface inside\n"}
Nov 23 10:38:38 asd filebeat[1448]: 2021-11-23T10:38:38.028+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:38:05: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.117 to 17.253.123.201 on interface inside\n"}
Nov 23 10:38:39 asd filebeat[1448]: 2021-11-23T10:38:39.949+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:38:06: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.126 to 173.194.221.94 on interface inside\n"}
Nov 23 10:38:42 asd filebeat[1448]: 2021-11-23T10:38:42.268+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<161>Nov 23 2021 10:38:09: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.117 to 45.82.107.56 on interface inside\n"}
Nov 23 10:38:44 asd filebeat[1448]: 2021-11-23T10:38:44.753+0500        INFO        [monitoring]        log/log.go:145        Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cgroup":{"cpuacct":{"total":{"ns":37435077}}},"cpu":{"system":{"ticks":43290,"time":{"ms":17}},"total":{"ticks":136560,"time":{"ms":32},"value":136560},"user":{"ticks":93270,"time":{"ms":15}}},"handles":{"limit":{"hard":524288,"soft":1024},"open":13},"info":{"ephemeral_id":"2141e610-9e32-4f98-852f-c76217b2dde7","uptime":{"ms":253230073},"version":"7.14.0"},"memstats":{"gc_next":20539424,"memory_alloc":17554888,"memory_total":9321651576,"rss":111796224},"runtime":{"goroutines":39}},"filebeat":{"events":{"active":-1,"added":10,"done":11},"harvester":{"open_files":1,"running":1}},"libbeat":{"config":{"module":{"running":1}},"output":{"events":{"acked":11,"active":0,"batches":9,"total":11},"read":{"bytes":6292},"write":{"bytes":11410}},"pipeline":{"clients":3,"events":{"active":0,"published":10,"total":10},"queue":{"acked":11}}},"registrar":{"states":{"current":16}},"system":{"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}}}}
Nov 23 10:38:48 asd filebeat[1448]: 2021-11-23T10:38:48.768+0500        ERROR        [syslog]        syslog/input.go:285        can't parse event as syslog rfc3164        {"message": "<162>Nov 23 2021 10:38:15: %ASA-2-106001: Inbound TCP connection denied from 95.100.153.88/443 to 192.168.8.10/53130 flags RST  on interface outside\n"}
7

I think I get it ... If Look close at the docs, you can not feed all the the different logs types the same port otherwise filebeat does not know which messages to process with which ingest pipelines... aka parser. So I think you are going need to send those ios and asa logs to different ports.

Technically the parsing is happening on Elasticsearch with a feature call Ingest Pipelines.

The good things is I ran this which you can in Kibana - Dev Tools

This basically says parse this message with the correct pipeline

POST _ingest/pipeline/filebeat-7.15.2-cisco-asa-asa-ftd-pipeline/_simulate
{
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "message": """<161>Nov 23 2021 10:36:49: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.117 to 17.253.39.207 on interface inside
"""
      }
    }
  ]
}

Run it and you should see this result which looks parsed

{
  "docs" : [
    {
      "doc" : {
        "_index" : "index",
        "_type" : "_doc",
        "_id" : "id",
        "_source" : {
          "log" : {
            "level" : "alert"
          },
          "destination" : {
            "geo" : {
              "continent_name" : "Europe",
              "region_iso_code" : "SE-AB",
              "city_name" : "Kista",
              "country_iso_code" : "SE",
              "country_name" : "Sweden",
              "region_name" : "Stockholm County",
              "location" : {
                "lon" : 17.91,
                "lat" : 59.4478
              }
            },
            "as" : {
              "number" : 6185,
              "organization" : {
                "name" : "APPLE-AUSTIN"
              }
            },
            "address" : "17.253.39.207",
            "ip" : "17.253.39.207"
          },
          "syslog" : {
            "facility" : 161
          },
          "source" : {
            "address" : "10.4.240.117",
            "ip" : "10.4.240.117"
          },
          "network" : {
            "iana_number" : 6,
            "transport" : "tcp"
          },
          "observer" : {
            "ingress" : {
              "interface" : {
                "name" : "inside"
              }
            },
            "product" : "asa",
            "type" : "firewall",
            "vendor" : "Cisco"
          },
          "@timestamp" : "2021-11-23T10:36:49.000Z",
          "related" : {
            "ip" : [
              "10.4.240.117",
              "17.253.39.207"
            ]
          },
          "event" : {
            "severity" : 1,
            "ingested" : "2021-11-24T06:01:02.669694Z",
            "code" : 106021,
            "original" : "%ASA-1-106021: Deny TCP reverse path check from 10.4.240.117 to 17.253.39.207 on interface inside",
            "kind" : "event",
            "action" : "firewall-rule",
            "category" : [
              "network"
            ],
            "type" : [
              "info",
              "denied"
            ],
            "outcome" : "failure"
          },
          "cisco" : {
            "asa" : {
              "source_interface" : "inside",
              "message_id" : "106021"
            }
          }
        },
        "_ingest" : {
          "timestamp" : "2021-11-24T06:01:02.669694Z"
        }
      }
    }
  ]
}
8

I tried / simulated 2 different messages they both parse

POST _ingest/pipeline/filebeat-7.15.2-cisco-asa-asa-ftd-pipeline/_simulate
{
  "docs": [
    {
      "_index": "index",
      "_id": "id1",
      "_source": {
        "message": """<161>Nov 23 2021 10:36:49: %ASA-1-106021: Deny TCP reverse path check from 10.4.240.117 to 17.253.39.207 on interface inside
"""
      }
    },
    {
      "_index": "index",
      "_id": "id2",
      "_source": {
        "message": """<162>Nov 23 2021 10:37:39: %ASA-2-106001: Inbound TCP connection denied from 169.136.114.97/443 to 192.168.8.251/47690 flags RST  on interface outside
"""
      }
    }
  ]
}
9

Yeah, Dev Tools shows exactly the same data. -) But it doesn't make sense for me, sorry. What should I do next to get parsed messages?

10

Exactly what I said put the asa and ios on separate ports.

11

Oops double phone post

Puts the asa and ios on separate ports

12

I left ASA only and removed cisco-ios pipeline. It works -) but still miss data for couple of windows in ASA dashboard. Now I can use filters and build own graphs. Next, I'll try to change syslog to netflow and see what will happen. I appreciate you help, many thanks -)

Безымянный
Безымянный1899×864 41 KB

1 Like
13

The problem with this implementation is that it ignores the fact that a Cisco ASA device is also a Cisco IOS device, at least from a logging perspective. Not all logs from a Cisco ASA firewall start with the ASA facility. They will also send various other logs that are common to all Cisco devices.

Additionally the Filebeat modules only handle a portion of the syslog output variations that can be seen from Cisco. They seem to be based on a sampling of logs that is too small.

For example, the dissect pattern provided for %ASA-6-106100 only handles one of many common patterns for this message. Depending on the ASA software version and enabled features this log may contain to rather than -> between the source and destination. It might also include information about the user associated with the source, and when it does this may be in one of two different formats.

While these two modules provide examples of how one might approach supporting Cisco logs, most users will need to invest additional pipeline work (or hand off to Logstash) to make these fully functional in their environments.

14

Hello,
Now I'm facing another issue. I can't get cisco-ios messages correctly. I removed ASA, so filebeat is sending logs from cisco switches only. Again, I'm getting the logs but they are parced incorrectly. In Kibana I have only cisco-ios pipeline. I'm not a programmer so I can't use DEV TOOLS. I tried to change the request that you provided for asa but it didn't help.

I tried the filters and only one works:
cisco.ios.facility:*

15

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.