# Problems parsing Cisco ASA logs using filebeat **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [August 15, 2019, 10:08am UTC](https://discuss.elastic.co/t/problems-parsing-cisco-asa-logs-using-filebeat/195326 "2019-08-15T10:08:33Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![candlerb](https://avatars.discourse-cdn.com/v4/letter/c/c5a1d2/32.png) [@candlerb](https://discuss.elastic.co/u/candlerb)\ **Post date:** [August 15, 2019, 10:08am UTC](https://discuss.elastic.co/t/problems-parsing-cisco-asa-logs-using-filebeat/195326/1 "2019-08-15T10:08:34Z")
I am testing ASA syslog parsing with filebeat 7.3.0. Using tcpdump I have captured some real packets generated by a Cisco ASA (running firmware 9.6) to make this easy to reproduce. I have a trivial filebeat configuration with ```auto output.console: enabled: true codec.json: pretty: true processors: [] ``` and have done `filebeat modules enable cisco` so that the [ASA listener](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-cisco.html) is on the default port 9001. Here is an example captured message in the default output format, which doesn't have timestamp or hostname: ```auto echo -n "<163>%ASA-3-106100: access-list global_access denied tcp ToLch/10.26.1.88(56561) -> Service/10.0.0.86(135) hit-cnt 1 first hit [0xeea701c8, 0x6bb63cea]" | nc -w0 -u localhost 9001 ``` And another example generated after adding configuration `logging timestamp` and `logging device-id hostname` on the ASA: ```auto echo -n "<163>Aug 14 2019 14:19:01 ix-asa1 : %ASA-3-106100: access-list global_access denied tcp ToLch/10.26.1.88(56557) -> Service/10.0.0.86(135) hit-cnt 4 300-second interval [0xeea701c8, 0x6bb63cea]" | nc -w0 -u localhost 9001 ``` In both cases, the message is completely unparsed by filebeat and remains in the "message" element: ```auto { "@timestamp": "2019-08-15T09:46:15.085Z", "@metadata": { "beat": "filebeat", "type": "_doc", "version": "7.3.0", "truncated": false, "pipeline": "filebeat-7.3.0-cisco-asa-pipeline" }, "event": { "module": "cisco", "timezone": "+00:00", "dataset": "cisco.asa" }, "fileset": { "name": "asa" }, "service": { "type": "cisco" }, "message": "<163>Aug 14 2019 14:19:01 ix-asa1 : %ASA-3-106100: access-list global_access denied tcp ToLch/10.26.1.88(56557) -> Service/10.0.0.86(135) hit-cnt 4 300-second interval [0xeea701c8, 0x6bb63cea]", "tags": [ "cisco-asa" ], "input": { "type": "syslog" }, "agent": { "version": "7.3.0", "type": "filebeat", "ephemeral_id": "3d8a0384-1518-49d3-8802-b2d9ae896cfc", "hostname": "test1", "id": "5ebedd24-b8a6-4445-bd8a-055a026404e3" }, "ecs": { "version": "1.0.1" }, "host": { "name": "test1" } } ``` Now, I can see the first problem: the Cisco includes the year in the timestamp (not standard syslog format). Clearly filebeat could be updated to accept an optional year. But for now I'll remove it from the source message: ```auto echo -n "<163>Aug 14 14:19:01 ix-asa1 : %ASA-3-106100: access-list global_access denied tcp ToLch/10.26.1.88(56557) -> Service/10.0.0.86(135) hit-cnt 4 300-second interval [0xeea701c8, 0x6bb63cea]" | nc -w0 -u localhost 9001 ``` Result: ```auto { "@timestamp": "2019-08-14T14:19:01.000Z", "@metadata": { "beat": "filebeat", "type": "_doc", "version": "7.3.0", "truncated": false, "pipeline": "filebeat-7.3.0-cisco-asa-pipeline" }, "input": { "type": "syslog" }, "message": ": %ASA-3-106100: access-list global_access denied tcp ToLch/10.26.1.88(56557) -> Service/10.0.0.86(135) hit-cnt 4 300-second interval [0xeea701c8, 0x6bb63cea]", "log": { "source": { "address": "127.0.0.1:58569" } }, "syslog": { "priority": 163, "severity_label": "Error", "facility": 20, "facility_label": "local4" }, "event": { "severity": 3, "module": "cisco", "dataset": "cisco.asa", "timezone": "+00:00" }, "tags": [ "cisco-asa" ], "fileset": { "name": "asa" }, "service": { "type": "cisco" }, "host": { "name": "test1" }, "hostname": "ix-asa1", "ecs": { "version": "1.0.1" }, "agent": { "version": "7.3.0", "type": "filebeat", "ephemeral_id": "3d8a0384-1518-49d3-8802-b2d9ae896cfc", "hostname": "test1", "id": "5ebedd24-b8a6-4445-bd8a-055a026404e3" } } ``` Now the priority/facility, timestamp and hostname are decoded, but the rest of the message is not. According to the [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/exported-fields-cisco.html) I was expecting to see fields like `cisco.asa.message_id` Has anybody else got this to work? Looking at the [test cases](https://github.com/elastic/beats/blob/master/x-pack/filebeat/module/cisco/asa/test/asa.log) I believe that perhaps these ASA messages have been received by some other syslog daemon (rsyslog?) and reformatted before forwarding to filebeat. I certainly don't know where the tag `CiscoASA[999]:` has come from, since this isn't provided in the messages generated by the ASA I'm testing here. Now, those test cases include the year. If I send the first one of those test cases over UDP, it is not parsed: ```auto echo -n "<163>Oct 10 2018 12:34:56 localhost CiscoASA[999]: %ASA-6-305011: Built dynamic TCP translation from inside:172.31.98.44/1772 to outside:100.66.98.44/8256" | nc -w0 -u localhost 9001 ``` And even if I remove the year, it's only partially parsed: ```auto { "@timestamp": "2019-10-10T12:34:56.000Z", "@metadata": { "beat": "filebeat", "type": "_doc", "version": "7.3.0", "truncated": false, "pipeline": "filebeat-7.3.0-cisco-asa-pipeline" }, "agent": { "type": "filebeat", "ephemeral_id": "3d8a0384-1518-49d3-8802-b2d9ae896cfc", "hostname": "test1", "id": "5ebedd24-b8a6-4445-bd8a-055a026404e3", "version": "7.3.0" }, "message": "%ASA-6-305011: Built dynamic TCP translation from inside:172.31.98.44/1772 to outside:100.66.98.44/8256", "log": { "source": { "address": "127.0.0.1:34523" } }, "syslog": { "severity_label": "Error", "facility": 20, "facility_label": "local4", "priority": 163 }, "process": { "pid": 999, "program": "CiscoASA" }, "service": { "type": "cisco" }, "input": { "type": "syslog" }, "fileset": { "name": "asa" }, "hostname": "localhost", "event": { "severity": 3, "module": "cisco", "dataset": "cisco.asa", "timezone": "+00:00" }, "tags": [ "cisco-asa" ], "host": { "name": "test1" }, "ecs": { "version": "1.0.1" } } ``` According to the [expected test results](https://github.com/elastic/beats/blob/master/x-pack/filebeat/module/cisco/asa/test/asa.log-expected.json) I should have seen: ```auto { "@timestamp": "2018-10-10T12:34:56.000Z", "cisco.asa.message_id": "305011", "event.action": "firewall-rule", "event.dataset": "cisco.asa", "event.module": "cisco", "event.severity": 6, "event.timezone": "+00:00", "fileset.name": "asa", "input.type": "log", "log.level": "informational", "log.offset": 0, "log.original": "Built dynamic TCP translation from inside:172.31.98.44/1772 to outside:100.66.98.44/8256", "service.type": "cisco", "tags": [ "cisco-asa" ] }, ``` So clearly something is going wrong here, but I don't know what it is. Thanks for any clues! ---
**Author:** ![candlerb](https://avatars.discourse-cdn.com/v4/letter/c/c5a1d2/32.png) [@candlerb](https://discuss.elastic.co/u/candlerb)\ **Post date:** [August 15, 2019, 10:16am UTC](https://discuss.elastic.co/t/problems-parsing-cisco-asa-logs-using-filebeat/195326/2 "2019-08-15T10:16:25Z")
(Aside: notice also that `log.source.address` is missing in the first example I posted, where the timestamp could not be parsed. I think that's a bug) ---
**Author:** ![candlerb](https://avatars.discourse-cdn.com/v4/letter/c/c5a1d2/32.png) [@candlerb](https://discuss.elastic.co/u/candlerb)\ **Post date:** [August 15, 2019, 11:23am UTC](https://discuss.elastic.co/t/problems-parsing-cisco-asa-logs-using-filebeat/195326/3 "2019-08-15T11:23:25Z")
I think I solved it. The ASA "module" in filebeat writes this [ingest pipeline](https://github.com/elastic/beats/blob/master/x-pack/filebeat/module/cisco/asa/ingest/pipeline.yml) to Elasticsearch. That uses the [grok processor](https://www.elastic.co/guide/en/elasticsearch/reference/master/grok-processor.html) along with the [date processor](https://www.elastic.co/guide/en/elasticsearch/reference/master/date-processor.html) to handle a range of date formats, together with the [dissect processor](https://www.elastic.co/guide/en/elasticsearch/reference/master/dissect-processor.html) to match various log lines and parse fields out of them. All this means that messages are only parsed if and when they are written into Elasticsearch. ---
**Author:** ![Joginders](https://avatars.discourse-cdn.com/v4/letter/j/73ab20/32.png) [@Joginders](https://discuss.elastic.co/u/Joginders)\ **Post date:** [August 29, 2019, 10:53am UTC](https://discuss.elastic.co/t/problems-parsing-cisco-asa-logs-using-filebeat/195326/4 "2019-08-29T10:53:28Z")
Can you help me the solution you applied? ---
**Author:** ![candlerb](https://avatars.discourse-cdn.com/v4/letter/c/c5a1d2/32.png) [@candlerb](https://discuss.elastic.co/u/candlerb)\ **Post date:** [August 29, 2019, 11:23am UTC](https://discuss.elastic.co/t/problems-parsing-cisco-asa-logs-using-filebeat/195326/5 "2019-08-29T11:23:59Z")
Well, I was trying to use filebeat on its own to parse ASA logs, but it doesn't. The solution, in principle, is to write the logs to elasticsearch and read them out of there. This requires you to use the filebeat ASA "module" to [upload the pipeline definition](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-modules-quickstart.html#load-ingest-pipelines) to elasticsearch. It also requires the [pipeline to be selected](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html) for each document written; filebeat can do this if it's writing directly to elasticsearch, but if you're going via logstash or fluentd you need to configure them to [add the pipeline parameter](https://www.elastic.co/guide/en/logstash/current/use-ingest-pipelines.html) to their writes. All of this is documented, but I've not tested it. ---
**Author:** ![Joginders](https://avatars.discourse-cdn.com/v4/letter/j/73ab20/32.png) [@Joginders](https://discuss.elastic.co/u/Joginders)\ **Post date:** [August 29, 2019, 11:45am UTC](https://discuss.elastic.co/t/problems-parsing-cisco-asa-logs-using-filebeat/195326/6 "2019-08-29T11:45:35Z")
I am using the built in pipeline for Cisco ASA Module in filebeat to write logs into elasticsearch. But the problem is similar, I am not getting Cisco ASA's hostname in Kibana or Elasticsearch. If you could help me in that. ---
**Author:** ![candlerb](https://avatars.discourse-cdn.com/v4/letter/c/c5a1d2/32.png) [@candlerb](https://discuss.elastic.co/u/candlerb)\ **Post date:** [August 29, 2019, 12:13pm UTC](https://discuss.elastic.co/t/problems-parsing-cisco-asa-logs-using-filebeat/195326/7 "2019-08-29T12:13:09Z")
Like I said I haven't built this myself. I suggest you look at: - the raw packets coming into filebeat (tcpdump is good for this) - the HTTP requests being written into elasticsearch (tcpdump is also helpful here, unless you've enabled SSL encryption). Check they invoke the pipeline. - the documents after they are stored in elasticsearch - the functioning of the ingest pipeline - see [https://www.elastic.co/guide/en/elasticsearch/reference/master/simulate-pipeline-api.html](https://www.elastic.co/guide/en/elasticsearch/reference/master/simulate-pipeline-api.html) Then you can work out where the problem is. ---
**Author:** ![Lantsel](https://avatars.discourse-cdn.com/v4/letter/l/c77e96/32.png) [@Lantsel](https://discuss.elastic.co/u/Lantsel)\ **Post date:** [September 4, 2019, 3:00pm UTC](https://discuss.elastic.co/t/problems-parsing-cisco-asa-logs-using-filebeat/195326/8 "2019-09-04T15:00:40Z")
Same problem, anyone have a fix? I am working on a fix as well. ---
**Author:** ![candlerb](https://avatars.discourse-cdn.com/v4/letter/c/c5a1d2/32.png) [@candlerb](https://discuss.elastic.co/u/candlerb)\ **Post date:** [September 4, 2019, 3:54pm UTC](https://discuss.elastic.co/t/problems-parsing-cisco-asa-logs-using-filebeat/195326/9 "2019-09-04T15:54:35Z")
Something to beware of is that ASAs can generate syslogs in a variety of formats: with and without timestamp, with and without hostname. There are configuration options which can control this: ```auto logging timestamp logging device-id hostname ``` Filebeat does not like [messages without hostname](https://github.com/elastic/beats/issues/13270) (which is non-standard anyway) If you set those two options you should get something that approximates standard syslog format, although I found I got a spurious colon by itself. ```auto <164>Sep 04 2019 16:50:52 ldex-asa1 : %ASA-4-402116: IPSEC: Received an ESP packet... ``` IOS is [worse](https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3750x_3560x/software/release/12-2_55_se/configuration/guide/3750xscg/swlog.html): messages can also be prefixed by sequence numbers (`[no] service sequence-numbers`) Since the parsing of messages is split between filebeat and ES pipelines, you'll have to check how filebeat does the initial breakdown of the message into JSON, and what fields the ES pipeline expects in its input. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 2, 2019, 3:54pm UTC](https://discuss.elastic.co/t/problems-parsing-cisco-asa-logs-using-filebeat/195326/10 "2019-10-02T15:54:36Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.