Cisco IOS Filebeat Indexing Error

Elastic Stack Beats
1

I am saving logs to a file via syslog-ng (for other reasons). Filebeat is using that file as a source for cisco-ios logs. I am using the official Cisco module but when I am loading cisco-ios logs, I am getting a parsing error.

2020 - 11 - 12T14: 43: 29.949 - 0600 WARN[elasticsearch]elasticsearch / client.go: 408 Cannot index event publisher.Event {
    Content: beat.Event {
        Timestamp: time.Time {
            wall: 0xbfe3857ba37168e6,
            ext: 161536153122,
            loc: ( * time.Location)(0x60807a0)
        },
        Meta: {
            "pipeline": "filebeat-7.9.3-cisco-ios-pipeline"
        },
        Fields: {
            "_tmp": {
                "timestamp": "10.246.127.240",
                "ts": "Nov 12 14:43:25"
            },
            "agent": {
                "ephemeral_id": "f55cccec-a26d-4aef-8ff4-1eddad041206",
                "hostname": "chi-bcs-cibc-slg-01.burwoodcloud.com",
                "id": "56633e39-6e52-405e-b0ac-cf2e50604d0c",
                "name": "chi-bcs-cibc-slg-01.burwoodcloud.com",
                "type": "filebeat",
                "version": "7.9.3"
            },
            "ecs": {
                "version": "1.5.0"
            },
            "error": {
                "message": "GoError: failed in processor.convert: conversion of field [event.sequence] to type [long] failed: unable to convert value [\u003c165\u003e47948]: strconv.ParseInt: parsing \"\u003c165\u003e47948\": invalid syntax"
            },
            "event": {
                "dataset": "cisco.ios",
                "module": "cisco",
                "sequence": "\u003c165\u003e47948",
                "timezone": "-06:00"
            },
            "fileset": {
                "name": "ios"
            },
            "input": {
                "type": "log"
            },
            "log": {
                "file": {
                    "path": "/var/log/network/cisco-ios"
                },
                "offset": 752225,
                "original": "Nov 12 14:43:25 10.246.127.240 \u003c165\u003e47948: 10.246.127.240: ]: Nov 12 2020 20:43:26.052 UTC: %PARSER-5-CFGLOG_LOGGEDCMD: User:SVC-NETBR-RONetAdm2  logged command:!exec: enable",
                "source": {
                    "address": "10.246.127.240"
                }
            },
            "message": "]: Nov 12 2020 20:43:26.052 UTC: %PARSER-5-CFGLOG_LOGGEDCMD: User:SVC-NETBR-RONetAdm2  logged command:!exec: enable",
            "service": {
                "type": "cisco"
            },
            "tags": ["cisco-ios", "forwarded", "_js_exception"]
        },
        Private: file.State {
            Id: "native::1193-64770",
            PrevId: "",
            Finished: false,
            Fileinfo: ( * os.fileStat)(0xc0003cad00),
            Source: "/var/log/network/cisco-ios",
            Offset: 752400,
            Timestamp: time.Time {
                wall: 0xbfe3857ba356977d,
                ext: 161534395584,
                loc: ( * time.Location)(0x60807a0)
            },
            TTL: -1,
            Type: "log",
            Meta: map[string]string(nil),
            FileStateOS: file.StateOS {
                Inode: 0x4a9,
                Device: 0xfd02
            },
            IdentifierName: "native"
        },
        TimeSeries: false
    },
    Flags: 0x1,
    Cache: publisher.EventCache {
        m: common.MapStr(nil)
    }
}
(status = 400): {
    "type": "mapper_parsing_exception",
    "reason": "failed to parse field [event.sequence] of type [long] in document with id 'meQyvnUBRBx9xBvX7YEQ'. Preview of field's value: '<165>47948'",
    "caused_by": {
        "type": "illegal_argument_exception",
        "reason": "For input string: \"<165>47948\""
    }
}

From a packet capture, I am receiving the angle brackets as well. Is this something weird with pulling the logs from the file?

Here is the packet capture of the raw logs.

<189>73: *Nov 12 03:56:54.184: %SYS-5-CONFIG_I: Configured from console by admin on vty1 (172.20.250.44)
<189>74: *Nov 12 03:57:17.429: %BGP_SESSION-5-ADJCHANGE: Message from tty2(user id: admin): "LINE"
<189>75: *Nov 12 03:57:23.735: %BGP_SESSION-5-ADJCHANGE: Message from tty2(user id: admin): "LINE"

Has anyone else experienced this or found a workaround?

Thank you!

2

That format looks a little different that what it was expecting when reading from a log file. Yours has the original syslog pri and some additional number at the start. This is what it expects.

The good news is that you can edit that dissect pattern in your Filebeat install and it should work.

I think the right tokenizer for the format your logs are in would be

tokenizer: "<%{_tmp.pri}>%{event.sequence}: *%{_tmp.timestamp->} %{+_tmp.timestamp} %{+_tmp.timestamp->}: %{_tmp.message}"

3

Thank you. That definitely fixed it. The format was a bit different from the wireshark that was written to disk.

I added this (just changed where the placeholder for the priority was):
tokenizer: "%{_tmp.ts->} %{+_tmp.ts} %{+_tmp.ts->} %{log.source.address} <%{_tmp.pri}>%{event.sequence}: %{_tmp.timestamp}: %{_tmp.message}"

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.