Converting Cisco Module

Elastic Stack Beats
1

I'm learning to use filebeat, I was wondering if there is a way to convert the ingest pipeline from elasticsearch to logstash config

I've had a look here
https://www.elastic.co/guide/en/logstash/current/ingest-converter.html
Can't really find a json for Cisco module in filebeat

Should I be able to copy this to a logstash conf?

curl -XGET localhost:9200/_ingest/pipeline?pretty
filebeat-7.4.2-cisco-ftd-asa-ftd-pipeline" : {
    "description" : "Pipeline for Cisco FTD logs",
..truncated

I have also tried to send output directly to elasticsearch but for some reason the timestamp is in the future.

Running 7.4.2 on Ubuntu 18.04

Thanks

Is there a reason why I can't install the Cisco ASA dashboard when I install through the linux filebeat from the server

It only works if i Install from a windows version of filebeat???

2

Hi @VamPikmin,

I don't know much about the ingest converter, but in principle it should be possible to use the ingest pipeline you are getting with curl. You cannot find the json pipeline in the filebeat repository because some pipelines, including the Cisco ones are developed in YAML format, for readability and improved maintainability.

If you are not using UTC this is probably related to an issue with timezones we had in some modules. This will be solved in 7.5.0 (see Some filebeat modules incorrectly parse timestamps · Issue #13877 · elastic/beats · GitHub).

It should work the same with Linux or Windows following these instructions.

3

Hi @jsoriano

Thanks for your help, I'm not using UTC so that makes sense now, will look into it
08:39:48.249 AEDT Fri Nov 8 2019

Once I've used the -e switch in filebeat I saw why it wasn't being installed

2019-11-08T09:25:52.456+1100 ERROR fileset/modules.go:125 Not loading modules. Module directory not found: /usr/share/filebeat/bin/module

Once I added --path.home /usr/share/filebeat/ it installed okay

It seems ingest converter doesn't support painless script and I think I see it in the cisco asa pipeline

Thanks for the explanation about the pipelines, do you have any suggestions on how to proceed

4

I think that you could try to remove the unsupported parts of the pipeline and manually migrate them after the conversion.

5

Thanks @jsoriano

I've also tried to test the filebeat IIS module and almost every single log was not being parsed correctly, so I've decided to go with logstash again and create my own patterns.

I was hoping it would just work since it's a Windows server 2012 but things are never simple

6

There are some issues reported for the IIS module for Filebeat, including one (#13799) about supporting more log formats. It'd be great if you could provide there some of example lines of the logs you see that are not being correctly parsed. Thanks!

7

Thanks for all your help @jsoriano
There's some great examples there that will come in handy

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.