CISCO FTD Integration

Elastic Stack Kibana
1

Thank you fore reading my message.

Situation:
After installing Cisco FTD Integration with Fleet Manager, it seems that the Dataset is not created.

  • Looking in Fleet -> Data streams, searching for cisco* and nothing is shown.

Any thoughts in how to fix this issue?

Background:
The Elastic Agent gets installed on the Ingestion server. Purpose of this ingestion server is to receive logs from a CISCO FTD firewall device via UDP protocol Port say 9003 or 514 and then send it to Elasticsearch.

Done so far:
Attempted the Integration on two different Windows server without success.

Assessment:
The Ingestion server is a Windows server and its Firewall settings have the ports open for inbound/outbound UDP port 514 (also tried 9003).

The Elastic Agent installation gets installed on the ingestion server during the Integration and enrolling to Fleet Manager with an Enrollment Token without errors.

The Fleet manager got enrolled to Elasticsearch Cluster with CA certificates (We have a valid CA); this got completed without any errors.

Fleet shows a healthy Integration with Agent policy in good standing nonetheless.

The Fleet Manager server is managed with Kibana on same version level as Elasticsearch.

Therefore can not get to the point to try Discovering data since No Dataset is created for such Data streams (Cisco FTD). In addition, checking on Index Management, No Data Stream Nor Indices for Cisco* (enabled hidden) are shown.

Even so other integrations have been installed on this Fleet Manager Server such as Crowdstrike, AbuseCH and WindowsOS and are working well; Datasets are visible in these cases.

Elasticsearch environment here:

  • ELK version: 8.13.2
  • The account used in Kibana is a super user with all rights to the cluster.

Next Steps:

  • Thank you very much for your inputs in trying to fix this issue.

-Cheers

2

Hello,

Can you share the configuration you are using for the integration?

3

ciscoftd-intgr-f1
ciscoftd-intgr-f1862×1052 84.2 KB

ciscoftd-intgr-f1b
ciscoftd-intgr-f1b862×1052 84.5 KB

Ciscoftd-integr-policy-1
Ciscoftd-integr-policy-11085×260 24.4 KB

ciscoftd-fleet-agents-f1
ciscoftd-fleet-agents-f11077×417 38.4 KB

Agent Policy:
#Note: I've maked all IDs and domain names below.
#Agent Policy (NetworkDevices1) assigned to Ingestion server (stptest2)

id:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx            
revision: 2
outputs:
  default:
    type: elasticsearch
    hosts:
      - 'https://xx.xx.xx.xxx:9200'
    ssl.ca_trusted_fingerprint:                                                                 
    preset: balanced
fleet:
  hosts:
    - 'https://<fleetserverhost>.<domainname>.<domain>.<domain>:8220'
output_permissions:
  default:
    _elastic_agent_monitoring:
      indices:
        - names:
            - logs-elastic_agent.apm_server-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.apm_server-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.auditbeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.auditbeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.cloud_defend-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.cloudbeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.cloudbeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.elastic_agent-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.endpoint_security-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.endpoint_security-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.filebeat_input-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.filebeat_input-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.filebeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.filebeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.fleet_server-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.fleet_server-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.heartbeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.heartbeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.metricbeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.metricbeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.osquerybeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.osquerybeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.packetbeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.packetbeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.pf_elastic_collector-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.pf_elastic_symbolizer-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.pf_host_agent-default
          privileges:
            - auto_configure
            - create_doc
    _elastic_agent_checks:
      cluster:
        - monitor
    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx:
      indices:
        - names:
            - logs-system.auth-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-system.syslog-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-system.application-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-system.security-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-system.system-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.cpu-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.diskio-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.filesystem-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.fsstat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.load-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.memory-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.network-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.process-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.process.summary-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.socket_summary-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.uptime-default
          privileges:
            - auto_configure
            - create_doc
    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx:
      indices:
        - names:
            - logs-cisco_ftd.log-default
          privileges:
            - auto_configure
            - create_doc
agent:
  download:
    sourceURI: 'https://artifacts.elastic.co/downloads/'
  monitoring:
    enabled: true
    use_output: default
    namespace: default
    logs: true
    metrics: true
  features: {}
  protection:
    enabled: false
    uninstall_token_hash: /<xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>
    signing_key: >-
      xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
inputs:
  - id: logfile-system-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    name: system-6
    revision: 1
    type: logfile
    use_output: default
    meta:
      package:
        name: system
        version: 1.58.1
    data_stream:
      namespace: default
    package_policy_id: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    streams:
      - id: logfile-system.auth-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        data_stream:
          dataset: system.auth
          type: logs
        ignore_older: 72h
        paths:
          - /var/log/auth.log*
          - /var/log/secure*
        exclude_files:
          - \.gz$
        multiline:
          pattern: ^\s
          match: after
        tags:
          - system-auth
        processors:
          - add_locale: null
          - rename:
              fields:
                - from: message
                  to: event.original
              ignore_missing: true
              fail_on_error: false
          - syslog:
              field: event.original
              ignore_missing: true
              ignore_failure: true
      - id: logfile-system.syslog-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        data_stream:
          dataset: system.syslog
          type: logs
        paths:
          - /var/log/messages*
          - /var/log/syslog*
          - /var/log/system*
        exclude_files:
          - \.gz$
        multiline:
          pattern: ^\s
          match: after
        processors:
          - add_locale: null
        tags: null
        ignore_older: 72h
  - id: winlog-system-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    name: system-6
    revision: 1
    type: winlog
    use_output: default
    meta:
      package:
        name: system
        version: 1.58.1
    data_stream:
      namespace: default
    package_policy_id: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    streams:
      - id: winlog-system.application-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        name: Application
        data_stream:
          dataset: system.application
          type: logs
        condition: '${host.platform} == ''windows'''
        ignore_older: 72h
      - id: winlog-system.security-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        name: Security
        data_stream:
          dataset: system.security
          type: logs
        condition: '${host.platform} == ''windows'''
        ignore_older: 72h
      - id: winlog-system.system-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        name: System
        data_stream:
          dataset: system.system
          type: logs
        condition: '${host.platform} == ''windows'''
        ignore_older: 72h
  - id: system/metrics-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    name: system-6
    revision: 1
    type: system/metrics
    use_output: default
    meta:
      package:
        name: system
        version: 1.58.1
    data_stream:
      namespace: default
    package_policy_id: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    streams:
      - id: system/metrics-system.cpu-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        data_stream:
          dataset: system.cpu
          type: metrics
        metricsets:
          - cpu
        cpu.metrics:
          - percentages
          - normalized_percentages
        period: 10s
      - id: system/metrics-system.diskio-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        data_stream:
          dataset: system.diskio
          type: metrics
        metricsets:
          - diskio
        diskio.include_devices: null
        period: 10s
      - id: system/metrics-system.filesystem-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        data_stream:
          dataset: system.filesystem
          type: metrics
        metricsets:
          - filesystem
        period: 1m
        processors:
          - drop_event.when.regexp:
              system.filesystem.mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)
      - id: system/metrics-system.fsstat-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        data_stream:
          dataset: system.fsstat
          type: metrics
        metricsets:
          - fsstat
        period: 1m
        processors:
          - drop_event.when.regexp:
              system.fsstat.mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)
      - id: system/metrics-system.load-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        data_stream:
          dataset: system.load
          type: metrics
        metricsets:
          - load
        condition: '${host.platform} != ''windows'''
        period: 10s
      - id: system/metrics-system.memory-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        data_stream:
          dataset: system.memory
          type: metrics
        metricsets:
          - memory
        period: 10s
      - id: system/metrics-system.network-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        data_stream:
          dataset: system.network
          type: metrics
        metricsets:
          - network
        period: 10s
        network.interfaces: null
      - id: system/metrics-system.process-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        data_stream:
          dataset: system.process
          type: metrics
        metricsets:
          - process
        period: 10s
        process.include_top_n.by_cpu: 5
        process.include_top_n.by_memory: 5
        process.cmdline.cache.enabled: true
        process.cgroups.enabled: false
        process.include_cpu_ticks: false
        processes:
          - .*
      - id: >-
          system/metrics-system.process.summary-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        data_stream:
          dataset: system.process.summary
          type: metrics
        metricsets:
          - process_summary
        period: 10s
      - id: >-
          system/metrics-system.socket_summary-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        data_stream:
          dataset: system.socket_summary
          type: metrics
        metricsets:
          - socket_summary
        period: 10s
      - id: system/metrics-system.uptime-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        data_stream:
          dataset: system.uptime
          type: metrics
        metricsets:
          - uptime
        period: 10s
  - id: udp-cisco_ftd-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    name: cisco_ftd-1
    revision: 1
    type: udp
    use_output: default
    meta:
      package:
        name: cisco_ftd
        version: 3.2.5
    data_stream:
      namespace: default
    package_policy_id: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    streams:
      - id: udp-cisco_ftd.log-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        data_stream:
          dataset: cisco_ftd.log
          type: logs
        host: 'xx.xxx.xx.xxx:514'                 #My note: IP address here is for the stptest2 server
        tags:
          - private_is_internal
          - cisco-ftd
          - forwarded
        publisher_pipeline.disable_host: true
        fields_under_root: true
        fields:
          _conf:
            tz_offset: UTC
        processors:
          - add_locale: null
signed:
  data: >-
    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
  signature: >-
    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
secret_references: []

#End of document.

Thanks for your inputs.

NOTE Stephen Formatted code, please format code in the future

4

Hello, I've replied with some screenshots for such configuration. Not sure why the images are not posted just yet.

5

ciscoftd-fleet-agents-f1
ciscoftd-fleet-agents-f11077×417 38.4 KB

ciscoftd-fleet-agents-f1
ciscoftd-fleet-agents-f11077×417 38.4 KB

Ciscoftd-integr-policy-1
Ciscoftd-integr-policy-11085×260 24.4 KB

ciscoftd-intgr-f1
ciscoftd-intgr-f1862×1052 84.2 KB

I hope these images help.

Thanks

6

Agent-Policy-1
Agent-Policy-1586×1175 18.4 KB

Agent-Policy-7-out-of-8
Agent-Policy-7-out-of-8654×1281 39.5 KB

Agent-policy-8-of-8
Agent-policy-8-of-8874×377 12.6 KB

Agent-policy-4-of-8
Agent-policy-4-of-8749×973 23.9 KB

So there is a sequence of 8 files (1 to 8) for that, listing here the more relevant for this particular integration. I hope that is Ok. If at all needed the rest of them let me know. Thanks.

7

Same here. Still don't see this integration to be working.

8

I did a Wireshark analysis from the target server. No UDP traffic is observed coming from the FTD device. So, I don’t know if is just such device not sending udp traffic at all or what else.

If you do a wireshark analysis on your target host, do you see the udp traffic coming from the Cisco FTD device?

Thanks,

Cheers

9

I used tcpdump on port that supposed to be receiving udp packets from FTD. Traffic was incoming non stop. But the agent is still not creating data on elastic.
Thank you

10

In my case, using Kibana, Fleet menu option, I installed Fleet manager on a separate server and got it enrolled to elasticsearch using our CA certificates. Agent policies were created in such process; default settings. The Settings Tab here show Elasticsearch connections settings and trusted fingerprints stuff.

Then as part of that process, installed the Elastic Agent on another sever and got it enrolled to the Fleet manager with its enrollment token.

Then, In Kibana, Created integrations such as Crowdstrike which are ingesting logs to elasticsearch.

In your case, Are other integrations sending logs to Elasticsearch?

Check firewall ports on the Fleet manager to have ports open inbound/outbound #8220 (windows) or the port number in your case. Not sure if you are using Windows or Linux.

In the Elasticagent server open ports 514 or your port number inbound outbound Cisco FTD.

Cheers

11

I have around 20 agents running collecting logs and sending them to the Fleet server. The only issue is with FTD logs at the moment. The same agent has more than 3 integrations running, and all of them are providing logs at elk. Firewall and ports are configured (as mentioned tcpdump successfully captured UDP traffic incoming from FTD at agent). I dont see the integration to be working at all.

12

I would recommend that you open a different topic and share how your integration is configured, if you are using standalone or fleet managed, and some logs you may have.

Your issue may be completely different from this one.