Hi Community,
I am trying to integrate Symantec endpoint protection with elastic stack but not able to fetch Symantec data into elastic.
I had tried with log input file option which elastic agent read the file in the given path send data to logstash and from there sending data to elasticstack.
Kindly help me why elastic agent is not able to read given path in the yml file.
I am sharing elastic agent yml file and logstash configuration file.
Regards,
Eshwar
id: e15dd11b-a973-46f0-8567-9039178281f6
revision: 2
outputs:
default:
type: logstash
hosts:
- 'xx.xx.xx.xx:5044'
output_permissions:
default:
_elastic_agent_monitoring:
indices:
- names:
- logs-elastic_agent.apm_server-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.apm_server-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.auditbeat-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.auditbeat-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.cloud_defend-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.cloudbeat-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.cloudbeat-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.elastic_agent-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.endpoint_security-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.endpoint_security-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.filebeat_input-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.filebeat_input-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.filebeat-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.filebeat-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.fleet_server-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.fleet_server-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.heartbeat-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.heartbeat-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.metricbeat-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.metricbeat-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.osquerybeat-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.osquerybeat-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.packetbeat-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.packetbeat-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.pf_elastic_collector-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.pf_elastic_symbolizer-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.pf_host_agent-default
privileges:
- auto_configure
- create_doc
_elastic_agent_checks:
cluster:
- monitor
ae0f472b-a70a-4ac0-ade3-fea21617109a:
indices:
- names:
- logs-system.auth-default
privileges:
- auto_configure
- create_doc
- names:
- logs-system.syslog-default
privileges:
- auto_configure
- create_doc
- names:
- logs-system.auth-default
privileges:
- auto_configure
- create_doc
- names:
- logs-system.syslog-default
privileges:
- auto_configure
- create_doc
- names:
- logs-system.application-default
privileges:
- auto_configure
- create_doc
- names:
- logs-system.security-default
privileges:
- auto_configure
- create_doc
- names:
- logs-system.system-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.cpu-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.diskio-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.filesystem-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.fsstat-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.load-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.memory-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.network-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.process-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.process.summary-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.socket_summary-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.uptime-default
privileges:
- auto_configure
- create_doc
ec9e6be5-6271-4465-8314-5297d4bfcd1a:
indices:
- names:
- logs-symantec_endpoint.log-default
privileges:
- auto_configure
- create_doc
agent:
download:
sourceURI: 'https://artifacts.elastic.co/downloads/'
monitoring:
enabled: true
use_output: default
logs: true
metrics: true
traces: true
namespace: default
features: {}
protection:
enabled: false
uninstall_token_hash: 1OEbBROjsrHKsGr5rC2gTWFZ869/EqmJiBFl0RM9Ktg=
signing_key: >-
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE/HJJt/XCOLouHvP9vNdzr2Z616O6F23g4elIK8VxaNdCGAKDfEBxC6S4/0SzwHIL7PqzNPsEw0Z2VBhPAGdQ+Q==
inputs:
- id: logfile-system-ae0f472b-a70a-4ac0-ade3-fea21617109a
name: system-1
revision: 1
type: logfile
use_output: default
meta:
package:
name: system
version: 1.63.1
data_stream:
namespace: default
package_policy_id: ae0f472b-a70a-4ac0-ade3-fea21617109a
streams:
- id: logfile-system.auth-ae0f472b-a70a-4ac0-ade3-fea21617109a
data_stream:
dataset: system.auth
type: logs
condition: >-
${host.os_version} != "12 (bookworm)" and (${host.os_platform} !=
"amzn" or ${host.os_version} != "2023")
ignore_older: 72h
paths:
- /var/log/auth.log*
- /var/log/secure*
exclude_files:
- \.gz$
multiline:
pattern: ^\s
match: after
tags:
- system-auth
processors:
- add_locale: null
- rename:
fields:
- from: message
to: event.original
ignore_missing: true
fail_on_error: false
- syslog:
field: event.original
ignore_missing: true
ignore_failure: true
- id: logfile-system.syslog-ae0f472b-a70a-4ac0-ade3-fea21617109a
data_stream:
dataset: system.syslog
type: logs
condition: >-
${host.os_version} != "12 (bookworm)" and (${host.os_platform} !=
"amzn" or ${host.os_version} != "2023")
paths:
- /var/log/messages*
- /var/log/syslog*
- /var/log/system*
exclude_files:
- \.gz$
multiline:
pattern: ^\s
match: after
processors:
- add_locale: null
tags: null
ignore_older: 72h
- id: journald-system-ae0f472b-a70a-4ac0-ade3-fea21617109a
name: system-1
revision: 1
type: journald
use_output: default
meta:
package:
name: system
version: 1.63.1
data_stream:
namespace: default
package_policy_id: ae0f472b-a70a-4ac0-ade3-fea21617109a
streams:
- id: journald-system.auth-ae0f472b-a70a-4ac0-ade3-fea21617109a
type: journald
data_stream:
dataset: system.auth
type: logs
facilities:
- 4
- 10
condition: >-
${host.os_version} == "12 (bookworm)" or (${host.os_platform} ==
"amzn" and ${host.os_version} == "2023")
tags: null
- id: journald-system.syslog-ae0f472b-a70a-4ac0-ade3-fea21617109a
type: journald
data_stream:
dataset: system.syslog
type: logs
facilities:
- 0
- 1
- 2
- 3
- 5
- 6
- 7
- 8
- 9
- 11
- 12
- 15
condition: >-
${host.os_version} == "12 (bookworm)" or (${host.os_platform} ==
"amzn" and ${host.os_version} == "2023")
tags: null
- id: winlog-system-ae0f472b-a70a-4ac0-ade3-fea21617109a
name: system-1
revision: 1
type: winlog
use_output: default
meta:
package:
name: system
version: 1.63.1
data_stream:
namespace: default
package_policy_id: ae0f472b-a70a-4ac0-ade3-fea21617109a
streams:
- id: winlog-system.application-ae0f472b-a70a-4ac0-ade3-fea21617109a
name: Application
data_stream:
dataset: system.application
type: logs
condition: '${host.platform} == ''windows'''
ignore_older: 72h
- id: winlog-system.security-ae0f472b-a70a-4ac0-ade3-fea21617109a
name: Security
data_stream:
dataset: system.security
type: logs
condition: '${host.platform} == ''windows'''
ignore_older: 72h
- id: winlog-system.system-ae0f472b-a70a-4ac0-ade3-fea21617109a
name: System
data_stream:
dataset: system.system
type: logs
condition: '${host.platform} == ''windows'''
ignore_older: 72h
- id: system/metrics-system-ae0f472b-a70a-4ac0-ade3-fea21617109a
name: system-1
revision: 1
type: system/metrics
use_output: default
meta:
package:
name: system
version: 1.63.1
data_stream:
namespace: default
package_policy_id: ae0f472b-a70a-4ac0-ade3-fea21617109a
streams:
- id: system/metrics-system.cpu-ae0f472b-a70a-4ac0-ade3-fea21617109a
data_stream:
dataset: system.cpu
type: metrics
metricsets:
- cpu
cpu.metrics:
- percentages
- normalized_percentages
period: 10s
- id: system/metrics-system.diskio-ae0f472b-a70a-4ac0-ade3-fea21617109a
data_stream:
dataset: system.diskio
type: metrics
metricsets:
- diskio
diskio.include_devices: null
period: 10s
- id: system/metrics-system.filesystem-ae0f472b-a70a-4ac0-ade3-fea21617109a
data_stream:
dataset: system.filesystem
type: metrics
metricsets:
- filesystem
period: 1m
processors:
- drop_event.when.regexp:
system.filesystem.mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)
- id: system/metrics-system.fsstat-ae0f472b-a70a-4ac0-ade3-fea21617109a
data_stream:
dataset: system.fsstat
type: metrics
metricsets:
- fsstat
period: 1m
processors:
- drop_event.when.regexp:
system.fsstat.mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)
- id: system/metrics-system.load-ae0f472b-a70a-4ac0-ade3-fea21617109a
data_stream:
dataset: system.load
type: metrics
metricsets:
- load
condition: '${host.platform} != ''windows'''
period: 10s
- id: system/metrics-system.memory-ae0f472b-a70a-4ac0-ade3-fea21617109a
data_stream:
dataset: system.memory
type: metrics
metricsets:
- memory
period: 10s
- id: system/metrics-system.network-ae0f472b-a70a-4ac0-ade3-fea21617109a
data_stream:
dataset: system.network
type: metrics
metricsets:
- network
period: 10s
network.interfaces: null
- id: system/metrics-system.process-ae0f472b-a70a-4ac0-ade3-fea21617109a
data_stream:
dataset: system.process
type: metrics
metricsets:
- process
period: 10s
process.include_top_n.by_cpu: 5
process.include_top_n.by_memory: 5
process.cmdline.cache.enabled: true
process.cgroups.enabled: false
process.include_cpu_ticks: false
processes:
- .*
- id: >-
system/metrics-system.process.summary-ae0f472b-a70a-4ac0-ade3-fea21617109a
data_stream:
dataset: system.process.summary
type: metrics
metricsets:
- process_summary
period: 10s
- id: >-
system/metrics-system.socket_summary-ae0f472b-a70a-4ac0-ade3-fea21617109a
data_stream:
dataset: system.socket_summary
type: metrics
metricsets:
- socket_summary
period: 10s
- id: system/metrics-system.uptime-ae0f472b-a70a-4ac0-ade3-fea21617109a
data_stream:
dataset: system.uptime
type: metrics
metricsets:
- uptime
period: 10s
- id: logfile-symantec-ec9e6be5-6271-4465-8314-5297d4bfcd1a
name: symantec_endpoint-1
revision: 1
type: logfile
use_output: default
meta:
package:
name: symantec_endpoint
version: 2.18.0
data_stream:
namespace: default
package_policy_id: ec9e6be5-6271-4465-8314-5297d4bfcd1a
streams:
- id: logfile-symantec_endpoint.log-ec9e6be5-6271-4465-8314-5297d4bfcd1a
data_stream:
dataset: symantec_endpoint.log
type: logs
paths:
- >-
**C:\Program Files (x86)\Symantec\Symantec Endpoint Protection**
** Manager\data\dump\*.log**
exclude_files:
- \.gz$
tags:
- symantec-endpoint-log
- forwarded
publisher_pipeline.disable_host: true
fields_under_root: true
fields:
_conf:
tz_offset: UTC
remove_mapped_fields: false
signed:
data: >-
eyJpZCI6ImUxNWRkMTFiLWE5NzMtNDZmMC04NTY3LTkwMzkxNzgyODFmNiIsImFnZW50Ijp7ImZlYXR1cmVzIjp7fSwicHJvdGVjdGlvbiI6eyJlbmFibGVkIjpmYWxzZSwidW5pbnN0YWxsX3Rva2VuX2hhc2giOiIxT0ViQlJPanNySEtzR3I1ckMyZ1RXRlo4NjkvRXFtSmlCRmwwUk05S3RnPSIsInNpZ25pbmdfa2V5IjoiTUZrd0V3WUhLb1pJemowQ0FRWUlLb1pJemowREFRY0RRZ0FFL0hKSnQvWENPTG91SHZQOXZOZHpyMlo2MTZPNkYyM2c0ZWxJSzhWeGFOZENHQUtEZkVCeEM2UzQvMFN6d0hJTDdQcXpOUHNFdzBaMlZCaFBBR2RRK1E9PSJ9fSwiaW5wdXRzIjpbeyJpZCI6ImxvZ2ZpbGUtc3lzdGVtLWFlMGY0NzJiLWE3MGEtNGFjMC1hZGUzLWZlYTIxNjE3MTA5YSIsIm5hbWUiOiJzeXN0ZW0tMSIsInJldmlzaW9uIjoxLCJ0eXBlIjoibG9nZmlsZSJ9LHsiaWQiOiJqb3VybmFsZC1zeXN0ZW0tYWUwZjQ3MmItYTcwYS00YWMwLWFkZTMtZmVhMjE2MTcxMDlhIiwibmFtZSI6InN5c3RlbS0xIiwicmV2aXNpb24iOjEsInR5cGUiOiJqb3VybmFsZCJ9LHsiaWQiOiJ3aW5sb2ctc3lzdGVtLWFlMGY0NzJiLWE3MGEtNGFjMC1hZGUzLWZlYTIxNjE3MTA5YSIsIm5hbWUiOiJzeXN0ZW0tMSIsInJldmlzaW9uIjoxLCJ0eXBlIjoid2lubG9nIn0seyJpZCI6InN5c3RlbS9tZXRyaWNzLXN5c3RlbS1hZTBmNDcyYi1hNzBhLTRhYzAtYWRlMy1mZWEyMTYxNzEwOWEiLCJuYW1lIjoic3lzdGVtLTEiLCJyZXZpc2lvbiI6MSwidHlwZSI6InN5c3RlbS9tZXRyaWNzIn0seyJpZCI6ImxvZ2ZpbGUtc3ltYW50ZWMtZWM5ZTZiZTUtNjI3MS00NDY1LTgzMTQtNTI5N2Q0YmZjZDFhIiwibmFtZSI6InN5bWFudGVjX2VuZHBvaW50LTEiLCJyZXZpc2lvbiI6MSwidHlwZSI6ImxvZ2ZpbGUifV19
signature: >-
MEUCIEsZ8SlGHzwL/RBzV6NhOBtBblwXhkQ5h/4NqWVD+zE3AiEA3pF/oOwe+08Zjjn5Lu+GK8ak9sewo2/AOdJWnd1utUI=
secret_references: []
namespaces: []
It is unclear which path you are referring to.... is this it?
If so I am not sure what the * are ...
How do you know it is not read?
Is there a log or just missing data?
Did you try checking the agent status?
That all said can you try to put in with forward slashes
C:/Program Files (x86)/Symantec/Symantec Endpoint Protection Manager/data/dump/*.log
Hi @stephenb,
Thank you for your response.
Yes, I could see data for other system logs and service logs except below mentioned logs.
The path would be as below
C:\Program Files (x86)\Symantec\Symantec Endpoint Protection Manager\data\dump*.log
I had copied configuration from agent policy so same thing I had mentioned in the agent yml file. Let me try with the recommended path by you.
Regards,
Eshwar
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.
