I inherited an ELK stack and recently upgraded my firewall. We can send syslog to filebeat, which I'm doing, and then filebeat should be sending to ES. When I try and run the filebeat setup -e from the remote server I get errors. I worked through them and got to this point of almost success:
Loading dashboards (Kibana must be running and reachable)
2021-09-21T23:00:55.054Z INFO kibana/client.go:119 Kibana url: http://10.10.10.245:5601
2021-09-21T23:00:57.264Z INFO kibana/client.go:119 Kibana url: http://10.10.10.245:5601
2021-09-21T23:02:14.785Z ERROR instance/beat.go:971 Exiting: Failed to import dashboard: Failed to load directory /usr/share/filebeat/kibana/7/dashboard:
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-nats-overview.json: returned 500 to import file: <nil>. Response: {"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-abuse-url.json: returned 500 to import file: <nil>. Response: {"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-alienvault-otx.json: returned 500 to import file: <nil>. Response: {"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-anomali.json: returned 500 to import file: <nil>. Response: {"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-aubse-malware.json: returned 500 to import file: <nil>. Response: {"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-misp.json: returned 500 to import file: <nil>. Response: {"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-overview.json: returned 500 to import file: <nil>. Response: {"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
Exiting: Failed to import dashboard: Failed to load directory /usr/share/filebeat/kibana/7/dashboard:
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-nats-overview.json: returned 500 to import file: <nil>. Response: {"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-abuse-url.json: returned 500 to import file: <nil>. Response: {"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-alienvault-otx.json: returned 500 to import file: <nil>. Response: {"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-anomali.json: returned 500 to import file: <nil>. Response: {"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-aubse-malware.json: returned 500 to import file: <nil>. Response: {"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-misp.json: returned 500 to import file: <nil>. Response: {"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-overview.json: returned 500 to import file: <nil>. Response: {"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
Can someone help? I feel like I'm just missing something silly.
Thanks Mark! I checked for anything error-wise in kibana and I didn't see anything that lined up. Kibana logs to the default syslog, and I didn't see anything. Elastic seems to not be logging, as the Elasticsearch.log only returns up to the end of last year so I'm not sure what's going on.
I did notice that filebeat is disabled on ELK as well, in case that helps.
So the logs are there, named my cluster name. lol, learning as I go! I see this error in my kibana logs relevant to the time I just ran filebeat setup -e.
"tags":["debug","plugins","usageCollection","collector-set"],"pid":16495,"message":"not sending [kibana_settings] monitoring document because [undefined] is null or invalid."}
One question is do I have to run the filebeat setup -e locally on ES or can I do it with a remote server configured in filebeat? I can't get filebeat to run on ELK.
Thanks. I followed some instructions online on how to get a secondary admin setup and that worked fine. I can now authenticate via X-Pack local file creds for accessing ES via CURL and validating access. I continue to get this error:
2021-09-27T12:20:09.226Z ERROR instance/beat.go:971 Exiting: Failed to import dashboard: Failed to load directory /usr/share/fi lebeat/kibana/7/dashboard:
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-nats-overview.json: returned 500 to import file: <nil>. Response: {"statusC ode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-abuse-url.json: returned 500 to import file: <nil>. Response: { "statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-alienvault-otx.json: returned 500 to import file: <nil>. Respon se: {"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-anomali.json: returned 500 to import file: <nil>. Response: {"s tatusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-aubse-malware.json: returned 500 to import file: <nil>. Respons e: {"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-misp.json: returned 500 to import file: <nil>. Response: {"stat usCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-overview.json: returned 500 to import file: <nil>. Response: {" statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
Exiting: Failed to import dashboard: Failed to load directory /usr/share/filebeat/kibana/7/dashboard:
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-nats-overview.json: returned 500 to import file: <nil>. Response: {"statusC ode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-abuse-url.json: returned 500 to import file: <nil>. Response: { "statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-alienvault-otx.json: returned 500 to import file: <nil>. Respon se: {"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-anomali.json: returned 500 to import file: <nil>. Response: {"s tatusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-aubse-malware.json: returned 500 to import file: <nil>. Respons e: {"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-misp.json: returned 500 to import file: <nil>. Response: {"stat usCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
error loading /usr/share/filebeat/kibana/7/dashboard/Filebeat-threatintel-overview.json: returned 500 to import file: <nil>. Response: {" statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
Not sure if there's anything else to do or if I'm just stuck at this point.
OK, Update. I was able to get the module to install, enable, and I'm no longer getting filebeat errors, When I run filebeat setup -e to create the fortinet indices, I get this error now:
filebeat[80319]: Exiting: Error reading fileset fortinet/firewall: Variable internal_interfaces doesn't have a 'default' key
I'm so close I can taste it. I have logs successfully being sent but the shards are failing until I can get the indices added.
It's because var.internal_interfaces doesn't have a value. Your need to set something since there isn't a default. What's weird is it shouldn't need a value as it will just not set the config that depends on it so idk why it's complaining. What version are u using?
So I think I got the indexes loaded for the FG module. I'm actually running it successfully from another filebeat agent. At this point I'm seeing the fortinet filters in Kibana. I still have 2 shards from the syslogger that are still showing an illegal exception. I've restarted the service and will monitor. Thanks for your assistance @legoguy1000
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
