Hi team.
im new to elasticsearch, kindly help me to resolve with below mentioned problem in integartion.
I configured cisco ise, fortinet and paloalto firewalls to push logs to elasticsearch via Load balancer . Logs are receiving @Load balancer node and getting distributed to elasticsearch node -1 and node -2, verified with tcpdump command.
i created fortigate policy to node-1 and cisco ise policy to node-2, and even added paloalto firewall integration policy to both above policies !. but in Kibana i can visualize only fortigate and cisco ISE logs not palo alto logs eventhough the index template is created for palo alto.
so kindly recommend how to solve this integration issue!
Thank you
I'm sorry, but it is still not clear how you are sending the data.
Elasticsearch is REST based and only receives data in json format, Network Devices does not send the data in the format that Elasticsearch expects, so you need something in between like Logstash, Filebeat or Elastic Agent.
What do you have between your load balancer and your Elasticsearch nodes? This is not clear.
Im using Filebeat to forward logs to elasticsearch.
Can you brief me how to integrate with network devices?(we have 6 different firewall vendors)
The method i followed:
configure in firewall with load balancer IP >>> LB distribute traffic to 2 elastic nodes>>> In Elastic nodes - configured rsyslog and installed filebeat Agent>>> can visualize only 1 configured firewall logs among 3
Thank you, it would be very helpful if i get guidance !!
It would be easier and fast to use the Elastic Agent instead of Filebeat.
This is all already documented by Elastic you need to look at the documentations and follow the instructions.
Not sure why are you using rsyslog, but with the Elastic Agent you can configure the Agent to receive the data directly, so your network device would send the logs to your load balancer and the load balance would send the tcp or udp conection to the Elastic Agent.
But this would need you to make changes on how you are receiving your data.
Which one you receive? You said before that you can see logs from ISE and Fortigate, what changed?
What do you mean by that? Do you have an index created for your Palo Alto data?
As you said i installed elastic agent to one of the elastic node after configuring fortinet firewall integration. now i can see only system logs from that node, i can not see any fortinet logs!!
It is pretty hard to know what you did if you do not share any configuration or the return of the comands as asked.
Also, are you using Elastic Agent or Filebeat? You mentioned filebeat before, now you are talking about Elastic Agent and integrations, what are you really using? Please share any configuration screen from the Elastic Agent integration or any filebeat configuration file you are using.
When you add an integration for Fortinet firewall for example you need to configure this integration with the port that it will listen and then configure your firewall to send logs to that IP and port.
In fortinet firewall, we configured to push logs for 514 port with IP ...(LB IP address)!
we configured LB to distribute traffic between elasticsearch node -1 and node-2 !
now i installed elastic-agent in node-2 and added fortinet integration policy to the same agent policy. while configuring integration listening IP is set to 0.0.0.0 and port is 514
And what is not working? Do you have anything in the Elastic Agent logs?
First, I would not use the port 514 in the agent as this port is used already by rsyslog, if you have rsyslog running in the same machine as the Elastic Agent then the agent is probably not working as the port would be already in use.
Is the agent correctly listening for the logs? You need to validate that in the Elastic Agent logs.
And is everything else configured to use this same port?
If the Agent is listening on the configured port, then you need to validate the other parts in your ingestion flow.
You said you have this:
Fireall > LB > Elastic Agent, is your LB already configured to send the data from your firewall to the port you are using? Is the data arriving in the LB? Is your firewall correctly sending the data to your LB?
As you said i followed same troubleshoot method. The results are:
Logs from fortinet. cisco and paloalto firewall getting to elastic Agent. But when i look in kibana, pan-os index is created and all cisco, fortinet and palo alto logs are reporting to same Pan-os index.
only Pan-os logs are parsed.
fortinet events also reporting under pan-os data stream itself and im getting parsing error for fortigate logs .
my integration configuration is as follows:
Added palo alto integration policy to Agent X -
listen Address: 0.0.0.0
port : 514
Added fortigate firewall integration policy to same Agent X:
listen Address: 0.0.0.0
port : 514
Added Cisco ISE integration policy to same Agent X:
listen Address: 0.0.0.0
port : 514
So my question is : Can i use same listening port for all 3 firewalls?, as i mentioned above
if so how to segregate different firewall logs?
Thank you for your quick response! Feel free to ask if any more info needed from my side.
No, you cannot use the same port for all 3 firewalls, you need to use a different port for each firewall, so for logs of the Palo Alto firewall you use the Palo Alto integration, for Logs for your Fortigate firewall you will use the Fortigate integration etc.
Each integration will have a different port, and you will need to configure your firewall and LB to send the logs of each device to its respective integration port.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
