I found this tutorial regarding how to configure logstash to include geo data.
But per what I was recommended, to try filebeat directly to Elasticsearch which is now working for my new setup, I do not have Logstash. It is possible to get geo data working without Logstash in my hosted (non-cloud) environment? If so, can you point me to where the instruction are for that configuration?
Hi @elastirroot
Yes and please use our documentation not 3rd party
And there may already be a filebeat module that will do all this sooo.
What version Elasticsearch/ filebeat are you on?
And
what is the source of the data?
If is is custom data
Using an ingest pipeline with the geoip processor
And finally you may already have it working but those IPs in your example are internal IPS and so there will be no geolocation information for them
I'm currently running 8.17 of both ELasticsearch and Filebeat. My source data is Suricata. That being the case which of your recommendations should I focus on if not all?
And yes, the IP's that you are able to see are from local traffic, but definitely some go out and come in to this network.
I only have one host that is reported as the hostname but that is because all of my traffic is forwarded to one machine (applepi) that host Suricata and the logs are forwarded from there to my Elasticsearch server. If have not yet figured out how to capture my various device names in my dashboard yet.
All that said, does the geo features require a special license?
Your help is appreciated as always!
No geoip does not require a commercial license, the Basic / Free license supports GEOIP
However you get the open source version of the goip databases
You can purchase and install your own commercial from the MaxMind which have better coverage and accuracy.
Can you share one of the documents ? Do you see the source hostname anywhere in the document? Or does it not show at all?
The source name is not in the document at the time. I suppose I will have to find a creative way to get that. Perhaps sourcing that in from my pihole logs. Either way that is a secondary issue. For right now, I'd like to get the maps working with the source/destination IP information available. I'm trying to figure that out with the previous recommendations you gave.
So far, I found that I have "pipeline: geoip-info" included in my filebeat.yml file. And when I go to Stack Management > Ingest Pipeline, I can find geoip-info. Not sure what I'm missing yet. Still looking.
Share an entire document of one of the events that has the geoip info like that GB one.
Exactly which map? Screenshot?
I just loaded the test alerts from our repo and The Destination Map comes up
All I did is enable the suricate module
set the path to my sample suricata alert data found here (note I updated the dates from 1028 to 1024)
ran
./filebeat setup -e
then ran
./filebeat -e
And the data is loaded and shows up on the map.. .I did nothing else
I added some data with valid source ips and got them too...
I just sent direct message. That said, I didn't even notice that record you pointed out. When I went to that data, it appears I do have some map info. I guess I was just expecting more visible mapping like the following:
But I do seem to have at least 1 point on my map.
GB data point. On my map.
I think I'm on my way. I just need to filter the data to see where my traffic is going, coming from.
Thank you again!.
Nope those Default Dashboards do not show the connection just source and destination
You can also go to Elastic Security and look at the network map there.
But those source are internal so you would not see the line because the source would not show... there are ways to enrich them setup your own data and enrich.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.
