Hello everyone,
I am new to Elastic Search and I've been referred to this solution to solve the following problem. However, given the tons of documentation, I barely know where to start. Here is my problem:
- I want to set up a virtual network as a network lab. Inside this network, I will have different hosts, servers, and security points.
- I will run some attacks and collect the event-type data (network packets, system logs, metrics, audit logs, etc.) from each network element possible. I will pre-process these logs in such way that I believe is proper for later ML classification algorithms I am trying to study or develop.
- I understand that Elastic Search has its own pre-processing technology as well as its own ML algorithms. But I just want the files so I can study them by myself.
The question I am raising here is: Is it possible to collect this kind of data with the different Beat shippers (packetbeat, filebeat, metricbeat, etc.)? And if so, can you please refer to me some reference that shows closely on how to achieve this data collection in such way that I can physically manipulate the data and not through Elastic Search neither Kibana?
Thanks so much in advance