I want to use Beats to generate files with event-type data to later analyze on my own

Hello everyone,

I am new to Elastic Search and I've been referred to this solution to solve the following problem. However, given the tons of documentation, I barely know where to start. Here is my problem:

  • I want to set up a virtual network as a network lab. Inside this network, I will have different hosts, servers, and security points.
  • I will run some attacks and collect the event-type data (network packets, system logs, metrics, audit logs, etc.) from each network element possible. I will pre-process these logs in such way that I believe is proper for later ML classification algorithms I am trying to study or develop.
  • I understand that Elastic Search has its own pre-processing technology as well as its own ML algorithms. But I just want the files so I can study them by myself.

The question I am raising here is: Is it possible to collect this kind of data with the different Beat shippers (packetbeat, filebeat, metricbeat, etc.)? And if so, can you please refer to me some reference that shows closely on how to achieve this data collection in such way that I can physically manipulate the data and not through Elastic Search neither Kibana?

Thanks so much in advance

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.