I'm setting an elasticsearch as a datalake for AI/ML purposes, I want to have all data in text blobs unparsed, so the AI scripts can work with unstructured data for an NLP project.
I have no problems with the custom logs/data I send using custom index/mappings since I put everything in a specific flattened field with type text.
Now my problem arises when I try to send windows logs extracted with winbeat, since it uses its own processors to parse/map all data into multiple fields, and I don't think re-writting all processors is the way to go, specially since I plan to use more out-of-the-box beats to ingest logs from other providers.
I'm trying to look for a solution to put all data of a document into a single field, doesn't have to be in the same index, preferible without using logstash due to HW constrains, any sugestions?
I'm thinking into moving the data from the pre-made winbeat index to a new index with the flattened mapping, but I'm not sure on how to do that either and if it would work or it would be the best way to do so.