How Kibana Transforms work?

ok, let me get this right.

Your stream of events contains data like:

{
    "field1":xyz,
    "field2": "something",
...
    "field50": "something_else"
}

in the new index you want:

{
    "field1":xyz,
    "field2": "something",
 ...
    "field10": "abcd"
}

You actually don't want to do something like sum(field1) to get e.g. the sum of all of those fields, right? That's what you mean, you don't want to aggregate?

If you only want the latest, there is a special transform function to do this called, well latest. :wink:

Latest will copy whatever is latest to the destination index, e.g. based on a timestamps. However it won't drop fields, but copy full documents. You can drop fields, if you really need to, in an ingest pipeline. However, I think we can ignore that for the 1st iteration.

If I still did not get it, please explain what you mean by "summarized data".

For me it seems like transform is the way to go, but I am not sure about the function that is right here, there is pivot - which is a group by - and latest. Have a look at the docs, e.g. this tutorial. The description for latest starts in point 7. It's also important you read about continuous vs. batch, continuous requires an extra step, see point 4 in the tutorial. There is more great docs around this tutorial, please check it out as well. The docs also explain what happens on the technical level.