Dealing with "malformed" attributes

Elastic Stack Elasticsearch
1

I'm using Elasticsearch to process user logs and clickstream data.

Each such log contains several primitive attributes (version, environment, userID, etc.), but the heart of it is a JSON "data" object, which holds all the relevant information regarding the desired log.

There are various types of logs, and their "data" object differs greatly from one another.

I'm looking for a solution that will allow me to index these documents with a different structure.

Potential solutions

  1. I know about the ignore_malformed parameter, that will allow me to suppress the error, but as I want the "data" object to be indexed, that's not a solution.

  2. I can Strinfigy the entire "date" object, which will also suppress the error, but then searching through the JSON will be inferior.

Would appreciate additional insights.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.