I'm trying to send http logs from AWS Cloudwatch to ES (Kibana), and a large chunk of logs are not being sent with the following error message:
mapper_parsing_exception, "reason":"object mapping for [responseData.client] tried to parse field [client] as object, but found a concrete value
I've tried to force my logs to stringify responseData.client so that it's always a string, however the error is persisting.
The index that AWS is sending to is incremented every day (it's called cwl-<today's date>) so my understanding was that it would create a new index after the change I implemented and keep the responseData.client field as a string, but it appears not to be working.
Is there a way to force ES to keep this field as a string?
Yes, create an index template with the desired mapping.
Thanks, there are two things I am concerned about before attempting this change:
- there are 2000+ different fields in that index: do I need to include them all in the index template? Can I send a mapping for the problematic field only?
- a new index is being created every day - my understanding is that its mappings are a function of whatever data it is sent from AWS (apologies for my lack of knowledge but I inherited this process and am not sure how it was initially set up). Could my mapping that I submit for, for example “cwl-01-10-2019”, get overwritten the next day when the index “cwl-02-10-2019” is initialised?
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.