Cannot index event publisher ERROR

Elastic Stack Elasticsearch
1

Getting this error

{"type":"mapper_parsing_exception","reason":"failed to parse field [network.forwarded_ip] of type [ip] in document with id 'msD2cIUBIsfwhIgCSPy9'. Preview of field's value: '0'","caused_by":{"type":"illegal_argument_exception","reason":"'0' is not an IP string literal."}}, dropping event!

2

Hmmm pretty straight forward you are trying to put the value 0 into the field which is mapped to data type ip see here

That field is expected to be a valid IP address... So it is dropping the event.

You need to fix the IP or change the mapping to ignore malformed ...which may be a quick fix but may be cause issues when trying to query

3

it's strange that it's ingesting 0 in an IP-mapped field.

I was just checking a spike in logs and they were full of this event. I haven't modified any mappings or anything like that. I guess this might be due to the update from 8.2 to 8.5 coz that's when issues started occurring.

Can you please tell me how can I fix this issue or how can I set the data type to keyword or string so that the error stops and the document shows proper information and does not drop the event?

4

It's actually not indexing this log into elasticsearch because of the mapper error... the error says that...

Moving forward,

What integration/ module are you using?...

What is the source of the log? perhaps there is a bug in the ingest pipeline.

Also you need to show the rest / more error or turn on debug so you can see what the original message is.

So in short

What is the source? Can you show us?
And what module are you using?
And if there's any more error logs before or after that would help.

5

I found out that the issue is with this field network.forwarded_ip which is a part of IIS integration.

chrome_FKODo3hlqq
chrome_FKODo3hlqq1025×358 28.1 KB

the entire event is shown in the message field followed by

Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Date(2023, time.January, 4, ********
*****full event************
{"type":"mapper_parsing_exception","reason":"failed to parse field [network.forwarded_ip] of type [ip] in document with id 'orWAe4UBIsfwhIgCi1lm'. Preview of field's value: '921'","caused_by":{"type":"illegal_argument_exception","reason":"'921' is not an IP string literal."}}, dropping event!

p.s. I'm using Elastic agent not any particular beat

6

@theacodes I suspect that your IIS logs are not in the standard format ... so there is no real easy fix for this.

Either you fix your IIS logs to be standard

Or You will need to go into the IIS ingest pipeline and edit / fix the pipeline...

Can you provide a couple of sample IIS log lines?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.