"this field is present in your elasticsearch mapping" on _source field

Elastic Stack Logstash
1

Upgraded my ELK stack to 6.8 finally. Beats are on 6.4 still. Zero issues all last week and all this week. Today I noticed one of my filters for one of my beats was experiencing some parsing errors on certain log entries. Made an updated filter, tested, parsing correctly and everything is working.

Get call for details on some logins. Start going through logs using Kibana. Everything is working fantastic. I change to the next timeframe and suddenly one of my Kibana columns is now showing _source right next to Time. To make things worse it seems to me the data for entire log record AND I cannot remove it as as column.

I then notice it is listed to the left under the Selected Fields and has a "?" next to it. Expanding this gives me the "This field is present in your Elasticsearch mapping but not in the 94 documents shown in the doc table. You may still be able to visualize or search on it."

No idea why this suddenly showed up. I was literally searching my indices one moment then like I said all I did was change timeframe and it was there. Everything seems to still be technically working, however I want to remove this column, and my greater concern is why I am getting this error on what I thought was a default meta field all of sudden?

Any advice would be greatly appreciated as I am starting to use this system more and more.

2

Just bumping this so it is not closed. Surely I am not the first to encounter this problem.

3

You can try refreshing the index pattern in Kibana.

4

Thanks Christian. I did attempt this. It unfortunately has not resolved the issue.

5

I have built an entirely new 7.2 cluster ingesting data from the same nodes but creating new indecies. The problem still exists. I know the _source field is a "default?" field containing the formatted JSON I believe. Is this required? I see I can remove it from the mapping, but I am too ignorant to fully understand the ramifications of doing so. Surely I cannot be the first one to experience this?

6

Going to bump one more time before I wipe the indices and start over. Would appreciate any direction on this. Thanks.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.