How to deal with mapping conflicts?

Elastic Stack Elasticsearch
1

Hi,

I have mutliple indices, one per logfile type. All application log indices are starting with the same prefix. I have an index pattern with that prefix, so that I am searching through all application indices at once so that I am able to merge the different logs by timestamp.

When I query in kibana for "exception" String, then I get the error message "2 of 110 shards failed".
Checking the index_patterns in management tab I can see a conflict for the field details.

I suppose that one index is using "details" as string and another one as structure.

What is the fastest way to find out where my mapping issue comes from?

Thanks, Andreas

2

seems if this are 2 different issuses here.

  • mapping conflict which kibana is showing.
  • the shard issue. I get same issue when only querying a single index.

So I created an index template pointing to a specific index: plx_session_fail-2019.20

When querying it without any filter, discover is showing me data as expected.
When I query for message: *exception* it works well.
When I query for *execption* (without pointing to a specific) field, now kibana is showing the following error in GUI:

Discover: bytes can be at most 32766 in length; got 71073

 Less Info
OK
Error: Request to Elasticsearch failed: {"error":{"root_cause":[{"type":"max_bytes_length_exceeded_exception","reason":"bytes can be at most 32766 in length; got 71073"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"plx_session_fail-2019.20","node":"GLHe8R8qS3uZmuPOZVGFnQ","reason":{"type":"max_bytes_length_exceeded_exception","reason":"bytes can be at most 32766 in length; got 71073"}}],"caused_by":{"type":"max_bytes_length_exceeded_exception","reason":"bytes can be at most 32766 in length; got 71073","caused_by":{"type":"max_bytes_length_exceeded_exception","reason":"bytes can be at most 32766 in length; got 71073"}}},"status":500}
    at https://qdetju/poc-kibana-0/bundles/commons.bundle.js:3:1268369
    at Function.Promise.try (https://qdetju/poc-kibana-0/bundles/commons.bundle.js:3:969152)
    at https://qdetju/poc-kibana-0/bundles/commons.bundle.js:3:968521
    at Array.map (<anonymous>)
    at Function.Promise.map (https://qdetju/poc-kibana-0/bundles/commons.bundle.js:3:968479)
    at callResponseHandlers (https://qdetju/poc-kibana-0/bundles/commons.bundle.js:3:1267381)
    at https://qdetju/poc-kibana-0/bundles/commons.bundle.js:3:1250429
    at processQueue (https://qdetju/poc-kibana-0/dlls/vendors.bundle.dll.js:434:199687)
    at https://qdetju/poc-kibana-0/dlls/vendors.bundle.dll.js:434:200650
    at Scope.$digest (https://qdetju/poc-kibana-0/dlls/vendors.bundle.dll.js:434:210412)
    at https://qdetju/poc-kibana-0/dlls/vendors.bundle.dll.js:434:212944
    at completeOutstandingRequest (https://qdetju/poc-kibana-0/dlls/vendors.bundle.dll.js:434:64425)
    at https://qdetju/poc-kibana-0/dlls/vendors.bundle.dll.js:434:67267

What is the issue here?

My target aim is the following:

I have multiple logs. Some are really structured yet, so exceptions, classes, stacktrace are stored in a structured way, splitted to separate fields. And on the other hand, we have logs, which are not really structured yet. So we multiline the logs (timestamp is indicator for new event) and only parse the timestamp. The rest is greedydata and goes to message.

Now I have all application logs (of different quality) in ES. I want to search for any event which contains the substring exception, regardless if it can be found in message or in a structured field.

3

no one who can shed a bit light here?

4

Hey,

can you run the same search with enabled stack traces and paste those in a gist? See https://www.elastic.co/guide/en/elasticsearch/reference/current/common-options.html#common-options-error-options how to enable it

--Alex

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.