When ES updated from 7.3.1 to 7.10.2, search slow 2x in fetch phase

Our product environment had updated es from 7.3.1 to 7.10.2, the index mapping and search sentence has not changed anymore. We specify include and exclude fileds by using that


    sourceBuilder.fetchSource(getIncludeFetch(), getExcludeFetch());

And we found the response slow 2x in 7.10.2 than in 7.3.1.

We are confused that and I used JProfile to watch the runnable processes. We found in 7.10.2, much time used in function convertToMap which is used to convert bufferByte to map.

And 7.3.1 deals with normal in this process.

in 7.10.2


98.4% - 984 s - 41 调用 java.util.concurrent.ThreadPoolExecutor$Worker.run

    98.0% - 979 s - 2,161 调用 org.elasticsearch.common.util.concurrent.AbstractRunnable.run

        98.0% - 979 s - 2,161 调用 org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun

            98.0% - 979 s - 2,062 调用 org.elasticsearch.common.util.concurrent.TimedRunnable.doRun

                97.9% - 979 s - 2,062 调用 org.elasticsearch.common.util.concurrent.AbstractRunnable.run

                    96.5% - 964 s - 1,040 调用 org.elasticsearch.action.ActionRunnable$2.doRun

                        96.5% - 964 s - 1,040 调用 org.elasticsearch.action.ActionRunnable.lambda$supply$0

                            96.4% - 964 s - 1,040 调用 org.elasticsearch.search.SearchService$2.lambda$onResponse$0

                                96.4% - 964 s - 1,040 调用 org.elasticsearch.search.SearchService.executeQueryPhase

                                    89.4% - 894 s - 1,039 调用 org.elasticsearch.search.SearchService.executeFetchPhase

                                        89.4% - 894 s - 1,039 调用 org.elasticsearch.search.fetch.FetchPhase.execute

                                            85.8% - 858 s - 103,403 调用 org.elasticsearch.search.fetch.subphase.FetchSourcePhase$1.process

                                                85.8% - 857 s - 103,403 调用 org.elasticsearch.search.fetch.subphase.FetchSourcePhase.hitExecute

                                                    74.9% - 749 s - 102,603 调用 org.elasticsearch.search.lookup.SourceLookup.filter

                                                        57.0% - 570 s - 102,603 调用 org.elasticsearch.search.lookup.SourceLookup.loadSourceIfNeeded

                                                            57.0% - 570 s - 102,603 调用 org.elasticsearch.search.lookup.SourceLookup.sourceAsMapAndType

                                                                57.0% - 570 s - 102,603 调用 org.elasticsearch.common.xcontent.XContentHelper.convertToMap

                                                                    57.0% - 570 s - 102,603 调用 org.elasticsearch.common.xcontent.XContentHelper.convertToMap

                                                                        56.9% - 569 s - 102,603 调用 org.elasticsearch.common.xcontent.XContentHelper.convertToMap

                                                                            56.9% - 568 s - 102,603 调用 org.elasticsearch.common.xcontent.support.AbstractXContentParser.map

                                                                                56.9% - 568 s - 102,603 调用 org.elasticsearch.common.xcontent.support.AbstractXContentParser.readMapSafe

                                                                                    56.8% - 568 s - 102,603 调用 org.elasticsearch.common.xcontent.support.AbstractXContentParser.readMapEntries

                                                                                        53.8% - 538 s - 6,466,288 调用 org.elasticsearch.common.xcontent.support.AbstractXContentParser.readValueUnsafe

in 7.3.1


4.7% - 311 s - 447 调用 org.elasticsearch.search.SearchService.executeFetchPhase

    4.7% - 311 s - 447 调用 org.elasticsearch.search.fetch.FetchPhase.execute

        4.5% - 301 s - 31,064 调用 org.elasticsearch.search.fetch.subphase.FetchSourceSubPhase.hitExecute

            4.1% - 270 s - 30,653 调用 org.elasticsearch.search.lookup.SourceLookup.filter

                3.3% - 220 s - 30,653 调用 org.elasticsearch.search.lookup.SourceLookup.loadSourceIfNeeded

                    3.3% - 220 s - 30,653 调用 org.elasticsearch.search.lookup.SourceLookup.sourceAsMapAndType

                        3.3% - 220 s - 30,653 调用 org.elasticsearch.common.xcontent.XContentHelper.convertToMap

                            3.3% - 220 s - 30,653 调用 org.elasticsearch.common.xcontent.XContentHelper.convertToMap

                                3.3% - 219 s - 30,653 调用 org.elasticsearch.common.xcontent.XContentHelper.convertToMap

                                    3.3% - 219 s - 30,653 调用 org.elasticsearch.common.xcontent.support.AbstractXContentParser.map

I had returned to normal because I found the source code which can avoid to using function convertToMap.


        // If this is a parent document and there are no source filters, then add the source as-is.

        if (nestedHit == false && containsFilters(fetchSourceContext) == false) {

            hitContext.hit().sourceRef(source.internalSourceRef());

            return;

        }

So I stored common returned fields in _source and stored optional fields in stored_field to avoid using function convertToMap, but I also want for helpping to find the reason which result in search slow in 7.10.2, thank you.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.