Kibana 6.2.4 performance issue on complex table aggregation

[asp]

Hi all,

i am running kibana 6.2.4 on top of ES 6.2.3 and I have performance issues on specific

we are logging each user request and put it into elasticsearch. There we have parsed different fields:

• userName
• serviceCall
• serviceCallSub
• intervalLength
• vehicleFilterCound
• driverFilterCount

So we can extract the information how often does a user call service with a specific function / run mode and the time interval a user requested, how many vehicles or drivers are added to the filter.

Currently I am building a performance test for that application. To be as near as possible to production as baseline, I created following visualization, which I will export as csv and use it as input for my loadtest:

As filters following is configured

{
  "filter": [
    {
      "meta": {
        "index": "tux-*",
        "negate": false,
        "disabled": false,
        "alias": "Prod",
        "type": "phrase",
        "key": "stage.keyword",
        "value": "Production",
        "params": {
          "query": "Production",
          "type": "phrase"
        }
      },
      "query": {
        "match": {
          "stage.keyword": {
            "query": "Production",
            "type": "phrase"
          }
        }
      },
      "$state": {
        "store": "appState"
      }
    },
    {
      "meta": {
        "index": "tux-*",
        "negate": true,
        "disabled": false,
        "alias": null,
        "type": "phrase",
        "key": "userName",
        "value": "system",
        "params": {
          "query": "system",
          "type": "phrase"
        }
      },
      "query": {
        "match": {
          "userName": {
            "query": "system",
            "type": "phrase"
          }
        }
      },
      "$state": {
        "store": "appState"
      }
    },
    {
      "meta": {
        "index": "tux-*",
        "negate": true,
        "disabled": false,
        "alias": null,
        "type": "phrase",
        "key": "userName",
        "value": "DE0400_gutrix",
        "params": {
          "query": "DE0400_gutrix",
          "type": "phrase"
        }
      },
      "query": {
        "match": {
          "userName": {
            "query": "DE0400_gutrix",
            "type": "phrase"
          }
        }
      },
      "$state": {
        "store": "appState"
      }
    },
    {
      "meta": {
        "index": "tux-*",
        "type": "phrases",
        "key": "serviceCall.keyword",
        "value": "DB_SCR_REQUEST, DB_SCR_RESPONSE, DB_SCR_STATUS, WATCHCLIENTS, DB_LOC_POSITION",
        "params": [
          "DB_SCR_REQUEST",
          "DB_SCR_RESPONSE",
          "DB_SCR_STATUS",
          "WATCHCLIENTS",
          "DB_LOC_POSITION"
        ],
        "negate": true,
        "disabled": false,
        "alias": "exluded services"
      },
      "query": {
        "bool": {
          "should": [
            {
              "match_phrase": {
                "serviceCall.keyword": "DB_SCR_REQUEST"
              }
            },
            {
              "match_phrase": {
                "serviceCall.keyword": "DB_SCR_RESPONSE"
              }
            },
            {
              "match_phrase": {
                "serviceCall.keyword": "DB_SCR_STATUS"
              }
            },
            {
              "match_phrase": {
                "serviceCall.keyword": "WATCHCLIENTS"
              }
            },
            {
              "match_phrase": {
                "serviceCall.keyword": "DB_LOC_POSITION"
              }
            }
          ],
          "minimum_should_match": 1
        }
      },
      "$state": {
        "store": "appState"
      }
    }
  ],
  "query": {
    "query": "",
    "language": "lucene"
  }
}

Now my problem is, that kibana's web interface is extremely slow, when I run this.

Data for an hour needs more than a minute to come back in kibana. The visualization is also often laggy on scrolling, etc. Opening the statistics takes a long long time, ...

The statistics of the visualization are showing the following:

|Query Duration|5020ms|
|Request Duration|6143ms|
|Hits|58478|
|Index|tux-*|
|Id|tux-*|

-> so querying the data is fast. I also reproduced these times by querying the request via dev tools. It also took about 5 seconds, so elasticsearch should be not the issue here.

Kibana monitoring shows this:

. Query took place at about 09:15

On client side, cpu is at 20 %, top scorer is browser, but no cpu core is at 100%.

I tried Chrome, Vivaldi and Firefox as browsers. None of them is doing well.
Any Ideas how I can get kibana in my browser more responsitive?

Thanks, Andreas

[asp]

seems to be this way: the more lines I get back in all pages together, the slower it gets.

[timroes]

Hi,

this issue is unfortunately happens if you have deep nested responses (i.e. tables with a lot of columns), due to the cloning of these objects in the browser. We are trying to improve that, and 6.3 and 6.4 should already improve performance for that large nested responses, and we'll continue looking into improving performance of the rendering pipeline.

Cheers,
Tim