Kibana Slow Perfomance Script Fields and many fields


(Rodrigo Tornis) #1

Hello Everyone!

I am doing a proof of concept with ELK and I have the following doubts regarding specifically Kibana.

My hardware

Core I5 ​​quad core
16GB RAM
Linux OS Ubuntu Server
JVM for Elasticsearch 5.4 8GB Heap
Kibana 5.4

1 - In my tests, I see that Kibana takes a lot of time to load indexes above 1000 fields. It takes time to set the indices on Kibana and the discovery process. One tactic I used was to load a document into the index to speed up loading in kibana to set the timestamp field. Is there a way to improve this load in the first configuration of the index and in the discovery process?

2- On scripts fields, I saw that the community recommendation is to use scripts moderately because it can impact performance depending on the complexity of the script. In my case I am doing a test using 7 fields scripts with the following code:

Painless

If (doc ['FIELD1.keyword']. Value == "Y") {return "OUTPUT"} else If (doc ['FIELD1.keyword']. Value == "Z") {return "OTHER"}

I was surprised by the delay in the loading time of only 2,000 documents (5 minutes approx). Is this behavior normal or is there really a performance problem when I use relatively simple
scripts? How can I improve this?

3 - On Painless for parameters, can you set the parameter value to a value of a field?

4 - Another doubt and about functions in Painless. How can I create a function and within that function read fields. My example below the error because the function does not know the field I'm trying to read:

Painless

String my_test (x) {
If (doc ['field1']. Value == X) {
Return 0
} Else {
Return 1
}

In this example error returns because the function does not recognize the doc ['field1']. Value. How can I resolve this?

Thanks in advance to everyone.

Best regards

Rodrigo Tornis


(Mark Walkom) #2

Is everything running on the single host?

For the second one, you should definitely do this before ingestion.


(Rodrigo Tornis) #3

Hello @warkolm,

Thanks for response :slight_smile:

Yes I,m running single host!

One way to solve slowness during discover was to disable the "doc_table: highlight" and "doc_table: highlight: all_fields" option. This greatly improved the response time.

One way to resolve slowness during discover was to disable the "doc_table: highlight" and "doc_table: highlight: all_fields" option in Kibana. This greatly improved the response time.

Now I have another problem!

I have an ALIASASE of name CRITICALDATASET linked with 4 index (index: SMF014, SMF015, SMF017, SMF030). I inserted 1000 docs in each index. In Kibana, I configured the index for the CRITICALDATASET Alias and all the fields appear in the Kibana mapping (figure 1).

When you query the Alias in Discover, the SMF030 index is not found (Figure 2).

However, when I query the SMF030 index separately, the data is presented (Figure 3)

Even if a query is performed by the "type" field smf030 in the Alias CRITICALDATASET no result is displayed.

Can you help me?


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.