Logstah performance issues

Elastic Stack Logstash
21

It would be better to use:

dissect {
  mapping => {
    "message" => "...,%{?onething}" 
}

Effectively, field onething will never be added, so it would not have to be removed. Of course, if the message field is no longer needed, it would be appropriate to have remove_field => ["message"] still, as it will only remove if the conversion is successful and has no errors.

22

Good, ty.

I test dissect filter and i have a probleme.

(Sometimes my fields are empty)

Few fields are shifted and take the wrong value...

even so, in Excel, my conf is good on log lines. I don't know where the problem comes from.

23

Caused by: java.lang.NumberFormatException: For input string: "FU_USAGE_OP"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) ~[?:1.8.0_131]

Here one of error. i have full error of this type.

Ok @theuntergeek , If my fields are empty, the values are assigned to the wrong field. How to resolve it ?

24

You may be compelled to use the csv filter if you have empty fields. I believe it handles those properly, where the dissect filter does not. However, that may or may not be true until you test it and find out.

25

I have empty filed only sometimes not always :confused:

26

Indeed, but the cost of trying to figure out which ones need to be shifted is too high. You should use the csv filter to see if it catches those properly. Otherwise, you're going to have a lot of work to try to figure out how to either omit or reprocess lines that have missing fields.

27

FYI, I have a change to Dissect that handles empty fields better but it has not been released - the Pull Request is in the review stage.

28

I changed the topic to be more constructive.

29

Good, i think swith to dissect afeter this changement.

Thank you for help all

30

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.