Cannot filter data in elastic SIEM

[lusynda]

Hi all
I have a problems in the elastic siem.
The filter function for me when i tried to filter for some field in the rule that i have created. those field seems to be not recognized by elastic so they do not allow me to filter those data. event though i have tried to create the index pattern and still the data is not recognized.
Is there a way to make siem understand those data since i really need to filter those data out.

Thanks for your time.

[spong (Garrett)]

HI there @lusynda!

Would you be willing to provide a little more detail around the filter you're seeing not take effect? When you say you tried to filter for some field in the rule that you created, do you mean you're not seeing the rule create any alerts with this filter? Do you see any errors under the Failure History tab on the Rule Details page / the Monitoring tab on the main Rules page, or is the rule running successfully? Are you seeing the query + filter work within Discover? What type of Rule are you trying to create, and did you ensure the correct index pattern is specified in the first step? Also, what version do you happen to be running?

With the above information we should be better able to help identify why your filters aren't working.

Thanks!
Garrett

[lusynda]

@spong thanks for your responce,

• the "Add filter" button under custom query is not working for me in SIEM and only SIEM, and it seem i misinform you on th not working thing, it's not that is not working, but it no longer have any suggestion on the field to filter on any more so i cannot use it.
• The rule still gen alert and there are no failure for the rule.
• the index patterns is correct since when i created the rule there are still alert for it.
• all type of rule have this problems.
• Some time i see the Unexpected token < in JSON at position 0 in SIEM and only SIEM part of the system.

[AngelaChuang]

Hello @lusynda,

Does it mean that you don't have the suggestions after putting the filter field?

Could you please check if the index patterns you are using exists in Stack Management / index patterns?
Not sure if this reproduced your case, but I found that if I left an index pattern which does not exist, I'll have no suggested fields available.

[lusynda]

Yes well like i said when i create the rule for it. It still generate alert for me so the index patterns is corrected. So the index pattern does exsist.

[Frank_Hassanabad]

If you're on plain version of 7.9.0, I would consider upgrading to the latest 7.9.2 as we implemented several performance enhancements which makes getting the fields magnitudes faster. I think from your error description you're timing out getting the fields. This can happen if you have a lot of different indexes or one of your indexes has a "mapping explosion" where it has a lot of fields that are auto-generated or just a lot in general.

The two tickets we fixed for 7.9.1+ with regards to perf here:

[Security Solution] Optimizes the index queries to not block the NodeJS event loop

Reduces field capabilities event loop block times by scaling linearly using hashes