Hi All,
I am having a problem with the search functionality on my website, which uses Elasticsearch to retrieve records based on visitors' search terms and selected filters. The basic search works correctly when a user enters a keyword without applying any additional conditions, but the results become unexpectedly incomplete when multiple filters are selected together. For example, a visitor might search for a particular product category and then narrow the results by availability, price range, or another attribute. Each filter appears to work correctly when tested individually, but combining two or three filters sometimes removes records that should still satisfy all the selected conditions. The missing records are present in Elasticsearch, and I can find them when I run a simpler query, so I am trying to understand why the combined query excludes them. This affects the usefulness of the website's search because visitors may assume that no matching items exist when the application is actually failing to retrieve some valid matches. I want to identify whether the problem comes from the structure of my Elasticsearch query, the mapping of the fields being filtered, or the way the frontend constructs the final request when users select multiple options.
The issue is particularly noticeable when filters are combined with a text search rather than used on their own. A keyword search may return the expected set of records, and selecting a single category may also produce correct results, but adding another filter can cause the number of returned records to drop unexpectedly. In some cases, the result count changes much more than I would expect based on the selected conditions, even though I can identify individual documents that appear to meet every requirement. I am reviewing whether the website is combining its filters using the intended logical relationships or accidentally requiring a document to satisfy conditions that should have been alternatives. For example, multiple values within one filter may need to be treated differently from conditions across separate filter groups, and I am concerned that the current query-building logic does not preserve that distinction. I have not confirmed that this is the cause, but I would like to understand how Elasticsearch evaluates nested bool queries containing must, filter, should, and must_not clauses when several filtering options are combined. A clear explanation of the query structure would help me determine whether the unexpected exclusions originate in Elasticsearch's matching behavior or in the request generated by my application.
I have started comparing the requests sent by the website with the results returned when I run equivalent queries directly against Elasticsearch. My intention is to take one record that disappears from the filtered results, inspect its indexed fields, and then compare those values with the exact conditions included in the search request. This should help establish whether the document genuinely fails one of the conditions or whether the query is structured incorrectly. I am also checking whether the relevant fields use mappings that are appropriate for filtering, particularly where a field contains text, a category name, a numeric value, or a list of possible values. I understand that analyzed text fields and exact-match keyword fields behave differently, and that using the wrong field type in a term-level query can lead to unexpected results. However, I do not want to change mappings without first identifying a specific mismatch between the stored document and the filter condition. If there is a recommended approach for debugging a combined Elasticsearch query one clause at a time, I would appreciate guidance on how to use query inspection tools or the Explain API to determine why an otherwise relevant document does not match.
Another part of the problem involves the way the website builds its query dynamically as visitors select and remove filters. The frontend sends the selected options to the backend, which constructs the Elasticsearch request using the search phrase and the current filter values. I am reviewing whether a filter that was previously selected remains in the request after the user changes another option, or whether an empty or missing value is being interpreted as an actual condition rather than being ignored. If the application accidentally preserves an outdated filter, the resulting query could be more restrictive than the options currently displayed on the page. I also want to verify whether multiple selections within the same category are combined using the intended behavior, such as matching any selected value instead of requiring a document to match every value simultaneously. At present, the problem appears to be related to the final combination of filters rather than a complete failure of Elasticsearch, but I need a reliable way to confirm that assumption. I would appreciate advice on how to structure the backend query-building logic so that each selected filter is translated consistently and the generated request accurately represents the conditions the visitor expects.
I would like to reproduce the problem using a small collection of known documents rather than relying only on the full dataset from the live website. For example, I could create several test records with clearly defined categories, availability values, and numeric attributes, then verify the expected results for each individual filter and combination. This would allow me to compare the expected matches with the actual Elasticsearch response and determine which combination introduces the unexpected exclusion. I am also interested in whether the Profile API or other query profiling tools would help identify an issue in query execution, although I understand that a query can execute efficiently and still return the wrong set of documents because its logical conditions are incorrect. My priority is to verify the correctness of the matching behavior before investigating performance or changing the index structure. If there are established testing patterns for complex filter combinations, especially where the application dynamically generates Boolean queries, I would appreciate recommendations that could help me turn the current intermittent reports into a repeatable test case.
Has anyone in the Elasticsearch community encountered a situation where individual search filters work correctly but combining them causes valid documents to disappear from the results? I would appreciate help identifying the best way to inspect the generated query, verify the relevant field mappings, and determine which clause excludes a document that should match. If necessary, I can share an anonymized example document, its mapping, the query generated by the website, and the expected versus actual results for a specific combination of filters. I am particularly interested in understanding how to handle multiple selected values within one filter group while combining separate filter groups with the intended logical behavior. My goal is to correct the query-building or field-matching issue responsible for the incomplete results without weakening the filtering rules or making unrelated changes to the search implementation. Any practical advice on isolating the faulty condition and validating the corrected query would be greatly appreciated. Sorry for long post!