Este artículo está disponible en Español.
Have you already tried ES|QL? ES|QL is a language designed from the ground up for Elasticsearch. It is designed to provide the best experience for filtering, analyzing and transforming your data.
Text filtering is critical for narrowing down results. More often than not you'll need text search to:
- Finding specific log messages
- Obtaining results tagged with a label
- Match all hosts that contain a specific string
ES|QL is here to help! There are multiple text filtering functions that you can use to search for specific text in your data.
Exact matching
You can use the equality operator (==) to check for an exact string match:
FROM logs
| WHERE environment == "production"
This will only find results that match exactly the provided string.
Of course, you can use all ES|QL expressiveness to transform both the string you're looking for and the values that will be checked:
FROM logs
| WHERE LEFT(environment, 10) == TO_UPPER("production")
Partial matching
Pattern matching allows to search for a specific substring:
FROM hosts
| WHERE STARTS_WITH(host, "prod-")
FROM access
| WHERE ENDS_WITH(username, "-test")
You can also use string patterns via wildcards...
FROM logs
| WHERE message LIKE """*NullPointerException*"""
... or regular expressions...
FROM users
| WHERE email RLIKE """.+\.com$"""
Triple quotes help with escaping, so you can focus on what you want to search for.
Text filtering limitations
Matching strings is great for discoverability and filtering your data! But there are some things that are inconvenient with the methods above:
- Text is not analyzed. You may want to use analyzers to process your text and your query to ensure you get the best match possible.
- Multivalues. Functions in ES|QL return null when applied to a multivalued field - you need to process your multivalues to make them searchable first, for example using
MV_CONCAT.
Enter full text search
Full text search is coming to ES|QL to enable better search results for text queries.
Two new functions have been added to Elastic Serverless and are coming on Elasticsearch 8.17:
match(field, query): Provides a match query to query specific fields.qstr(query): Uses the query string query syntax to perform complex queries.
These functions provide text analysis for the specified queries, and are also able to match on individual values for multivalued fields without extra work on your side.
Now you can do full text searches like the following:
FROM logs
| WHERE match(message, "this is a test")
You can simplify more a match function by using the match operator ( : )
FROM logs
| WHERE message : "this is a test"
You can also use qstr for doing complex queries that involve multiple fields and conditions:
FROM emails
| WHERE qstr("subject:(new york city) OR text:(big apple)")
qstr also allows to perform fuzzy matching:
FROM emails
| WHERE qstr("subject: york~2")
New functions and capabilities will be added to full text functions - stay tuned!
Give it a try!
There are multiple ways to perform text filtering in ES|QL to find your data. Be it full match, pattern matching, or full text search capabilities, ES|QL has got you covered.
Start finding your text data, and let us know your feedback!