I am curious when sending data from logstash to elasticsearch

Elastic Stack Elasticsearch
1

I pass custom logs to elastocsearch using logstash

For example, the string "A+B=AB" contains the field name in "fields1".

If I search for "A+B=AB" literally, other fields containing A,B,AB are partially searched. This is my problem.

I will list the methods I would like to solve in order.

  1. I know that the problem I have is because elasticsearch automatically analyzes the search terms, so can I pass the type of "fileds1" from logstash to elasticsearch as "keyword"?
    After checking, the type of fields1 is automatically text.

  2. When I searched a lot of documents, I found that the reserved words (+,-...) specified in elasticsearch are processed as blanks.
    Is that correct?

  3. Question 2 is suspicious. I searched "₩₩+" but nothing was found. As a workaround I found, you can replace "+" with a character like "plus" in logstash and pass it to Elastic.
    However, I haven't been able to find a way to change it in Logstash, which function should I use to change it?

2

How fields are mapped in Elasticsearch is determined by whether you have matching index templates in place or rely on dynamic mappings. By default mappings create a fields1.keyword subfield that can be used to search for exact matches.

3

Thanks for the reply.

The most curious thing, suppose I put the string "abcd+efgh-hijk" into elasticsearch through logstash.

Is the string that I put in elasticsearch stored as it is? (including special characters such as +,-)
If so, why can't I search for anything when I search for "₩₩+"?

4

How are you searching?

What is the mapping for the field you are searching?

5

I am searching using queryStringqueryBuilder.

And mapping comes out like this.
{"mapping":{
"doc":{
"properties":{
"myfield":{"type":"text",fields":{"keyword":{"type":"keyword","ignore_above":256}

I have never set anything for the field. I only put data from logstash from kafka into elasticsearch.

6

Try searching the myfield.keyword field.

7

Thank you!! :rofl: it's almost solved, but I'll ask you one more thing

I use queryStringqueryBuilder and highlightBuilder together for SearchSourceBuilder.

However, highlightBuilder is not applied to the phrase searched by filed.keyword.

Instead, search by filed minus .keyword will apply highlightBuilder What's the difference between the two?

8

That I do not know.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.