Elasticsearch NEST Highlight

[Ramon_Zaro (Ramon)]

Hi all,
I am running an ES 5.5.0 implementation and use the NEST api in a custom .NET app.
I am able to get correct highlight results while using exact word search. But after activating the full lucene query the highlight function returns no results.

Simply put, how do I get back the correct highlight response to lucene queries (using AND/OR, field : value combinations etc) using NEST ?? which function do I use ?

thanks in advance.

[Ramon_Zaro (Ramon)]

help, assistance please ?

anyone ?

[forloop (Russ Cam)]

Can you provide a minimal, reproducible example of what you're running, what the results are, and what you expect to see?

[Ramon_Zaro (Ramon)]

Hello Russ,
the part I am having a problem with is something like this :

var response = elasticclient.search < >
.From(Int32.Parse(elast_fromind.tostring()))
.Size(Int32.Parse(elast_count.tostring()))
.Type("doc")
.Query(q=> q
	.Querystring(qs=>qs
		.Query(txt2search)))
.Highlight(h=>h
	.Pretags("<tag>")
	.Posttags("</tag>")
	.Fields(fs=> fs
		.Field(f=>f.content)
		.Numberoffragments(1)
		.Fragmentsize(200)))
		);

I am using an index created by fscrawler. Now, if I pass a search string, say "boat AND race AND file.extension : pdf" "boat NOT race" the highlight function doesn't return anything.
The search results work fine though.
Ideally, I would like it to behave like it does in kibana, highlight terms like boat & race in the first case, only boat in the second case while ignoring AND, file.extension and such terms.

[Ramon_Zaro (Ramon)]

Hi, @forloop
If it helps, here's a search that does return highlight hits :

.From(Int32.Parse(elast_fromind.tostring()))
.Size(Int32.Parse(elast_count.tostring()))
.Type("doc")
.Query(q=> q
	.Match (m=> m
		.Field(f => f.content)
		.Query (txt2search)))
.Highlight(h=>h
	.Pretags("<tag>")
	.Posttags("</tag>")
	.Fields(fs=> fs
		.Field(f=>f.content)
		.Numberoffragments(1)
		.Fragmentsize(200)
	)));

IOW, the highlight option works if I search inside a specific field, but if I am using lucene query any field in the index might be called. and in that case the highlight function doesn't return any hits.

[forloop (Russ Cam)]

I think you may need to include the highlight_query in the highlighting; the highlight_query in this case would simply be the same as the search query.

When using the query_string query in 5.5 without specifying any fields, you'll be running the query against the _all field, but higlighting is configured to run against the "content" field, so you need to indicate what query to run against this field for highlighting.

[Ramon_Zaro (Ramon)]

Hi @forloop, I tried this but didn't getdesired result.
might have misunderstood what you meant.

could you provide me a sample few lines of what you mean by include highlight_query in highlighting ?
thanks & regards.

[forloop (Russ Cam)]

It would look something like the following (adapt to your data):

var response = client.Search<Question>(s => s
	.From(0)
	.Size(20)
	.Type("doc")
	.Query(q => q
		.QueryString(qs => qs
			.Query("text to search")
		)
	)
	.Highlight(h => h
		.PreTags("<tag>")
		.PostTags("</tag>")
		.Fields(fs => fs
			.Field(f => f.Body)
			.HighlightQuery(hq => hq
			    // repeat the search query here
				.QueryString(qs => qs
					.Query("text to search")
				)
			)
		)
		.NumberOfFragments(1)
		.FragmentSize(200)
	)
);

Note that it's possible that this does not highlight anything, if the documents that match the search query don't match for the highlight query in the target field i.e. "text to search" might be a match for a document in the "_all" field with a value that has originated from the "Title" field but not the "Body" field, so a highlight on "Body" field won't have anything to highlight.

[Ramon_Zaro (Ramon)]

thanks. I think I am having the exact problem you warned of. How does kibana handle this ?
Because, as far as I can understand this feature works fine there, IOW it is able to highlight successfully without needing the field to be declared explicitly in the query.

[forloop (Russ Cam)]

@Ramon_Zaro

How does kibana handle this ?

Take a look at the request and response sent by Kibana for the given query:

Kibana sends a highlight against * fields

which you could also send in the client

var response = client.Search<Question>(s => s
	.From(0)
	.Size(20)
	.Type("doc")
	.Query(q => q
		.QueryString(qs => qs
			.Query("text to search")
		)
	)
	.Highlight(h => h
		.PreTags("<tag>")
		.PostTags("</tag>")
		.Fields(fs => fs
			.Field("*")
		)
	)
);

[Ramon_Zaro (Ramon)]

thanks a lot, that's really useful !