How to create an index template with NGramTokenizer using .NET client 8.0

Elastic Stack Elasticsearch
1

I try to convert our C# code from Nest to the new Elastic.Clients.Elasticsearch client. Got stuck on many details. Currently I can't see how to build a request for creating an index template.

Code snippet:

var createIndexTemplateResponse = await _elasticClient.Indices.PutIndexTemplateAsync(templateName,
	d => d
	.IndexPatterns($"{_idxName}-*")
	.ComposedOf(new Collection<Name>())
	.Template(t => t
		.Settings(s => s
			.Index(i => i
				.DefaultPipeline(_pipelineName)
				.Analysis(a => a
					.Analyzer(ad => ad
						.Custom("fta", c => c
							.Filter(new[] { "lowercase" })
							.Tokenizer("ftt")))
					.Tokenizer(td => td
						.Add("ftt", ...))))))// TODO: Complete the code to use an NGramTokenizer
);

I want to use an NGramTokenizer like this:

var ngt = new NGramTokenizer
{
	MaxGram = 3,
	MinGram = 3,
	TokenChars = new[] { TokenChar.Letter, TokenChar.Digit, TokenChar.Punctuation, TokenChar.Symbol }
};

How to complete my request?

2

Hi, @UmLao.

The current release has a bug making analysis settings difficult to configure and causing bad serialization. This has been fixed this week and will go out in 8.0.4, hopefully in the next day or so. Once that ships, you'll be able to configure you template as follows:

var createIndexTemplateResponse = await client.Indices.PutIndexTemplateAsync(templateName,	d => d
	.IndexPatterns($"{_idxName}-*")
	.ComposedOf(new Collection<Name>())
	.Template(t => t
		.Settings(s => s
			.Index(i => i
				.DefaultPipeline(_pipelineName)
				.Analysis(a => a
					.Analyzers(ad => ad
						.Custom("fta", c => c
							.Filter(new[] { "lowercase" })
							.Tokenizer("ftt")))
					.Tokenizers(td => td
						.NGram("ftt", n => n
							.MaxGram(3)
							.MinGram(3)
							.TokenChars(new[] { TokenChar.Letter, TokenChar.Digit, TokenChar.Punctuation, TokenChar.Symbol }))))))));
3

8.0.4 is now available on NuGet.

4

Thanks Steve, now I can create this template as expected.

Follow-up: I also want to read this template in order to verify if its contents are correct. But the following request for getting this template throws an Exception:

var getIndexTemplateResponse = await _elasticClient.Indices.GetIndexTemplateAsync(d => d
	.Name(templateName));

The templateName is myindex_template.

The exception is

Elastic.Transport.UnexpectedTransportException: The JSON value could not be converted to Elastic.Clients.Elasticsearch.Names. Path: $.index_templates[0].index_template.index_patterns | LineNumber: 5 | BytePositionInLine: 28. ---> System.Text.Json.JsonException: The JSON value could not be converted to Elastic.Clients.Elasticsearch.Names. Path: $.index_templates[0].index_template.index_patterns | LineNumber: 5 | BytePositionInLine: 28.
   bei System.Text.Json.ThrowHelper.ThrowJsonException_DeserializeUnableToConvertValue(Type propertyType)
   bei System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   bei System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   bei System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   bei System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   bei System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   bei System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   bei System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   bei System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   bei System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value)
   bei System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   bei System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   bei System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   bei System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   bei System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   bei System.Text.Json.JsonSerializer.ReadCore[TValue](JsonReaderState& readerState, Boolean isFinalBlock, ReadOnlySpan`1 buffer, JsonSerializerOptions options, ReadStack& state, JsonConverter converterBase)
   bei System.Text.Json.JsonSerializer.ContinueDeserialize[TValue](ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack, JsonConverter converter, JsonSerializerOptions options)
   bei System.Text.Json.JsonSerializer.<ReadAllAsync>d__65`1.MoveNext()

This is the template as seen in Kibana (Kibana and ES 8.2.3)

{
  "template": {
    "settings": {
      "index": {
        "analysis": {
          "analyzer": {
            "fta": {
              "filter": [
                "lowercase"
              ],
              "type": "custom",
              "tokenizer": "ftt"
            }
          },
          "tokenizer": {
            "ftt": {
              "token_chars": [
                "letter",
                "digit",
                "punctuation",
                "symbol"
              ],
              "min_gram": "3",
              "type": "ngram",
              "max_gram": "3"
            }
          }
        },
        "routing": {
          "allocation": {
            "include": {
              "_tier_preference": "data_content"
            }
          }
        },
        "default_pipeline": "myindex_pipeline"
      }
    },
    "aliases": {},
    "mappings": {}
  }
}
5

Thanks for highlighting this bug. I've raised an issue to track it. For exceptions and bugs like this one, please raise those directly as issues on the repository so they get my attention. The forum is best for general usage questions.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.