ElasticSearch: Mapping not applied to AWS ELK

Elastic Stack Elasticsearch
1

While applying below mapping to my local ElasticSearch 7.4.1

private static void addIndexMapping(RestHighLevelClient client, String indexName) throws IOException {
        PutMappingRequest request = new PutMappingRequest(indexName);
        XContentBuilder builder = XContentFactory.jsonBuilder();
        builder.startObject();
        {
            builder.startObject("properties");
            {
                builder.startObject("modifiedDate");
                {
                    builder.field("type", "date").field("format","yyyy-MM-dd HH:mm:ss.SSS");

                }
                builder.endObject();
            }
            builder.endObject();
        }
        builder.endObject();
        request.source(builder);
        client.indices().putMapping(request, RequestOptions.DEFAULT);
    }

i can see below mapping got created

{
  "sandbox" : {
    "mappings" : {
      "modifiedDate" : {
        "full_name" : "modifiedDate",
        "mapping" : {
          "modifiedDate" : {
            "type" : "date",
            "format" : "yyyy-MM-dd HH:mm:ss.SSS"
          }
        }
      }
    }
  }
}

but when applying same mapping on AWS elk 7.4.2 i am seeing below mapping

{
  "sandbox" : {
    "mappings" : {
      "modifiedDate" : {
        "full_name" : "modifiedDate",
        "mapping" : {
          "modifiedDate" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          }
        }
      }
    }
  }
}

In my local i have installed ElasticSearch 7.4.1 and in production ElasticSearch 7.4.2. Couldn't understand what is wrong with my configuration.

2

May be the index already exists with a mapping before you are calling the addIndexMapping method?

You should may be check the result of

client.indices().putMapping(request, RequestOptions.DEFAULT);

?

BTW did you look at Cloud by Elastic, also available if needed from AWS Marketplace ?

Cloud by elastic is one way to have access to all features, all managed by us. Think about what is there yet like Security, Monitoring, Reporting, SQL, Canvas, Maps UI, Alerting and built-in solutions named Observability, Security, Enterprise Search and what is coming next :slight_smile: ...

3

@dadoonet May be the index already exists with a mapping before you are calling the addIndexMapping method?
No. I verified this.

Just before inserting the first data i am creating index with required mapping.

This is not in my hand to suggest.

4

Is there a chance your code is multithreaded and an index operation is added before the index is actually created?

In general, I'd recommend using index templates instead. So if by mistake an index is created by a PUT document request, the right mapping will be created.

5

After applying below template still seeing same issue,Could you suggest me the required template.

PUT _template/template_1
{
  "index_patterns": [
    "te*",
    "bar*"
  ],
  "settings": {
    "number_of_shards": 3
  },
  "mappings": {
    "_source": {
      "enabled": false
    },
    "properties": {
      "host_name": {
        "type": "keyword"
      },
      "created_at": {
        "type": "date",
        "format": "EEE MMM dd HH:mm:ss Z yyyy"
      },
      "@timestamp": {
        "type": "date"
      },
      "@version": {
        "type": "keyword"
      }
    }
  }
}
6

What is the index name?

Could you run:

GET /_cat/indices?v
7

my index name is sandbox

8

Could you share the full output please?

9

After applying above index template, i see data is not pushing to index
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open sandbox xZnFTDadQwWQUienyA38AA 3 2 0 0 2kb 690b

10

The index template above does not apply to the sandbox index as the pattern specified does not match. What is the relevance of this template?

11

sorry for the confusion i actually modified the pattern

{
  "template_1" : {
    "order" : 0,
    "index_patterns" : [
      "san*",
      "bar*"
    ],
    "settings" : {
      "index" : {
        "number_of_shards" : "3"
      }
    },
    "mappings" : {
      "_source" : {
        "enabled" : false
      },
      "properties" : {
        "@timestamp" : {
          "type" : "date"
        },
        "@version" : {
          "type" : "keyword"
        },
        "created_at" : {
          "format" : "EEE MMM dd HH:mm:ss Z yyyy",
          "type" : "date"
        },
        "host_name" : {
          "type" : "keyword"
        }
      }
    },
    "aliases" : { }
  }
}
12

What mapping do you get if you recreate the index once this template has been uploaded?

1 Like
13

thank you very much for your support. I could resolve the issue by applying proper index template.

14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.