Kibana index pattern don't show data with time filter field name

[gyana_nayak]

i am trying to create an index from Java code to index some data but if i am creating index pattern with time filter Kibana never shows any data. Following below order while creating index with some setting and adding an index template for date filed

CreateIndexRequest request = new CreateIndexRequest(indexName);
        request.settings(Settings.builder()
                .put("index.max_inner_result_window", 250)
                .put("index.write.wait_for_active_shards", 1)
                .put("index.query.default_field", "paragraph")
                .put("index.number_of_shards", 3)
                .put("index.number_of_replicas", 2)
                .loadFromSource(Strings.toString(
                        XContentFactory.jsonBuilder()
                        .startObject()
                        .startObject("analysis")
                        .startObject("analyzer")
                        .startObject("englishAnalyzer")
                        .field("tokenizer", "standard")
                        .field("char_filter", "html_strip")
                        .field("filter", new String[] { "snowball", "standard", "lowercase" })
                        .endObject()
                        .startObject("email")
                        .field("filter", new String[] { "standard", "lowercase" })
                        .field("type", "custom")
                        .field("tokenizer", "uax_url_email")
                        .endObject()
                        .endObject()
                        .endObject()
                        .endObject()), XContentType.JSON));
        CreateIndexResponse createIndexResponse = client.indices().create(request, RequestOptions.DEFAULT);
        

PutIndexTemplateRequest request = new PutIndexTemplateRequest(templateName);
            request.patterns(templateList);
            request.settings(Settings.builder().put("index.number_of_shards", 3).put("index.number_of_replicas", 1));
            XContentBuilder builder = XContentFactory.jsonBuilder();
            builder.startObject();
            {
                builder.startObject("properties");
                {
                    builder.startObject("createdDate");
                    {
                        builder.field("type", "date").field("format", "yyyy-MM-dd HH:mm:ss.SSS");
                    }
                    builder.endObject();
                }
                builder.endObject();
            }
            builder.endObject();
            request.mapping(builder);
            request.version();
            client.indices().putTemplate(request, RequestOptions.DEFAULT);

Then i am inserting below record as JSON but when creating an index pattern with time filter keep on showing empty result in Kibana discovery

{
  "createdDate": "2020-11-25 13:33:36.782",
  "userId": "raw.man@bee.com",
}

[dadoonet (David Pilato)]

Hi.

Few things to check.

What is the index mapping?

Run this from Kibana dev console:

GET /YOURINDEX/_mapping

Then, if it's correct, check that you selected the createdDate as the time field when you created the index pattern in Kibana.

[gyana_nayak]

please find index mapping

{
  "httplog" : {
    "mappings" : {
      "properties" : {
        "createdDate" : {
          "type" : "date",
          "format" : "yyyy-MM-dd HH:mm:ss.SSS"
        },
        "userId" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        }
      }
    }
  }
}

Below is the JSON i am pushing

{
createdDate=2020-11-27 11:55:02.930, 
userId=abcd@bee.com
}

using ELK 7.4 version

[dadoonet (David Pilato)]

Did you check the last part of my answer?

[gyana_nayak]

could you tell me how would i check check that you selected the createdDate as the time field when you created the index pattern in Kibana

while creating index pattern i am getting below things

if i select createdDate then Kibana not showing any data.

[dadoonet (David Pilato)]

Could you change the time period in Kibana?

Instead of last 15 minutes, use last 2 weeks.
If nothing shows up, share the output of:

GET httplog/_search

[gyana_nayak]

Nothing is Showing

After firing GET httplog/_search below record is showing

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 3,
    "successful" : 3,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "httplog",
        "_type" : "_doc",
        "_id" : "Rh2-CHYBRk3R1lxIdodk",
        "_score" : 1.0,
        "_source" : {
          "createdDate" : "2020-11-27 13:37:48.575",
          "userId" : "abcd@bee.com"
        }
      }
    ]
  }
}

Additional information i am using Joda date to set createdDate value

Map<String, Object> searchableMap = jsonObject.toMap();
			DateTime dt = new DateTime();
			DateTimeFormatter fmt = DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ss.SSS");
			String str = fmt.print(dt);
			searchableMap.put("createdDate", str);
			request.source(searchableMap, XContentType.JSON);
			IndexResponse response = client.index(request, RequestOptions.DEFAULT); 

[dadoonet (David Pilato)]

I suspect that you have to look into the future to see your content.

There's a timezone problem here. Elasticsearch is using UTC when the timezone is not specified.

Kibana is using the browser timezone.

Try to change the date in Kibana and look at data from yesterday to next week for example and you should see your hit.

[gyana_nayak]

if it would be the case then when i am selecting last 2 weeks as filter it should show.

[dadoonet (David Pilato)]

That's my only guess. Did you try ?

Could you try to create the index pattern without any date filter as well?