Problem with TimeZone formatter (DateMathExpressionResolver)

[Thales_Valias]

Hi Everybody.

I'm running my tests on elasticsearch and facing one problem about DateFormatter/Resolver. Basically when we start the
ES 'Node' for the tests, it has some Resolvers e.g: IndexNameExpressionResolver and inside it has a DateMathExpressionResolver.

When it starts, uses the default "date_math_expression_resolver.default_time_zone" setting to configure the Timezone, if not present, uses the default "UTC" value.

String defaultTimeZoneId = settings.get("date_math_expression_resolver.default_time_zone", "UTC");

When parsing the data(from ES query) it was parsing with 000Z for Europe/London(if I change my machine to london)
but its parsing with +01:00 for Europe/Lisbon (My current location). It uses JODA Formatters/Parsers and ISOChronology objects to parse the value.

Can be something related to DateMapper? UTC settings? Any idea?

[spinscale (Alexander Reelsen)]

Hey,

can you please provide a reproduction instead of screenshots of code? Provide the elasticsearch.yml configuration, the mapping, a sample document you indexed and a query, that is not working. And also what you expectations are. What is even more important is the elasticsearch version you are using, as date/time handling has drastically changed in Elasticsearch 7 and above.

Thanks!

--Alex

[almeidajava (Fernando Almeida)]

Hi Alexander, I work with Thales in same project. We are using ElasticSearch 6.
<elastic.version>6.0.0</elastic.version>

My settings for the node creation are:

cluster.name - Cluster_3988051938047629400
discovery.zen.ping.unicast.hosts - 127.0.0.1:9400
http.port - 9400
http.type - netty4
network.host - _local_
node.name - docstore
path.data - /var/folders/ng/jkfx481x58l1y81nt18x7l8h0000gn/T/1561624675783-0
path.home - /var/folders/ng/jkfx481x58l1y81nt18x7l8h0000gn/T/1561624675783-0
path.repo - /var/folders/ng/jkfx481x58l1y81nt18x7l8h0000gn/T/1561624675783-0
transport.bind_host - _local_

My current query for the test is:

{
  “bool” : {
    “must” : [
      {
        “bool” : {
          “must” : [
            {
              “bool” : {
                “must” : [
                  {
                    “query_string” : {
                      “query” : ” sci_IND_sci”,
                      “fields” : [
                        “authors^1.0",
                        “citation^1.0”,
                        “doi^1.0",
                        “organisation^1.0”,
                        “section_matrix.text_markup^1.0",
                        “section_matrix.text_plain^1.0”
                      ],
                      “type” : “best_fields”,
                      “default_operator” : “and”,
                      “analyzer” : “ds_nostem”,
                      “max_determinized_states” : 10000,
                      “enable_position_increments” : true,
                      “fuzziness” : “AUTO”,
                      “fuzzy_prefix_length” : 0,
                      “fuzzy_max_expansions” : 50,
                      “phrase_slop” : 0,
                      “escape” : false,
                      “boost” : 1.0
                    }
                  }
                ],
                “adjust_pure_negative” : true,
                “boost” : 1.0
              }
            }
          ],
          “filter” : [
            {
              “bool” : {
                “adjust_pure_negative” : true,
                “boost” : 1.0
              }
            },
            {
              “bool” : {
                “adjust_pure_negative” : true,
                “boost” : 1.0
              }
            },
            {
              “bool” : {
                “adjust_pure_negative” : true,
                “boost” : 1.0
              }
            },
            {
              “bool” : {
                “adjust_pure_negative” : true,
                “boost” : 1.0
              }
            }
          ],
          “adjust_pure_negative” : true,
          “boost” : 1.0
        }
      }
    ],
    “adjust_pure_negative” : true,
    “boost” : 1.0
  }
}

I've tried to put .timezone() as a query param but it had no effect...

[almeidajava (Fernando Almeida)]

I've tried to start my test cluster setting a property for timezone, and it didn't work, This is the message that fails on settings validation:

unknown setting [date_math_expression_resolver.default_time_zone] please check that any required plugins are installed, or check the breaking changes documentation for removed settings

[spinscale (Alexander Reelsen)]

I must have missed something, where would a timezone be involved in the above query? I did not see the indices you are querying, are you putting a date math expression in there?

[almeidajava (Fernando Almeida)]

Sorry, I didn't explained it well...
I'm not sending the TimeZone in the query, it's using the default JODA Timezone(Inside DateTimeZone.java), and it's using the 'user.timezone' property from the system.

I'm not using the datemathresolver, my bad...(I was debugging the code and looked too much code about it). I guess it's not related to the index anyway.

One thing that I tried its the following:

I realized that the problem can be related to the specific DateTimeZone.class from JodaTime:

Before I start the cluster for the test I set the
DateTimeZone.setDefault(DateTimeZone.forID("Europe/London"));

Doing this, the code inside ElasticSearch formatter(that uses JODA) use this value and format data with correct timezone.

But if I don't set anything, it uses my default TimeZone("Europe/Lisbon") and my tests fail, since it is expecting a "000Z" time and not "+01:00". I'm not sure if its an ES problem, since its directly related to JodaTime used by ES.

If is non related, feel free to close it.
Thanks again

[spinscale (Alexander Reelsen)]

Hey,

sorry, but I still have zero understanding what you are trying to do. When you run Elasticsearch as a server, you cannot call DateTimeZone.setDefault either, so why do you do it in a test before starting your cluster?

a proper reproduction instead of snippets and screenshots might be helpful.

--Alex