High level REST client Java and Date serialization

Elastic Stack Elasticsearch
1

Hello,
Is there a way to make serialization work with Date classes ?
I have a simple example:

.startObject().field("port_id", portAlias.getPortId())
		.field("portname", portAlias.getPort().getName())
		.field("time_created", portAlias.getTimeCreated())
.endObject();

Which fails with
cannot write time value xcontent for unknown value of type class java.util.Date

I know i can just send the timestamp in millis, but this is getting annoying.
I thought XcontentBuilders were meant to help...

2

Is there a way to call something like portAlias.getTimeCreated().toString() or something like this?

Here is what I'm doing myself. Not that I'm not using the xcontent to generate that but Jackson directly. That might help though.

and

Based on that I can imagine that you could do something like:

SimpleDateFormat formatter = new SimpleDateFormat("yyyy-MM-dd");

// later
.startObject().field("port_id", portAlias.getPortId())
		.field("portname", portAlias.getPort().getName())
		.field("time_created", formatter.format(portAlias.getTimeCreated()))
.endObject();

And that should work OOTB.

3

This is exactly what i meant. It is cumbersome.
It is not like there are no solutions.

This worked fine with the old Java client, but with the REST client is simply cumbersome.

calling portAlias.getTimeCreated().toString() is also something that i don;t like as it depends on the type of the Date object and it's toString() method. Can be a Date, can be a joda.DateTime can be java.time.ZonedDate etc...

There is also a .timeField in XContentBuider which is supposed to handle this, but it does not

For now, i guess the easiest is to just use the unix epoch...

4

I honestly don't know if it's a bug or not. @spinscale do you know?

5

you could try new Date().toInstant() in combination with the timefield

6

Will try.
For me the biggest problem is the documentation which does not work.
For example:
https://www.elastic.co/guide/en/elasticsearch/client/java-rest/6.6/java-rest-high-document-index.html

 XContentBuilder builder = XContentFactory.jsonBuilder();
 builder.startObject();
{
    builder.field("user", "kimchy");
    builder.timeField("postDate", new Date());
    builder.field("message", "trying out Elasticsearch");
}
builder.endObject();
IndexRequest indexRequest = new IndexRequest("posts", "doc", "1")
        .source(builder);

People are using those examples, and they often do not work

7

There is something weird. This actually works.

It does not work though if I use the shaded REST client.

The shaded REST client is just the same REST client but with classes moved to other packages. I had to introduce this as i already have the old ES client in my app and the new and the old client have overlapping classes.

This is the only thing i have in my pom.xml for the shaded version

 <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>3.1.0</version>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals><goal>shade</goal></goals>
                    <configuration>
                        <relocations>
                            <relocation>
                                <pattern>org.elasticsearch</pattern>
                                <shadedPattern>org.elasticsearch.${package.path}</shadedPattern>
                            </relocation>
                            <relocation>
                                <pattern>org.apache.lucene</pattern>
                                <shadedPattern>hidden.org.apache.lucene</shadedPattern>
                            </relocation>
                            <relocation>
                                <pattern>org.apache.http</pattern>
                                <shadedPattern>hidden.org.apache.http</shadedPattern>
                            </relocation>
                            <relocation>
                                <pattern>org.apache.logging</pattern>
                                <shadedPattern>hidden.org.apache.logging</shadedPattern>
                            </relocation>
                            <relocation>
                                <pattern>org.apache.commons.codec</pattern>
                                <shadedPattern>hidden.org.apache.commons.codec</shadedPattern>
                            </relocation>
                            <relocation>
                                <pattern>org.apache.commons.logging</pattern>
                                <shadedPattern>hidden.org.apache.commons.logging</shadedPattern>
                            </relocation>
                        </relocations>
                    </configuration>
                </execution>
            </executions>
        </plugin>
8

I am not sure if shading within this context is supported. I think it may make sense to open a github issue for that in order to clarify or maybe update the docs in that regard.

Also, the other issue looks to me like a solid documentation bug, feel free to open an issue as well.

1 Like
9

Hi,

What version are you running this test against? I tried the same, below, and ended up with data in a local elasticsearch cluster. I am running from a build of our code in the master branch.

    public static void main(String[] args) throws IOException {
        RestHighLevelClient client = new RestHighLevelClient(
            RestClient.builder(
                new HttpHost("localhost", 9200, "http")));

        XContentBuilder builder = XContentFactory.jsonBuilder();
        builder.startObject();
        {
            builder.field("user", "kimchy");
            builder.timeField("postDate", new java.util.Date());
            builder.field("message", "trying out Elasticsearch");
        }
        builder.endObject();

        IndexRequest indexRequest = new IndexRequest("posts", "doc", "1")
            .source(builder);

        IndexResponse indexResponse = client.index(indexRequest, RequestOptions.DEFAULT);
    }

yields

 % curl localhost:9200/posts/doc/1
{"_index":"posts","_type":"doc","_id":"1","_version":1,"_seq_no":0,"_primary_term":1,"found":true,"_source":{"user":"kimchy","postDate":"2019-02-27T18:44:30.669Z","message":"trying out Elasticsearch"}}%

and generates one warning due to types removal (the docs are different in 7.0 but i wanted to use the docs you linked to ensure i got the correct data.

[2019-02-27T12:44:30,892][WARN ][o.e.c.RestClient         ] [[main]] request [PUT http://localhost:9200/posts/doc/1?timeout=1m] returned 1 warnings: [299 Elasticsearch-8.0.0-SNAPSHOT-bc60ca8 "[types removal] Specifying types in document index requests is deprecated, use the typeless endpoints instead (/{index}/_doc/{id}, /{index}/_doc, or /{index}/_create/{id})."]

and a call to verify the mapping

 % curl localhost:9200/posts/_mapping
{"posts":{"mappings":{"properties":{"message":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"postDate":{"type":"date"},"user":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}}}}}}%

Can you post a full snippet that shows the failure and tell me the version you are using as well?

1 Like
10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.