# How to automatically set the server date/time when adding data? **URL:** \ **Category:** Elasticsearch\ **Created:** [February 9, 2022, 10:28am UTC](https://discuss.elastic.co/t/how-to-automatically-set-the-server-date-time-when-adding-data/296708 "2022-02-09T10:28:02Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![vmassol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vmassol/32/17398_2.png) [@vmassol](https://discuss.elastic.co/u/vmassol)\ **Post date:** [February 9, 2022, 10:28am UTC](https://discuss.elastic.co/t/how-to-automatically-set-the-server-date-time-when-adding-data/296708/1 "2022-02-09T10:28:02Z")
Hi there, In old ES versions, I was using the `_timestamp` field but it's been removed and it's now recommended to set the date on the client side. However, I'm surprised to not find any easy way to continue to set it on the server side. Setting the date on the client side is risky for our use case since we don't control the clients (they can be anywhere in the world) and they can put wrong date or in the wrong timezone. Our use case is to record pings from clients and I'd much prefer to set the date using the ES server date/time. Is there an easy way to do this? Thanks a lot -Vincent ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 9, 2022, 10:49am UTC](https://discuss.elastic.co/t/how-to-automatically-set-the-server-date-time-when-adding-data/296708/2 "2022-02-09T10:49:10Z")
Bonjour Vincent! 😉 I believe you can do something like this using an ingest pipeline with a `Set` processor. And you can make the pipeline a default pipeline for your index. (See [Ingest pipelines | Elasticsearch Guide [7.17] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/ingest.html#set-default-pipeline)) Full example (from this example [Ingest pipelines | Elasticsearch Guide [7.17] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/ingest.html#access-ingest-metadata)): ```auto PUT _ingest/pipeline/set-timestamp { "description": "sets the timestamp", "processors": [ { "set": { "field": "timestamp", "value": "{{{_ingest.timestamp}}}" } } ] } DELETE test PUT test { "settings": { "index.default_pipeline": "set-timestamp" }, "mappings": { "properties": { "timestamp": { "type": "date" } } } } PUT test/_doc/1 { "foo": "bar" } GET test/_doc/1 ``` Gives: ```auto { "_index" : "test", "_type" : "_doc", "_id" : "1", "_version" : 1, "_seq_no" : 0, "_primary_term" : 1, "found" : true, "_source" : { "foo" : "bar", "timestamp" : "2022-02-09T10:47:34.880561Z" } } ``` ---
**Author:** ![vmassol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vmassol/32/17398_2.png) [@vmassol](https://discuss.elastic.co/u/vmassol)\ **Post date:** [February 9, 2022, 12:38pm UTC](https://discuss.elastic.co/t/how-to-automatically-set-the-server-date-time-when-adding-data/296708/3 "2022-02-09T12:38:50Z")
hey, hi David! 🙂 Thanks a lot for your answer. I should have mentioned that I saw this answer in some answer on stackoverflow but I thought it looked a bit complex, hence why I asked the question here. So I gather that there's no simpler way and I'll go in this direction, thanks again! PS: The concept of pipelines is pretty cool! ---
**Author:** ![vmassol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vmassol/32/17398_2.png) [@vmassol](https://discuss.elastic.co/u/vmassol)\ **Post date:** [February 9, 2022, 12:45pm UTC](https://discuss.elastic.co/t/how-to-automatically-set-the-server-date-time-when-adding-data/296708/4 "2022-02-09T12:45:35Z")
@dadoonet Sorry, another question related to this. Do you know if this ingest pipeline will overwrite any value set in the `timestamp ` field from your example? Is it possible to say to replace the field value only if it's empty or not set? Thanks ---
**Author:** ![vmassol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vmassol/32/17398_2.png) [@vmassol](https://discuss.elastic.co/u/vmassol)\ **Post date:** [February 9, 2022, 12:51pm UTC](https://discuss.elastic.co/t/how-to-automatically-set-the-server-date-time-when-adding-data/296708/5 "2022-02-09T12:51:54Z")
To be more precise, I'd like to set 2 dates: - `pingDate`: always set it with an ingest pipeline to be the current server date - `firstPingDate`: I do some query to find the first ping entry's `firstPingDate` for a given instance id. If it exists, I set it to that value. If it doesn't exist, I'd like to set it to the current server date. I'd like it to be the same value as the `pingDate` too. So 3 questions: 1. Do I need 2 pipelines? 2. How can I set `firstPingDate` in a pipeline, only if not set already? 3. How can I be sure to use the same value as the `pingDate` provided by the ES server? Lots of questions, sorry... Thanks! ---
**Author:** ![vmassol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vmassol/32/17398_2.png) [@vmassol](https://discuss.elastic.co/u/vmassol)\ **Post date:** [February 9, 2022, 12:53pm UTC](https://discuss.elastic.co/t/how-to-automatically-set-the-server-date-time-when-adding-data/296708/6 "2022-02-09T12:53:02Z")
> [@vmassol](#): > > Do I need 2 pipelines? Actually I'm using the wrong terminology. I need 2 processors I think. ---
**Author:** ![vmassol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vmassol/32/17398_2.png) [@vmassol](https://discuss.elastic.co/u/vmassol)\ **Post date:** [February 9, 2022, 12:53pm UTC](https://discuss.elastic.co/t/how-to-automatically-set-the-server-date-time-when-adding-data/296708/7 "2022-02-09T12:53:50Z")
> [@vmassol](#): > > How can I set `firstPingDate` in a pipeline, only if not set already? Found the answer with the `override` config param in [Set processor | Elasticsearch Guide [7.17] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/set-processor.html) ---
**Author:** ![Tomo\_M](https://avatars.discourse-cdn.com/v4/letter/t/848f3c/32.png) [@Tomo\_M](https://discuss.elastic.co/u/Tomo_M)\ **Post date:** [February 9, 2022, 12:54pm UTC](https://discuss.elastic.co/t/how-to-automatically-set-the-server-date-time-when-adding-data/296708/8 "2022-02-09T12:54:43Z")
Af far as you are using Update API, `override` parameter or `if` paramter in [set processors](https://www.elastic.co/guide/en/elasticsearch/reference/master/set-processor.html) can detect whether there is existing value or not. But if you use Index API, both paramters does not work. ---
**Author:** ![vmassol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vmassol/32/17398_2.png) [@vmassol](https://discuss.elastic.co/u/vmassol)\ **Post date:** [February 9, 2022, 12:55pm UTC](https://discuss.elastic.co/t/how-to-automatically-set-the-server-date-time-when-adding-data/296708/9 "2022-02-09T12:55:00Z")
> [@vmassol](#): > > How can I be sure to use the same value as the `pingDate` provided by the ES server? Seems I coud use the `copy_from` config param from [Set processor | Elasticsearch Guide [7.17] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/set-processor.html) 🙂 ---
**Author:** ![vmassol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vmassol/32/17398_2.png) [@vmassol](https://discuss.elastic.co/u/vmassol)\ **Post date:** [February 10, 2022, 10:07am UTC](https://discuss.elastic.co/t/how-to-automatically-set-the-server-date-time-when-adding-data/296708/10 "2022-02-10T10:07:07Z")
@dadoonet hmm I can't find aJava API to use `copy_from`, it seems to be missing. This is what I'm trying: ```auto client.ingest().putPipeline(b0 -> b0 .id("set-timestamp") .description("Set pingDate to be the server date/time and fill firstPingDate if empty") .processors(b1 -> b1 .set(b2 -> b2 .field(PROPERTY_PING_DATE) .value(JsonData.of("{{{_ingest.timestamp}}}"))) .set(b3 -> b3 .override(false) .field(PROPERTY_FIRST_PING_DATE) .copy_from(PROPERTY_PING_DATE) ))); ``` BTW the second `set()` isn't correct syntactically. How do I setup 2 "set" processors using the java API? Thx! ---
**Author:** ![vmassol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vmassol/32/17398_2.png) [@vmassol](https://discuss.elastic.co/u/vmassol)\ **Post date:** [February 10, 2022, 10:10am UTC](https://discuss.elastic.co/t/how-to-automatically-set-the-server-date-time-when-adding-data/296708/11 "2022-02-10T10:10:42Z")
For the second question re multiple processors, it seems the API is not finished but I've found a workaround with: ```auto Processor processor1 = Processor.of(b0 -> b0 .set(b1 -> b1 .field(PROPERTY_PING_DATE) .value(JsonData.of("{{{_ingest.timestamp}}}")))); Processor processor2 = Processor.of(b0 -> b0 .set(b3 -> b3 .override(false) .field(PROPERTY_FIRST_PING_DATE) .copy_from(PROPERTY_PING_DATE))); client.ingest().putPipeline(b0 -> b0 .id("set-timestamp") .description("Set pingDate to be the server date/time and fill firstPingDate if empty") .processors(processor1, processor2); ``` Note that `copy_from()` is not working as it doesn't exist. So I still don't know how to achieve that part. Thx ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 10, 2022, 11:04am UTC](https://discuss.elastic.co/t/how-to-automatically-set-the-server-date-time-when-adding-data/296708/12 "2022-02-10T11:04:19Z")
Bringing in @swallez as this part is the one he is working on. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 10, 2022, 11:06am UTC](https://discuss.elastic.co/t/how-to-automatically-set-the-server-date-time-when-adding-data/296708/13 "2022-02-10T11:06:00Z")
That said, why not using `value` in that case? Like `"value": "{{{timestamp}}}"` ---
**Author:** ![vmassol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vmassol/32/17398_2.png) [@vmassol](https://discuss.elastic.co/u/vmassol)\ **Post date:** [February 10, 2022, 2:03pm UTC](https://discuss.elastic.co/t/how-to-automatically-set-the-server-date-time-when-adding-data/296708/14 "2022-02-10T14:03:38Z")
> [@dadoonet](#): > > Like `"value": "{{{timestamp}}}"` Yes that works fine, as a workaround (I guess it would be cleaner to use `copy_from` since it's meant for that) 🙂 Thanks! For future readers, this is what I got to: ```auto Processor processor1 = Processor.of(b0 -> b0 .set(b1 -> b1 .field(PROPERTY_PING_DATE) .value(JsonData.of("{{{_ingest.timestamp}}}")))); Processor processor2 = Processor.of(b0 -> b0 .set(b3 -> b3 .override(false) .field(PROPERTY_FIRST_PING_DATE) .value(JsonData.of(String.format("{{{%s}}}", PROPERTY_PING_DATE))))); client.ingest().putPipeline(b0 -> b0 .id(PIPELINE_NAME) .description("Set pingDate to be the server date/time and fill firstPingDate if empty") .processors(processor1, processor2)); ``` ---
**Author:** ![swallez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/swallez/32/11972_2.png) [@swallez](https://discuss.elastic.co/u/swallez)\ **Post date:** [February 10, 2022, 4:46pm UTC](https://discuss.elastic.co/t/how-to-automatically-set-the-server-date-time-when-adding-data/296708/15 "2022-02-10T16:46:47Z")
Thank @dadoonet. It's actually missing in the API specification and I opened an issue to fix this. > > > The ingest \`set\` processor (\[type SetProcessor\](https://github.com/elastic/elast…icsearch-specification/blob/025d85d754f8e91a9bdc2409631a66f87340096f/specification/ingest/\_types/Processors.ts#L305)) is missing a lot of fields \[described in the docs\](https://www.elastic.co/guide/en/elasticsearch/reference/current/set-processor.html). > > Reported in \[this Discuss topic\](https://discuss.elastic.co/t/how-to-automatically-set-the-server-date-time-when-adding-data/296708/10) related to using \`copy\_to\` in the Java Client ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 10, 2022, 6:14pm UTC](https://discuss.elastic.co/t/how-to-automatically-set-the-server-date-time-when-adding-data/296708/16 "2022-02-10T18:14:13Z")
To me `copy_from` is helpful when you want to copy a full structure of fields, like an object. For a single field, I think that `value` is the way to go. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 10, 2022, 6:14pm UTC](https://discuss.elastic.co/t/how-to-automatically-set-the-server-date-time-when-adding-data/296708/17 "2022-03-10T18:14:15Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.