# Preventing date\_time\_parse\_exception in ingest pipeline
**URL:** https://discuss.elastic.co/t/preventing-date-time-parse-exception-in-ingest-pipeline/382707
**Category:** Elasticsearch
**Tags:** language-clients, ingest-pipeline
**Created:** [October 14, 2025, 1:30pm UTC](https://discuss.elastic.co/t/preventing-date-time-parse-exception-in-ingest-pipeline/382707 "2025-10-14T13:30:36Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@marekott](https://discuss.elastic.co/u/marekott)
#### Post date: [October 14, 2025, 1:30pm UTC](https://discuss.elastic.co/t/preventing-date-time-parse-exception-in-ingest-pipeline/382707/1 "2025-10-14T13:30:36Z")
Hi, I have encountered date parsing related problem in my ingest pipeline. Full details can be found here: [[Ingest pipeline] Occasionally ends with `date_time_parse_exception` when pipeline is trying to store time without seconds part · Issue #97314 · elastic/elasticsearch · GitHub](https://github.com/elastic/elasticsearch/issues/97314) . However the main problem is that ingest pipeline with configuration like this (configured via Nest):
```auto
new SetProcessor
{
Field = new Field("ingest_timestamp"),
Description = "Generic pipeline tracking ingest timestamp.",
Value = "{{_ingest.timestamp}}"
}
```
can lead to errors because `{{_ingest.timestamp}}` relies on Mustache to-String-ing which can omit zeros when date is for example `2023-06-02T13:57:00.000Z`. When such value is being inserted into property with given mapping:
```auto
"ingest_timestamp": {
"type": "date",
"format": "strict_date_optional_time_nanos"
},
```
it will fail. In the mentioned issue there is a proposed fix for that but I am unable to configure it via Nest. When below code is executed:
```auto
var putResponse = await elasticClient.Ingest.PutPipelineAsync(pipelineName, p => p
.Processors(pr => pr
.Script(s => s
.Source("ctx.ingest_timestamp = metadata().now.format(DateTimeFormatter.ISO_INSTANT);")
)
), cancellationToken);
```
it ends up with: `Elasticsearch.Net.ElasticsearchClientException: Request failed to execute. Call: Status code 400 from: PUT /_ingest/pipeline/my-pipeline-name?pretty=true. ServerError: Type: script_exception Reason: "compile error" CausedBy: "Type: illegal_argument_exception Reason: "Unknown call [metadata] with [0] arguments.""`
Can someone suggest how to configure it using Nest?
Elasticsearch version: 7.17.25 (but soon we will be updating to 8.X where this pipeline will still be required)
Nest version: 7.17.5
---
### Author:  [@joegallo](https://discuss.elastic.co/u/joegallo)
#### Post date: [October 14, 2025, 4:45pm UTC](https://discuss.elastic.co/t/preventing-date-time-parse-exception-in-ingest-pipeline/382707/2 "2025-10-14T16:45:31Z")
The Nest client has nothing to do with it. The `metadata()` call doesn’t exist in 7.x, it was added three years ago in v8.4.0.
---
### Author:  [@joegallo](https://discuss.elastic.co/u/joegallo)
#### Post date: [October 14, 2025, 5:08pm UTC](https://discuss.elastic.co/t/preventing-date-time-parse-exception-in-ingest-pipeline/382707/3 "2025-10-14T17:08:28Z")
```auto
POST _ingest/pipeline/_simulate?verbose=true
{
"pipeline": {
"processors": [
{
"set": {
"field": "ingest_timestamp",
"value": "{{_ingest.timestamp}}"
}
},
{
"date": {
"field": "ingest_timestamp",
"formats": ["strict_date_optional_time",
"strict_date_optional_time_nanos"]
}
}
]
},
"docs": [
{
"_index": "index",
"_id": "id",
"_source": {
}
}
]
}
```
This seems to work for me on 7.17.25, but I haven’t tested it especially rigorously. Another approach would be to grab the current time with a set processor like I’ve done here and then to manipulate that using data parsing and formatting in a script processor.
---
### Author:  [@marekott](https://discuss.elastic.co/u/marekott)
#### Post date: [October 15, 2025, 1:39pm UTC](https://discuss.elastic.co/t/preventing-date-time-parse-exception-in-ingest-pipeline/382707/4 "2025-10-15T13:39:04Z")
If in your example `ingest_timestamp` is of type `strict_date_optional_time_nanos` and this execution will happen at time without seconds (e.g. 12:25:00) would’t it fail like in mentioned Github issue? I am asking because this set processor looks exactly like the one I was using before.
---
### Author:  [@joegallo](https://discuss.elastic.co/u/joegallo)
#### Post date: [October 15, 2025, 2:12pm UTC](https://discuss.elastic.co/t/preventing-date-time-parse-exception-in-ingest-pipeline/382707/5 "2025-10-15T14:12:30Z")
Yes, the set processor is the same. The date processor that I’m using here parses the date with two different formats, and emits a date in ISO-8601 format. So, no, it wouldn’t fail like mentioned in the github issue.
How about you try it a few dozen thousand times and see?
---
### Author:  [@marekott](https://discuss.elastic.co/u/marekott)
#### Post date: [October 16, 2025, 7:52am UTC](https://discuss.elastic.co/t/preventing-date-time-parse-exception-in-ingest-pipeline/382707/6 "2025-10-16T07:52:46Z")
Thanks, so if I will execute request like this
```auto
POST _ingest/pipeline/_simulate?verbose=true
{
"pipeline": {
"processors": [
{
"set": {
"field": "ingest_timestamp",
"value": "2023-06-02T13:57Z" //Hardcoded date without seconds
}
},
{
"date": {
"field": "ingest_timestamp",
"formats": ["strict_date_optional_time",
"strict_date_optional_time_nanos"]
}
}
]
},
"docs": [
{
"_index": "index",
"_id": "id",
"_source": {
}
}
]
}
```
This should produce date in ISO-8601 format on document right? I am asking because in response I can see:
```auto
"doc" : {
"_index" : "index",
"_type" : "_doc",
"_id" : "id",
"_source" : {
"ingest_timestamp" : "2023-06-02T13:57Z"
},
"_ingest" : {
"pipeline" : "_simulate_pipeline",
"timestamp" : "2025-10-16T07:48:33.376594276Z"
}
}
```
---
### Author:  [@joegallo](https://discuss.elastic.co/u/joegallo)
#### Post date: [November 11, 2025, 9:05pm UTC](https://discuss.elastic.co/t/preventing-date-time-parse-exception-in-ingest-pipeline/382707/7 "2025-11-11T21:05:00Z")
I tried this pipeline:
```auto
PUT _ingest/pipeline/timestamp-demo
{
"processors": [
{
"set": {
"field": "ingest_timestamp",
"value": "{{_ingest.timestamp}}"
}
},
{
"date": {
"field": "ingest_timestamp",
"target_field": "ingest_timestamp",
"formats": [
"strict_date_optional_time",
"strict_date_optional_time_nanos"
],
"output_format": "strict_date_optional_time_nanos"
}
}
]
}
```
With this mapping:
```auto
PUT my-index
{
"mappings": {
"properties": {
"ingest_timestamp": {
"type": "date",
"format": "strict_date_optional_time_nanos"
}
}
}
}
```
And with this change to Elasticsearch itself in order to make interesting timestamps more frequent (for testing purposes):
```auto
diff --git a/server/src/main/java/org/elasticsearch/ingest/IngestDocument.java b/server/src/main/java/org/elasticsearch/ingest/IngestDocument.java
index d0d7687fcbde..39ccc4f4842f 100644
--- a/server/src/main/java/org/elasticsearch/ingest/IngestDocument.java
+++ b/server/src/main/java/org/elasticsearch/ingest/IngestDocument.java
@@ -44,6 +44,7 @@ import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.BiConsumer;
import java.util.stream.Collectors;
@@ -101,8 +102,18 @@ public final class IngestDocument {
private boolean reroute = false;
private boolean terminate = false;
+ private static final AtomicInteger counter = new AtomicInteger(-1);
+
public IngestDocument(String index, String id, long version, String routing, VersionType versionType, Map source) {
- this.ctxMap = new IngestCtxMap(index, id, version, routing, versionType, ZonedDateTime.now(ZoneOffset.UTC), source);
+ final var now = switch (counter.incrementAndGet() % 4) {
+ case 0 -> ZonedDateTime.now(ZoneOffset.UTC).withNano(0).withSecond(0);
+ case 1 -> ZonedDateTime.now(ZoneOffset.UTC).withNano(0);
+ case 2 -> ZonedDateTime.now(ZoneOffset.UTC).withNano(123_000_000);
+ case 3 -> ZonedDateTime.now(ZoneOffset.UTC);
+ default -> throw new IllegalStateException();
+ };
+
+ this.ctxMap = new IngestCtxMap(index, id, version, routing, versionType, now, source);
this.ingestMetadata = new HashMap<>();
this.ingestMetadata.put(TIMESTAMP, ctxMap.getMetadata().getNow());
this.templateModel = initializeTemplateModel();
```
And I think it seems like it’s all working fine to me. In my previous pipeline I missed that the `target_field` and `output_format` are probably important.