# Ingestion pipeline milliseconds

**URL:** https://discuss.elastic.co/t/ingestion-pipeline-milliseconds/275967
**Category:** Kibana
**Tags:** ingest-pipeline
**Created:** [June 15, 2021, 9:13am UTC](https://discuss.elastic.co/t/ingestion-pipeline-milliseconds/275967 "2021-06-15T09:13:22Z")
**Posts on this page:** 12
**Page:** 1

<div class="post-metadata">

### Author: ![accateo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/accateo/32/85332_2.png) [@accateo](https://discuss.elastic.co/u/accateo)
#### Post date: [June 15, 2021, 9:13am UTC](https://discuss.elastic.co/t/ingestion-pipeline-milliseconds/275967/1 "2021-06-15T09:13:22Z")

</div>

Hi,  
is it possible to add, via ingestion pipeline, the current timestamp, but, in milliseconds? (Unix time)

Thanks

---

<div class="post-metadata">

### Author: ![flash1293](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/flash1293/32/41227_2.png) [@flash1293](https://discuss.elastic.co/u/flash1293)
#### Post date: [June 15, 2021, 9:31am UTC](https://discuss.elastic.co/t/ingestion-pipeline-milliseconds/275967/2 "2021-06-15T09:31:46Z")

</div>

I think you are looking for the `_ingest.timestamp` meta data: [Ingest pipelines | Elasticsearch Guide [master] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/master/ingest.html#access-ingest-metadata)

---

<div class="post-metadata">

### Author: ![accateo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/accateo/32/85332_2.png) [@accateo](https://discuss.elastic.co/u/accateo)
#### Post date: [June 15, 2021, 9:40am UTC](https://discuss.elastic.co/t/ingestion-pipeline-milliseconds/275967/3 "2021-06-15T09:40:01Z")

</div>

Yes, i Know....but in milliseconds unix time

---

<div class="post-metadata">

### Author: ![flash1293](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/flash1293/32/41227_2.png) [@flash1293](https://discuss.elastic.co/u/flash1293)
#### Post date: [June 15, 2021, 9:42am UTC](https://discuss.elastic.co/t/ingestion-pipeline-milliseconds/275967/4 "2021-06-15T09:42:42Z")

</div>

Can you elaborate on what you are trying to do? This will give you a date field which should be in millisecond precision

---

<div class="post-metadata">

### Author: ![accateo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/accateo/32/85332_2.png) [@accateo](https://discuss.elastic.co/u/accateo)
#### Post date: [June 15, 2021, 9:49am UTC](https://discuss.elastic.co/t/ingestion-pipeline-milliseconds/275967/5 "2021-06-15T09:49:45Z")

</div>

I would add a milliseconds timestamp field to my index.

If i use \_ingest.timestamp I have a date+time field.

---

<div class="post-metadata">

### Author: ![flash1293](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/flash1293/32/41227_2.png) [@flash1293](https://discuss.elastic.co/u/flash1293)
#### Post date: [June 15, 2021, 3:52pm UTC](https://discuss.elastic.co/t/ingestion-pipeline-milliseconds/275967/6 "2021-06-15T15:52:02Z")

</div>

It contains the same information - what do you want to do with milliseconds timestamp you can't do with a `date` field? It seems like your problem is not related to how the field is stored but with how you access it.

---

<div class="post-metadata">

### Author: ![accateo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/accateo/32/85332_2.png) [@accateo](https://discuss.elastic.co/u/accateo)
#### Post date: [June 15, 2021, 5:30pm UTC](https://discuss.elastic.co/t/ingestion-pipeline-milliseconds/275967/7 "2021-06-15T17:30:37Z")

</div>

I need milliseconds in a query in Canvas. I prefer to create an ingestion pipeline that add directly the value in the index...

---

<div class="post-metadata">

### Author: ![flash1293](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/flash1293/32/41227_2.png) [@flash1293](https://discuss.elastic.co/u/flash1293)
#### Post date: [June 16, 2021, 8:43am UTC](https://discuss.elastic.co/t/ingestion-pipeline-milliseconds/275967/8 "2021-06-16T08:43:08Z")

</div>

I'm still not sure what you are planning to do exactly. If you are using ESQL, there are functions to do duration calculations with dates: [Date/Time and Interval Functions and Operators | Elasticsearch Guide [7.13] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/sql-functions-datetime.html#sql-functions-datetime-format) This seems like a more stable way to do calculations with dates than working with raw millisecond timestamps.

That being said, I guess you can add a second step to your ingest pipeline and use the [Date processor | Elasticsearch Guide [master] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/master/date-processor.html) and setting the `output_format` to `UNIX_MS`

---

<div class="post-metadata">

### Author: ![accateo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/accateo/32/85332_2.png) [@accateo](https://discuss.elastic.co/u/accateo)
#### Post date: [June 16, 2021, 9:45am UTC](https://discuss.elastic.co/t/ingestion-pipeline-milliseconds/275967/9 "2021-06-16T09:45:39Z")

</div>

> [@flash1293](#):
>
> UNIX\_MS

Like this?

 ![Cattura](https://us1.discourse-cdn.com/elastic/original/3X/7/c/7c3ed101d939c758b84a654c16404978378ad800.png)

---

<div class="post-metadata">

### Author: ![flash1293](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/flash1293/32/41227_2.png) [@flash1293](https://discuss.elastic.co/u/flash1293)
#### Post date: [June 16, 2021, 10:13am UTC](https://discuss.elastic.co/t/ingestion-pipeline-milliseconds/275967/10 "2021-06-16T10:13:30Z")

</div>

It seems like the UI doesn't let you specify the output\_format so you have to specify the pipeline using JSON and import it. However I tried to do this and the pipeline fails for the `UNIX_MS` output format, not sure why:

 ![Screenshot 2021-06-16 at 12.10.32](https://us1.discourse-cdn.com/elastic/original/3X/1/f/1fec14b2052074e4550ad10eb7ae9f89ae7ab322.png)

Still, this looks an awful lot like an [xy problem](https://en.wikipedia.org/wiki/XY_problem) - getting a timestamp formatted as milliseconds in there is very likely not what you actually want to do. I'll be able to help you much better if you would explain what you actually want to do with this value eventually.

> The **XY problem** is a communication problem encountered in [help desk](https://en.wikipedia.org/wiki/Help_desk) and similar situations in which the person asking for help obscures the real issue, _X_ , because instead of asking directly about issue _X_ , they ask how to solve a secondary issue, _Y_ , which they believe will allow them to resolve issue _X_ . However, resolving issue _Y_ often does not resolve issue _X_ , or is a poor way to resolve it, and the obscuring of the real issue and the introduction of the potentially strange secondary issue can lead to the person trying to help having unnecessary difficulties in communication and/or offering poor solutions.

---

<div class="post-metadata">

### Author: ![accateo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/accateo/32/85332_2.png) [@accateo](https://discuss.elastic.co/u/accateo)
#### Post date: [June 16, 2021, 10:51am UTC](https://discuss.elastic.co/t/ingestion-pipeline-milliseconds/275967/11 "2021-06-16T10:51:26Z")

</div>

Sorry but yes, I need the current timestamp in UNIX format, every time I receive a document...

I dont know how to explain in another way.

The problem I have is this:

> [@Kibana essql query date diff](https://discuss.elastic.co/t/kibana-essql-query-date-diff/271483/15):
>
> Hi Felix, I already tried the sort function....nothing change. Current situation: with expression: filters group="group3" ungrouped=true | essql query="SELECT DAY(date) as dd, MIN(date) as realDate, DATE\_DIFF('hours',MIN(date),MAX(date)) as diff FROM \"application\_data\_bridgenuzoo\" GROUP BY DAY(date)" | mapColumn name="formatted\_date" expression={getCell "realDate" | formatdate format="DD/MM/YY"} | pointseries x="formatted\_date" y="diff" | plot defaultStyle={seri…

but no one has helped me anymore...

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 14, 2021, 10:52am UTC](https://discuss.elastic.co/t/ingestion-pipeline-milliseconds/275967/12 "2021-07-14T10:52:07Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
