Ingestion pipeline milliseconds

Hi,
is it possible to add, via ingestion pipeline, the current timestamp, but, in milliseconds? (Unix time)

Thanks

I think you are looking for the _ingest.timestamp meta data: Ingest pipelines | Elasticsearch Guide [master] | Elastic

Yes, i Know....but in milliseconds unix time

Can you elaborate on what you are trying to do? This will give you a date field which should be in millisecond precision

I would add a milliseconds timestamp field to my index.

If i use _ingest.timestamp I have a date+time field.

It contains the same information - what do you want to do with milliseconds timestamp you can't do with a date field? It seems like your problem is not related to how the field is stored but with how you access it.

I need milliseconds in a query in Canvas. I prefer to create an ingestion pipeline that add directly the value in the index...

I'm still not sure what you are planning to do exactly. If you are using ESQL, there are functions to do duration calculations with dates: Date/Time and Interval Functions and Operators | Elasticsearch Guide [7.13] | Elastic This seems like a more stable way to do calculations with dates than working with raw millisecond timestamps.

That being said, I guess you can add a second step to your ingest pipeline and use the Date processor | Elasticsearch Guide [master] | Elastic and setting the output_format to UNIX_MS

1 Like

Like this?

It seems like the UI doesn't let you specify the output_format so you have to specify the pipeline using JSON and import it. However I tried to do this and the pipeline fails for the UNIX_MS output format, not sure why:

Still, this looks an awful lot like an xy problem - getting a timestamp formatted as milliseconds in there is very likely not what you actually want to do. I'll be able to help you much better if you would explain what you actually want to do with this value eventually.

The XY problem is a communication problem encountered in help desk and similar situations in which the person asking for help obscures the real issue, X , because instead of asking directly about issue X , they ask how to solve a secondary issue, Y , which they believe will allow them to resolve issue X . However, resolving issue Y often does not resolve issue X , or is a poor way to resolve it, and the obscuring of the real issue and the introduction of the potentially strange secondary issue can lead to the person trying to help having unnecessary difficulties in communication and/or offering poor solutions.

Sorry but yes, I need the current timestamp in UNIX format, every time I receive a document...

I dont know how to explain in another way.

The problem I have is this:

but no one has helped me anymore...

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.