The documentation says that documents must contain a @timestamp field in order to be accepted by a datastream.
I'm trying to make it optional by declaring a default ingest_pipeline in a datastream template that will automatically add a timestamp field. It works perfectly when I use a simple index template instead. Unfortunatly, it doesn't seem to work with a datastream template ("data stream timestamp field [@timestamp] is missing").
So my question is : Do you think it's possible to automatically add a @timestamp to a document with an ingest pipeline so that a datastream can accept it ?
Well, after several attempts, I can say that not only it's indeed possible, but it works perfectly.
My problem was only due to a mistake in my ingest pipeline (it was creating the field "ingest_timestamp" instead of "@timestamp"). Now I'm able to send simple status documents without timestamp to my datastream which adds itself the @timestamp field
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.