# Auto incremental property **URL:** \ **Category:** Elasticsearch\ **Created:** [March 17, 2011, 2:07pm UTC](https://discuss.elastic.co/t/auto-incremental-property/4121 "2011-03-17T14:07:05Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![Mustafa\_Sener](https://avatars.discourse-cdn.com/v4/letter/m/ed8c4c/32.png) [@Mustafa\_Sener](https://discuss.elastic.co/u/Mustafa_Sener)\ **Post date:** [March 17, 2011, 2:07pm UTC](https://discuss.elastic.co/t/auto-incremental-property/4121/1 "2011-03-17T14:07:05Z")
Hi, Is it possible to add a feature to configure auto increment properties for a type. -- Mustafa Sener [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 17, 2011, 6:34pm UTC](https://discuss.elastic.co/t/auto-incremental-property/4121/2 "2011-03-17T18:34:05Z")
No, there isn't. I guess you mean one that will increment for each index operation? Not now, but possible in the future. On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote: > Hi, > Is it possible to add a feature to configure auto increment properties for a type. > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![Berkay\_Mollamustafao](https://avatars.discourse-cdn.com/v4/letter/b/22d042/32.png) [@Berkay\_Mollamustafao](https://discuss.elastic.co/u/Berkay_Mollamustafao)\ **Post date:** [March 17, 2011, 7:25pm UTC](https://discuss.elastic.co/t/auto-incremental-property/4121/3 "2011-03-17T19:25:31Z")
Along the same lines, if ES could insert a timestamp, it would be useful as well (may be easier than incremental id?) Providing the timestamp by the client side introduces potential problems if the time at client machine does not match the time on the ES servers. Regards, Berkay Mollamustafaoglu mberkay on yahoo, google and skype On Thu, Mar 17, 2011 at 2:34 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > No, there isn't. I guess you mean one that will increment for each index > operation? Not now, but possible in the future. > > On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote: > > Hi, > Is it possible to add a feature to configure auto increment properties for > a type. > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 17, 2011, 7:27pm UTC](https://discuss.elastic.co/t/auto-incremental-property/4121/4 "2011-03-17T19:27:27Z")
Timestamp field is certainly possible, but the same problems with client machines exists for "server" machines, as oyu will need to sync the time between them. On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote: > Along the same lines, if ES could insert a timestamp, it would be useful as well (may be easier than incremental id?) Providing the timestamp by the client side introduces potential problems if the time at client machine does not match the time on the ES servers. > > Regards, > Berkay Mollamustafaoglu > mberkay on yahoo, google and skype > > On Thu, Mar 17, 2011 at 2:34 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > No, there isn't. I guess you mean one that will increment for each index operation? Not now, but possible in the future. > > On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote: > > > > > Hi, > > > Is it possible to add a feature to configure auto increment properties for a type. > > > > > > -- > > > Mustafa Sener > > > [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![Mustafa\_Sener](https://avatars.discourse-cdn.com/v4/letter/m/ed8c4c/32.png) [@Mustafa\_Sener](https://discuss.elastic.co/u/Mustafa_Sener)\ **Post date:** [March 17, 2011, 8:00pm UTC](https://discuss.elastic.co/t/auto-incremental-property/4121/5 "2011-03-17T20:00:37Z")
Yes, I want to increment after each index operation or possibly after an update operation. We may use this property instead of a timestamp value. On Thu, Mar 17, 2011 at 9:27 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > Timestamp field is certainly possible, but the same problems with client > machines exists for "server" machines, as oyu will need to sync the time > between them. > > On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote: > > Along the same lines, if ES could insert a timestamp, it would be useful as > well (may be easier than incremental id?) Providing the timestamp by the > client side introduces potential problems if the time at client machine does > not match the time on the ES servers. > > Regards, > Berkay Mollamustafaoglu > mberkay on yahoo, google and skype > > On Thu, Mar 17, 2011 at 2:34 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > > No, there isn't. I guess you mean one that will increment for each index > operation? Not now, but possible in the future. > > On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote: > > Hi, > Is it possible to add a feature to configure auto increment properties for > a type. > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) -- Mustafa Sener [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 17, 2011, 8:02pm UTC](https://discuss.elastic.co/t/auto-incremental-property/4121/6 "2011-03-17T20:02:37Z")
What are you going to use it for? The \_version does exactly that internally (though you can't search on it), it auto increments when something changes. On Thursday, March 17, 2011 at 10:00 PM, Mustafa Sener wrote: > Yes, I want to increment after each index operation or possibly after an update operation. We may use this property instead of a timestamp value. > > On Thu, Mar 17, 2011 at 9:27 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > Timestamp field is certainly possible, but the same problems with client machines exists for "server" machines, as oyu will need to sync the time between them. > > On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote: > > > > > Along the same lines, if ES could insert a timestamp, it would be useful as well (may be easier than incremental id?) Providing the timestamp by the client side introduces potential problems if the time at client machine does not match the time on the ES servers. > > > > > > Regards, > > > Berkay Mollamustafaoglu > > > mberkay on yahoo, google and skype > > > > > > On Thu, Mar 17, 2011 at 2:34 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > > > > > No, there isn't. I guess you mean one that will increment for each index operation? Not now, but possible in the future. > > > > On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote: > > > > > > > > > Hi, > > > > > Is it possible to add a feature to configure auto increment properties for a type. > > > > > > > > > > -- > > > > > Mustafa Sener > > > > > [www.ifountain.com](http://www.ifountain.com) > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![Mustafa\_Sener](https://avatars.discourse-cdn.com/v4/letter/m/ed8c4c/32.png) [@Mustafa\_Sener](https://discuss.elastic.co/u/Mustafa_Sener)\ **Post date:** [March 17, 2011, 8:22pm UTC](https://discuss.elastic.co/t/auto-incremental-property/4121/7 "2011-03-17T20:22:49Z")
We have many clients which populate data to ES and we want to process this data in the order they received by ES. If we assign this incremental or timestamp value from client each client should be synchronized and even this is not enough since the network time will affect the result. Think that we have a client which process the data according to this timestamp or incremental value. And assume the data has not arrived to ES while the processor client continue to process the records inserted at time. In this case a data loss will occur from the perspective of processor client. I think instead of timestamp solution and incremental field setting support per index or type will be much better. Is it possible to implement such a feature? On Thu, Mar 17, 2011 at 10:02 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > What are you going to use it for? The \_version does exactly that > internally (though you can't search on it), it auto increments when > something changes. > > On Thursday, March 17, 2011 at 10:00 PM, Mustafa Sener wrote: > > Yes, I want to increment after each index operation or possibly after an > update operation. We may use this property instead of a timestamp value. > > On Thu, Mar 17, 2011 at 9:27 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > > > Timestamp field is certainly possible, but the same problems with client > > machines exists for "server" machines, as oyu will need to sync the time > > between them. > > > > On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote: > > > > Along the same lines, if ES could insert a timestamp, it would be useful > > as well (may be easier than incremental id?) Providing the timestamp by the > > client side introduces potential problems if the time at client machine does > > not match the time on the ES servers. > > > > Regards, > > Berkay Mollamustafaoglu > > mberkay on yahoo, google and skype > > > > On Thu, Mar 17, 2011 at 2:34 PM, Shay Banon \<[shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) > > > > > wrote: > > > > No, there isn't. I guess you mean one that will increment for each > > index operation? Not now, but possible in the future. > > > > On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote: > > > > Hi, > > Is it possible to add a feature to configure auto increment properties > > for a type. > > > > -- > > Mustafa Sener > > [www.ifountain.com](http://www.ifountain.com) > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) -- Mustafa Sener [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 17, 2011, 9:43pm UTC](https://discuss.elastic.co/t/auto-incremental-property/4121/8 "2011-03-17T21:43:18Z")
Global incremental time not. but timestamp field is possible (there is already an issue for that). But, you will have to make sure that time is sync'ed between elasticsearch servers. Though you can do it now, by just adding a timestamp field yourself. On Thursday, March 17, 2011 at 10:22 PM, Mustafa Sener wrote: > We have many clients which populate data to ES and we want to process this data in the order they received by ES. If we assign this incremental or timestamp value from client each client should be synchronized and even this is not enough since the network time will affect the result. Think that we have a client which process the data according to this timestamp or incremental value. And assume the data has not arrived to ES while the processor client continue to process the records inserted at time. In this case a data loss will occur from the perspective of processor client. > > I think instead of timestamp solution and incremental field setting support per index or type will be much better. Is it possible to implement such a feature? > > On Thu, Mar 17, 2011 at 10:02 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > What are you going to use it for? The \_version does exactly that internally (though you can't search on it), it auto increments when something changes. > > On Thursday, March 17, 2011 at 10:00 PM, Mustafa Sener wrote: > > > > > Yes, I want to increment after each index operation or possibly after an update operation. We may use this property instead of a timestamp value. > > > > > > On Thu, Mar 17, 2011 at 9:27 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > > > > > Timestamp field is certainly possible, but the same problems with client machines exists for "server" machines, as oyu will need to sync the time between them. > > > > On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote: > > > > > > > > > Along the same lines, if ES could insert a timestamp, it would be useful as well (may be easier than incremental id?) Providing the timestamp by the client side introduces potential problems if the time at client machine does not match the time on the ES servers. > > > > > > > > > > Regards, > > > > > Berkay Mollamustafaoglu > > > > > mberkay on yahoo, google and skype > > > > > > > > > > On Thu, Mar 17, 2011 at 2:34 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > > > > > > > > > No, there isn't. I guess you mean one that will increment for each index operation? Not now, but possible in the future. > > > > > > On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote: > > > > > > > > > > > > > Hi, > > > > > > > Is it possible to add a feature to configure auto increment properties for a type. > > > > > > > > > > > > > > -- > > > > > > > Mustafa Sener > > > > > > > [www.ifountain.com](http://www.ifountain.com) > > > > > > -- > > > Mustafa Sener > > > [www.ifountain.com](http://www.ifountain.com) > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![Administrator\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/administrator_2/32/3211_2.png) [@Administrator\_2](https://discuss.elastic.co/u/Administrator_2)\ **Post date:** [March 17, 2011, 9:46pm UTC](https://discuss.elastic.co/t/auto-incremental-property/4121/9 "2011-03-17T21:46:13Z")
Shay, ``` It’s most definitely because I’m a newb at ES, but how do you guarantee atomicity in doing so if you have multiple clients in multiple processes? You’d have to do a query and an update at the same time and have it be an ACID operation, which I don’t know that ES supports. Or does it? - Nick ``` From: Shay Banon [[mailto:shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)] Sent: Thursday, March 17, 2011 5:43 PM To: Mustafa Sener Cc: [users@elasticsearch.com](mailto:users@elasticsearch.com); Berkay Mollamustafaoglu Subject: Re: Auto incremental property Global incremental time not. but timestamp field is possible (there is already an issue for that). But, you will have to make sure that time is sync'ed between elasticsearch servers. Though you can do it now, by just adding a timestamp field yourself. On Thursday, March 17, 2011 at 10:22 PM, Mustafa Sener wrote: We have many clients which populate data to ES and we want to process this data in the order they received by ES. If we assign this incremental or timestamp value from client each client should be synchronized and even this is not enough since the network time will affect the result. Think that we have a client which process the data according to this timestamp or incremental value. And assume the data has not arrived to ES while the processor client continue to process the records inserted at time. In this case a data loss will occur from the perspective of processor client. I think instead of timestamp solution and incremental field setting support per index or type will be much better. Is it possible to implement such a feature? On Thu, Mar 17, 2011 at 10:02 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: What are you going to use it for? The \_version does exactly that internally (though you can't search on it), it auto increments when something changes. On Thursday, March 17, 2011 at 10:00 PM, Mustafa Sener wrote: Yes, I want to increment after each index operation or possibly after an update operation. We may use this property instead of a timestamp value. On Thu, Mar 17, 2011 at 9:27 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: Timestamp field is certainly possible, but the same problems with client machines exists for "server" machines, as oyu will need to sync the time between them. On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote: Along the same lines, if ES could insert a timestamp, it would be useful as well (may be easier than incremental id?) Providing the timestamp by the client side introduces potential problems if the time at client machine does not match the time on the ES servers. Regards, Berkay Mollamustafaoglu mberkay on yahoo, google and skype On Thu, Mar 17, 2011 at 2:34 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: No, there isn't. I guess you mean one that will increment for each index operation? Not now, but possible in the future. On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote: Hi, Is it possible to add a feature to configure auto increment properties for a type. -- Mustafa Sener [www.ifountain.com](http://www.ifountain.com) -- Mustafa Sener [www.ifountain.com](http://www.ifountain.com) -- Mustafa Sener [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 17, 2011, 10:27pm UTC](https://discuss.elastic.co/t/auto-incremental-property/4121/10 "2011-03-17T22:27:50Z")
Automaticity on what? On Thursday, March 17, 2011 at 11:46 PM, Administrator wrote: > Shay, > > It’s most definitely because I’m a newb at ES, but how do you guarantee atomicity in doing so if you have multiple clients in multiple processes? You’d have to do a query and an update at the same time and have it be an ACID operation, which I don’t know that ES supports. > > Or does it? > > - Nick > > From: Shay Banon [[mailto:shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)] > Sent: Thursday, March 17, 2011 5:43 PM > To: Mustafa Sener > Cc: [users@elasticsearch.com](mailto:users@elasticsearch.com); Berkay Mollamustafaoglu > Subject: Re: Auto incremental property > > Global incremental time not. but timestamp field is possible (there is already an issue for that). But, you will have to make sure that time is sync'ed between elasticsearch servers. Though you can do it now, by just adding a timestamp field yourself. > On Thursday, March 17, 2011 at 10:22 PM, Mustafa Sener wrote: > > > We have many clients which populate data to ES and we want to process this data in the order they received by ES. If we assign this incremental or timestamp value from client each client should be synchronized and even this is not enough since the network time will affect the result. Think that we have a client which process the data according to this timestamp or incremental value. And assume the data has not arrived to ES while the processor client continue to process the records inserted at time. In this case a data loss will occur from the perspective of processor client. > > > > I think instead of timestamp solution and incremental field setting support per index or type will be much better. Is it possible to implement such a feature? > > On Thu, Mar 17, 2011 at 10:02 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > What are you going to use it for? The \_version does exactly that internally (though you can't search on it), it auto increments when something changes. > > On Thursday, March 17, 2011 at 10:00 PM, Mustafa Sener wrote: > > > > > Yes, I want to increment after each index operation or possibly after an update operation. We may use this property instead of a timestamp value. > > > On Thu, Mar 17, 2011 at 9:27 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > Timestamp field is certainly possible, but the same problems with client machines exists for "server" machines, as oyu will need to sync the time between them. > > > On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote: > > > > > > > Along the same lines, if ES could insert a timestamp, it would be useful as well (may be easier than incremental id?) Providing the timestamp by the client side introduces potential problems if the time at client machine does not match the time on the ES servers. > > > > > > > > Regards, > > > > Berkay Mollamustafaoglu > > > > mberkay on yahoo, google and skype > > > > > > > > On Thu, Mar 17, 2011 at 2:34 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > > > > > > No, there isn't. I guess you mean one that will increment for each index operation? Not now, but possible in the future. > > > > On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote: > > > > > > > > > Hi, > > > > > Is it possible to add a feature to configure auto increment properties for a type. > > > > > > > > > > -- > > > > > Mustafa Sener > > > > > [www.ifountain.com](http://www.ifountain.com) > > > > > > -- > > > Mustafa Sener > > > [www.ifountain.com](http://www.ifountain.com) > > > > -- > > Mustafa Sener > > [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![Mustafa\_Sener](https://avatars.discourse-cdn.com/v4/letter/m/ed8c4c/32.png) [@Mustafa\_Sener](https://discuss.elastic.co/u/Mustafa_Sener)\ **Post date:** [March 17, 2011, 11:32pm UTC](https://discuss.elastic.co/t/auto-incremental-property/4121/11 "2011-03-17T23:32:06Z")
Shay, It does not have to be a timestamp value. It can be a long value which is incremented after every write operation. This can be index based or global. On Fri, Mar 18, 2011 at 12:27 AM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > Automaticity on what? > > On Thursday, March 17, 2011 at 11:46 PM, Administrator wrote: > > Shay, > > ``` > It’s most definitely because I’m a newb at ES, but how do > > ``` > > you guarantee atomicity in doing so if you have multiple clients in multiple > processes? You’d have to do a query and an update at the same time and have > it be an ACID operation, which I don’t know that ES supports. > > ``` > Or does it? > > - Nick > > ``` > > _From:_ Shay Banon [[mailto:shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)[shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)] > > _Sent:_ Thursday, March 17, 2011 5:43 PM > _To:_ Mustafa Sener > _Cc:_ [users@elasticsearch.com](mailto:users@elasticsearch.com); Berkay Mollamustafaoglu > _Subject:_ Re: Auto incremental property > > Global incremental time not. but timestamp field is possible (there is > already an issue for that). But, you will have to make sure that time is > sync'ed between elasticsearch servers. Though you can do it now, by just > adding a timestamp field yourself. > > On Thursday, March 17, 2011 at 10:22 PM, Mustafa Sener wrote: > > We have many clients which populate data to ES and we want to process this > data in the order they received by ES. If we assign this incremental or > timestamp value from client each client should be synchronized and even this > is not enough since the network time will affect the result. Think that we > have a client which process the data according to this timestamp or > incremental value. And assume the data has not arrived to ES while the > processor client continue to process the records inserted at time. In this > case a data loss will occur from the perspective of processor client. > > I think instead of timestamp solution and incremental field setting support > per index or type will be much better. Is it possible to implement such a > feature? > > On Thu, Mar 17, 2011 at 10:02 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) > wrote: > > What are you going to use it for? The \_version does exactly that internally > (though you can't search on it), it auto increments when something changes. > > On Thursday, March 17, 2011 at 10:00 PM, Mustafa Sener wrote: > > Yes, I want to increment after each index operation or possibly after an > update operation. We may use this property instead of a timestamp value. > > On Thu, Mar 17, 2011 at 9:27 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) > wrote: > > Timestamp field is certainly possible, but the same problems with client > machines exists for "server" machines, as oyu will need to sync the time > between them. > > On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote: > > Along the same lines, if ES could insert a timestamp, it would be useful as > well (may be easier than incremental id?) Providing the timestamp by the > client side introduces potential problems if the time at client machine does > not match the time on the ES servers. > > Regards, > Berkay Mollamustafaoglu > mberkay on yahoo, google and skype > > On Thu, Mar 17, 2011 at 2:34 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) > wrote: > > No, there isn't. I guess you mean one that will increment for each index > operation? Not now, but possible in the future. > > On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote: > > Hi, > Is it possible to add a feature to configure auto increment properties for > a type. > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) -- Mustafa Sener [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 17, 2011, 11:34pm UTC](https://discuss.elastic.co/t/auto-incremental-property/4121/12 "2011-03-17T23:34:05Z")
I understand, but thats not going to be implemented because of the overhead involved in implementing it. Use timestamps. On Friday, March 18, 2011 at 1:32 AM, Mustafa Sener wrote: > Shay, > It does not have to be a timestamp value. It can be a long value which is incremented after every write operation. This can be index based or global. > > On Fri, Mar 18, 2011 at 12:27 AM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > Automaticity on what? > > On Thursday, March 17, 2011 at 11:46 PM, Administrator wrote: > > > > > Shay, > > > > > > It’s most definitely because I’m a newb at ES, but how do you guarantee atomicity in doing so if you have multiple clients in multiple processes? You’d have to do a query and an update at the same time and have it be an ACID operation, which I don’t know that ES supports. > > > > > > Or does it? > > > > > > - Nick > > > > > > From: Shay Banon [[mailto:shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)] > > > Sent: Thursday, March 17, 2011 5:43 PM > > > To: Mustafa Sener > > > Cc: [users@elasticsearch.com](mailto:users@elasticsearch.com); Berkay Mollamustafaoglu > > > Subject: Re: Auto incremental property > > > > > > Global incremental time not. but timestamp field is possible (there is already an issue for that). But, you will have to make sure that time is sync'ed between elasticsearch servers. Though you can do it now, by just adding a timestamp field yourself. > > > On Thursday, March 17, 2011 at 10:22 PM, Mustafa Sener wrote: > > > > > > > We have many clients which populate data to ES and we want to process this data in the order they received by ES. If we assign this incremental or timestamp value from client each client should be synchronized and even this is not enough since the network time will affect the result. Think that we have a client which process the data according to this timestamp or incremental value. And assume the data has not arrived to ES while the processor client continue to process the records inserted at time. In this case a data loss will occur from the perspective of processor client. > > > > > > > > I think instead of timestamp solution and incremental field setting support per index or type will be much better. Is it possible to implement such a feature? > > > > On Thu, Mar 17, 2011 at 10:02 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > > What are you going to use it for? The \_version does exactly that internally (though you can't search on it), it auto increments when something changes. > > > > On Thursday, March 17, 2011 at 10:00 PM, Mustafa Sener wrote: > > > > > > > > > Yes, I want to increment after each index operation or possibly after an update operation. We may use this property instead of a timestamp value. > > > > > On Thu, Mar 17, 2011 at 9:27 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > > > Timestamp field is certainly possible, but the same problems with client machines exists for "server" machines, as oyu will need to sync the time between them. > > > > > On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote: > > > > > > > > > > > Along the same lines, if ES could insert a timestamp, it would be useful as well (may be easier than incremental id?) Providing the timestamp by the client side introduces potential problems if the time at client machine does not match the time on the ES servers. > > > > > > > > > > > > Regards, > > > > > > Berkay Mollamustafaoglu > > > > > > mberkay on yahoo, google and skype > > > > > > > > > > > > On Thu, Mar 17, 2011 at 2:34 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > > > > > > > > > > No, there isn't. I guess you mean one that will increment for each index operation? Not now, but possible in the future. > > > > > > On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote: > > > > > > > > > > > > > Hi, > > > > > > > Is it possible to add a feature to configure auto increment properties for a type. > > > > > > > > > > > > > > -- > > > > > > > Mustafa Sener > > > > > > > [www.ifountain.com](http://www.ifountain.com) > > > > > > > > > > -- > > > > > Mustafa Sener > > > > > [www.ifountain.com](http://www.ifountain.com) > > > > > > > > -- > > > > Mustafa Sener > > > > [www.ifountain.com](http://www.ifountain.com) > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![Mustafa\_Sener](https://avatars.discourse-cdn.com/v4/letter/m/ed8c4c/32.png) [@Mustafa\_Sener](https://discuss.elastic.co/u/Mustafa_Sener)\ **Post date:** [March 18, 2011, 12:33am UTC](https://discuss.elastic.co/t/auto-incremental-property/4121/13 "2011-03-18T00:33:36Z")
Ok, Thanks... On Fri, Mar 18, 2011 at 1:34 AM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > I understand, but thats not going to be implemented because of the > overhead involved in implementing it. Use timestamps. > > On Friday, March 18, 2011 at 1:32 AM, Mustafa Sener wrote: > > Shay, > It does not have to be a timestamp value. It can be a long value which is > incremented after every write operation. This can be index based or global. > > On Fri, Mar 18, 2011 at 12:27 AM, Shay Banon \<[shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) > > > wrote: > > > Automaticity on what? > > > > On Thursday, March 17, 2011 at 11:46 PM, Administrator wrote: > > > > Shay, > > > > ``` > > It’s most definitely because I’m a newb at ES, but how do > > > > ``` > > > > you guarantee atomicity in doing so if you have multiple clients in multiple > > processes? You’d have to do a query and an update at the same time and have > > it be an ACID operation, which I don’t know that ES supports. > > > > ``` > > Or does it? > > > > - Nick > > > > ``` > > > > _From:_ Shay Banon [[mailto:shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)[shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)] > > > > _Sent:_ Thursday, March 17, 2011 5:43 PM > > _To:_ Mustafa Sener > > _Cc:_ [users@elasticsearch.com](mailto:users@elasticsearch.com); Berkay Mollamustafaoglu > > _Subject:_ Re: Auto incremental property > > > > Global incremental time not. but timestamp field is possible (there is > > already an issue for that). But, you will have to make sure that time is > > sync'ed between elasticsearch servers. Though you can do it now, by just > > adding a timestamp field yourself. > > > > On Thursday, March 17, 2011 at 10:22 PM, Mustafa Sener wrote: > > > > We have many clients which populate data to ES and we want to process this > > data in the order they received by ES. If we assign this incremental or > > timestamp value from client each client should be synchronized and even this > > is not enough since the network time will affect the result. Think that we > > have a client which process the data according to this timestamp or > > incremental value. And assume the data has not arrived to ES while the > > processor client continue to process the records inserted at time. In this > > case a data loss will occur from the perspective of processor client. > > > > I think instead of timestamp solution and incremental field setting > > support per index or type will be much better. Is it possible to implement > > such a feature? > > > > On Thu, Mar 17, 2011 at 10:02 PM, Shay Banon \< > > [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)\> wrote: > > > > What are you going to use it for? The \_version does exactly that > > internally (though you can't search on it), it auto increments when > > something changes. > > > > On Thursday, March 17, 2011 at 10:00 PM, Mustafa Sener wrote: > > > > Yes, I want to increment after each index operation or possibly after an > > update operation. We may use this property instead of a timestamp value. > > > > On Thu, Mar 17, 2011 at 9:27 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) > > wrote: > > > > Timestamp field is certainly possible, but the same problems with client > > machines exists for "server" machines, as oyu will need to sync the time > > between them. > > > > On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote: > > > > Along the same lines, if ES could insert a timestamp, it would be useful > > as well (may be easier than incremental id?) Providing the timestamp by the > > client side introduces potential problems if the time at client machine does > > not match the time on the ES servers. > > > > Regards, > > Berkay Mollamustafaoglu > > mberkay on yahoo, google and skype > > > > On Thu, Mar 17, 2011 at 2:34 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) > > wrote: > > > > No, there isn't. I guess you mean one that will increment for each index > > operation? Not now, but possible in the future. > > > > On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote: > > > > Hi, > > Is it possible to add a feature to configure auto increment properties > > for a type. > > > > -- > > Mustafa Sener > > [www.ifountain.com](http://www.ifountain.com) > > > > -- > > Mustafa Sener > > [www.ifountain.com](http://www.ifountain.com) > > > > -- > > Mustafa Sener > > [www.ifountain.com](http://www.ifountain.com) > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) -- Mustafa Sener [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![Administrator\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/administrator_2/32/3211_2.png) [@Administrator\_2](https://discuss.elastic.co/u/Administrator_2)\ **Post date:** [March 18, 2011, 12:54pm UTC](https://discuss.elastic.co/t/auto-incremental-property/4121/14 "2011-03-18T12:54:42Z")
``` Perhaps this is implied by "timestamp", but most users think ``` of timestamps as date/time pairs. ``` For systems with high throughputs, depending on the ``` frequency of requests, you might run into an issue with the date/time being the same. ``` However, you can easily use GUIDs as your timestamp if you ``` run into this problem. ``` While an auto-incremented id would be nice, I'm neutral ``` about it, but I think GUIDs would be a good solution for those that want to guarantee some sort of concurrency. ``` Perhaps the proper request here would be to have the ``` timestamp provided by elasticsearch to be a GUID, or offer two timestamps, one being the number of milliseconds since epoch, as well as a time-neutral stamp which is a GUID. ``` - Nick ``` From: Mustafa Sener [[mailto:mustafa.sener@gmail.com](mailto:mustafa.sener@gmail.com)] Sent: Thursday, March 17, 2011 8:34 PM To: Shay Banon Cc: [users@elasticsearch.com](mailto:users@elasticsearch.com) Subject: Re: RE: Auto incremental property Ok, Thanks... On Fri, Mar 18, 2011 at 1:34 AM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: I understand, but thats not going to be implemented because of the overhead involved in implementing it. Use timestamps. On Friday, March 18, 2011 at 1:32 AM, Mustafa Sener wrote: Shay, It does not have to be a timestamp value. It can be a long value which is incremented after every write operation. This can be index based or global. On Fri, Mar 18, 2011 at 12:27 AM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: Automaticity on what? On Thursday, March 17, 2011 at 11:46 PM, Administrator wrote: Shay, ``` It's most definitely because I'm a newb at ES, but how do ``` you guarantee atomicity in doing so if you have multiple clients in multiple processes? You'd have to do a query and an update at the same time and have it be an ACID operation, which I don't know that ES supports. ``` Or does it? - Nick ``` From: Shay Banon [[mailto:shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)] Sent: Thursday, March 17, 2011 5:43 PM To: Mustafa Sener Cc: [users@elasticsearch.com](mailto:users@elasticsearch.com); Berkay Mollamustafaoglu Subject: Re: Auto incremental property Global incremental time not. but timestamp field is possible (there is already an issue for that). But, you will have to make sure that time is sync'ed between elasticsearch servers. Though you can do it now, by just adding a timestamp field yourself. On Thursday, March 17, 2011 at 10:22 PM, Mustafa Sener wrote: We have many clients which populate data to ES and we want to process this data in the order they received by ES. If we assign this incremental or timestamp value from client each client should be synchronized and even this is not enough since the network time will affect the result. Think that we have a client which process the data according to this timestamp or incremental value. And assume the data has not arrived to ES while the processor client continue to process the records inserted at time. In this case a data loss will occur from the perspective of processor client. I think instead of timestamp solution and incremental field setting support per index or type will be much better. Is it possible to implement such a feature? On Thu, Mar 17, 2011 at 10:02 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: What are you going to use it for? The \_version does exactly that internally (though you can't search on it), it auto increments when something changes. On Thursday, March 17, 2011 at 10:00 PM, Mustafa Sener wrote: Yes, I want to increment after each index operation or possibly after an update operation. We may use this property instead of a timestamp value. On Thu, Mar 17, 2011 at 9:27 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: Timestamp field is certainly possible, but the same problems with client machines exists for "server" machines, as oyu will need to sync the time between them. On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote: Along the same lines, if ES could insert a timestamp, it would be useful as well (may be easier than incremental id?) Providing the timestamp by the client side introduces potential problems if the time at client machine does not match the time on the ES servers. Regards, Berkay Mollamustafaoglu mberkay on yahoo, google and skype On Thu, Mar 17, 2011 at 2:34 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: No, there isn't. I guess you mean one that will increment for each index operation? Not now, but possible in the future. On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote: Hi, Is it possible to add a feature to configure auto increment properties for a type. -- Mustafa Sener [www.ifountain.com](http://www.ifountain.com) -- Mustafa Sener [www.ifountain.com](http://www.ifountain.com) -- Mustafa Sener [www.ifountain.com](http://www.ifountain.com) -- Mustafa Sener [www.ifountain.com](http://www.ifountain.com) -- Mustafa Sener [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:09am UTC](https://discuss.elastic.co/t/auto-incremental-property/4121/15 "2017-07-06T04:09:40Z")