Hi,
Is it possible to add a feature to configure auto increment properties for
a type.
--
Mustafa Sener
www.ifountain.com
No, there isn't. I guess you mean one that will increment for each index operation? Not now, but possible in the future.
On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote:
Hi,
Is it possible to add a feature to configure auto increment properties for a type.--
Mustafa Sener
www.ifountain.com
Along the same lines, if ES could insert a timestamp, it would be useful as
well (may be easier than incremental id?) Providing the timestamp by the
client side introduces potential problems if the time at client machine does
not match the time on the ES servers.
Regards,
Berkay Mollamustafaoglu
mberkay on yahoo, google and skype
On Thu, Mar 17, 2011 at 2:34 PM, Shay Banon shay.banon@elasticsearch.comwrote:
No, there isn't. I guess you mean one that will increment for each index
operation? Not now, but possible in the future.On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote:
Hi,
Is it possible to add a feature to configure auto increment properties for
a type.--
Mustafa Sener
www.ifountain.com
Timestamp field is certainly possible, but the same problems with client machines exists for "server" machines, as oyu will need to sync the time between them.
On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote:
Along the same lines, if ES could insert a timestamp, it would be useful as well (may be easier than incremental id?) Providing the timestamp by the client side introduces potential problems if the time at client machine does not match the time on the ES servers.
Regards,
Berkay Mollamustafaoglu
mberkay on yahoo, google and skypeOn Thu, Mar 17, 2011 at 2:34 PM, Shay Banon shay.banon@elasticsearch.com wrote:
No, there isn't. I guess you mean one that will increment for each index operation? Not now, but possible in the future.
On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote:Hi,
Is it possible to add a feature to configure auto increment properties for a type.--
Mustafa Sener
www.ifountain.com
Yes, I want to increment after each index operation or possibly after an
update operation. We may use this property instead of a timestamp value.
On Thu, Mar 17, 2011 at 9:27 PM, Shay Banon shay.banon@elasticsearch.comwrote:
Timestamp field is certainly possible, but the same problems with client
machines exists for "server" machines, as oyu will need to sync the time
between them.On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote:
Along the same lines, if ES could insert a timestamp, it would be useful as
well (may be easier than incremental id?) Providing the timestamp by the
client side introduces potential problems if the time at client machine does
not match the time on the ES servers.Regards,
Berkay Mollamustafaoglu
mberkay on yahoo, google and skypeOn Thu, Mar 17, 2011 at 2:34 PM, Shay Banon shay.banon@elasticsearch.comwrote:
No, there isn't. I guess you mean one that will increment for each index
operation? Not now, but possible in the future.On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote:
Hi,
Is it possible to add a feature to configure auto increment properties for
a type.--
Mustafa Sener
www.ifountain.com
--
Mustafa Sener
www.ifountain.com
What are you going to use it for? The _version does exactly that internally (though you can't search on it), it auto increments when something changes.
On Thursday, March 17, 2011 at 10:00 PM, Mustafa Sener wrote:
Yes, I want to increment after each index operation or possibly after an update operation. We may use this property instead of a timestamp value.
On Thu, Mar 17, 2011 at 9:27 PM, Shay Banon shay.banon@elasticsearch.com wrote:
Timestamp field is certainly possible, but the same problems with client machines exists for "server" machines, as oyu will need to sync the time between them.
On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote:Along the same lines, if ES could insert a timestamp, it would be useful as well (may be easier than incremental id?) Providing the timestamp by the client side introduces potential problems if the time at client machine does not match the time on the ES servers.
Regards,
Berkay Mollamustafaoglu
mberkay on yahoo, google and skypeOn Thu, Mar 17, 2011 at 2:34 PM, Shay Banon shay.banon@elasticsearch.com wrote:
No, there isn't. I guess you mean one that will increment for each index operation? Not now, but possible in the future.
On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote:Hi,
Is it possible to add a feature to configure auto increment properties for a type.--
Mustafa Sener
www.ifountain.com--
Mustafa Sener
www.ifountain.com
We have many clients which populate data to ES and we want to process this
data in the order they received by ES. If we assign this incremental or
timestamp value from client each client should be synchronized and even this
is not enough since the network time will affect the result. Think that we
have a client which process the data according to this timestamp or
incremental value. And assume the data has not arrived to ES while the
processor client continue to process the records inserted at time. In this
case a data loss will occur from the perspective of processor client.
I think instead of timestamp solution and incremental field setting support
per index or type will be much better. Is it possible to implement such a
feature?
On Thu, Mar 17, 2011 at 10:02 PM, Shay Banon
shay.banon@elasticsearch.comwrote:
What are you going to use it for? The _version does exactly that
internally (though you can't search on it), it auto increments when
something changes.On Thursday, March 17, 2011 at 10:00 PM, Mustafa Sener wrote:
Yes, I want to increment after each index operation or possibly after an
update operation. We may use this property instead of a timestamp value.On Thu, Mar 17, 2011 at 9:27 PM, Shay Banon shay.banon@elasticsearch.comwrote:
Timestamp field is certainly possible, but the same problems with client
machines exists for "server" machines, as oyu will need to sync the time
between them.On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote:
Along the same lines, if ES could insert a timestamp, it would be useful
as well (may be easier than incremental id?) Providing the timestamp by the
client side introduces potential problems if the time at client machine does
not match the time on the ES servers.Regards,
Berkay Mollamustafaoglu
mberkay on yahoo, google and skypeOn Thu, Mar 17, 2011 at 2:34 PM, Shay Banon <shay.banon@elasticsearch.com
wrote:
No, there isn't. I guess you mean one that will increment for each
index operation? Not now, but possible in the future.On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote:
Hi,
Is it possible to add a feature to configure auto increment properties
for a type.--
Mustafa Sener
www.ifountain.com--
Mustafa Sener
www.ifountain.com
--
Mustafa Sener
www.ifountain.com
Global incremental time not. but timestamp field is possible (there is already an issue for that). But, you will have to make sure that time is sync'ed between elasticsearch servers. Though you can do it now, by just adding a timestamp field yourself.
On Thursday, March 17, 2011 at 10:22 PM, Mustafa Sener wrote:
We have many clients which populate data to ES and we want to process this data in the order they received by ES. If we assign this incremental or timestamp value from client each client should be synchronized and even this is not enough since the network time will affect the result. Think that we have a client which process the data according to this timestamp or incremental value. And assume the data has not arrived to ES while the processor client continue to process the records inserted at time. In this case a data loss will occur from the perspective of processor client.
I think instead of timestamp solution and incremental field setting support per index or type will be much better. Is it possible to implement such a feature?
On Thu, Mar 17, 2011 at 10:02 PM, Shay Banon shay.banon@elasticsearch.com wrote:
What are you going to use it for? The _version does exactly that internally (though you can't search on it), it auto increments when something changes.
On Thursday, March 17, 2011 at 10:00 PM, Mustafa Sener wrote:Yes, I want to increment after each index operation or possibly after an update operation. We may use this property instead of a timestamp value.
On Thu, Mar 17, 2011 at 9:27 PM, Shay Banon shay.banon@elasticsearch.com wrote:
Timestamp field is certainly possible, but the same problems with client machines exists for "server" machines, as oyu will need to sync the time between them.
On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote:Along the same lines, if ES could insert a timestamp, it would be useful as well (may be easier than incremental id?) Providing the timestamp by the client side introduces potential problems if the time at client machine does not match the time on the ES servers.
Regards,
Berkay Mollamustafaoglu
mberkay on yahoo, google and skypeOn Thu, Mar 17, 2011 at 2:34 PM, Shay Banon shay.banon@elasticsearch.com wrote:
No, there isn't. I guess you mean one that will increment for each index operation? Not now, but possible in the future.
On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote:Hi,
Is it possible to add a feature to configure auto increment properties for a type.--
Mustafa Sener
www.ifountain.com--
Mustafa Sener
www.ifountain.com--
Mustafa Sener
www.ifountain.com
Shay,
It’s most definitely because I’m a newb at ES, but how do you guarantee atomicity in doing so if you have multiple clients in multiple processes? You’d have to do a query and an update at the same time and have it be an ACID operation, which I don’t know that ES supports.
Or does it?
- Nick
From: Shay Banon [mailto:shay.banon@elasticsearch.com]
Sent: Thursday, March 17, 2011 5:43 PM
To: Mustafa Sener
Cc: users@elasticsearch.com; Berkay Mollamustafaoglu
Subject: Re: Auto incremental property
Global incremental time not. but timestamp field is possible (there is already an issue for that). But, you will have to make sure that time is sync'ed between elasticsearch servers. Though you can do it now, by just adding a timestamp field yourself.
On Thursday, March 17, 2011 at 10:22 PM, Mustafa Sener wrote:
We have many clients which populate data to ES and we want to process this data in the order they received by ES. If we assign this incremental or timestamp value from client each client should be synchronized and even this is not enough since the network time will affect the result. Think that we have a client which process the data according to this timestamp or incremental value. And assume the data has not arrived to ES while the processor client continue to process the records inserted at time. In this case a data loss will occur from the perspective of processor client.
I think instead of timestamp solution and incremental field setting support per index or type will be much better. Is it possible to implement such a feature?
On Thu, Mar 17, 2011 at 10:02 PM, Shay Banon shay.banon@elasticsearch.com wrote:
What are you going to use it for? The _version does exactly that internally (though you can't search on it), it auto increments when something changes.
On Thursday, March 17, 2011 at 10:00 PM, Mustafa Sener wrote:
Yes, I want to increment after each index operation or possibly after an update operation. We may use this property instead of a timestamp value.
On Thu, Mar 17, 2011 at 9:27 PM, Shay Banon shay.banon@elasticsearch.com wrote:
Timestamp field is certainly possible, but the same problems with client machines exists for "server" machines, as oyu will need to sync the time between them.
On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote:
Along the same lines, if ES could insert a timestamp, it would be useful as well (may be easier than incremental id?) Providing the timestamp by the client side introduces potential problems if the time at client machine does not match the time on the ES servers.
Regards,
Berkay Mollamustafaoglu
mberkay on yahoo, google and skype
On Thu, Mar 17, 2011 at 2:34 PM, Shay Banon shay.banon@elasticsearch.com wrote:
No, there isn't. I guess you mean one that will increment for each index operation? Not now, but possible in the future.
On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote:
Hi,
Is it possible to add a feature to configure auto increment properties for a type.
--
Mustafa Sener
www.ifountain.com
--
Mustafa Sener
www.ifountain.com
--
Mustafa Sener
www.ifountain.com
Automaticity on what?
On Thursday, March 17, 2011 at 11:46 PM, Administrator wrote:
Shay,
Itâs most definitely because Iâm a newb at ES, but how do you guarantee atomicity in doing so if you have multiple clients in multiple processes? Youâd have to do a query and an update at the same time and have it be an ACID operation, which I donât know that ES supports.
Or does it?
- Nick
From: Shay Banon [mailto:shay.banon@elasticsearch.com]
Sent: Thursday, March 17, 2011 5:43 PM
To: Mustafa Sener
Cc: users@elasticsearch.com; Berkay Mollamustafaoglu
Subject: Re: Auto incremental propertyGlobal incremental time not. but timestamp field is possible (there is already an issue for that). But, you will have to make sure that time is sync'ed between elasticsearch servers. Though you can do it now, by just adding a timestamp field yourself.
On Thursday, March 17, 2011 at 10:22 PM, Mustafa Sener wrote:We have many clients which populate data to ES and we want to process this data in the order they received by ES. If we assign this incremental or timestamp value from client each client should be synchronized and even this is not enough since the network time will affect the result. Think that we have a client which process the data according to this timestamp or incremental value. And assume the data has not arrived to ES while the processor client continue to process the records inserted at time. In this case a data loss will occur from the perspective of processor client.
I think instead of timestamp solution and incremental field setting support per index or type will be much better. Is it possible to implement such a feature?
On Thu, Mar 17, 2011 at 10:02 PM, Shay Banon shay.banon@elasticsearch.com wrote:
What are you going to use it for? The _version does exactly that internally (though you can't search on it), it auto increments when something changes.
On Thursday, March 17, 2011 at 10:00 PM, Mustafa Sener wrote:Yes, I want to increment after each index operation or possibly after an update operation. We may use this property instead of a timestamp value.
On Thu, Mar 17, 2011 at 9:27 PM, Shay Banon shay.banon@elasticsearch.com wrote:
Timestamp field is certainly possible, but the same problems with client machines exists for "server" machines, as oyu will need to sync the time between them.
On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote:Along the same lines, if ES could insert a timestamp, it would be useful as well (may be easier than incremental id?) Providing the timestamp by the client side introduces potential problems if the time at client machine does not match the time on the ES servers.
Regards,
Berkay Mollamustafaoglu
mberkay on yahoo, google and skypeOn Thu, Mar 17, 2011 at 2:34 PM, Shay Banon shay.banon@elasticsearch.com wrote:
No, there isn't. I guess you mean one that will increment for each index operation? Not now, but possible in the future.
On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote:Hi,
Is it possible to add a feature to configure auto increment properties for a type.--
Mustafa Sener
www.ifountain.com--
Mustafa Sener
www.ifountain.com--
Mustafa Sener
www.ifountain.com
Shay,
It does not have to be a timestamp value. It can be a long value which is
incremented after every write operation. This can be index based or global.
On Fri, Mar 18, 2011 at 12:27 AM, Shay Banon
shay.banon@elasticsearch.comwrote:
Automaticity on what?
On Thursday, March 17, 2011 at 11:46 PM, Administrator wrote:
Shay,
It’s most definitely because I’m a newb at ES, but how doyou guarantee atomicity in doing so if you have multiple clients in multiple
processes? You’d have to do a query and an update at the same time and have
it be an ACID operation, which I don’t know that ES supports.Or does it? - NickFrom: Shay Banon [mailto:shay.banon@elasticsearch.comshay.banon@elasticsearch.com]
Sent: Thursday, March 17, 2011 5:43 PM
To: Mustafa Sener
Cc: users@elasticsearch.com; Berkay Mollamustafaoglu
Subject: Re: Auto incremental propertyGlobal incremental time not. but timestamp field is possible (there is
already an issue for that). But, you will have to make sure that time is
sync'ed between elasticsearch servers. Though you can do it now, by just
adding a timestamp field yourself.On Thursday, March 17, 2011 at 10:22 PM, Mustafa Sener wrote:
We have many clients which populate data to ES and we want to process this
data in the order they received by ES. If we assign this incremental or
timestamp value from client each client should be synchronized and even this
is not enough since the network time will affect the result. Think that we
have a client which process the data according to this timestamp or
incremental value. And assume the data has not arrived to ES while the
processor client continue to process the records inserted at time. In this
case a data loss will occur from the perspective of processor client.I think instead of timestamp solution and incremental field setting support
per index or type will be much better. Is it possible to implement such a
feature?On Thu, Mar 17, 2011 at 10:02 PM, Shay Banon shay.banon@elasticsearch.com
wrote:What are you going to use it for? The _version does exactly that internally
(though you can't search on it), it auto increments when something changes.On Thursday, March 17, 2011 at 10:00 PM, Mustafa Sener wrote:
Yes, I want to increment after each index operation or possibly after an
update operation. We may use this property instead of a timestamp value.On Thu, Mar 17, 2011 at 9:27 PM, Shay Banon shay.banon@elasticsearch.com
wrote:Timestamp field is certainly possible, but the same problems with client
machines exists for "server" machines, as oyu will need to sync the time
between them.On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote:
Along the same lines, if ES could insert a timestamp, it would be useful as
well (may be easier than incremental id?) Providing the timestamp by the
client side introduces potential problems if the time at client machine does
not match the time on the ES servers.Regards,
Berkay Mollamustafaoglu
mberkay on yahoo, google and skypeOn Thu, Mar 17, 2011 at 2:34 PM, Shay Banon shay.banon@elasticsearch.com
wrote:No, there isn't. I guess you mean one that will increment for each index
operation? Not now, but possible in the future.On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote:
Hi,
Is it possible to add a feature to configure auto increment properties for
a type.--
Mustafa Sener
www.ifountain.com--
Mustafa Sener
www.ifountain.com--
Mustafa Sener
www.ifountain.com
--
Mustafa Sener
www.ifountain.com
I understand, but thats not going to be implemented because of the overhead involved in implementing it. Use timestamps.
On Friday, March 18, 2011 at 1:32 AM, Mustafa Sener wrote:
Shay,
It does not have to be a timestamp value. It can be a long value which is incremented after every write operation. This can be index based or global.On Fri, Mar 18, 2011 at 12:27 AM, Shay Banon shay.banon@elasticsearch.com wrote:
Automaticity on what?
On Thursday, March 17, 2011 at 11:46 PM, Administrator wrote:Shay,
Itâs most definitely because Iâm a newb at ES, but how do you guarantee atomicity in doing so if you have multiple clients in multiple processes? Youâd have to do a query and an update at the same time and have it be an ACID operation, which I donât know that ES supports.
Or does it?
- Nick
From: Shay Banon [mailto:shay.banon@elasticsearch.com]
Sent: Thursday, March 17, 2011 5:43 PM
To: Mustafa Sener
Cc: users@elasticsearch.com; Berkay Mollamustafaoglu
Subject: Re: Auto incremental propertyGlobal incremental time not. but timestamp field is possible (there is already an issue for that). But, you will have to make sure that time is sync'ed between elasticsearch servers. Though you can do it now, by just adding a timestamp field yourself.
On Thursday, March 17, 2011 at 10:22 PM, Mustafa Sener wrote:We have many clients which populate data to ES and we want to process this data in the order they received by ES. If we assign this incremental or timestamp value from client each client should be synchronized and even this is not enough since the network time will affect the result. Think that we have a client which process the data according to this timestamp or incremental value. And assume the data has not arrived to ES while the processor client continue to process the records inserted at time. In this case a data loss will occur from the perspective of processor client.
I think instead of timestamp solution and incremental field setting support per index or type will be much better. Is it possible to implement such a feature?
On Thu, Mar 17, 2011 at 10:02 PM, Shay Banon shay.banon@elasticsearch.com wrote:
What are you going to use it for? The _version does exactly that internally (though you can't search on it), it auto increments when something changes.
On Thursday, March 17, 2011 at 10:00 PM, Mustafa Sener wrote:Yes, I want to increment after each index operation or possibly after an update operation. We may use this property instead of a timestamp value.
On Thu, Mar 17, 2011 at 9:27 PM, Shay Banon shay.banon@elasticsearch.com wrote:
Timestamp field is certainly possible, but the same problems with client machines exists for "server" machines, as oyu will need to sync the time between them.
On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote:Along the same lines, if ES could insert a timestamp, it would be useful as well (may be easier than incremental id?) Providing the timestamp by the client side introduces potential problems if the time at client machine does not match the time on the ES servers.
Regards,
Berkay Mollamustafaoglu
mberkay on yahoo, google and skypeOn Thu, Mar 17, 2011 at 2:34 PM, Shay Banon shay.banon@elasticsearch.com wrote:
No, there isn't. I guess you mean one that will increment for each index operation? Not now, but possible in the future.
On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote:Hi,
Is it possible to add a feature to configure auto increment properties for a type.--
Mustafa Sener
www.ifountain.com--
Mustafa Sener
www.ifountain.com--
Mustafa Sener
www.ifountain.com--
Mustafa Sener
www.ifountain.com
Ok,
Thanks...
On Fri, Mar 18, 2011 at 1:34 AM, Shay Banon shay.banon@elasticsearch.comwrote:
I understand, but thats not going to be implemented because of the
overhead involved in implementing it. Use timestamps.On Friday, March 18, 2011 at 1:32 AM, Mustafa Sener wrote:
Shay,
It does not have to be a timestamp value. It can be a long value which is
incremented after every write operation. This can be index based or global.On Fri, Mar 18, 2011 at 12:27 AM, Shay Banon <shay.banon@elasticsearch.com
wrote:
Automaticity on what?
On Thursday, March 17, 2011 at 11:46 PM, Administrator wrote:
Shay,
It’s most definitely because I’m a newb at ES, but how doyou guarantee atomicity in doing so if you have multiple clients in multiple
processes? You’d have to do a query and an update at the same time and have
it be an ACID operation, which I don’t know that ES supports.Or does it? - NickFrom: Shay Banon [mailto:shay.banon@elasticsearch.comshay.banon@elasticsearch.com]
Sent: Thursday, March 17, 2011 5:43 PM
To: Mustafa Sener
Cc: users@elasticsearch.com; Berkay Mollamustafaoglu
Subject: Re: Auto incremental propertyGlobal incremental time not. but timestamp field is possible (there is
already an issue for that). But, you will have to make sure that time is
sync'ed between elasticsearch servers. Though you can do it now, by just
adding a timestamp field yourself.On Thursday, March 17, 2011 at 10:22 PM, Mustafa Sener wrote:
We have many clients which populate data to ES and we want to process this
data in the order they received by ES. If we assign this incremental or
timestamp value from client each client should be synchronized and even this
is not enough since the network time will affect the result. Think that we
have a client which process the data according to this timestamp or
incremental value. And assume the data has not arrived to ES while the
processor client continue to process the records inserted at time. In this
case a data loss will occur from the perspective of processor client.I think instead of timestamp solution and incremental field setting
support per index or type will be much better. Is it possible to implement
such a feature?On Thu, Mar 17, 2011 at 10:02 PM, Shay Banon <
shay.banon@elasticsearch.com> wrote:What are you going to use it for? The _version does exactly that
internally (though you can't search on it), it auto increments when
something changes.On Thursday, March 17, 2011 at 10:00 PM, Mustafa Sener wrote:
Yes, I want to increment after each index operation or possibly after an
update operation. We may use this property instead of a timestamp value.On Thu, Mar 17, 2011 at 9:27 PM, Shay Banon shay.banon@elasticsearch.com
wrote:Timestamp field is certainly possible, but the same problems with client
machines exists for "server" machines, as oyu will need to sync the time
between them.On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote:
Along the same lines, if ES could insert a timestamp, it would be useful
as well (may be easier than incremental id?) Providing the timestamp by the
client side introduces potential problems if the time at client machine does
not match the time on the ES servers.Regards,
Berkay Mollamustafaoglu
mberkay on yahoo, google and skypeOn Thu, Mar 17, 2011 at 2:34 PM, Shay Banon shay.banon@elasticsearch.com
wrote:No, there isn't. I guess you mean one that will increment for each index
operation? Not now, but possible in the future.On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote:
Hi,
Is it possible to add a feature to configure auto increment properties
for a type.--
Mustafa Sener
www.ifountain.com--
Mustafa Sener
www.ifountain.com--
Mustafa Sener
www.ifountain.com--
Mustafa Sener
www.ifountain.com
--
Mustafa Sener
www.ifountain.com
Perhaps this is implied by "timestamp", but most users think
of timestamps as date/time pairs.
For systems with high throughputs, depending on the
frequency of requests, you might run into an issue with the date/time being
the same.
However, you can easily use GUIDs as your timestamp if you
run into this problem.
While an auto-incremented id would be nice, I'm neutral
about it, but I think GUIDs would be a good solution for those that want to
guarantee some sort of concurrency.
Perhaps the proper request here would be to have the
timestamp provided by elasticsearch to be a GUID, or offer two timestamps,
one being the number of milliseconds since epoch, as well as a time-neutral
stamp which is a GUID.
- Nick
From: Mustafa Sener [mailto:mustafa.sener@gmail.com]
Sent: Thursday, March 17, 2011 8:34 PM
To: Shay Banon
Cc: users@elasticsearch.com
Subject: Re: RE: Auto incremental property
Ok,
Thanks...
On Fri, Mar 18, 2011 at 1:34 AM, Shay Banon shay.banon@elasticsearch.com
wrote:
I understand, but thats not going to be implemented because of the overhead
involved in implementing it. Use timestamps.
On Friday, March 18, 2011 at 1:32 AM, Mustafa Sener wrote:
Shay,
It does not have to be a timestamp value. It can be a long value which is
incremented after every write operation. This can be index based or global.
On Fri, Mar 18, 2011 at 12:27 AM, Shay Banon shay.banon@elasticsearch.com
wrote:
Automaticity on what?
On Thursday, March 17, 2011 at 11:46 PM, Administrator wrote:
Shay,
It's most definitely because I'm a newb at ES, but how do
you guarantee atomicity in doing so if you have multiple clients in multiple
processes? You'd have to do a query and an update at the same time and have
it be an ACID operation, which I don't know that ES supports.
Or does it?
- Nick
From: Shay Banon [mailto:shay.banon@elasticsearch.com]
Sent: Thursday, March 17, 2011 5:43 PM
To: Mustafa Sener
Cc: users@elasticsearch.com; Berkay Mollamustafaoglu
Subject: Re: Auto incremental property
Global incremental time not. but timestamp field is possible (there is
already an issue for that). But, you will have to make sure that time is
sync'ed between elasticsearch servers. Though you can do it now, by just
adding a timestamp field yourself.
On Thursday, March 17, 2011 at 10:22 PM, Mustafa Sener wrote:
We have many clients which populate data to ES and we want to process this
data in the order they received by ES. If we assign this incremental or
timestamp value from client each client should be synchronized and even this
is not enough since the network time will affect the result. Think that we
have a client which process the data according to this timestamp or
incremental value. And assume the data has not arrived to ES while the
processor client continue to process the records inserted at time. In this
case a data loss will occur from the perspective of processor client.
I think instead of timestamp solution and incremental field setting support
per index or type will be much better. Is it possible to implement such a
feature?
On Thu, Mar 17, 2011 at 10:02 PM, Shay Banon shay.banon@elasticsearch.com
wrote:
What are you going to use it for? The _version does exactly that internally
(though you can't search on it), it auto increments when something changes.
On Thursday, March 17, 2011 at 10:00 PM, Mustafa Sener wrote:
Yes, I want to increment after each index operation or possibly after an
update operation. We may use this property instead of a timestamp value.
On Thu, Mar 17, 2011 at 9:27 PM, Shay Banon shay.banon@elasticsearch.com
wrote:
Timestamp field is certainly possible, but the same problems with client
machines exists for "server" machines, as oyu will need to sync the time
between them.
On Thursday, March 17, 2011 at 9:25 PM, Berkay Mollamustafaoglu wrote:
Along the same lines, if ES could insert a timestamp, it would be useful as
well (may be easier than incremental id?) Providing the timestamp by the
client side introduces potential problems if the time at client machine does
not match the time on the ES servers.
Regards,
Berkay Mollamustafaoglu
mberkay on yahoo, google and skype
On Thu, Mar 17, 2011 at 2:34 PM, Shay Banon shay.banon@elasticsearch.com
wrote:
No, there isn't. I guess you mean one that will increment for each index
operation? Not now, but possible in the future.
On Thursday, March 17, 2011 at 4:07 PM, Mustafa Sener wrote:
Hi,
Is it possible to add a feature to configure auto increment properties for
a type.
--
Mustafa Sener
www.ifountain.com
--
Mustafa Sener
www.ifountain.com
--
Mustafa Sener
www.ifountain.com
--
Mustafa Sener
www.ifountain.com
--
Mustafa Sener
www.ifountain.com