How to create autoincrement id in ES using java_api in storm?

Elastic Stack Elasticsearch
1

Hi,

I am trying to create autoincrement id in ES in storm using TransportClient client. first i am checking the max id present in ES and then inserting existing max id +1 . But, its replicating same id for 3-4 times. Pelase help me if somebody would have created auto-increment id in ES using java API.

2

There are a couple of problems with this.

  • The probable reason for what you're seeing is that new documents only become searchable after a refresh. Refreshes take place periodically but can also be explicitly triggered (at a cost).
  • Checking the highest value and incrementing it by one requires that there at any time is no more than one thread or process that inserts values, or that you have a global lock preventing more than one of doing it at a time.
  • Unless you have a very low insertion rate these queries for each insert operation are going to be prohibitively expensive.

What problem are you trying to solve? Do you really need this kind of auto-incrementing id? How are you going to use this field?

3

Thank you Mognus, I think i will have to rethink on autoincrement id, actually we need this kind of unique autoincrement id for searching purpose, search by ID functionality.

4

Searching by id makes total sense, but I don't see why it has to be an auto-incrementing id. Is there anything wrong with the ids that Elasticsearch generates automatically?

5

To get the auto increment functionality you require
I would suggest using the scripting mechanism
Groovy script can do this atomic increment

6

I am fine with ids generated by ES but it is not readable , is there a way i can make it readable ?
autogenerated ids by ES are like ..AU-tUX0H6tLZvOocuO00 , AU-tUX0e6tLZvOocuO01

7

@whattheheck,

Please elaborate and example please, Thank you.