Es node security config/ java api

Hello,

yesterday I finally had time to get started with ES. The install
initially was quite easy, but what puzzles me is the config, currently
I have:

name: shopCluster
network:
host: 127.0.0.1

and it works nice, but I have some questions regarding security,
because after my tests I let it run for quite a while and during this
period OTHER nodes showed up?!

I didnt start anything beside 1 client node and also didn't create the
then shown indexes - so I assume there was any other one ding same as
me - so how can I limit this to only certain IP adresses?

Beside that, the java-api got me some headache:

first, why can't there be a limited client-only-api? The full one not
only makes the war bigger, but because of the superior work from the
lucene guys lucene 3 and 2 cant be sitting next to each other as they
are not compatible. Meaning I first had to fiddle quite a big a bit
for this to work.

second, the api itself is quite difficult to understand, as it often
wont work like expected, e.g:

Map<String, Object> m = new HashMap<String, Object>();

        m.put("a", o[0]);
        m.put("b", o[1]);

        client.prepareIndex(index, type)
                .setSource(m).execute().actionGet();

works, while

BulkRequest bulk = Requests.bulkRequest();
...
bulk.add(Requests.indexRequest().source(m));

spews out a "Index not found" error - now as I'm trying to index >1000
doc's at once in a loop it seems to me a bulk will b faster for this -
so how can one index using bulk?

Best,

K.

PS: whats realy cool is the es + es-head as it nicely shows whats
where and, compared to SOLRs admin, it also works quite nice - even
I'm still not that comforatble with json... btw: how to get the json-
builder in java? the doc (http://www.elasticsearch.org/guide/reference/
java-api/index_.html) says:
jsonBuilder()
.startObject()
.field("user", "kimchy")
.field("postDate", new Date())
.field("message", "trying out Elastic Search")
.endObject()
but not what kind jsonBuilder is and whre it's from.....

how to get jsonBuilder:

XContentBuilder b = XContentFactory.jsonBuilder();
thanks,
canal


From: K.B. korbinian.bachl@googlemail.com
To: users users@elasticsearch.com
Sent: Sat, March 5, 2011 5:35:08 PM
Subject: es node security config/ java api

Hello,

yesterday I finally had time to get started with ES. The install
initially was quite easy, but what puzzles me is the config, currently
I have:

name: shopCluster
network:
host: 127.0.0.1

and it works nice, but I have some questions regarding security,
because after my tests I let it run for quite a while and during this
period OTHER nodes showed up?!

I didnt start anything beside 1 client node and also didn't create the
then shown indexes - so I assume there was any other one ding same as
me - so how can I limit this to only certain IP adresses?

Beside that, the java-api got me some headache:

first, why can't there be a limited client-only-api? The full one not
only makes the war bigger, but because of the superior work from the
lucene guys lucene 3 and 2 cant be sitting next to each other as they
are not compatible. Meaning I first had to fiddle quite a big a bit
for this to work.

second, the api itself is quite difficult to understand, as it often
wont work like expected, e.g:

Map<String, Object> m = new HashMap<String, Object>();

        m.put("a", o[0]);
        m.put("b", o[1]);

        client.prepareIndex(index, type)
                .setSource(m).execute().actionGet();

works, while

BulkRequest bulk = Requests.bulkRequest();
...
bulk.add(Requests.indexRequest().source(m));

spews out a "Index not found" error - now as I'm trying to index >1000
doc's at once in a loop it seems to me a bulk will b faster for this -
so how can one index using bulk?

Best,

K.

PS: whats realy cool is the es + es-head as it nicely shows whats
where and, compared to SOLRs admin, it also works quite nice - even
I'm still not that comforatble with json... btw: how to get the json-
builder in java? the doc (http://www.elasticsearch.org/guide/reference/
java-api/index_.html) says:
jsonBuilder()
.startObject()
.field("user", "kimchy")
.field("postDate", new Date())
.field("message", "trying out Elastic Search")
.endObject()
but not what kind jsonBuilder is and whre it's from.....

Thank you canal!

Didnt notice the Factory :slight_smile:

any clue why I had these nodes?

On 5 Mrz., 10:41, go canal goca...@yahoo.com wrote:

how to get jsonBuilder:

XContentBuilder b = XContentFactory.jsonBuilder();
thanks,
canal


From: K.B. korbinian.ba...@googlemail.com
To: users us...@elasticsearch.com
Sent: Sat, March 5, 2011 5:35:08 PM
Subject: es node security config/ java api

Hello,

yesterday I finally had time to get started with ES. The install
initially was quite easy, but what puzzles me is the config, currently
I have:

name: shopCluster
network:
host: 127.0.0.1

and it works nice, but I have some questions regarding security,
because after my tests I let it run for quite a while and during this
period OTHER nodes showed up?!

I didnt start anything beside 1 client node and also didn't create the
then shown indexes - so I assume there was any other one ding same as
me - so how can I limit this to only certain IP adresses?

Beside that, the java-api got me some headache:

first, why can't there be a limited client-only-api? The full one not
only makes the war bigger, but because of the superior work from the
lucene guys lucene 3 and 2 cant be sitting next to each other as they
are not compatible. Meaning I first had to fiddle quite a big a bit
for this to work.

second, the api itself is quite difficult to understand, as it often
wont work like expected, e.g:

Map<String, Object> m = new HashMap<String, Object>();

        m.put("a", o[0]);
        m.put("b", o[1]);

        client.prepareIndex(index, type)
                .setSource(m).execute().actionGet();

works, while

BulkRequest bulk = Requests.bulkRequest();
...
bulk.add(Requests.indexRequest().source(m));

spews out a "Index not found" error - now as I'm trying to index >1000
doc's at once in a loop it seems to me a bulk will b faster for this -
so how can one index using bulk?

Best,

K.

PS: whats realy cool is the es + es-head as it nicely shows whats
where and, compared to SOLRs admin, it also works quite nice - even
I'm still not that comforatble with json... btw: how to get the json-
builder in java? the doc (Elasticsearch Platform — Find real-time answers at scale | Elastic
java-api/index_.html) says:
jsonBuilder()
.startObject()
.field("user", "kimchy")
.field("postDate", new Date())
.field("message", "trying out Elastic Search")
.endObject()
but not what kind jsonBuilder is and whre it's from.....

Hi,

see my answers inlined.

Regards,
Lukas

On Sat, Mar 5, 2011 at 10:35 AM, K.B. korbinian.bachl@googlemail.comwrote:

Hello,

yesterday I finally had time to get started with ES. The install
initially was quite easy, but what puzzles me is the config, currently
I have:

name: shopCluster
network:
host: 127.0.0.1

and it works nice, but I have some questions regarding security,
because after my tests I let it run for quite a while and during this
period OTHER nodes showed up?!

I didnt start anything beside 1 client node and also didn't create the
then shown indexes - so I assume there was any other one ding same as
me - so how can I limit this to only certain IP adresses?

Interesting that somebody gave his/her cluster the same name. Anyway, you
can specify IP addresses that can connect to your cluster. For example the
following is one configuration I used some time ago (note: IP addresses were
cooked up for this example).

network :
host : 12.16.93.184

discovery.zen.ping.multicast :
enabled : flase

discovery.zen.ping.unicast :
hosts : ["12.16.93.208:9300","12.16.93.208:9301","12.16.93.184:9300"]

This way I am telling that the cluster can be made of three nodes, two
running on *.208 and one running on *.184 on the said ports.

Check here
Elasticsearch Platform — Find real-time answers at scale | Elastic if
you are using Zen discovery module.

Beside that, the java-api got me some headache:

first, why can't there be a limited client-only-api? The full one not
only makes the war bigger, but because of the superior work from the
lucene guys lucene 3 and 2 cant be sitting next to each other as they
are not compatible. Meaning I first had to fiddle quite a big a bit
for this to work.

Yea, this question gets asked sometimes. As far as I know there is no
"client-only" API package (and probably the reason is that you can have a
node client that becomes a full member of the cluster thus it needs a lot of
packages that are needed by standalone nodes too).

second, the api itself is quite difficult to understand, as it often
wont work like expected, e.g:

Map<String, Object> m = new HashMap<String, Object>();

       m.put("a", o[0]);
       m.put("b", o[1]);

       client.prepareIndex(index, type)
               .setSource(m).execute().actionGet();

works, while

BulkRequest bulk = Requests.bulkRequest();
...
bulk.add(Requests.indexRequest().source(m));

spews out a "Index not found" error - now as I'm trying to index >1000
doc's at once in a loop it seems to me a bulk will b faster for this -
so how can one index using bulk?

Best,

K.

PS: whats realy cool is the es + es-head as it nicely shows whats
where and, compared to SOLRs admin, it also works quite nice - even
I'm still not that comforatble with json... btw: how to get the json-
builder in java? the doc (Elasticsearch Platform — Find real-time answers at scale | Elastic
java-api/index_.html) says:
jsonBuilder()
.startObject()
.field("user", "kimchy")
.field("postDate", new Date())
.field("message", "trying out Elastic Search")
.endObject()
but not what kind jsonBuilder is and whre it's from.....

Answered by Go canal.

Thank you very much! This was exactly what I'm looking for to limit
my nodes to my IPs !

Best,

K.

PS: ES is really nice, but it takes so time to get used to the
principles behind it

On 5 Mrz., 12:15, Lukáš Vlček lukas.vl...@gmail.com wrote:

Hi,

see my answers inlined.

Regards,
Lukas

On Sat, Mar 5, 2011 at 10:35 AM, K.B. korbinian.ba...@googlemail.comwrote:

Hello,

yesterday I finally had time to get started with ES. The install
initially was quite easy, but what puzzles me is the config, currently
I have:

name: shopCluster
network:
host: 127.0.0.1

and it works nice, but I have some questions regarding security,
because after my tests I let it run for quite a while and during this
period OTHER nodes showed up?!

I didnt start anything beside 1 client node and also didn't create the
then shown indexes - so I assume there was any other one ding same as
me - so how can I limit this to only certain IP adresses?

Interesting that somebody gave his/her cluster the same name. Anyway, you
can specify IP addresses that can connect to your cluster. For example the
following is one configuration I used some time ago (note: IP addresses were
cooked up for this example).

network :
host : 12.16.93.184

discovery.zen.ping.multicast :
enabled : flase

discovery.zen.ping.unicast :
hosts : ["12.16.93.208:9300","12.16.93.208:9301","12.16.93.184:9300"]

This way I am telling that the cluster can be made of three nodes, two
running on *.208 and one running on *.184 on the said ports.

Check herehttp://www.elasticsearch.org/guide/reference/modules/discovery/zen.htmlif
you are using Zen discovery module.

Beside that, the java-api got me some headache:

first, why can't there be a limited client-only-api? The full one not
only makes the war bigger, but because of the superior work from the
lucene guys lucene 3 and 2 cant be sitting next to each other as they
are not compatible. Meaning I first had to fiddle quite a big a bit
for this to work.

Yea, this question gets asked sometimes. As far as I know there is no
"client-only" API package (and probably the reason is that you can have a
node client that becomes a full member of the cluster thus it needs a lot of
packages that are needed by standalone nodes too).

second, the api itself is quite difficult to understand, as it often
wont work like expected, e.g:

Map<String, Object> m = new HashMap<String, Object>();

       m.put("a", o[0]);
       m.put("b", o[1]);
       client.prepareIndex(index, type)
               .setSource(m).execute().actionGet();

works, while

BulkRequest bulk = Requests.bulkRequest();
...
bulk.add(Requests.indexRequest().source(m));

spews out a "Index not found" error - now as I'm trying to index >1000
doc's at once in a loop it seems to me a bulk will b faster for this -
so how can one index using bulk?

Best,

K.

PS: whats realy cool is the es + es-head as it nicely shows whats
where and, compared to SOLRs admin, it also works quite nice - even
I'm still not that comforatble with json... btw: how to get the json-
builder in java? the doc (Elasticsearch Platform — Find real-time answers at scale | Elastic
java-api/index_.html) says:
jsonBuilder()
.startObject()
.field("user", "kimchy")
.field("postDate", new Date())
.field("message", "trying out Elastic Search")
.endObject()
but not what kind jsonBuilder is and whre it's from.....

Answered by Go canal.

  • deleted -