Es node security config/ java api


(K.B.) #1

Hello,

yesterday I finally had time to get started with ES. The install
initially was quite easy, but what puzzles me is the config, currently
I have:

name: shopCluster
network:
host: 127.0.0.1

and it works nice, but I have some questions regarding security,
because after my tests I let it run for quite a while and during this
period OTHER nodes showed up?!

I didnt start anything beside 1 client node and also didn't create the
then shown indexes - so I assume there was any other one ding same as
me - so how can I limit this to only certain IP adresses?

Beside that, the java-api got me some headache:

first, why can't there be a limited client-only-api? The full one not
only makes the war bigger, but because of the superior work from the
lucene guys lucene 3 and 2 cant be sitting next to each other as they
are not compatible. Meaning I first had to fiddle quite a big a bit
for this to work.

second, the api itself is quite difficult to understand, as it often
wont work like expected, e.g:

Map<String, Object> m = new HashMap<String, Object>();

        m.put("a", o[0]);
        m.put("b", o[1]);

        client.prepareIndex(index, type)
                .setSource(m).execute().actionGet();

works, while

BulkRequest bulk = Requests.bulkRequest();
...
bulk.add(Requests.indexRequest().source(m));

spews out a "Index not found" error - now as I'm trying to index >1000
doc's at once in a loop it seems to me a bulk will b faster for this -
so how can one index using bulk?

Best,

K.

PS: whats realy cool is the es + es-head as it nicely shows whats
where and, compared to SOLRs admin, it also works quite nice - even
I'm still not that comforatble with json... btw: how to get the json-
builder in java? the doc (http://www.elasticsearch.org/guide/reference/
java-api/index_.html) says:
jsonBuilder()
.startObject()
.field("user", "kimchy")
.field("postDate", new Date())
.field("message", "trying out Elastic Search")
.endObject()
but not what kind jsonBuilder is and whre it's from.....


(canal) #2

how to get jsonBuilder:

XContentBuilder b = XContentFactory.jsonBuilder();
thanks,
canal


From: K.B. korbinian.bachl@googlemail.com
To: users users@elasticsearch.com
Sent: Sat, March 5, 2011 5:35:08 PM
Subject: es node security config/ java api

Hello,

yesterday I finally had time to get started with ES. The install
initially was quite easy, but what puzzles me is the config, currently
I have:

name: shopCluster
network:
host: 127.0.0.1

and it works nice, but I have some questions regarding security,
because after my tests I let it run for quite a while and during this
period OTHER nodes showed up?!

I didnt start anything beside 1 client node and also didn't create the
then shown indexes - so I assume there was any other one ding same as
me - so how can I limit this to only certain IP adresses?

Beside that, the java-api got me some headache:

first, why can't there be a limited client-only-api? The full one not
only makes the war bigger, but because of the superior work from the
lucene guys lucene 3 and 2 cant be sitting next to each other as they
are not compatible. Meaning I first had to fiddle quite a big a bit
for this to work.

second, the api itself is quite difficult to understand, as it often
wont work like expected, e.g:

Map<String, Object> m = new HashMap<String, Object>();

        m.put("a", o[0]);
        m.put("b", o[1]);

        client.prepareIndex(index, type)
                .setSource(m).execute().actionGet();

works, while

BulkRequest bulk = Requests.bulkRequest();
...
bulk.add(Requests.indexRequest().source(m));

spews out a "Index not found" error - now as I'm trying to index >1000
doc's at once in a loop it seems to me a bulk will b faster for this -
so how can one index using bulk?

Best,

K.

PS: whats realy cool is the es + es-head as it nicely shows whats
where and, compared to SOLRs admin, it also works quite nice - even
I'm still not that comforatble with json... btw: how to get the json-
builder in java? the doc (http://www.elasticsearch.org/guide/reference/
java-api/index_.html) says:
jsonBuilder()
.startObject()
.field("user", "kimchy")
.field("postDate", new Date())
.field("message", "trying out Elastic Search")
.endObject()
but not what kind jsonBuilder is and whre it's from.....


(K.B.) #3

Thank you canal!

Didnt notice the Factory :slight_smile:

any clue why I had these nodes?

On 5 Mrz., 10:41, go canal goca...@yahoo.com wrote:

how to get jsonBuilder:

XContentBuilder b = XContentFactory.jsonBuilder();
thanks,
canal


From: K.B. korbinian.ba...@googlemail.com
To: users us...@elasticsearch.com
Sent: Sat, March 5, 2011 5:35:08 PM
Subject: es node security config/ java api

Hello,

yesterday I finally had time to get started with ES. The install
initially was quite easy, but what puzzles me is the config, currently
I have:

name: shopCluster
network:
host: 127.0.0.1

and it works nice, but I have some questions regarding security,
because after my tests I let it run for quite a while and during this
period OTHER nodes showed up?!

I didnt start anything beside 1 client node and also didn't create the
then shown indexes - so I assume there was any other one ding same as
me - so how can I limit this to only certain IP adresses?

Beside that, the java-api got me some headache:

first, why can't there be a limited client-only-api? The full one not
only makes the war bigger, but because of the superior work from the
lucene guys lucene 3 and 2 cant be sitting next to each other as they
are not compatible. Meaning I first had to fiddle quite a big a bit
for this to work.

second, the api itself is quite difficult to understand, as it often
wont work like expected, e.g:

Map<String, Object> m = new HashMap<String, Object>();

        m.put("a", o[0]);
        m.put("b", o[1]);

        client.prepareIndex(index, type)
                .setSource(m).execute().actionGet();

works, while

BulkRequest bulk = Requests.bulkRequest();
...
bulk.add(Requests.indexRequest().source(m));

spews out a "Index not found" error - now as I'm trying to index >1000
doc's at once in a loop it seems to me a bulk will b faster for this -
so how can one index using bulk?

Best,

K.

PS: whats realy cool is the es + es-head as it nicely shows whats
where and, compared to SOLRs admin, it also works quite nice - even
I'm still not that comforatble with json... btw: how to get the json-
builder in java? the doc (http://www.elasticsearch.org/guide/reference/
java-api/index_.html) says:
jsonBuilder()
.startObject()
.field("user", "kimchy")
.field("postDate", new Date())
.field("message", "trying out Elastic Search")
.endObject()
but not what kind jsonBuilder is and whre it's from.....


(Lukáš Vlček) #4

Hi,

see my answers inlined.

Regards,
Lukas

On Sat, Mar 5, 2011 at 10:35 AM, K.B. korbinian.bachl@googlemail.comwrote:

Hello,

yesterday I finally had time to get started with ES. The install
initially was quite easy, but what puzzles me is the config, currently
I have:

name: shopCluster
network:
host: 127.0.0.1

and it works nice, but I have some questions regarding security,
because after my tests I let it run for quite a while and during this
period OTHER nodes showed up?!

I didnt start anything beside 1 client node and also didn't create the
then shown indexes - so I assume there was any other one ding same as
me - so how can I limit this to only certain IP adresses?

Interesting that somebody gave his/her cluster the same name. Anyway, you
can specify IP addresses that can connect to your cluster. For example the
following is one configuration I used some time ago (note: IP addresses were
cooked up for this example).

network :
host : 12.16.93.184

discovery.zen.ping.multicast :
enabled : flase

discovery.zen.ping.unicast :
hosts : ["12.16.93.208:9300","12.16.93.208:9301","12.16.93.184:9300"]

This way I am telling that the cluster can be made of three nodes, two
running on *.208 and one running on *.184 on the said ports.

Check here
http://www.elasticsearch.org/guide/reference/modules/discovery/zen.html if
you are using Zen discovery module.

Beside that, the java-api got me some headache:

first, why can't there be a limited client-only-api? The full one not
only makes the war bigger, but because of the superior work from the
lucene guys lucene 3 and 2 cant be sitting next to each other as they
are not compatible. Meaning I first had to fiddle quite a big a bit
for this to work.

Yea, this question gets asked sometimes. As far as I know there is no
"client-only" API package (and probably the reason is that you can have a
node client that becomes a full member of the cluster thus it needs a lot of
packages that are needed by standalone nodes too).

second, the api itself is quite difficult to understand, as it often
wont work like expected, e.g:

Map<String, Object> m = new HashMap<String, Object>();

       m.put("a", o[0]);
       m.put("b", o[1]);

       client.prepareIndex(index, type)
               .setSource(m).execute().actionGet();

works, while

BulkRequest bulk = Requests.bulkRequest();
...
bulk.add(Requests.indexRequest().source(m));

spews out a "Index not found" error - now as I'm trying to index >1000
doc's at once in a loop it seems to me a bulk will b faster for this -
so how can one index using bulk?

Best,

K.

PS: whats realy cool is the es + es-head as it nicely shows whats
where and, compared to SOLRs admin, it also works quite nice - even
I'm still not that comforatble with json... btw: how to get the json-
builder in java? the doc (http://www.elasticsearch.org/guide/reference/
java-api/index_.html) says:
jsonBuilder()
.startObject()
.field("user", "kimchy")
.field("postDate", new Date())
.field("message", "trying out Elastic Search")
.endObject()
but not what kind jsonBuilder is and whre it's from.....

Answered by Go canal.


(K.B.) #5

Thank you very much! This was exactly what I'm looking for to limit
my nodes to my IPs !

Best,

K.

PS: ES is really nice, but it takes so time to get used to the
principles behind it

On 5 Mrz., 12:15, Lukáš Vlček lukas.vl...@gmail.com wrote:

Hi,

see my answers inlined.

Regards,
Lukas

On Sat, Mar 5, 2011 at 10:35 AM, K.B. korbinian.ba...@googlemail.comwrote:

Hello,

yesterday I finally had time to get started with ES. The install
initially was quite easy, but what puzzles me is the config, currently
I have:

name: shopCluster
network:
host: 127.0.0.1

and it works nice, but I have some questions regarding security,
because after my tests I let it run for quite a while and during this
period OTHER nodes showed up?!

I didnt start anything beside 1 client node and also didn't create the
then shown indexes - so I assume there was any other one ding same as
me - so how can I limit this to only certain IP adresses?

Interesting that somebody gave his/her cluster the same name. Anyway, you
can specify IP addresses that can connect to your cluster. For example the
following is one configuration I used some time ago (note: IP addresses were
cooked up for this example).

network :
host : 12.16.93.184

discovery.zen.ping.multicast :
enabled : flase

discovery.zen.ping.unicast :
hosts : ["12.16.93.208:9300","12.16.93.208:9301","12.16.93.184:9300"]

This way I am telling that the cluster can be made of three nodes, two
running on *.208 and one running on *.184 on the said ports.

Check herehttp://www.elasticsearch.org/guide/reference/modules/discovery/zen.htmlif
you are using Zen discovery module.

Beside that, the java-api got me some headache:

first, why can't there be a limited client-only-api? The full one not
only makes the war bigger, but because of the superior work from the
lucene guys lucene 3 and 2 cant be sitting next to each other as they
are not compatible. Meaning I first had to fiddle quite a big a bit
for this to work.

Yea, this question gets asked sometimes. As far as I know there is no
"client-only" API package (and probably the reason is that you can have a
node client that becomes a full member of the cluster thus it needs a lot of
packages that are needed by standalone nodes too).

second, the api itself is quite difficult to understand, as it often
wont work like expected, e.g:

Map<String, Object> m = new HashMap<String, Object>();

       m.put("a", o[0]);
       m.put("b", o[1]);
       client.prepareIndex(index, type)
               .setSource(m).execute().actionGet();

works, while

BulkRequest bulk = Requests.bulkRequest();
...
bulk.add(Requests.indexRequest().source(m));

spews out a "Index not found" error - now as I'm trying to index >1000
doc's at once in a loop it seems to me a bulk will b faster for this -
so how can one index using bulk?

Best,

K.

PS: whats realy cool is the es + es-head as it nicely shows whats
where and, compared to SOLRs admin, it also works quite nice - even
I'm still not that comforatble with json... btw: how to get the json-
builder in java? the doc (http://www.elasticsearch.org/guide/reference/
java-api/index_.html) says:
jsonBuilder()
.startObject()
.field("user", "kimchy")
.field("postDate", new Date())
.field("message", "trying out Elastic Search")
.endObject()
but not what kind jsonBuilder is and whre it's from.....

Answered by Go canal.


(fashionalwallet) #6
  • deleted -

(system) #7