Autocomplete with ES (newbie)

Elastic Stack Elasticsearch
1

Hi,

I have an autocomplete system in my application that works fine with ES.
It exposes all search results via REST service written in Java with Spring
MVC.
So I'm using TransportClient to fetch my data stored on my local machine
and
I'm running ES with this config :
index.number_of_shards: 1
index.number_of_replicas: 0
BUT what I'm not understanding is why each time the autocomplete service is
called,
(when someone type a letter)
there is a new node that is created with a new name ?
Is this the normal behavior of ES ?

Thx

--

2

Hello!

Do you reuse a single instance of TransportClient or create a new one with each call ?

--

Regards,

Rafał Kuć

Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch

Hi,

I have an autocomplete system in my application that works fine with ES.

It exposes all search results via REST service written in Java with Spring MVC.

So I'm using TransportClient to fetch my data stored on my local machine and

I'm running ES with this config :

 index.number_of_shards: 1


 index.number_of_replicas: 0

BUT what I'm not understanding is why each time the autocomplete service is called,

(when someone type a letter)

there is a new node that is created with a new name ?

Is this the normal behavior of ES ?

Thx

--

3

Now I create a new one with each call.

So I'll try to reuse a single instance !
I'll let u know how it is.

--

4

I try now to reuse a single instance but it raises a No Node available
Exception

--

5

Hello!

Do you have ElasticSearch running ? Try sending a single query to it to see if it responds.

--

Regards,

Rafał Kuć

Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch

I try now to reuse a single instance but it raises a No Node available Exception

--

6

My ES instance is running (0.19.10): I have done :
http://localhost:9200/_cluster/nodes/_local
and I get this response :

{

  • ok: true,
  • cluster_name: "elasticsearch",
  • nodes:
    {
    • 7k5aeHwoSd6LHv2erbcJCg:
      {
      • name: "Marko, Cain",
      • transport_address: "inet[/10.0.2.113:9300]",
      • hostname: "MOE005X",
      • http_address: "inet[/10.0.2.113:9200]"
        }
        }

}

Here is how I set my TransportClient :

private Client client; public ElasticSearchController() { super();
initESClient(); } public void initESClient(){ Settings settings =
ImmutableSettings.settingsBuilder() .put("client.transport.sniff", true) .
put("client.transport.ignore_cluster_name", true) .build(); client = new
TransportClient(settings).addTransportAddress(new InetSocketTransportAddress
(ES_INET.getValue(), 9300)); }

Then I just call the client object in the others methods like this :

QueryBuilder qb = QueryBuilders.queryString("" + term.toLowerCase() + "")
.useDisMax(true)
.defaultOperator(QueryStringQueryBuilder.Operator.AND)
.field("firstname")
.field("lastname")
.field("zip")
.field("city");
SearchResponse response = client.prepareSearch("peopledb")
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setQuery(qb)
.setFrom(0)
.setSize(10)
.setExplain(true)
.execute()
.actionGet();

Maybe I have missed something ?

--

7

Hello!

Try running http://localhost:9200/_cluster/health and please share the result.

And one more thing - what is TransportClient saying when connecting to your cluster ?

--

Regards,

Rafał Kuć

Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch

My ES instance is running (0.19.10): I have done : http://localhost:9200/_cluster/nodes/_local

and I get this response :

{

ok: true,
cluster_name: "elasticsearch",
nodes:

{

7k5aeHwoSd6LHv2erbcJCg:

{

name: "Marko, Cain",
transport_address: "inet[/10.0.2.113:9300]",
hostname: "MOE005X",
http_address: "inet[/10.0.2.113:9200]"

}

}

}

Here is how I set my TransportClient :

private Client client;

public ElasticSearchController() {

super();

initESClient();

}

public void initESClient(){

Settings settings = ImmutableSettings.settingsBuilder()

.put("client.transport.sniff", true)



.put("client.transport.ignore_cluster_name", true)



.build();

client = new TransportClient(settings).addTransportAddress(new InetSocketTransportAddress(ES_INET.getValue(), 9300));

}

Then I just call the client object in the others methods like this :

QueryBuilder qb = QueryBuilders.queryString("" + term.toLowerCase() + "")

       .useDisMax(true)


                   .defaultOperator(QueryStringQueryBuilder.Operator.AND)    


                   .field("firstname")


                   .field("lastname")


                   .field("zip")


                   .field("city");

SearchResponse response = client.prepareSearch("peopledb")

       .setSearchType(SearchType.DFS_QUERY_THEN_FETCH)


       .setQuery(qb)


       .setFrom(0)


       .setSize(10)


       .setExplain(true)


       .execute()


       .actionGet();

Maybe I have missed something ?

--

8

Result of http://localhost:9200/_cluster/healthhttp://localhost:9200/_cluster/nodes/_local
:
{

  • cluster_name: "elasticsearch",
  • status: "green",
  • timed_out: false,
  • number_of_nodes: 1,
  • number_of_data_nodes: 1,
  • active_primary_shards: 2,
  • active_shards: 2,
  • relocating_shards: 0,
  • initializing_shards: 0,
  • unassigned_shards: 0

}

Here the log from TransportClient :

2012-10-17 16:12:10,355 [elasticsearch[Carolyn Parmenter][generic][T#3]]
INFO org.elasticsearch.client.transport - [Carolyn Parmenter] failed to
get local cluster state for [#transport#-1][inet[/10.0.2.113:9200]],
disconnecting...
org.elasticsearch.transport.ReceiveTimeoutTransportException: [][inet[
/10.0.2.113:9200]][cluster/state] request_id [24] timed out after [5001ms]
at org.elasticsearch.transport.TransportService$TimeoutHandler.run(
TransportService.java:347)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(
ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

Here the log from the ES console :

[2012-10-17 16:07:23,401][INFO ][node ] [Charcoal] {0.19.10}[8732]:
initializing ... [2012-10-17 16:07:23,411][INFO ][plugins ] [Charcoal]
loaded [], sites [] [2012-10-17 16:07:27,726][INFO ][node ] [Charcoal]
{0.19.10}[8732]: initialized [2012-10-17 16:07:27,727][INFO ][node ]
[Charcoal] {0.19.10}[8732]: starting ... [2012-10-17 16:07:28,006][INFO ][transport
] [Charcoal] bound_address {inet[/0.0.0.0:9300]}, publish_address {inet[/10.0.2.113:9300]}
[2012-10-17 16:07:31,224][INFO ][cluster.service ] [Charcoal] new_master
[Charcoal][U6xpMVUERYmc0s0ZZahXCw][inet[/10.0.2.113:9300]], reason: zen-disco-join
(elected_as_master) [2012-10-17 16:07:31,272][INFO ][discovery ] [Charcoal]
elasticsearch/U6xpMVUERYmc0s0ZZahXCw [2012-10-17 16:07:31,422][INFO ][http
] [Charcoal] bound_address {inet[/0.0.0.0:9200]}, publish_address {inet[/10.0.2.113:9200]}
[2012-10-17 16:07:31,423][INFO ][node ] [Charcoal] {0.19.10}[8732]: started
[2012-10-17 16:07:32,590][INFO ][gateway ] [Charcoal] recovered [2] indices
into cluster_state [2012-10-17 16:09:50,263][WARN ][http.netty ] [Charcoal]
Caught exception while handling client http traffic, closing connection
[id: 0x7e2b5f3a, /10.0 .2.113:53013 => /10.0.2.113:9200] java.lang.IllegalArgumentException:
empty text atorg.elasticsearch.common.netty.handler.codec.http.HttpVersion.<
init>(HttpVersion.java:97) atorg.elasticsearch.common.netty.handler.codec.http.HttpVersion.valueOf(HttpVersion.java:62)
atorg.elasticsearch.common.netty.handler.codec.http.HttpRequestDecoder.createMessage(HttpRequestDecoder.java:76)
atorg.elasticsearch.common.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:189)
atorg.elasticsearch.common.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:101)
atorg.elasticsearch.common.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:502)
atorg.elasticsearch.common.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:437)
atorg.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:75)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:565)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:793)
at
org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:74)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:565)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:84)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:471)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:332)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
at
org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102)
at
org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(
ThreadPoolExecutor.java:886) at java.util.concurrent.
ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.
Thread.run(Thread.java:662) [2012-10-17 16:09:50,268][WARN ][http.netty ]
[Charcoal] Caught exception while handling client http traffic, closing
connection [id: 0x7e2b5f3a, /10.0 .2.113:53013 :> /10.0.2.113:9200] java.
lang.IllegalArgumentException: empty text

--

9

Hello!

As you can see there is an error when TransportClient tries to connect to your cluster. Do you use 0.19.10 version of ES library in your application ?

--

Regards,

Rafał Kuć

Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch

Result of http://localhost:9200/_cluster/health :

{

cluster_name: "elasticsearch",
status: "green",
timed_out: false,
number_of_nodes: 1,
number_of_data_nodes: 1,
active_primary_shards: 2,
active_shards: 2,
relocating_shards: 0,
initializing_shards: 0,
unassigned_shards: 0

}

Here the log from TransportClient :

2012-10-17 16:12:10,355 [elasticsearch[Carolyn Parmenter][generic][T#3]] INFO org.elasticsearch.client.transport - [Carolyn Parmenter] failed to get local cluster state for [#transport#-1][inet[/10.0.2.113:9200]], disconnecting...

org.elasticsearch.transport.ReceiveTimeoutTransportException: [][inet[/10.0.2.113:9200]][cluster/state] request_id [24] timed out after [5001ms]

at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:347)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:662)

Here the log from the ES console :

[2012-10-17 16:07:23,401][INFO ][node ] [Charcoal] {0.19.10}[8732]: initializing ...

[2012-10-17 16:07:23,411][INFO ][plugins ] [Charcoal] loaded [], sites []

[2012-10-17 16:07:27,726][INFO ][node ] [Charcoal] {0.19.10}[8732]: initialized

[2012-10-17 16:07:27,727][INFO ][node ] [Charcoal] {0.19.10}[8732]: starting ...

[2012-10-17 16:07:28,006][INFO ][transport ] [Charcoal] bound_address {inet[/0.0.0.0:9300]}, publish_address {inet[/10.0.2.113:9300]}

[2012-10-17 16:07:31,224][INFO ][cluster.service ] [Charcoal] new_master [Charcoal][U6xpMVUERYmc0s0ZZahXCw][inet[/10.0.2.113:9300]], reason: zen-disco-join

(elected_as_master)

[2012-10-17 16:07:31,272][INFO ][discovery ] [Charcoal] elasticsearch/U6xpMVUERYmc0s0ZZahXCw

[2012-10-17 16:07:31,422][INFO ][http ] [Charcoal] bound_address {inet[/0.0.0.0:9200]}, publish_address {inet[/10.0.2.113:9200]}

[2012-10-17 16:07:31,423][INFO ][node ] [Charcoal] {0.19.10}[8732]: started

[2012-10-17 16:07:32,590][INFO ][gateway ] [Charcoal] recovered [2] indices into cluster_state

[2012-10-17 16:09:50,263][WARN ][http.netty ] [Charcoal] Caught exception while handling client http traffic, closing connection [id: 0x7e2b5f3a, /10.0

.2.113:53013 => /10.0.2.113:9200]

java.lang.IllegalArgumentException: empty text

    at org.elasticsearch.common.netty.handler.codec.http.HttpVersion.&lt;init&gt;(HttpVersion.java:97)



    at org.elasticsearch.common.netty.handler.codec.http.HttpVersion.valueOf(HttpVersion.java:62)



    at org.elasticsearch.common.netty.handler.codec.http.HttpRequestDecoder.createMessage(HttpRequestDecoder.java:76)



    at org.elasticsearch.common.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:189)



    at org.elasticsearch.common.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:101)



    at org.elasticsearch.common.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:502)



    at org.elasticsearch.common.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:437)



    at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:75)



    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:565)



    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:793)



    at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:74)



    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:565)



    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560)



    at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)



    at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)



    at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:84)



    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:471)



    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:332)



    at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)



    at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102)



    at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)



    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)



    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)



    at java.lang.Thread.run(Thread.java:662)

[2012-10-17 16:09:50,268][WARN ][http.netty ] [Charcoal] Caught exception while handling client http traffic, closing connection [id: 0x7e2b5f3a, /10.0

.2.113:53013 :> /10.0.2.113:9200]

java.lang.IllegalArgumentException: empty text

--

10

YES ! I'm using Maven 3.

--

11

Just in case you are using Spring, you can use the spring factories for Elasticsearch: https://github.com/dadoonet/spring-elasticsearch

It could help you to connect to your running node using a TransportClient.

HTH

David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 17 oct. 2012 à 18:17, antoine philippe chaker philippe.chaker@gmail.com a écrit :

YES ! I'm using Maven 3.

--

12

Thanks David,

I have just discovered this today. Indeed I'm using Spring MVC 3.0.1
but is it possible use it with annotation ?

(Comment je fait pour vous rejoindre sur le site de votre factory,
plus simple pour moi en Fr :wink: )

--

13

SORRY, it's possible !

Ok let me give it a try and I'll make a feedback here later.

Thx all !

--

14

There's no dedicated mailing list for the factory but there's a french elasticsearch user group where you can ask for help in french: http://groups.google.com/group/elasticsearch-fr

That said, you can also send me direct mail to david (at) pilato.fr

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 17 oct. 2012 à 18:53, antoine philippe chaker philippe.chaker@gmail.com a écrit :

Thanks David,

I have just discovered this today. Indeed I'm using Spring MVC 3.0.1
but is it possible use it with annotation ?

(Comment je fait pour vous rejoindre sur le site de votre factory,
plus simple pour moi en Fr :wink: )

--

--