TransportClient not connecting


(IronMike) #1

I can't seem to figure out this problem, Node from NodeBuilder works, but
If I use transportclient like below, I get an exception.
//I am using all default settings
//elasticsearch-0.90.9

Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name",
"elasticsearch").build();
InetSocketTransportAddress("localhost", 9300));

Exception in thread "main" org.elasticsearch.client.transport.
NoNodeAvailableException: No node available

at org.elasticsearch.client.transport.TransportClientNodesService.execute(
TransportClientNodesService.java:213)

at org.elasticsearch.client.transport.support.InternalTransportClient.
execute(InternalTransportClient.java:106)

at org.elasticsearch.client.support.AbstractClient.bulk(AbstractClient.java
:149)

at org.elasticsearch.client.transport.TransportClient.bulk(TransportClient.
java:346)

at org.elasticsearch.action.bulk.BulkRequestBuilder.doExecute(
BulkRequestBuilder.java:165)

at org.elasticsearch.action.ActionRequestBuilder.execute(
ActionRequestBuilder.java:85)

at org.elasticsearch.action.ActionRequestBuilder.execute(
ActionRequestBuilder.java:59)

at EntryPoint.createBulkIndexes(EntryPoint.java:305)

at EntryPoint.main(EntryPoint.java:147)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6f8941ac-11df-4f8d-9670-cc609c7aca6e%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(IronMike) #2

Anyone using transportclient from java?

On Wednesday, January 22, 2014 12:04:30 PM UTC-5, ZenMaster80 wrote:

I can't seem to figure out this problem, Node from NodeBuilder works, but
If I use transportclient like below, I get an exception.
//I am using all default settings
//elasticsearch-0.90.9

Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name"
, "elasticsearch").build();
InetSocketTransportAddress("localhost", 9300));

Exception in thread "main" org.elasticsearch.client.transport.
NoNodeAvailableException: No node available

at org.elasticsearch.client.transport.TransportClientNodesService.execute
(TransportClientNodesService.java:213)

at org.elasticsearch.client.transport.support.InternalTransportClient.
execute(InternalTransportClient.java:106)

at org.elasticsearch.client.support.AbstractClient.bulk(AbstractClient.
java:149)

at org.elasticsearch.client.transport.TransportClient.bulk(
TransportClient.java:346)

at org.elasticsearch.action.bulk.BulkRequestBuilder.doExecute(
BulkRequestBuilder.java:165)

at org.elasticsearch.action.ActionRequestBuilder.execute(
ActionRequestBuilder.java:85)

at org.elasticsearch.action.ActionRequestBuilder.execute(
ActionRequestBuilder.java:59)

at EntryPoint.createBulkIndexes(EntryPoint.java:305)

at EntryPoint.main(EntryPoint.java:147)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4731270f-60ca-4d41-9056-9179c23d6a53%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Brian Yoder) #3

ImmutableSettings.Builder settingsBuilder =
ImmutableSettings.settingsBuilder();
settingsBuilder.put("cluster.name", clusterName);
TransportClient client = new TransportClient(settingsBuilder.build());

for (String host : hostNames)
{
InetSocketTransportAddress server_address = new
InetSocketTransportAddress(
host, portTransport);

client.addTransportAddress(server_address);
}

Brian

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4f02b8f3-3571-4746-9797-68df679501b6%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(IronMike) #4

Brian,

This is no different from what I have. I googled the problem, and I guess
this may come from the fact that ES js using a different java version. I
have added the es 0.90.0.jar to java from the es installation folder. I
have no clue what I am missing.

On Wednesday, January 22, 2014 2:02:57 PM UTC-5, InquiringMind wrote:

ImmutableSettings.Builder settingsBuilder =
ImmutableSettings.settingsBuilder();
settingsBuilder.put("cluster.name", clusterName);
TransportClient client = new TransportClient(settingsBuilder.build());

for (String host : hostNames)
{
InetSocketTransportAddress server_address = new
InetSocketTransportAddress(
host, portTransport);

client.addTransportAddress(server_address);
}

Brian

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ebaf9d9d-808a-4fc3-a94f-4f7c5564c1dd%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #5

You don't use Maven for your project?
If not, don't forget to add all needed dependencies.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 22 janv. 2014 à 20:23, ZenMaster80 sabdalla80@gmail.com a écrit :

Brian,

This is no different from what I have. I googled the problem, and I guess this may come from the fact that ES js using a different java version. I have added the es 0.90.0.jar to java from the es installation folder. I have no clue what I am missing.

On Wednesday, January 22, 2014 2:02:57 PM UTC-5, InquiringMind wrote:
ImmutableSettings.Builder settingsBuilder = ImmutableSettings.settingsBuilder();
settingsBuilder.put("cluster.name", clusterName);
TransportClient client = new TransportClient(settingsBuilder.build());

for (String host : hostNames)
{
InetSocketTransportAddress server_address = new InetSocketTransportAddress(
host, portTransport);

client.addTransportAddress(server_address);
}

Brian

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ebaf9d9d-808a-4fc3-a94f-4f7c5564c1dd%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/62A8DCBC-0AC7-4EA0-A206-D34D57AB04FC%40pilato.fr.
For more options, visit https://groups.google.com/groups/opt_out.


(IronMike) #6

This is drving me crazy! No, I am not using Maven, just a plain simple java
sample project/eclipse. It works fine with Node builder and getting client
from node, but if i use transport client instead, it gives the above
mentioned exception. As far as I know I have included all the possible
dependencies. and here are the imports:

import static org.elasticsearch.node.NodeBuilder.*;

import java.util.ArrayList;

import java.util.Date;

import java.util.HashMap;

import java.util.Iterator;

import java.util.Map;

import java.io.File;

import java.io.FileNotFoundException;

import java.io.FileReader;

import org.elasticsearch.action.delete.DeleteResponse;

import org.elasticsearch.action.get.GetResponse;

import org.elasticsearch.action.search.SearchResponse;

import org.elasticsearch.action.search.SearchType;

import org.elasticsearch.client.Client;

import org.elasticsearch.client.transport.TransportClient;

import static org.elasticsearch.index.query.QueryBuilders.fieldQuery;

import org.elasticsearch.index.query.*;

import org.elasticsearch.index.query.CommonTermsQueryBuilder.Operator;

import org.elasticsearch.index.query.MatchQueryBuilder.Type;

import org.elasticsearch.rest.RestStatus;

import org.elasticsearch.search.SearchHit;

import org.elasticsearch.search.SearchHits;

import org.elasticsearch.node.Node;

import org.elasticsearch.node.NodeBuilder;

import org.elasticsearch.cluster.metadata.AliasMetaData;

import
org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequestBuilder;

importorg.elasticsearch.action.admin.indices.alias.get.IndicesGetAliasesResponse;

//import static org.elasticsearch.node.NodeBuilder.nodeBuilder;

import org.hamcrest.EasyMock2Matchers;

import
org.elasticsearch.action.admin.indices.create.CreateIndexRequestBuilder;

import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;

import org.elasticsearch.action.admin.indices.delete.DeleteIndexResponse;

import org.elasticsearch.action.index.IndexRequestBuilder;

import org.elasticsearch.action.index.IndexResponse;

import org.elasticsearch.action.search.SearchPhaseExecutionException;

import org.elasticsearch.action.search.ShardSearchFailure;

import org.elasticsearch.common.settings.ImmutableSettings;

import org.elasticsearch.common.settings.Settings;

import org.elasticsearch.common.transport.InetSocketTransportAddress;

import org.elasticsearch.common.xcontent.XContentFactory;

import org.elasticsearch.search.facet.FacetBuilders;

import org.elasticsearch.action.bulk.*;

import java.io.IOException;

import java.net.InetAddress;

import java.nio.ByteBuffer;

import java.nio.charset.Charset;

import java.nio.charset.StandardCharsets;

import java.nio.file.Files;

import java.nio.file.Paths;

import java.util.Random;

import java.util.concurrent.ExecutionException;

import static org.elasticsearch.cluster.metadata.IndexMetaData.
SETTING_NUMBER_OF_REPLICAS;

import static org.elasticsearch.cluster.metadata.IndexMetaData.
SETTING_NUMBER_OF_SHARDS;

import static
org.elasticsearch.common.settings.ImmutableSettings.settingsBuilder;

import static org.elasticsearch.common.xcontent.XContentFactory.jsonBuilder;

import static org.elasticsearch.index.query.FilterBuilders.*;

import static org.elasticsearch.index.query.QueryBuilders.*;

import static
org.elasticsearch.index.query.functionscore.ScoreFunctionBuilders.scriptFunction
;

import org.json.simple.JSONObject;

import org.json.simple.JSONArray;

import org.json.simple.parser.ParseException;

import org.json.simple.parser.JSONParser;

On Wednesday, January 22, 2014 2:59:00 PM UTC-5, David Pilato wrote:

You don't use Maven for your project?
If not, don't forget to add all needed dependencies.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 22 janv. 2014 à 20:23, ZenMaster80 <sabda...@gmail.com <javascript:>>
a écrit :

Brian,

This is no different from what I have. I googled the problem, and I guess
this may come from the fact that ES js using a different java version. I
have added the es 0.90.0.jar to java from the es installation folder. I
have no clue what I am missing.

On Wednesday, January 22, 2014 2:02:57 PM UTC-5, InquiringMind wrote:

ImmutableSettings.Builder settingsBuilder =
ImmutableSettings.settingsBuilder();
settingsBuilder.put("cluster.name", clusterName);
TransportClient client = new TransportClient(settingsBuilder.build());

for (String host : hostNames)
{
InetSocketTransportAddress server_address = new
InetSocketTransportAddress(
host, portTransport);

client.addTransportAddress(server_address);
}

Brian

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ebaf9d9d-808a-4fc3-a94f-4f7c5564c1dd%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a8df498b-49a6-424c-9754-6a587b77cf13%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jörg Prante) #7

You wrote that you have a 0.90.9 cluster but you added 0.90.0 jars to the
client. Is that correct?

Please check:

  • if your cluster nodes and client node is using exactly the same JVM

  • if your cluster and client use exactly the same ES version

  • if your cluster and client use the same cluster name

  • reasons outside ES: IP blocking, network reachability, network
    interfaces, IPv4/IPv6 etc.

Then you should be able to connect with TransportClient.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGhimgwn38eBx6kmQ839eV_kiEz8AURtDyAc56PM_Z%2Bkg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Brian Yoder) #8

IMHO, you don't need maven. You just need the Java trick to include all of
the ES dependent jar files by setting the classpath to a colon-separated
list of directory names, but append "/*" to the end of each directory name.
If you do this, you don't need to list all of the jars in the directory:
The jvm will find them for you.

Assume that ES is installed in /somepath and the jar files are in the
/somepath/elasticsearch-0.90.9/lib directory. Then your jars are in the
/somepath/mystuff/lib directory. Putting them all together:

$ java -classpath "/somepath/elasticsearch-0.90.9/lib/:
/somepath/elasticsearch-0.90.9/sigar/lib/
:/somepath/mystuff/lib/*" ...

I wrap all this mumbo-jumbo inside Bash scripts. One Bash script to emit
the full classpath collection of colon-separated pieces. Then a Bash script
that calls it to set up the Java command. Dirt simple. Nice and easy. I can
even dynamically re-point to a current build or the installed version for
testing current changes and regression-testing the full installation as the
customer would see it (respectively).

Brian

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7562d544-0547-472b-a1db-4bd9d84fbc13%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(IronMike) #9

Yes, I do have 0.90.9 across the board.
I know 9300 is opened.
I am not sure how to check if both are using same JVM?
es.yml is default, default clustername, nodename .. I only have the default
(1 Node)... Do I need to specify unicast instead of the default which I
believe uses multiCast?

On Wednesday, January 22, 2014 3:25:26 PM UTC-5, Jörg Prante wrote:

You wrote that you have a 0.90.9 cluster but you added 0.90.0 jars to the
client. Is that correct?

Please check:

  • if your cluster nodes and client node is using exactly the same JVM

  • if your cluster and client use exactly the same ES version

  • if your cluster and client use the same cluster name

  • reasons outside ES: IP blocking, network reachability, network
    interfaces, IPv4/IPv6 etc.

Then you should be able to connect with TransportClient.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/60e88c0f-ed47-4437-95df-8e6c78fd3e56%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Simpsora) #10

java -version will tell you the exact version, patch level, vendor, and
architecture of that JVM. The tricky bit can be finding out which JVM
you're actually using (usually the value in $JAVA_HOME or which java will
lead you in the right direction). If you're running your example under an
IDE, it might well be using a different JVM than the ES server.

I've had similar troubles to what you describe, but not on the localhost.
Do you get the exception right away, or some time after starting up your
client?

Ross

On Thursday, 23 January 2014 10:34:40 UTC+11, ZenMaster80 wrote:

Yes, I do have 0.90.9 across the board.
I know 9300 is opened.
I am not sure how to check if both are using same JVM?
es.yml is default, default clustername, nodename .. I only have the
default (1 Node)... Do I need to specify unicast instead of the default
which I believe uses multiCast?

On Wednesday, January 22, 2014 3:25:26 PM UTC-5, Jörg Prante wrote:

You wrote that you have a 0.90.9 cluster but you added 0.90.0 jars to the
client. Is that correct?

Please check:

  • if your cluster nodes and client node is using exactly the same JVM

  • if your cluster and client use exactly the same ES version

  • if your cluster and client use the same cluster name

  • reasons outside ES: IP blocking, network reachability, network
    interfaces, IPv4/IPv6 etc.

Then you should be able to connect with TransportClient.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1bbaa194-dd7b-41ba-99d7-c8570dacb23a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(IronMike) #11

java version "1.7.0_11"

Java(TM) SE Runtime Environment (build 1.7.0_11-b21)

Java HotSpot(TM) 64-Bit Server VM (build 23.6-b04, mixed mode)

I spent too much time on this, I gave up. I'll ask the question
differently, I wanted to use the transport client at 9300 so I can index a
file, and the intent to search it with http://localhost:9300/_search...for
demo purposes since I didn't want to search it using java code. I am able
to index the file with nodeBuilder, is there a way I can query it using
http? My understanding is that the node is local, can I query it somehow
over http?

On Wednesday, January 22, 2014 10:02:14 PM UTC-5, Ross Simpson wrote:

java -version will tell you the exact version, patch level, vendor, and
architecture of that JVM. The tricky bit can be finding out which JVM
you're actually using (usually the value in $JAVA_HOME or which javawill lead you in the right direction). If you're running your example
under an IDE, it might well be using a different JVM than the ES server.

I've had similar troubles to what you describe, but not on the localhost.
Do you get the exception right away, or some time after starting up your
client?

Ross

On Thursday, 23 January 2014 10:34:40 UTC+11, ZenMaster80 wrote:

Yes, I do have 0.90.9 across the board.
I know 9300 is opened.
I am not sure how to check if both are using same JVM?
es.yml is default, default clustername, nodename .. I only have the
default (1 Node)... Do I need to specify unicast instead of the default
which I believe uses multiCast?

On Wednesday, January 22, 2014 3:25:26 PM UTC-5, Jörg Prante wrote:

You wrote that you have a 0.90.9 cluster but you added 0.90.0 jars to
the client. Is that correct?

Please check:

  • if your cluster nodes and client node is using exactly the same JVM

  • if your cluster and client use exactly the same ES version

  • if your cluster and client use the same cluster name

  • reasons outside ES: IP blocking, network reachability, network
    interfaces, IPv4/IPv6 etc.

Then you should be able to connect with TransportClient.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/15445d05-937e-45ce-800b-d57f82bf2af9%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jörg Prante) #12

If you do not post error message or log files, I'm afraid nobody can help
you.

NodeClient is not local as long as you do not set local(true). But I am
afraid you understand something different under "local".

If you want HTTP, neither NodeClient nor TransportClient is the right
thing. 9300 is the ES native transport protocol. You can not use
http://localhost:9300. Use port 9200 for HTTP.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGgJ1naZvRBP9rEkqUz95MrAq00SzmncCtPKmv%3DaU_BHg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(IronMike) #13

Jörg,
Thanks, I did understand "local" to be something different. I want to get
something straight, so for production with aws integration, I would like to
index tons of files using native protocol. I should be able to query the es
server from java script application using http API, at least this is my
understanding. Am I wrong?

On Thursday, January 23, 2014 2:58:28 AM UTC-5, Jörg Prante wrote:

If you do not post error message or log files, I'm afraid nobody can help
you.

NodeClient is not local as long as you do not set local(true). But I am
afraid you understand something different under "local".

If you want HTTP, neither NodeClient nor TransportClient is the right
thing. 9300 is the ES native transport protocol. You can not use
http://localhost:9300. Use port 9200 for HTTP.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fe83ab64-f121-4c5a-9a26-276947f5668e%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jörg Prante) #14

Yes, you can query ES with Javascript over HTTP.

You can also use HTTP to index tons of files, that does not depend on the
native protocol.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHEV1-xMy6erZxCNeE%3DYHAyznKBBmmXwJ5fy66bO3fiSg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(IronMike) #15

Ok great, I already prepare the files using java, so I thought it would be
a great spot to index it as well in java (slight better performance over
http I am guessing). I struggled to find decent examples on indexing files
via http, I wouldn't mind testing with it as well if you can point to some
example. How is this done, do we write curl script that run on the server?
So in my case, Assume I have files in some folder ready for indexing, do I
write a curl script that would go through the files and index them?

On Thursday, January 23, 2014 12:23:27 PM UTC-5, Jörg Prante wrote:

Yes, you can query ES with Javascript over HTTP.

You can also use HTTP to index tons of files, that does not depend on the
native protocol.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/eab032ba-0572-4c87-9dce-90bada20ee56%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #16

I don't get it. You have beans in memory in Java?
So basically yes, just serialize them to json (using for example Jackson) and send them to elasticsearch.

Have a look at this for an example: https://github.com/elasticsearchfr/hands-on/blob/answers/src/test/java/org/elasticsearchfr/handson/ex1/IndexTest.java#L107

You don't have to store it on filesystem and then index them…

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 23 janvier 2014 at 19:02:12, ZenMaster80 (sabdalla80@gmail.com) a écrit:

Ok great, I already prepare the files using java, so I thought it would be a great spot to index it as well in java (slight better performance over http I am guessing). I struggled to find decent examples on indexing files via http, I wouldn't mind testing with it as well if you can point to some example. How is this done, do we write curl script that run on the server? So in my case, Assume I have files in some folder ready for indexing, do I write a curl script that would go through the files and index them?

On Thursday, January 23, 2014 12:23:27 PM UTC-5, Jörg Prante wrote:
Yes, you can query ES with Javascript over HTTP.

You can also use HTTP to index tons of files, that does not depend on the native protocol.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/eab032ba-0572-4c87-9dce-90bada20ee56%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52e16c3d.2ae8944a.8a60%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.


(Jörg Prante) #17

I'm quite sure you want something like the filesystem river

http://www.pilato.fr/fsriver/

or nozzle

It is not that easy to push files to ES without such tools, you have to
analyze the content of the files. You have to consider what content you
want to analyze and store into what fields. Even with JSON, you have to
decide how to map the content (as long as you want structured search and
not a plain one-size-fits-all search).

So in short, it is not really a question whether to use HTTP or not HTTP,
but a question for the right tool (and your assumption "slight better
performance over http" is not correct but performance should not matter at
all at that point).

There are certainly more useful tools around... they are easy to find with
googling.

Jörg

On Thu, Jan 23, 2014 at 7:02 PM, ZenMaster80 sabdalla80@gmail.com wrote:

Ok great, I already prepare the files using java, so I thought it would be
a great spot to index it as well in java (slight better performance over
http I am guessing). I struggled to find decent examples on indexing files
via http, I wouldn't mind testing with it as well if you can point to some
example. How is this done, do we write curl script that run on the server?
So in my case, Assume I have files in some folder ready for indexing, do I
write a curl script that would go through the files and index them?

On Thursday, January 23, 2014 12:23:27 PM UTC-5, Jörg Prante wrote:

Yes, you can query ES with Javascript over HTTP.

You can also use HTTP to index tons of files, that does not depend on the
native protocol.

Jörg

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/eab032ba-0572-4c87-9dce-90bada20ee56%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFf91sNq_Ynq4FyGkFm1fPNAJzdEK_GF0oT4g0BMh3w0A%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(IronMike) #18

David,
Sorry for the confusion, I am doing exactly what you mentioned. I start
with our custom files, turn them into JSON and feed/BULK JSON in memory to
indexer via native Java API. This topic/thread started by my question about
having problem with transport client but not node builder. I haven't been
able to get fix the issue, but I have couple of things to check based on
the responses I got.

On Thursday, January 23, 2014 2:23:40 PM UTC-5, David Pilato wrote:

I don't get it. You have beans in memory in Java?
So basically yes, just serialize them to json (using for example Jackson)
and send them to elasticsearch.

Have a look at this for an example:
https://github.com/elasticsearchfr/hands-on/blob/answers/src/test/java/org/elasticsearchfr/handson/ex1/IndexTest.java#L107

You don't have to store it on filesystem and then index them…

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 23 janvier 2014 at 19:02:12, ZenMaster80 (sabda...@gmail.com<javascript:>)
a écrit:

Ok great, I already prepare the files using java, so I thought it would be
a great spot to index it as well in java (slight better performance over
http I am guessing). I struggled to find decent examples on indexing files
via http, I wouldn't mind testing with it as well if you can point to some
example. How is this done, do we write curl script that run on the server?
So in my case, Assume I have files in some folder ready for indexing, do I
write a curl script that would go through the files and index them?

On Thursday, January 23, 2014 12:23:27 PM UTC-5, Jörg Prante wrote:

Yes, you can query ES with Javascript over HTTP.

You can also use HTTP to index tons of files, that does not depend on the
native protocol.

Jörg

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/eab032ba-0572-4c87-9dce-90bada20ee56%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7e9a818f-8749-41e4-afee-cd36c4d3ecdb%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #19