How to insert data Elastic search war file?

Elastic Stack Elasticsearch
1

I tried to load my postgress data into elasticsearch using logstash but got an error. The same logstash conf file works fine when elastic search is running locally in my machine. But when depoyed in jboss Iam not able to insert data into it.Please anyone suggent me how to insert data in Elastic search war file.

below is my .conf file

file: books-index-logstash.conf

input {
jdbc {
jdbc_connection_string => 'jdbc:postgresql://localhost:5432/postgres'
jdbc_user => 'postgres'
jdbc_password => '*************'
jdbc_validate_connection => true
jdbc_driver_library => 'D:/logstash-2.2.0_new/logstash-2.2.0/bin/postgresql-9.4.1207.jre6.jar'
jdbc_driver_class => 'org.postgresql.Driver'
statement => 'SELECT * from fiction'
}
}
output {
elasticsearch { hosts => ["localhost:8080/elasticsearch-war-example-master"]
document_type => ["entry"]
document_id => ["%{id}"]
index => ["books"] }
}

2

Please format you code.

Why do you want to deploy elasticsearch within a JBoss container?
It's not designed for that.

Run it as a server, which is what elasticsearch is.

3

If I dont deploy in jboss..! How can I integrate with my web application..? My application is a separate war file in jboss..! I cant always run elasticsearch locally and get the result right..! Whenever my server gets started elastic search should also start automatically...!

4

Sorry but you wrote a logstash configuration which goal is to extract data from postgresql and send them into elasticsearch.

If you are running elasticsearch standalone, then, localhost:8080/elasticsearch-war-example-master is wrong. You need to send your data to elasticsearch localhost:9200.

I don't understand the relation with running a client from a webapp.

Can you explain exactly what you are trying to do?

5

My scenario is..
We have a spring based web application which we will deploy over jboss server. In this applicaion to perform global search for fetching particular user records I need to use elastic search. How can we achieve this..?Will this require elastic search server running along with jboss server..?? or Is it enough just to add spring elastic search dependencies without elastic search server running.??

6

Just to clarify something. If you want to use elasticsearch, it means that you have to index data into elasticsearch first.
Then you can use any client using spring or not to query elasticsearch.

Did you already index your data into elasticsearch?

Some links:

HTH

7

Pinky

It is a bit of a struggle, and understand your pain. The Java API documentation is pretty lacking and there are few things that are missing to complete the picture.

If you can think of Elasticsearch as your database, you need to have the database running in order to access it. Elasticsearch runs within it's own J2EE container, similar to your application.

In maven the dependencies are

<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>2.2.0</version>
</dependency>

I wrote the following to connect :

import java.net.InetAddress;
import java.net.UnknownHostException;
import org.elasticsearch.client.Client;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.transport.InetSocketTransportAddress;

public class ESClient {

public ESClient() {}

public static Client connectToES() throws UnknownHostException {
	Client client = TransportClient.builder().build()
			.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("localhost"), 9300));
	return client;			
}

}

This is a pretty basic example below. This is to get the min_time and max_time timestamp of an index, and returns JSON.

public static SearchResponse getMinMaxDateRangeofIndex(String indexName)  {
	SearchResponse esResponse = null;
	Client client;
	try {
		client = ESClient.connectToES();
		esResponse = client.prepareSearch(indexName)
		  .setSize(0)
			  .setQuery(QueryBuilders.matchAllQuery())
			  .addAggregation(AggregationBuilders.min("min_time").field("@timestamp").format("date_time"))
			  .addAggregation(AggregationBuilders.max("max_time").field("@timestamp").format("date_time"))
			.execute()
			.actionGet();
		client.close();
		

	} catch (UnknownHostException e) {
		// TODO Auto-generated catch block
		e.printStackTrace();
	}
		
	return esResponse;

I feel your pain. Something what would and should appear to be pretty simple is reasonably hard to find working examples of how and not to do things.

I am struggling at the moment getting my head around doing a dump and then do a re-import of data. If I get time and I understand this a little more, will try and put some tutorials together that give a complete example of how to do things, once I find out. I have found the documentation only shows half the story and not complete working examples from simple to the more complex things that you would want to do.

8

It's easy with my knapsack import/export tool GitHub - jprante/elasticsearch-knapsack: Knapsack plugin is an import/export tool for Elasticsearch

Use the source. It's all in there, especially in the test source code, you can find working examples.

9

Looks pretty good Jorg, thanks will have a look at it.

10

dadoonet
Yeah I indexed my data into elastic search using logstash. What I was expecting was when we deploy the elasticsearch war file in jboss elasticsearch server is running and we don't need to run the batch file again.But that is not the case right..! When our application is using elastic search we should run the elasticsearch server.

thanks for the links u posted previously..! It actually gave me an idea how to use elastic search in web application.