# How can I input data from java to ES in real time?

**URL:** <https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523>\
**Category:** Elasticsearch\
**Created:** [January 7, 2015, 1:46pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523 "2015-01-07T13:46:18Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Marian\_Valero](https://avatars.discourse-cdn.com/v4/letter/m/b9e5f3/32.png) [@Marian\_Valero](https://discuss.elastic.co/u/Marian_Valero)\
**Post date:** [January 7, 2015, 1:46pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523/1 "2015-01-07T13:46:18Z")

</div>

I'm reading data from my oracle database to java with JDBC but I want to  
know how can I input the data that I'm getting in Elasticsearch in real  
time.

Thanks.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b8010611-a842-4ab0-aad9-62f1edf747a5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b8010611-a842-4ab0-aad9-62f1edf747a5%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 7, 2015, 1:53pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523/2 "2015-01-07T13:53:28Z")

</div>

I'd recommend looking at [GitHub - dadoonet/legacy-search: Demo project showing how to add elasticsearch to a legacy application.](https://github.com/dadoonet/legacy-search)

Branch 01, basically does this: [https://github.com/dadoonet/legacy-search/blob/01-direct/src/main/java/fr/pilato/demo/legacysearch/dao/ElasticsearchDao.java#L44](https://github.com/dadoonet/legacy-search/blob/01-direct/src/main/java/fr/pilato/demo/legacysearch/dao/ElasticsearchDao.java#L44)

HTH

--  
David 😉  
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

> Le 7 janv. 2015 à 14:46, Marian Valero [marianv0992@gmail.com](mailto:marianv0992@gmail.com) a écrit :
> 
> I'm reading data from my oracle database to java with JDBC but I want to know how can I input the data that I'm getting in Elasticsearch in real time.
> 
> ## Thanks.
> 
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b8010611-a842-4ab0-aad9-62f1edf747a5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b8010611-a842-4ab0-aad9-62f1edf747a5%40googlegroups.com).  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/22ECDD92-5671-4EFE-A3DF-681C6D8D2FCC%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/22ECDD92-5671-4EFE-A3DF-681C6D8D2FCC%40pilato.fr).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Marian\_Valero](https://avatars.discourse-cdn.com/v4/letter/m/b9e5f3/32.png) [@Marian\_Valero](https://discuss.elastic.co/u/Marian_Valero)\
**Post date:** [January 7, 2015, 3:41pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523/3 "2015-01-07T15:41:17Z")

</div>

I have this:

public static void main(String args[]) throws SQLException {

```
int count=0;

    String url = "jdbc:oracle:thin:eso:1521:eso"; 
  
    Properties props = new Properties();
    props.setProperty("user", "eso");
    props.setProperty("password", "eso");
  
    Connection conn = DriverManager.getConnection(url,props);

    String sql ="select * from responselog";

    PreparedStatement preStatement = conn.prepareStatement(sql);

    ResultSet result = preStatement.executeQuery();
  
    Timestamp timestamp = null;
    
    while(result.next() && count<1000){
    count++;
    String id = result.getString("id");
    String deliveryid = result.getString("deliveryid");
    String msgid = result.getString("msgid");
    String rspdate = result.getString("rspdate");
    String parsedresponse = result.getString("parsedresponse");
    String shortcode = result.getString("shortcode");
    String insid = result.getString("insid");
    String mobilenumber = result.getString("mobilenumber");
    String rawresponse = result.getString("rawresponse");
          
        System.out.println("Responselog from Oracle : "+ 

```

result.getString("id")+" "+result.getString("deliveryid")+"  
"+result.getString("msgid")+" "+result.getString("rspdate")+"  
"+result.getString("parsedresponse")+" "+result.getString("shortcode")+"  
"+result.getString("insid")+" "+result.getString("mobilenumber")+"  
"+result.getString("rawresponse"));  
}  
System.out.println("done");  
}

I'm reading 1000 registers of the table and I want to insert this in  
Elasticsearch, but this is an example, I want to do that for all registers  
of my database, this is millions of registers. I don't understand very well  
that you send me.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/8391d264-86c0-48e5-ac90-613cc9f59628%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8391d264-86c0-48e5-ac90-613cc9f59628%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 7, 2015, 4:08pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523/4 "2015-01-07T16:08:32Z")

</div>

You should read this: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/index_.html) [http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/index\_.html](http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/index_.html)

You need to create a Client: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/client.html)  
[http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/client.html](http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/client.html)You need to create a JSON from your properties. See: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/index_.html#helpers) [http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/index\_.html#helpers](http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/index_.html#helpers)  
Then you should use a Bulk to index all those documents: See: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/bulk.html) [http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/bulk.html](http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/bulk.html)  
Something like this:  
BulkRequestBuilder bulkRequest = client.prepareBulk();  
while(result.next() && count\<1000){  
// YOUR ORACLE stuff here  
// either use client#prepare, or use Requests# to directly build index/delete requests  
bulkRequest.add(client.prepareIndex("indexname", "typename", id)  
.setSource(jsonBuilder()  
.startObject()  
.field("deliveryid", deliveryid)  
.field("msgid", msgid) // Add your other fields  
.endObject()  
)  
);  
}  
BulkResponse bulkResponse = bulkRequest.execute().actionGet();  
Do that every 1000 or 10000 documents. Don’t send too many requests within the same bulk. It will use too much memory and probably will slow down your indexation rate.

HTH

--  
David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com)  
@dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs)

> Le 7 janv. 2015 à 16:41, Marian Valero [marianv0992@gmail.com](mailto:marianv0992@gmail.com) a écrit :
> 
> I have this:
> 
> public static void main(String args) throws SQLException {
> 
> ```
> int count=0;
> 	
> String url = "jdbc:oracle:thin:eso:1521:eso"; 
>   
> Properties props = new Properties();
> props.setProperty("user", "eso");
> props.setProperty("password", "eso");
>   
> Connection conn = DriverManager.getConnection(url,props);
> 
> String sql ="select * from responselog";
> 
> PreparedStatement preStatement = conn.prepareStatement(sql);
> 
> ResultSet result = preStatement.executeQuery();
>   
> Timestamp timestamp = null;
>     
> while(result.next() && count<1000){
> count++;
> String id = result.getString("id");
> String deliveryid = result.getString("deliveryid");
> String msgid = result.getString("msgid");
> String rspdate = result.getString("rspdate");
> String parsedresponse = result.getString("parsedresponse");
> String shortcode = result.getString("shortcode");
> String insid = result.getString("insid");
> String mobilenumber = result.getString("mobilenumber");
> String rawresponse = result.getString("rawresponse");
>          	
> System.out.println("Responselog from Oracle : "+ result.getString("id")+" "+result.getString("deliveryid")+" "+result.getString("msgid")+" "+result.getString("rspdate")+" "+result.getString("parsedresponse")+" "+result.getString("shortcode")+" "+result.getString("insid")+" "+result.getString("mobilenumber")+" "+result.getString("rawresponse")); 
> }               
> System.out.println("done");
> }
> 
> ```
> 
> I'm reading 1000 registers of the table and I want to insert this in Elasticsearch, but this is an example, I want to do that for all registers of my database, this is millions of registers. I don't understand very well that you send me.
> 
> --  
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/8391d264-86c0-48e5-ac90-613cc9f59628%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8391d264-86c0-48e5-ac90-613cc9f59628%40googlegroups.com) [https://groups.google.com/d/msgid/elasticsearch/8391d264-86c0-48e5-ac90-613cc9f59628%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/8391d264-86c0-48e5-ac90-613cc9f59628%40googlegroups.com?utm_medium=email&utm_source=footer).  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout) [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/A69A19D5-CEC2-4A3E-9CBC-5024438288AE%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/A69A19D5-CEC2-4A3E-9CBC-5024438288AE%40pilato.fr).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Marian\_Valero](https://avatars.discourse-cdn.com/v4/letter/m/b9e5f3/32.png) [@Marian\_Valero](https://discuss.elastic.co/u/Marian_Valero)\
**Post date:** [January 7, 2015, 4:28pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523/5 "2015-01-07T16:28:55Z")

</div>

> Thanks!

I have a problem with jsonBuilder() method, eclipse don't found this, I do  
it that way:

```
public static void main(String args[]) throws SQLException {
     
int count=0;
Node node = nodeBuilder().client(true).node();
Client client = node.client();

    String url = "jdbc:oracle:thin:@eso:1521:eso"; 
  
    Properties props = new Properties();
    props.setProperty("user", "eso");
    props.setProperty("password", "eso");
  
    Connection conn = DriverManager.getConnection(url,props);

    String sql ="select * from responselog";

    PreparedStatement preStatement = conn.prepareStatement(sql);

    ResultSet result = preStatement.executeQuery();
  
    // Timestamp timestamp = null;
    
  BulkRequestBuilder bulkRequest = client.prepareBulk();
    
    while(result.next() && count<1000){
    count++;
        String id = result.getString("id");
    String deliveryid = result.getString("deliveryid");
    String msgid = result.getString("msgid");
    String rspdate = result.getString("rspdate");
    String parsedresponse = result.getString("parsedresponse");
    String shortcode = result.getString("shortcode");
    String insid = result.getString("insid");
    String mobilenumber = result.getString("mobilenumber");
    String rawresponse = result.getString("rawresponse");
          
    XContentBuilder builder = jsonBuilder()
    .startObject()
            .field("deliveryid", deliveryid)
            .field("msgid", msgid)
            .field("message", "trying out Elasticsearch")
        .endObject();

    // YOUR ORACLE stuff here
    // either use client#prepare, or use Requests# to directly build 

```

index/delete requests  
bulkRequest.add(client.prepareIndex("logs", "responselog", id)  
.setSource(jsonBuilder()  
.startObject()  
.field("deliveryid", deliveryid)  
.field("msgid", msgid)  
.field("rspdate", rspdate)  
.field("parsedresponse", parsedresponse)  
.field("shortcode", shortcode)  
.field("insid", insid)  
.field("mobilenumber", mobilenumber)  
.field("rawresponse", rawresponse)  
.endObject()  
)  
);

```
        System.out.println("Responselog from Oracle : "+ 

```

result.getString("id")+" "+result.getString("deliveryid")+"  
"+result.getString("msgid")+" "+result.getString("rspdate")+"  
"+result.getString("parsedresponse")+" "+result.getString("shortcode")+"  
"+result.getString("insid")+" "+result.getString("mobilenumber")+"  
"+result.getString("rawresponse"));  
}

```
BulkResponse bulkResponse = bulkRequest.execute().actionGet();
System.out.println("done");
} 

```

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [January 7, 2015, 4:35pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523/6 "2015-01-07T16:35:36Z")

</div>

You can use the JDBC plugin instead of reinventing the wheel.

> **[GitHub - jprante/elasticsearch-jdbc: JDBC importer for Elasticsearch](https://github.com/jprante/elasticsearch-jdbc)**
>
> JDBC importer for Elasticsearch. Contribute to jprante/elasticsearch-jdbc development by creating an account on GitHub.

Jörg

On Wed, Jan 7, 2015 at 5:28 PM, Marian Valero [marianv0992@gmail.com](mailto:marianv0992@gmail.com) wrote:

> Thanks!
> 
> > 
> 
> I have a problem with jsonBuilder() method, eclipse don't found this, I do  
> it that way:
> 
> ```
> public static void main(String args[]) throws SQLException {
> 
> int count=0;
> Node node = nodeBuilder().client(true).node();
> Client client = node.client();
> 
> String url = "jdbc:oracle:thin:@eso:1521:eso";
> 
> Properties props = new Properties();
> props.setProperty("user", "eso");
> props.setProperty("password", "eso");
> 
> Connection conn = DriverManager.getConnection(url,props);
> 
> String sql ="select * from responselog";
> 
> PreparedStatement preStatement = conn.prepareStatement(sql);
> 
> ResultSet result = preStatement.executeQuery();
> 
> // Timestamp timestamp = null;
> 
> BulkRequestBuilder bulkRequest = client.prepareBulk();
> 
> while(result.next() && count<1000){
> count++;
> String id = result.getString("id");
> String deliveryid = result.getString("deliveryid");
> String msgid = result.getString("msgid");
> String rspdate = result.getString("rspdate");
> String parsedresponse = result.getString("parsedresponse");
> String shortcode = result.getString("shortcode");
> String insid = result.getString("insid");
> String mobilenumber = result.getString("mobilenumber");
> String rawresponse = result.getString("rawresponse");
> 
> XContentBuilder builder = jsonBuilder()
> .startObject()
> .field("deliveryid", deliveryid)
> .field("msgid", msgid)
> .field("message", "trying out Elasticsearch")
> .endObject();
> 
> // YOUR ORACLE stuff here
> // either use client#prepare, or use Requests# to directly build
> 
> ```
> 
> index/delete requests  
> bulkRequest.add(client.prepareIndex("logs", "responselog", id)  
> .setSource(jsonBuilder()  
> .startObject()  
> .field("deliveryid", deliveryid)  
> .field("msgid", msgid)  
> .field("rspdate", rspdate)  
> .field("parsedresponse", parsedresponse)  
> .field("shortcode", shortcode)  
> .field("insid", insid)  
> .field("mobilenumber", mobilenumber)  
> .field("rawresponse", rawresponse)  
> .endObject()  
> )  
> );
> 
> ```
> System.out.println("Responselog from Oracle : "+
> 
> ```
> 
> result.getString("id")+" "+result.getString("deliveryid")+"  
> "+result.getString("msgid")+" "+result.getString("rspdate")+"  
> "+result.getString("parsedresponse")+" "+result.getString("shortcode")+"  
> "+result.getString("insid")+" "+result.getString("mobilenumber")+"  
> "+result.getString("rawresponse"));  
> }
> 
> ```
> BulkResponse bulkResponse = bulkRequest.execute().actionGet();
> System.out.println("done");
> }
> 
> ```
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .
> 
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcx7gXvXzg83cu9dFx7%3DYtW%2BA\_K%2BtBANUpdD-86uB8GQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcx7gXvXzg83cu9dFx7%3DYtW%2BA_K%2BtBANUpdD-86uB8GQ%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Marian\_Valero](https://avatars.discourse-cdn.com/v4/letter/m/b9e5f3/32.png) [@Marian\_Valero](https://discuss.elastic.co/u/Marian_Valero)\
**Post date:** [January 7, 2015, 4:39pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523/7 "2015-01-07T16:39:56Z")

</div>

Hi Jörg I have problems when I use that, I didn't insert the data, I create  
the river but I can get the data in ES.

El miércoles, 7 de enero de 2015 12:05:47 UTC-4:30, Jörg Prante escribió:

> You can use the JDBC plugin instead of reinventing the wheel.
> 
> [GitHub - jprante/elasticsearch-jdbc: JDBC importer for Elasticsearch](https://github.com/jprante/elasticsearch-river-jdbc)
> 
> Jörg
> 
> On Wed, Jan 7, 2015 at 5:28 PM, Marian Valero \<[maria...@gmail.com](mailto:maria...@gmail.com)  
> \<javascript:\>\> wrote:
> 
> > Thanks!
> > 
> > > 
> > 
> > I have a problem with jsonBuilder() method, eclipse don't found this, I  
> > do it that way:
> > 
> > ```
> > public static void main(String args[]) throws SQLException {
> >      
> > int count=0;
> > Node node = nodeBuilder().client(true).node();
> > Client client = node.client();
> > 
> > String url = "jdbc:oracle:thin:@eso:1521:eso"; 
> >   
> > Properties props = new Properties();
> > props.setProperty("user", "eso");
> > props.setProperty("password", "eso");
> >   
> > Connection conn = DriverManager.getConnection(url,props);
> > 
> > String sql ="select * from responselog";
> > 
> > PreparedStatement preStatement = conn.prepareStatement(sql);
> > 
> > ResultSet result = preStatement.executeQuery();
> >   
> > // Timestamp timestamp = null;
> >     
> > BulkRequestBuilder bulkRequest = client.prepareBulk();
> >     
> > while(result.next() && count<1000){
> > count++;
> > String id = result.getString("id");
> > String deliveryid = result.getString("deliveryid");
> > String msgid = result.getString("msgid");
> > String rspdate = result.getString("rspdate");
> > String parsedresponse = result.getString("parsedresponse");
> > String shortcode = result.getString("shortcode");
> > String insid = result.getString("insid");
> > String mobilenumber = result.getString("mobilenumber");
> > String rawresponse = result.getString("rawresponse");
> >           
> > XContentBuilder builder = jsonBuilder()
> > .startObject()
> > .field("deliveryid", deliveryid)
> > .field("msgid", msgid)
> > .field("message", "trying out Elasticsearch")
> > .endObject();
> > 
> > // YOUR ORACLE stuff here
> > // either use client#prepare, or use Requests# to directly build 
> > 
> > ```
> > 
> > index/delete requests  
> > bulkRequest.add(client.prepareIndex("logs", "responselog", id)  
> > .setSource(jsonBuilder()  
> > .startObject()  
> > .field("deliveryid", deliveryid)  
> > .field("msgid", msgid)  
> > .field("rspdate", rspdate)  
> > .field("parsedresponse", parsedresponse)  
> > .field("shortcode", shortcode)  
> > .field("insid", insid)  
> > .field("mobilenumber", mobilenumber)  
> > .field("rawresponse", rawresponse)  
> > .endObject()  
> > )  
> > );
> > 
> > ```
> > System.out.println("Responselog from Oracle : "+ 
> > 
> > ```
> > 
> > result.getString("id")+" "+result.getString("deliveryid")+"  
> > "+result.getString("msgid")+" "+result.getString("rspdate")+"  
> > "+result.getString("parsedresponse")+" "+result.getString("shortcode")+"  
> > "+result.getString("insid")+" "+result.getString("mobilenumber")+"  
> > "+result.getString("rawresponse"));  
> > }
> > 
> > ```
> > BulkResponse bulkResponse = bulkRequest.execute().actionGet();
> > System.out.println("done");
> > } 
> > 
> > ```
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com)  
> > [https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > .
> > 
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d90b58c2-fb33-4dc3-b630-183c12e0566c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d90b58c2-fb33-4dc3-b630-183c12e0566c%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [January 7, 2015, 4:48pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523/8 "2015-01-07T16:48:15Z")

</div>

Are there any error messages? What did you try to insert data?

Jörg

On Wed, Jan 7, 2015 at 5:39 PM, Marian Valero [marianv0992@gmail.com](mailto:marianv0992@gmail.com) wrote:

> Hi Jörg I have problems when I use that, I didn't insert the data, I  
> create the river but I can get the data in ES.
> 
> El miércoles, 7 de enero de 2015 12:05:47 UTC-4:30, Jörg Prante escribió:
> 
> > You can use the JDBC plugin instead of reinventing the wheel.
> > 
> > [GitHub - jprante/elasticsearch-jdbc: JDBC importer for Elasticsearch](https://github.com/jprante/elasticsearch-river-jdbc)
> > 
> > Jörg
> > 
> > On Wed, Jan 7, 2015 at 5:28 PM, Marian Valero [maria...@gmail.com](mailto:maria...@gmail.com) wrote:
> > 
> > > Thanks!
> > > 
> > > > 
> > > 
> > > I have a problem with jsonBuilder() method, eclipse don't found this, I  
> > > do it that way:
> > > 
> > > ```
> > > public static void main(String args[]) throws SQLException {
> > > 
> > > int count=0;
> > > Node node = nodeBuilder().client(true).node();
> > > Client client = node.client();
> > > 
> > > String url = "jdbc:oracle:thin:@eso:1521:eso";
> > > 
> > > Properties props = new Properties();
> > > props.setProperty("user", "eso");
> > > props.setProperty("password", "eso");
> > > 
> > > Connection conn = DriverManager.getConnection(url,props);
> > > 
> > > String sql ="select * from responselog";
> > > 
> > > PreparedStatement preStatement = conn.prepareStatement(sql);
> > > 
> > > ResultSet result = preStatement.executeQuery();
> > > 
> > > // Timestamp timestamp = null;
> > > 
> > > BulkRequestBuilder bulkRequest = client.prepareBulk();
> > > 
> > > while(result.next() && count<1000){
> > > count++;
> > > String id = result.getString("id");
> > > String deliveryid = result.getString("deliveryid");
> > > String msgid = result.getString("msgid");
> > > String rspdate = result.getString("rspdate");
> > > String parsedresponse = result.getString("parsedresponse");
> > > String shortcode = result.getString("shortcode");
> > > String insid = result.getString("insid");
> > > String mobilenumber = result.getString("mobilenumber");
> > > String rawresponse = result.getString("rawresponse");
> > > 
> > > XContentBuilder builder = jsonBuilder()
> > > .startObject()
> > > .field("deliveryid", deliveryid)
> > > .field("msgid", msgid)
> > > .field("message", "trying out Elasticsearch")
> > > .endObject();
> > > 
> > > // YOUR ORACLE stuff here
> > > // either use client#prepare, or use Requests# to directly
> > > 
> > > ```
> > > 
> > > build index/delete requests  
> > > bulkRequest.add(client.prepareIndex("logs", "responselog", id)  
> > > .setSource(jsonBuilder()  
> > > .startObject()  
> > > .field("deliveryid", deliveryid)  
> > > .field("msgid", msgid)  
> > > .field("rspdate", rspdate)  
> > > .field("parsedresponse", parsedresponse)  
> > > .field("shortcode", shortcode)  
> > > .field("insid", insid)  
> > > .field("mobilenumber", mobilenumber)  
> > > .field("rawresponse", rawresponse)  
> > > .endObject()  
> > > )  
> > > );
> > > 
> > > ```
> > > System.out.println("Responselog from Oracle : "+
> > > 
> > > ```
> > > 
> > > result.getString("id")+" "+result.getString("deliveryid")+"  
> > > "+result.getString("msgid")+" "+result.getString("rspdate")+"  
> > > "+result.getString("parsedresponse")+" "+result.getString("shortcode")+"  
> > > "+result.getString("insid")+" "+result.getString("mobilenumber")+"  
> > > "+result.getString("rawresponse"));  
> > > }
> > > 
> > > ```
> > > BulkResponse bulkResponse = bulkRequest.execute().actionGet();
> > > System.out.println("done");
> > > }
> > > 
> > > ```
> > > 
> > > --  
> > > You received this message because you are subscribed to the Google  
> > > Groups "elasticsearch" group.  
> > > To unsubscribe from this group and stop receiving emails from it, send  
> > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).  
> > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/)  
> > > msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%  
> > > [40googlegroups.com](http://40googlegroups.com)  
> > > [https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > > .
> > > 
> > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/elasticsearch/d90b58c2-fb33-4dc3-b630-183c12e0566c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d90b58c2-fb33-4dc3-b630-183c12e0566c%40googlegroups.com)  
> > [https://groups.google.com/d/msgid/elasticsearch/d90b58c2-fb33-4dc3-b630-183c12e0566c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/d90b58c2-fb33-4dc3-b630-183c12e0566c%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > .
> 
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFkE%2BO\_b6TstTgOZN8VMyrTxJg\_dCxic3XpfyYt8V0c6Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFkE%2BO_b6TstTgOZN8VMyrTxJg_dCxic3XpfyYt8V0c6Q%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 7, 2015, 4:50pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523/9 "2015-01-07T16:50:15Z")

</div>

Of course, but if you read carefully the links to the doc I pasted, you will see that you have to import a class.

Doc says:  
import static org.elasticsearch.common.xcontent.XContentFactory.\*;

XContentBuilder builder = jsonBuilder()  
.startObject()  
.field("user", "kimchy")  
.field("postDate", new Date())  
.field("message", "trying out Elasticsearch")  
.endObject()

Did you see it?

--  
David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com)  
@dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs)

> Le 7 janv. 2015 à 17:28, Marian Valero [marianv0992@gmail.com](mailto:marianv0992@gmail.com) a écrit :
> 
> Thanks!
> 
> I have a problem with jsonBuilder() method, eclipse don't found this, I do it that way:
> 
> ```
> public static void main(String args[]) throws SQLException {
> 	    	
> int count=0;
> Node node = nodeBuilder().client(true).node();
> Client client = node.client();
> 	
> String url = "jdbc:oracle:thin:@eso:1521:eso"; 
>   
> Properties props = new Properties();
> props.setProperty("user", "eso");
> props.setProperty("password", "eso");
>   
> Connection conn = DriverManager.getConnection(url,props);
> 
> String sql ="select * from responselog";
> 
> PreparedStatement preStatement = conn.prepareStatement(sql);
> 
> ResultSet result = preStatement.executeQuery();
>   
> // Timestamp timestamp = null;
>     
> BulkRequestBuilder bulkRequest = client.prepareBulk();
>     
> while(result.next() && count<1000){
> count++;
> String id = result.getString("id");
> String deliveryid = result.getString("deliveryid");
> String msgid = result.getString("msgid");
> String rspdate = result.getString("rspdate");
> String parsedresponse = result.getString("parsedresponse");
> String shortcode = result.getString("shortcode");
> String insid = result.getString("insid");
> String mobilenumber = result.getString("mobilenumber");
> String rawresponse = result.getString("rawresponse");
>           	
> XContentBuilder builder = jsonBuilder()
> .startObject()
> .field("deliveryid", deliveryid)
> .field("msgid", msgid)
> .field("message", "trying out Elasticsearch")
> .endObject();
> 
> // YOUR ORACLE stuff here
> // either use client#prepare, or use Requests# to directly build index/delete requests
> bulkRequest.add(client.prepareIndex("logs", "responselog", id)
> .setSource(jsonBuilder()
> .startObject()
> .field("deliveryid", deliveryid)
> .field("msgid", msgid) 
> .field("rspdate", rspdate)
> .field("parsedresponse", parsedresponse)
> .field("shortcode", shortcode)
> .field("insid", insid)
> .field("mobilenumber", mobilenumber)
> .field("rawresponse", rawresponse)
> .endObject()
> )
> );    	
>          	
> System.out.println("Responselog from Oracle : "+ result.getString("id")+" "+result.getString("deliveryid")+" "+result.getString("msgid")+" "+result.getString("rspdate")+" "+result.getString("parsedresponse")+" "+result.getString("shortcode")+" "+result.getString("insid")+" "+result.getString("mobilenumber")+" "+result.getString("rawresponse")); 
> }              
>     
> BulkResponse bulkResponse = bulkRequest.execute().actionGet();
> System.out.println("done");
> } 
> 
> ```
> 
> --  
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com) [https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com?utm_medium=email&utm_source=footer).  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout) [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/568DFD50-81A0-46EF-9B64-1EEE5159BDE5%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/568DFD50-81A0-46EF-9B64-1EEE5159BDE5%40pilato.fr).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 7, 2015, 4:52pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523/10 "2015-01-07T16:52:26Z")

</div>

Well. I did not mention your plugin because he basically asked for "I want to know how can I input the data that I'm getting in Elasticsearch in real time.".  
Real time aspect here seems to be important.

Elasticsearch is not real time but near real time.  
JDBC river is obviously not real time as IIRC it runs a job every x minutes or so.

--  
David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com)  
@dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs)

> Le 7 janv. 2015 à 17:35, [joergprante@gmail.com](mailto:joergprante@gmail.com) a écrit :
> 
> You can use the JDBC plugin instead of reinventing the wheel.
> 
> [GitHub - jprante/elasticsearch-jdbc: JDBC importer for Elasticsearch](https://github.com/jprante/elasticsearch-river-jdbc) [https://github.com/jprante/elasticsearch-river-jdbc](https://github.com/jprante/elasticsearch-river-jdbc)
> 
> Jörg
> 
> On Wed, Jan 7, 2015 at 5:28 PM, Marian Valero \<[marianv0992@gmail.com](mailto:marianv0992@gmail.com) [mailto:marianv0992@gmail.com](mailto:marianv0992@gmail.com)\> wrote:  
> Thanks!
> 
> I have a problem with jsonBuilder() method, eclipse don't found this, I do it that way:
> 
> ```
> public static void main(String args[]) throws SQLException {
> 	    	
> int count=0;
> Node node = nodeBuilder().client(true).node();
> Client client = node.client();
> 	
> String url = "jdbc:oracle:thin:@eso:1521:eso"; 
>   
> Properties props = new Properties();
> props.setProperty("user", "eso");
> props.setProperty("password", "eso");
>   
> Connection conn = DriverManager.getConnection(url,props);
> 
> String sql ="select * from responselog";
> 
> PreparedStatement preStatement = conn.prepareStatement(sql);
> 
> ResultSet result = preStatement.executeQuery();
>   
> // Timestamp timestamp = null;
>     
> BulkRequestBuilder bulkRequest = client.prepareBulk();
>     
> while(result.next() && count<1000){
> count++;
> String id = result.getString("id");
> String deliveryid = result.getString("deliveryid");
> String msgid = result.getString("msgid");
> String rspdate = result.getString("rspdate");
> String parsedresponse = result.getString("parsedresponse");
> String shortcode = result.getString("shortcode");
> String insid = result.getString("insid");
> String mobilenumber = result.getString("mobilenumber");
> String rawresponse = result.getString("rawresponse");
>           	
> XContentBuilder builder = jsonBuilder()
> .startObject()
> .field("deliveryid", deliveryid)
> .field("msgid", msgid)
> .field("message", "trying out Elasticsearch")
> .endObject();
> 
> // YOUR ORACLE stuff here
> // either use client#prepare, or use Requests# to directly build index/delete requests
> bulkRequest.add(client.prepareIndex("logs", "responselog", id)
> .setSource(jsonBuilder()
> .startObject()
> .field("deliveryid", deliveryid)
> .field("msgid", msgid) 
> .field("rspdate", rspdate)
> .field("parsedresponse", parsedresponse)
> .field("shortcode", shortcode)
> .field("insid", insid)
> .field("mobilenumber", mobilenumber)
> .field("rawresponse", rawresponse)
> .endObject()
> )
> );    	
>          	
> System.out.println("Responselog from Oracle : "+ result.getString("id")+" "+result.getString("deliveryid")+" "+result.getString("msgid")+" "+result.getString("rspdate")+" "+result.getString("parsedresponse")+" "+result.getString("shortcode")+" "+result.getString("insid")+" "+result.getString("mobilenumber")+" "+result.getString("rawresponse")); 
> }              
>     
> BulkResponse bulkResponse = bulkRequest.execute().actionGet();
> System.out.println("done");
> } 
> 
> ```
> 
> --  
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com) [https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com?utm_medium=email&utm_source=footer).
> 
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout) [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> 
> --  
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcx7gXvXzg83cu9dFx7%3DYtW%2BA\_K%2BtBANUpdD-86uB8GQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcx7gXvXzg83cu9dFx7%3DYtW%2BA_K%2BtBANUpdD-86uB8GQ%40mail.gmail.com) [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcx7gXvXzg83cu9dFx7%3DYtW%2BA\_K%2BtBANUpdD-86uB8GQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcx7gXvXzg83cu9dFx7%3DYtW%2BA_K%2BtBANUpdD-86uB8GQ%40mail.gmail.com?utm_medium=email&utm_source=footer).  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout) [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/AA7A49A7-5BFC-4DFD-9745-BBEAAD60E603%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/AA7A49A7-5BFC-4DFD-9745-BBEAAD60E603%40pilato.fr).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Marian\_Valero](https://avatars.discourse-cdn.com/v4/letter/m/b9e5f3/32.png) [@Marian\_Valero](https://discuss.elastic.co/u/Marian_Valero)\
**Post date:** [January 7, 2015, 5:24pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523/11 "2015-01-07T17:24:46Z")

</div>

Yes! I previously import this but the problem persist

El miércoles, 7 de enero de 2015 12:22:39 UTC-4:30, David Pilato escribió:

> Of course, but if you read carefully the links to the doc I pasted, you  
> will see that you have to import a class.
> 
> Doc says:
> 
> import static org.elasticsearch.common.xcontent.XContentFactory.\*;
> 
> XContentBuilder builder = jsonBuilder()  
> .startObject()  
> .field("user", "kimchy")  
> .field("postDate", new Date())  
> .field("message", "trying out Elasticsearch")  
> .endObject()
> 
> Did you see it?
> 
> --  
> _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)  
> [http://Elasticsearch.com](http://Elasticsearch.com)_  
> @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr  
> [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) | @scrutmydocs  
> [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs)
> 
> Le 7 janv. 2015 à 17:28, Marian Valero \<[maria...@gmail.com](mailto:maria...@gmail.com) \<javascript:\>\>  
> a écrit :
> 
> Thanks!
> 
> > 
> 
> I have a problem with jsonBuilder() method, eclipse don't found this, I do  
> it that way:
> 
> ```
> public static void main(String args[]) throws SQLException {
>      
> int count=0;
> Node node = nodeBuilder().client(true).node();
> Client client = node.client();
> 
> String url = "jdbc:oracle:thin:@eso:1521:eso"; 
>   
> Properties props = new Properties();
> props.setProperty("user", "eso");
> props.setProperty("password", "eso");
>   
> Connection conn = DriverManager.getConnection(url,props);
> 
> String sql ="select * from responselog";
> 
> PreparedStatement preStatement = conn.prepareStatement(sql);
> 
> ResultSet result = preStatement.executeQuery();
>   
> // Timestamp timestamp = null;
>     
> BulkRequestBuilder bulkRequest = client.prepareBulk();
>     
> while(result.next() && count<1000){
> count++;
> String id = result.getString("id");
> String deliveryid = result.getString("deliveryid");
> String msgid = result.getString("msgid");
> String rspdate = result.getString("rspdate");
> String parsedresponse = result.getString("parsedresponse");
> String shortcode = result.getString("shortcode");
> String insid = result.getString("insid");
> String mobilenumber = result.getString("mobilenumber");
> String rawresponse = result.getString("rawresponse");
>           
> XContentBuilder builder = jsonBuilder()
> .startObject()
> .field("deliveryid", deliveryid)
> .field("msgid", msgid)
> .field("message", "trying out Elasticsearch")
> .endObject();
> 
> // YOUR ORACLE stuff here
> // either use client#prepare, or use Requests# to directly build 
> 
> ```
> 
> index/delete requests  
> bulkRequest.add(client.prepareIndex("logs", "responselog", id)  
> .setSource(jsonBuilder()  
> .startObject()  
> .field("deliveryid", deliveryid)  
> .field("msgid", msgid)  
> .field("rspdate", rspdate)  
> .field("parsedresponse", parsedresponse)  
> .field("shortcode", shortcode)  
> .field("insid", insid)  
> .field("mobilenumber", mobilenumber)  
> .field("rawresponse", rawresponse)  
> .endObject()  
> )  
> );
> 
> ```
> System.out.println("Responselog from Oracle : "+ 
> 
> ```
> 
> result.getString("id")+" "+result.getString("deliveryid")+"  
> "+result.getString("msgid")+" "+result.getString("rspdate")+"  
> "+result.getString("parsedresponse")+" "+result.getString("shortcode")+"  
> "+result.getString("insid")+" "+result.getString("mobilenumber")+"  
> "+result.getString("rawresponse"));  
> }
> 
> ```
> BulkResponse bulkResponse = bulkRequest.execute().actionGet();
> System.out.println("done");
> } 
> 
> ```
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/73ca9160-f1b4-4ccc-808b-d640082f6ddb%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/73ca9160-f1b4-4ccc-808b-d640082f6ddb%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Marian\_Valero](https://avatars.discourse-cdn.com/v4/letter/m/b9e5f3/32.png) [@Marian\_Valero](https://discuss.elastic.co/u/Marian_Valero)\
**Post date:** [January 7, 2015, 5:31pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523/12 "2015-01-07T17:31:56Z")

</div>

I think that is problem of the jar but I import this well.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/63744158-4ecc-44cb-862c-29cec2097b11%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/63744158-4ecc-44cb-862c-29cec2097b11%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [January 7, 2015, 5:47pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523/13 "2015-01-07T17:47:26Z")

</div>

The definitions of "real time" are quite vague. I also tend to use "real  
time" for sub-second latencies with a guarantee of a maximum response time.  
Other just mean by "real time" everything that is not batching, pushing  
data over the wire as the changes happen, no matter how large the change is

- a process which can take huge delays in worst case, even minutes.

Although it is possible to run a select query every second with JDBC plugin  
crontab, this puts a heavy load on the system with not much gain, so I  
don't recommend it.

For instant replication, one would have to extend JDBC plugin with a  
special queue on the side of the RDBMS and use proprietary driver API.  
Even with such a queue the latency is often higher than a few seconds.

Jörg

On Wed, Jan 7, 2015 at 5:52 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote:

> Well. I did not mention your plugin because he basically asked for "I want  
> to know how can I input the data that I'm getting in Elasticsearch in real  
> time.".  
> Real time aspect here seems to be important.
> 
> Elasticsearch is not real time but near real time.  
> JDBC river is obviously not real time as IIRC it runs a job every x  
> minutes or so.
> 
> --  
> _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)  
> [http://Elasticsearch.com](http://Elasticsearch.com)_  
> @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr  
> [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) | @scrutmydocs  
> [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs)
> 
> Le 7 janv. 2015 à 17:35, [joergprante@gmail.com](mailto:joergprante@gmail.com) a écrit :
> 
> You can use the JDBC plugin instead of reinventing the wheel.
> 
> [GitHub - jprante/elasticsearch-jdbc: JDBC importer for Elasticsearch](https://github.com/jprante/elasticsearch-river-jdbc)
> 
> Jörg
> 
> On Wed, Jan 7, 2015 at 5:28 PM, Marian Valero [marianv0992@gmail.com](mailto:marianv0992@gmail.com)  
> wrote:
> 
> > Thanks!
> > 
> > > 
> > 
> > I have a problem with jsonBuilder() method, eclipse don't found this, I  
> > do it that way:
> > 
> > ```
> > public static void main(String args[]) throws SQLException {
> > 
> > int count=0;
> > Node node = nodeBuilder().client(true).node();
> > Client client = node.client();
> > 
> > String url = "jdbc:oracle:thin:@eso:1521:eso";
> > 
> > Properties props = new Properties();
> > props.setProperty("user", "eso");
> > props.setProperty("password", "eso");
> > 
> > Connection conn = DriverManager.getConnection(url,props);
> > 
> > String sql ="select * from responselog";
> > 
> > PreparedStatement preStatement = conn.prepareStatement(sql);
> > 
> > ResultSet result = preStatement.executeQuery();
> > 
> > // Timestamp timestamp = null;
> > 
> > BulkRequestBuilder bulkRequest = client.prepareBulk();
> > 
> > while(result.next() && count<1000){
> > count++;
> > String id = result.getString("id");
> > String deliveryid = result.getString("deliveryid");
> > String msgid = result.getString("msgid");
> > String rspdate = result.getString("rspdate");
> > String parsedresponse = result.getString("parsedresponse");
> > String shortcode = result.getString("shortcode");
> > String insid = result.getString("insid");
> > String mobilenumber = result.getString("mobilenumber");
> > String rawresponse = result.getString("rawresponse");
> > 
> > XContentBuilder builder = jsonBuilder()
> > .startObject()
> > .field("deliveryid", deliveryid)
> > .field("msgid", msgid)
> > .field("message", "trying out Elasticsearch")
> > .endObject();
> > 
> > // YOUR ORACLE stuff here
> > // either use client#prepare, or use Requests# to directly build
> > 
> > ```
> > 
> > index/delete requests  
> > bulkRequest.add(client.prepareIndex("logs", "responselog", id)  
> > .setSource(jsonBuilder()  
> > .startObject()  
> > .field("deliveryid", deliveryid)  
> > .field("msgid", msgid)  
> > .field("rspdate", rspdate)  
> > .field("parsedresponse", parsedresponse)  
> > .field("shortcode", shortcode)  
> > .field("insid", insid)  
> > .field("mobilenumber", mobilenumber)  
> > .field("rawresponse", rawresponse)  
> > .endObject()  
> > )  
> > );
> > 
> > ```
> > System.out.println("Responselog from Oracle : "+
> > 
> > ```
> > 
> > result.getString("id")+" "+result.getString("deliveryid")+"  
> > "+result.getString("msgid")+" "+result.getString("rspdate")+"  
> > "+result.getString("parsedresponse")+" "+result.getString("shortcode")+"  
> > "+result.getString("insid")+" "+result.getString("mobilenumber")+"  
> > "+result.getString("rawresponse"));  
> > }
> > 
> > ```
> > BulkResponse bulkResponse = bulkRequest.execute().actionGet();
> > System.out.println("done");
> > }
> > 
> > ```
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com)  
> > [https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > .
> > 
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcx7gXvXzg83cu9dFx7%3DYtW%2BA\_K%2BtBANUpdD-86uB8GQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcx7gXvXzg83cu9dFx7%3DYtW%2BA_K%2BtBANUpdD-86uB8GQ%40mail.gmail.com)  
> [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcx7gXvXzg83cu9dFx7%3DYtW%2BA\_K%2BtBANUpdD-86uB8GQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcx7gXvXzg83cu9dFx7%3DYtW%2BA_K%2BtBANUpdD-86uB8GQ%40mail.gmail.com?utm_medium=email&utm_source=footer)  
> .  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/AA7A49A7-5BFC-4DFD-9745-BBEAAD60E603%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/AA7A49A7-5BFC-4DFD-9745-BBEAAD60E603%40pilato.fr)  
> [https://groups.google.com/d/msgid/elasticsearch/AA7A49A7-5BFC-4DFD-9745-BBEAAD60E603%40pilato.fr?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/AA7A49A7-5BFC-4DFD-9745-BBEAAD60E603%40pilato.fr?utm_medium=email&utm_source=footer)  
> .
> 
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE0CUreVbvpQrtXE186yqjA%2BtiYPCfZkwK\_3QxfoOnOSA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE0CUreVbvpQrtXE186yqjA%2BtiYPCfZkwK_3QxfoOnOSA%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 7, 2015, 5:56pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523/14 "2015-01-07T17:56:07Z")

</div>

Agreed. Reading is database could be an heavy operation.  
That’s why I always recommend if possible to send to elasticsearch data in the same « transaction » as you persist your data to the database.  
You often have a bean ready to be persisted in memory. Serializing to JSON with Jackson and then sending to elasticsearch does not cost much in term of CPU usage/memory and in term of line of codes.

But for that, you need to have an access to the application which produces the data.

If not, I agree, you should not reinvent the wheel and use whatever ETL you want to read, transform and index data. Such as JDBC River.  
My advice about JDBC river would be to run it outside elasticsearch as a feeder: [GitHub - jprante/elasticsearch-jdbc: JDBC importer for Elasticsearch](https://github.com/jprante/elasticsearch-river-jdbc#two-flavors-river-or-feeder) [https://github.com/jprante/elasticsearch-river-jdbc#two-flavors-river-or-feeder](https://github.com/jprante/elasticsearch-river-jdbc#two-flavors-river-or-feeder)

Best

--  
David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com)  
@dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs)

> Le 7 janv. 2015 à 18:47, [joergprante@gmail.com](mailto:joergprante@gmail.com) a écrit :
> 
> The definitions of "real time" are quite vague. I also tend to use "real time" for sub-second latencies with a guarantee of a maximum response time. Other just mean by "real time" everything that is not batching, pushing data over the wire as the changes happen, no matter how large the change is - a process which can take huge delays in worst case, even minutes.
> 
> Although it is possible to run a select query every second with JDBC plugin crontab, this puts a heavy load on the system with not much gain, so I don't recommend it.
> 
> For instant replication, one would have to extend JDBC plugin with a special queue on the side of the RDBMS and use proprietary driver API. Even with such a queue the latency is often higher than a few seconds.
> 
> Jörg
> 
> On Wed, Jan 7, 2015 at 5:52 PM, David Pilato \<[david@pilato.fr](mailto:david@pilato.fr) [mailto:david@pilato.fr](mailto:david@pilato.fr)\> wrote:  
> Well. I did not mention your plugin because he basically asked for "I want to know how can I input the data that I'm getting in Elasticsearch in real time.".  
> Real time aspect here seems to be important.
> 
> Elasticsearch is not real time but near real time.  
> JDBC river is obviously not real time as IIRC it runs a job every x minutes or so.
> 
> --  
> David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com) [http://elasticsearch.com/](http://elasticsearch.com/)  
> @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs)
> 
> > Le 7 janv. 2015 à 17:35, [joergprante@gmail.com](mailto:joergprante@gmail.com) [mailto:joergprante@gmail.com](mailto:joergprante@gmail.com) a écrit :
> > 
> > You can use the JDBC plugin instead of reinventing the wheel.
> > 
> > [GitHub - jprante/elasticsearch-jdbc: JDBC importer for Elasticsearch](https://github.com/jprante/elasticsearch-river-jdbc) [https://github.com/jprante/elasticsearch-river-jdbc](https://github.com/jprante/elasticsearch-river-jdbc)
> > 
> > Jörg
> > 
> > On Wed, Jan 7, 2015 at 5:28 PM, Marian Valero \<[marianv0992@gmail.com](mailto:marianv0992@gmail.com) [mailto:marianv0992@gmail.com](mailto:marianv0992@gmail.com)\> wrote:  
> > Thanks!
> > 
> > I have a problem with jsonBuilder() method, eclipse don't found this, I do it that way:
> > 
> > ```
> > public static void main(String args[]) throws SQLException {
> > 	    	
> > int count=0;
> > Node node = nodeBuilder().client(true).node();
> > Client client = node.client();
> > 	
> > String url = "jdbc:oracle:thin:@eso:1521:eso"; 
> >   
> > Properties props = new Properties();
> > props.setProperty("user", "eso");
> > props.setProperty("password", "eso");
> >   
> > Connection conn = DriverManager.getConnection(url,props);
> > 
> > String sql ="select * from responselog";
> > 
> > PreparedStatement preStatement = conn.prepareStatement(sql);
> > 
> > ResultSet result = preStatement.executeQuery();
> >   
> > // Timestamp timestamp = null;
> >     
> > BulkRequestBuilder bulkRequest = client.prepareBulk();
> >     
> > while(result.next() && count<1000){
> > count++;
> > String id = result.getString("id");
> > String deliveryid = result.getString("deliveryid");
> > String msgid = result.getString("msgid");
> > String rspdate = result.getString("rspdate");
> > String parsedresponse = result.getString("parsedresponse");
> > String shortcode = result.getString("shortcode");
> > String insid = result.getString("insid");
> > String mobilenumber = result.getString("mobilenumber");
> > String rawresponse = result.getString("rawresponse");
> >           	
> > XContentBuilder builder = jsonBuilder()
> > .startObject()
> > .field("deliveryid", deliveryid)
> > .field("msgid", msgid)
> > .field("message", "trying out Elasticsearch")
> > .endObject();
> > 
> > // YOUR ORACLE stuff here
> > // either use client#prepare, or use Requests# to directly build index/delete requests
> > bulkRequest.add(client.prepareIndex("logs", "responselog", id)
> > .setSource(jsonBuilder()
> > .startObject()
> > .field("deliveryid", deliveryid)
> > .field("msgid", msgid) 
> > .field("rspdate", rspdate)
> > .field("parsedresponse", parsedresponse)
> > .field("shortcode", shortcode)
> > .field("insid", insid)
> > .field("mobilenumber", mobilenumber)
> > .field("rawresponse", rawresponse)
> > .endObject()
> > )
> > );    	
> >          	
> > System.out.println("Responselog from Oracle : "+ result.getString("id")+" "+result.getString("deliveryid")+" "+result.getString("msgid")+" "+result.getString("rspdate")+" "+result.getString("parsedresponse")+" "+result.getString("shortcode")+" "+result.getString("insid")+" "+result.getString("mobilenumber")+" "+result.getString("rawresponse")); 
> > }              
> >     
> > BulkResponse bulkResponse = bulkRequest.execute().actionGet();
> > System.out.println("done");
> > } 
> > 
> > ```
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com) [https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com?utm_medium=email&utm_source=footer).
> > 
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout) [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcx7gXvXzg83cu9dFx7%3DYtW%2BA\_K%2BtBANUpdD-86uB8GQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcx7gXvXzg83cu9dFx7%3DYtW%2BA_K%2BtBANUpdD-86uB8GQ%40mail.gmail.com) [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcx7gXvXzg83cu9dFx7%3DYtW%2BA\_K%2BtBANUpdD-86uB8GQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcx7gXvXzg83cu9dFx7%3DYtW%2BA_K%2BtBANUpdD-86uB8GQ%40mail.gmail.com?utm_medium=email&utm_source=footer).  
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout) [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> 
> --  
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/AA7A49A7-5BFC-4DFD-9745-BBEAAD60E603%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/AA7A49A7-5BFC-4DFD-9745-BBEAAD60E603%40pilato.fr) [https://groups.google.com/d/msgid/elasticsearch/AA7A49A7-5BFC-4DFD-9745-BBEAAD60E603%40pilato.fr?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/AA7A49A7-5BFC-4DFD-9745-BBEAAD60E603%40pilato.fr?utm_medium=email&utm_source=footer).
> 
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout) [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> 
> --  
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE0CUreVbvpQrtXE186yqjA%2BtiYPCfZkwK\_3QxfoOnOSA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE0CUreVbvpQrtXE186yqjA%2BtiYPCfZkwK_3QxfoOnOSA%40mail.gmail.com) [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE0CUreVbvpQrtXE186yqjA%2BtiYPCfZkwK\_3QxfoOnOSA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE0CUreVbvpQrtXE186yqjA%2BtiYPCfZkwK_3QxfoOnOSA%40mail.gmail.com?utm_medium=email&utm_source=footer).  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout) [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/8F8C7196-83E8-48BC-9AF3-94F866A65E8B%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/8F8C7196-83E8-48BC-9AF3-94F866A65E8B%40pilato.fr).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Marian\_Valero](https://avatars.discourse-cdn.com/v4/letter/m/b9e5f3/32.png) [@Marian\_Valero](https://discuss.elastic.co/u/Marian_Valero)\
**Post date:** [January 7, 2015, 6:29pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523/15 "2015-01-07T18:29:19Z")

</div>

I understand both opinions, for give you a reference of my project, it is  
based in migrate data from oracle to ES for analyze it with kibana,  
specially are logs, millions of logs (around 90.000.000 of lines) and I'm  
looking for the best way to do that, I don't know how many cluster I have  
to use because is inserted data all days, I just to insert data like David  
explain me and I can fix the error that I have and all worked very well  
also I can see my data in Kibana. I have another question, how can I insert  
not only 1000 registers like I'm doing now, else all my data that is  
around 90.000.000, like I told you before and this increment so fast.

Thank you both guys.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/30010282-64e5-4b94-9f9a-f16aa0a7659e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/30010282-64e5-4b94-9f9a-f16aa0a7659e%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [January 7, 2015, 10:21pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523/16 "2015-01-07T22:21:39Z")

</div>

You can change the logic for "count \< 1000", submit bulk request at each  
count of 1000 with condition "count % 1000 == 0" or something, and iterate  
with resultset.next() through all rows.

If you want to load rows with less stress on the Oracle read transactional  
store and less memory consumption, you should not use a single SQL  
statement but several ones to partition the SQL result set into smaller  
ones, each fetching parts of the 90m rows.

Jörg

On Wed, Jan 7, 2015 at 7:29 PM, Marian Valero [marianv0992@gmail.com](mailto:marianv0992@gmail.com) wrote:

> I understand both opinions, for give you a reference of my project, it is  
> based in migrate data from oracle to ES for analyze it with kibana,  
> specially are logs, millions of logs (around 90.000.000 of lines) and I'm  
> looking for the best way to do that, I don't know how many cluster I have  
> to use because is inserted data all days, I just to insert data like David  
> explain me and I can fix the error that I have and all worked very well  
> also I can see my data in Kibana. I have another question, how can I insert  
> not only 1000 registers like I'm doing now, else all my data that is  
> around 90.000.000, like I told you before and this increment so fast.
> 
> Thank you both guys.
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/30010282-64e5-4b94-9f9a-f16aa0a7659e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/30010282-64e5-4b94-9f9a-f16aa0a7659e%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/30010282-64e5-4b94-9f9a-f16aa0a7659e%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/30010282-64e5-4b94-9f9a-f16aa0a7659e%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .
> 
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEzH2MmRbL0Mt1LkLZ18PNCxjzvOGt8S2my%3DtwcsuoB6A%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEzH2MmRbL0Mt1LkLZ18PNCxjzvOGt8S2my%3DtwcsuoB6A%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Marian\_Valero](https://avatars.discourse-cdn.com/v4/letter/m/b9e5f3/32.png) [@Marian\_Valero](https://discuss.elastic.co/u/Marian_Valero)\
**Post date:** [January 8, 2015, 1:15pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523/17 "2015-01-08T13:15:04Z")

</div>

Ok! thank you, and referent to cluster, because is so many data an everyday  
this increment and I can't have all data in only one machine, how many  
cluster I have to use?

Thanks for all.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0886b197-b9e9-4cea-bce5-64f0a47b203d%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0886b197-b9e9-4cea-bce5-64f0a47b203d%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [January 8, 2015, 10:37pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523/18 "2015-01-08T22:37:56Z")

</div>

You have to try capacity planning for yourself, by adjusting configuration  
of index / shards / nodes and resource control parameters like heap size,  
how many documents your system can load. There is no fixed rule that can  
estimate the maximum amount of data volume an ES cluster is capable of.  
Note that you can always add nodes and create new indices, it is possible  
to grow a single cluster into hundreds or thousands of nodes ( = machines).

For a rough number, you should estimate your retention data volume, maybe  
you must keep the data for some days and you can drop it then. Then you  
have to estimate the resources for search (quer load, filter caches etc.)  
All these factors should be taken into consideration for the number of  
nodes you may need.

Jörg

On Thu, Jan 8, 2015 at 2:15 PM, Marian Valero [marianv0992@gmail.com](mailto:marianv0992@gmail.com) wrote:

> Ok! thank you, and referent to cluster, because is so many data an  
> everyday this increment and I can't have all data in only one machine, how  
> many cluster I have to use?
> 
> Thanks for all.
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/0886b197-b9e9-4cea-bce5-64f0a47b203d%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0886b197-b9e9-4cea-bce5-64f0a47b203d%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/0886b197-b9e9-4cea-bce5-64f0a47b203d%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0886b197-b9e9-4cea-bce5-64f0a47b203d%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .
> 
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHMBfCKmrjvgWRwwJP6SsHQqw9Q3qmz93azDWOGJWay4g%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHMBfCKmrjvgWRwwJP6SsHQqw9Q3qmz93azDWOGJWay4g%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Ed\_Kim](https://avatars.discourse-cdn.com/v4/letter/e/53a042/32.png) [@Ed\_Kim](https://discuss.elastic.co/u/Ed_Kim)\
**Post date:** [January 13, 2015, 8:52pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523/19 "2015-01-13T20:52:23Z")

</div>

Not sure if this is an option for you, but if an application is feeding  
that log data into oracle, you could consider having that application also  
index into Elasticsearch.

On Wednesday, January 7, 2015 at 5:46:18 AM UTC-8, Marian Valero wrote:

> I'm reading data from my oracle database to java with JDBC but I want to  
> know how can I input the data that I'm getting in Elasticsearch in real  
> time.
> 
> Thanks.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/86e937ac-af3f-43f7-8c1f-60c7517b8cbf%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/86e937ac-af3f-43f7-8c1f-60c7517b8cbf%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Marian\_Valero](https://avatars.discourse-cdn.com/v4/letter/m/b9e5f3/32.png) [@Marian\_Valero](https://discuss.elastic.co/u/Marian_Valero)\
**Post date:** [January 13, 2015, 9:07pm UTC](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523/20 "2015-01-13T21:07:12Z")

</div>

Thanks Ed.

Yes, but I have data from 10 years ago and I have to input this data too  
for analyze that logs with the data that is insert every day. There are  
millions of logs.

2015-01-13 16:22 GMT-04:30 Ed Kim [edkim81@gmail.com](mailto:edkim81@gmail.com):

> Not sure if this is an option for you, but if an application is feeding  
> that log data into oracle, you could consider having that application also  
> index into Elasticsearch.
> 
> On Wednesday, January 7, 2015 at 5:46:18 AM UTC-8, Marian Valero wrote:
> 
> > I'm reading data from my oracle database to java with JDBC but I want to  
> > know how can I input the data that I'm getting in Elasticsearch in real  
> > time.
> > 
> > Thanks.
> 
> --  
> You received this message because you are subscribed to a topic in the  
> Google Groups "elasticsearch" group.  
> To unsubscribe from this topic, visit  
> [https://groups.google.com/d/topic/elasticsearch/cnkq84aom1M/unsubscribe](https://groups.google.com/d/topic/elasticsearch/cnkq84aom1M/unsubscribe).  
> To unsubscribe from this group and all its topics, send an email to  
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/86e937ac-af3f-43f7-8c1f-60c7517b8cbf%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/86e937ac-af3f-43f7-8c1f-60c7517b8cbf%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/86e937ac-af3f-43f7-8c1f-60c7517b8cbf%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/86e937ac-af3f-43f7-8c1f-60c7517b8cbf%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .
> 
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CADT%2BR%3DZ6LNFP-0m%2BwHdwkj1MRupD2\_Ybbbo7A9dSKtE126v%3DbA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CADT%2BR%3DZ6LNFP-0m%2BwHdwkj1MRupD2_Ybbbo7A9dSKtE126v%3DbA%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

[Next page](https://discuss.elastic.co/t/how-can-i-input-data-from-java-to-es-in-real-time/21523.md?page=2)