Do I need the JDBC driver

Hi,

I'm setting up a system where I have a main SQL database which is synced
with elasticsearch. My plan is to use the main PHP library for
elasticsearch.

I was going to have a cron run every thirty minuets to check for items in
my database that not only have an "active" flag but that also do not have
an "indexed" flag, that means I need to add them to the index. Then I was
going to add that item to the index. Since I am using taking this path, it
doesn't seem like I need the JDBC driver, as I can add items to
elasticsearch using the PHP library.

So, my question is, can I get away without using the JDBC driver?

James

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/be682d05-bdad-45a4-8b00-2ecf26217534%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hello James ,

I didn't fully understand your question , but i feel JDBC river might be of
any use to you - https://github.com/jprante/elasticsearch-river-jdbc

Thanks
Vineeth

On Wed, Sep 10, 2014 at 3:29 PM, James mail@employ.com wrote:

Hi,

I'm setting up a system where I have a main SQL database which is synced
with elasticsearch. My plan is to use the main PHP library for
elasticsearch.

I was going to have a cron run every thirty minuets to check for items in
my database that not only have an "active" flag but that also do not have
an "indexed" flag, that means I need to add them to the index. Then I was
going to add that item to the index. Since I am using taking this path, it
doesn't seem like I need the JDBC driver, as I can add items to
elasticsearch using the PHP library.

So, my question is, can I get away without using the JDBC driver?

James

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/be682d05-bdad-45a4-8b00-2ecf26217534%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/be682d05-bdad-45a4-8b00-2ecf26217534%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGdPd5nFdiMxM7KDrN8-wZnpBtoBxX8BVbqiaSppanzKmNwz1g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

It just seems I can send data to elasticsearch using the php library, so I
don't seen the need to add the JDBC driver?

http://www.elasticsearch.org/guide/en/elasticsearch/client/php-api/current/_indexing_operations.html

Use a CRON operation to check for items that aren't indexed, and then use
the PHP library to send the non-indexed data to ES to get indexed?

On Wednesday, September 10, 2014 12:21:35 PM UTC+1, vineeth mohan wrote:

Hello James ,

I didn't fully understand your question , but i feel JDBC river might be
of any use to you - https://github.com/jprante/elasticsearch-river-jdbc

Thanks
Vineeth

On Wed, Sep 10, 2014 at 3:29 PM, James <ma...@employ.com <javascript:>>
wrote:

Hi,

I'm setting up a system where I have a main SQL database which is synced
with elasticsearch. My plan is to use the main PHP library for
elasticsearch.

I was going to have a cron run every thirty minuets to check for items in
my database that not only have an "active" flag but that also do not have
an "indexed" flag, that means I need to add them to the index. Then I was
going to add that item to the index. Since I am using taking this path, it
doesn't seem like I need the JDBC driver, as I can add items to
elasticsearch using the PHP library.

So, my question is, can I get away without using the JDBC driver?

James

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/be682d05-bdad-45a4-8b00-2ecf26217534%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/be682d05-bdad-45a4-8b00-2ecf26217534%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a9901f1a-2753-46d2-9113-056b8d996eb8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

You need JDBC (Java Database Connectivity) for Java.

Jörg

On Wed, Sep 10, 2014 at 1:30 PM, James mail@employ.com wrote:

It just seems I can send data to elasticsearch using the php library, so I
don't seen the need to add the JDBC driver?

http://www.elasticsearch.org/guide/en/elasticsearch/client/php-api/current/_indexing_operations.html

Use a CRON operation to check for items that aren't indexed, and then use
the PHP library to send the non-indexed data to ES to get indexed?

On Wednesday, September 10, 2014 12:21:35 PM UTC+1, vineeth mohan wrote:

Hello James ,

I didn't fully understand your question , but i feel JDBC river might be
of any use to you - https://github.com/jprante/elasticsearch-river-jdbc

Thanks
Vineeth

On Wed, Sep 10, 2014 at 3:29 PM, James ma...@employ.com wrote:

Hi,

I'm setting up a system where I have a main SQL database which is synced
with elasticsearch. My plan is to use the main PHP library for
elasticsearch.

I was going to have a cron run every thirty minuets to check for items
in my database that not only have an "active" flag but that also do not
have an "indexed" flag, that means I need to add them to the index. Then I
was going to add that item to the index. Since I am using taking this path,
it doesn't seem like I need the JDBC driver, as I can add items to
elasticsearch using the PHP library.

So, my question is, can I get away without using the JDBC driver?

James

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/be682d05-bdad-45a4-8b00-2ecf26217534%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/be682d05-bdad-45a4-8b00-2ecf26217534%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a9901f1a-2753-46d2-9113-056b8d996eb8%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/a9901f1a-2753-46d2-9113-056b8d996eb8%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoF33zrKTRqFhr%3D%3DMG2fYFYd04kezBqEcoXgYrFuaEyT6w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

I don't really see why though, can't the php library send JSON data to
elasticsearch?

On 10 September 2014 12:47, joergprante@gmail.com joergprante@gmail.com
wrote:

You need JDBC (Java Database Connectivity) for Java.

Jörg

On Wed, Sep 10, 2014 at 1:30 PM, James mail@employ.com wrote:

It just seems I can send data to elasticsearch using the php library, so
I don't seen the need to add the JDBC driver?

http://www.elasticsearch.org/guide/en/elasticsearch/client/php-api/current/_indexing_operations.html

Use a CRON operation to check for items that aren't indexed, and then use
the PHP library to send the non-indexed data to ES to get indexed?

On Wednesday, September 10, 2014 12:21:35 PM UTC+1, vineeth mohan wrote:

Hello James ,

I didn't fully understand your question , but i feel JDBC river might be
of any use to you - https://github.com/jprante/elasticsearch-river-jdbc

Thanks
Vineeth

On Wed, Sep 10, 2014 at 3:29 PM, James ma...@employ.com wrote:

Hi,

I'm setting up a system where I have a main SQL database which is
synced with elasticsearch. My plan is to use the main PHP library for
elasticsearch.

I was going to have a cron run every thirty minuets to check for items
in my database that not only have an "active" flag but that also do not
have an "indexed" flag, that means I need to add them to the index. Then I
was going to add that item to the index. Since I am using taking this path,
it doesn't seem like I need the JDBC driver, as I can add items to
elasticsearch using the PHP library.

So, my question is, can I get away without using the JDBC driver?

James

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/be682d05-bdad-45a4-8b00-2ecf26217534%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/be682d05-bdad-45a4-8b00-2ecf26217534%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a9901f1a-2753-46d2-9113-056b8d996eb8%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/a9901f1a-2753-46d2-9113-056b8d996eb8%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/0dzSMbARlks/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoF33zrKTRqFhr%3D%3DMG2fYFYd04kezBqEcoXgYrFuaEyT6w%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoF33zrKTRqFhr%3D%3DMG2fYFYd04kezBqEcoXgYrFuaEyT6w%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPng%3D3f6qkSEehuowD%3D1QVKnS2JgoGqqYjasL5gWsLkz%2Bq%3D29A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

I've also put this question up on stackoverflow for anyone who might be
able to help me understand.

On Wednesday, September 10, 2014 10:59:18 AM UTC+1, James wrote:

Hi,

I'm setting up a system where I have a main SQL database which is synced
with elasticsearch. My plan is to use the main PHP library for
elasticsearch.

I was going to have a cron run every thirty minuets to check for items in
my database that not only have an "active" flag but that also do not have
an "indexed" flag, that means I need to add them to the index. Then I was
going to add that item to the index. Since I am using taking this path, it
doesn't seem like I need the JDBC driver, as I can add items to
elasticsearch using the PHP library.

So, my question is, can I get away without using the JDBC driver?

James

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3d07653f-a62b-4e16-bab0-8a9b0008730d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

I think I answered the question, or I do not fully understand.

JDBC is Java Database Connectivity, it is Java only, it has nothing to do
with PHP. So if you choose PHP, it is obvious that you can not use it.

Jörg

On Thu, Sep 11, 2014 at 12:08 PM, James mail@employ.com wrote:

I've also put this question up on stackoverflow for anyone who might be
able to help me understand.

http://stackoverflow.com/questions/25763997/elasticsearch-do-i-need-the-jdbc-driver

On Wednesday, September 10, 2014 10:59:18 AM UTC+1, James wrote:

Hi,

I'm setting up a system where I have a main SQL database which is synced
with elasticsearch. My plan is to use the main PHP library for
elasticsearch.

I was going to have a cron run every thirty minuets to check for items in
my database that not only have an "active" flag but that also do not have
an "indexed" flag, that means I need to add them to the index. Then I was
going to add that item to the index. Since I am using taking this path, it
doesn't seem like I need the JDBC driver, as I can add items to
elasticsearch using the PHP library.

So, my question is, can I get away without using the JDBC driver?

James

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/3d07653f-a62b-4e16-bab0-8a9b0008730d%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/3d07653f-a62b-4e16-bab0-8a9b0008730d%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHY8cj8YoSh5wJN6MVK6UzjoJJEDdC7iDehBRqwxNjK-w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

I'm sorry but that doesn't answer my question. It's elasticsearch that is
Java. I need to sync elasticsearch with my SQL DB. I'm stuck between these
two scenarios:

Scenario 1:
PHP website adds data to the SQL DB
JDBC driver used by elasticsearch to grab values from SQL DB into index

Scenario 2:
PHP website adds data to SQL
CRON job uses PHP elasticsearch library to convert SQL to JSON and send it
to elasticsearch to be indexed.

Which one is the better implementation?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPng%3D3e2ykmb4S2aqaXiLbV1h39ZE_ZWvHkX4OmVEQV2hPK3_A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

How do you mean "PHP elasticsearch library can convert SQL to JSON"? How
can this be? It is only for Elasticsearch.

As a matter fact, there is no "JDBC driver used by elasticsearch", there is
a plugin elasticsearch-river-jdbc, a community effort - I assume you mean
this implementation?

What do you mean by "better" implementation, in regard to what requirements?

Jörg

On Thu, Sep 11, 2014 at 2:16 PM, James mail@employ.com wrote:

I'm sorry but that doesn't answer my question. It's elasticsearch that is
Java. I need to sync elasticsearch with my SQL DB. I'm stuck between these
two scenarios:

Scenario 1:
PHP website adds data to the SQL DB
JDBC driver used by elasticsearch to grab values from SQL DB into index

Scenario 2:
PHP website adds data to SQL
CRON job uses PHP elasticsearch library to convert SQL to JSON and send it
to elasticsearch to be indexed.

Which one is the better implementation?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAPng%3D3e2ykmb4S2aqaXiLbV1h39ZE_ZWvHkX4OmVEQV2hPK3_A%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAPng%3D3e2ykmb4S2aqaXiLbV1h39ZE_ZWvHkX4OmVEQV2hPK3_A%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG0p6P9TCSsjToewsEVdHnHY6jAWZPu0kP346vnxwVzMQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Hi Jorg,

Thank you for the reply. Yes I meant the elasticsearch river. Simply put, I
want to syncronize the entries in my SQL database with my elasticsearch, so
I can use elasicsearch for searching and not doing fulltext search. I want
to know that when a new item gets added or removed from that database that
it also gets added / removed from elasicsearch.

My understand, which might be wrong, is I can either use the PHP
elasticsearch library to push updates (adds / removes) to elasticsearch
when new items are added to SQL:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html

Or I can use the river JDBC river plugin for elasticsearch to connect to my
database directly and syncronize elasticsearch with the SQL database.

My two questions are:

  1. Is my understanding above correct
  2. Does one option have advantages over the other
  • James

On Wednesday, September 10, 2014 10:59:18 AM UTC+1, James wrote:

Hi,

I'm setting up a system where I have a main SQL database which is synced
with elasticsearch. My plan is to use the main PHP library for
elasticsearch.

I was going to have a cron run every thirty minuets to check for items in
my database that not only have an "active" flag but that also do not have
an "indexed" flag, that means I need to add them to the index. Then I was
going to add that item to the index. Since I am using taking this path, it
doesn't seem like I need the JDBC driver, as I can add items to
elasticsearch using the PHP library.

So, my question is, can I get away without using the JDBC driver?

James

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1d5fe901-fd0e-4663-9c68-5f7cf8092cf1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Synchronization of data is a very broad question. This is just because the
data organization in an RDBMS is very different from ES. You surely know
that. See also object-relational impedance mismatch

The JDBC river plugin allows you to define SQL statements so you can easily
construct JSON out if it, for indexing into ES.

If you can map identifiers from your RDBMS to JSON doc IDs and allocate the
_id field in the JDBC river plugin, you are lucky. In that case you can
just overwrite existing docs in ES to keep up with the most recent version.

Synchronization also includes modifications and deletions to avoid stale
docs, and transactional ACID properties. I have no general solution for
this. The best approach is to provide timewindowed indices and drop indices
that are too old, similar to what Logstash does.

Jörg

On Thu, Sep 11, 2014 at 3:39 PM, James mail@employ.com wrote:

Hi Jorg,

Thank you for the reply. Yes I meant the elasticsearch river. Simply put,
I want to syncronize the entries in my SQL database with my elasticsearch,
so I can use elasicsearch for searching and not doing fulltext search. I
want to know that when a new item gets added or removed from that database
that it also gets added / removed from elasicsearch.

My understand, which might be wrong, is I can either use the PHP
elasticsearch library to push updates (adds / removes) to elasticsearch
when new items are added to SQL:

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html

Or I can use the river JDBC river plugin for elasticsearch to connect to
my database directly and syncronize elasticsearch with the SQL database.

My two questions are:

  1. Is my understanding above correct
  2. Does one option have advantages over the other
  • James

On Wednesday, September 10, 2014 10:59:18 AM UTC+1, James wrote:

Hi,

I'm setting up a system where I have a main SQL database which is synced
with elasticsearch. My plan is to use the main PHP library for
elasticsearch.

I was going to have a cron run every thirty minuets to check for items in
my database that not only have an "active" flag but that also do not have
an "indexed" flag, that means I need to add them to the index. Then I was
going to add that item to the index. Since I am using taking this path, it
doesn't seem like I need the JDBC driver, as I can add items to
elasticsearch using the PHP library.

So, my question is, can I get away without using the JDBC driver?

James

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1d5fe901-fd0e-4663-9c68-5f7cf8092cf1%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/1d5fe901-fd0e-4663-9c68-5f7cf8092cf1%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFoUMAv3R0b1NN%2B9B7eBNCcqpjg7tTMuNQPzCgGGupkQw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Thank you, that answers a lot of my questions. There is still the point of using the Php library for elastic search, where I can send documents directly to elastic search in JSON format without needing a JDBC driver. Is this not a good option?

Sent from my iPhone

On 11 Sep 2014, at 16:33, "joergprante@gmail.com" joergprante@gmail.com wrote:

Synchronization of data is a very broad question. This is just because the data organization in an RDBMS is very different from ES. You surely know that. See also object-relational impedance mismatch http://en.wikipedia.org/wiki/Object-relational_impedance_mismatch

The JDBC river plugin allows you to define SQL statements so you can easily construct JSON out if it, for indexing into ES.

If you can map identifiers from your RDBMS to JSON doc IDs and allocate the _id field in the JDBC river plugin, you are lucky. In that case you can just overwrite existing docs in ES to keep up with the most recent version.

Synchronization also includes modifications and deletions to avoid stale docs, and transactional ACID properties. I have no general solution for this. The best approach is to provide timewindowed indices and drop indices that are too old, similar to what Logstash does.

Jörg

On Thu, Sep 11, 2014 at 3:39 PM, James mail@employ.com wrote:
Hi Jorg,

Thank you for the reply. Yes I meant the elasticsearch river. Simply put, I want to syncronize the entries in my SQL database with my elasticsearch, so I can use elasicsearch for searching and not doing fulltext search. I want to know that when a new item gets added or removed from that database that it also gets added / removed from elasicsearch.

My understand, which might be wrong, is I can either use the PHP elasticsearch library to push updates (adds / removes) to elasticsearch when new items are added to SQL:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html

Or I can use the river JDBC river plugin for elasticsearch to connect to my database directly and syncronize elasticsearch with the SQL database.

My two questions are:

  1. Is my understanding above correct
  2. Does one option have advantages over the other
  • James

On Wednesday, September 10, 2014 10:59:18 AM UTC+1, James wrote:
Hi,

I'm setting up a system where I have a main SQL database which is synced with elasticsearch. My plan is to use the main PHP library for elasticsearch.

I was going to have a cron run every thirty minuets to check for items in my database that not only have an "active" flag but that also do not have an "indexed" flag, that means I need to add them to the index. Then I was going to add that item to the index. Since I am using taking this path, it doesn't seem like I need the JDBC driver, as I can add items to elasticsearch using the PHP library.

So, my question is, can I get away without using the JDBC driver?

James

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1d5fe901-fd0e-4663-9c68-5f7cf8092cf1%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/0dzSMbARlks/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFoUMAv3R0b1NN%2B9B7eBNCcqpjg7tTMuNQPzCgGGupkQw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9F779070-9B92-4A0E-A09A-71848F551214%40employ.com.
For more options, visit https://groups.google.com/d/optout.

I do not know the PHP client in particular, but this is just another one of
the official Elasticsearch clients, like there are Elasticsearch clients
for other language, Perl, Python, Ruby, etc.

With an Elasticsearch client, you can use Elasticsearch, not an RDBMS
database.

Jörg

On Thu, Sep 11, 2014 at 6:57 PM, Employ mail@employ.com wrote:

Thank you, that answers a lot of my questions. There is still the point of
using the Php library for elastic search, where I can send documents
directly to elastic search in JSON format without needing a JDBC driver. Is
this not a good option?

Sent from my iPhone

On 11 Sep 2014, at 16:33, "joergprante@gmail.com" joergprante@gmail.com
wrote:

Synchronization of data is a very broad question. This is just because the
data organization in an RDBMS is very different from ES. You surely know
that. See also object-relational impedance mismatch
http://en.wikipedia.org/wiki/Object-relational_impedance_mismatch

The JDBC river plugin allows you to define SQL statements so you can
easily construct JSON out if it, for indexing into ES.

If you can map identifiers from your RDBMS to JSON doc IDs and allocate
the _id field in the JDBC river plugin, you are lucky. In that case you can
just overwrite existing docs in ES to keep up with the most recent version.

Synchronization also includes modifications and deletions to avoid stale
docs, and transactional ACID properties. I have no general solution for
this. The best approach is to provide timewindowed indices and drop indices
that are too old, similar to what Logstash does.

Jörg

On Thu, Sep 11, 2014 at 3:39 PM, James mail@employ.com wrote:

Hi Jorg,

Thank you for the reply. Yes I meant the elasticsearch river. Simply put,
I want to syncronize the entries in my SQL database with my elasticsearch,
so I can use elasicsearch for searching and not doing fulltext search. I
want to know that when a new item gets added or removed from that database
that it also gets added / removed from elasicsearch.

My understand, which might be wrong, is I can either use the PHP
elasticsearch library to push updates (adds / removes) to elasticsearch
when new items are added to SQL:

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html

Or I can use the river JDBC river plugin for elasticsearch to connect to
my database directly and syncronize elasticsearch with the SQL database.

My two questions are:

  1. Is my understanding above correct
  2. Does one option have advantages over the other
  • James

On Wednesday, September 10, 2014 10:59:18 AM UTC+1, James wrote:

Hi,

I'm setting up a system where I have a main SQL database which is synced
with elasticsearch. My plan is to use the main PHP library for
elasticsearch.

I was going to have a cron run every thirty minuets to check for items
in my database that not only have an "active" flag but that also do not
have an "indexed" flag, that means I need to add them to the index. Then I
was going to add that item to the index. Since I am using taking this path,
it doesn't seem like I need the JDBC driver, as I can add items to
elasticsearch using the PHP library.

So, my question is, can I get away without using the JDBC driver?

James

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1d5fe901-fd0e-4663-9c68-5f7cf8092cf1%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/1d5fe901-fd0e-4663-9c68-5f7cf8092cf1%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/0dzSMbARlks/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFoUMAv3R0b1NN%2B9B7eBNCcqpjg7tTMuNQPzCgGGupkQw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFoUMAv3R0b1NN%2B9B7eBNCcqpjg7tTMuNQPzCgGGupkQw%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9F779070-9B92-4A0E-A09A-71848F551214%40employ.com
https://groups.google.com/d/msgid/elasticsearch/9F779070-9B92-4A0E-A09A-71848F551214%40employ.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHjVQXhX403og_LCMB8OMTR%3DhZG9iYtJdWeTd7nN3umHg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

I'm sorry I'm such this is clear to a lot if people but not to me. Can I not use the official elastic search PHP client to add documents to elasticsearch? In which case my php website can use this library to grab data from the database, convert it to the right format and send it to elasticsearch to be indexed?

Sent from my iPhone

On 11 Sep 2014, at 19:52, "joergprante@gmail.com" joergprante@gmail.com wrote:

I do not know the PHP client in particular, but this is just another one of the official Elasticsearch clients, like there are Elasticsearch clients for other language, Perl, Python, Ruby, etc.

With an Elasticsearch client, you can use Elasticsearch, not an RDBMS database.

Jörg

On Thu, Sep 11, 2014 at 6:57 PM, Employ mail@employ.com wrote:
Thank you, that answers a lot of my questions. There is still the point of using the Php library for elastic search, where I can send documents directly to elastic search in JSON format without needing a JDBC driver. Is this not a good option?

Sent from my iPhone

On 11 Sep 2014, at 16:33, "joergprante@gmail.com" joergprante@gmail.com wrote:

Synchronization of data is a very broad question. This is just because the data organization in an RDBMS is very different from ES. You surely know that. See also object-relational impedance mismatch http://en.wikipedia.org/wiki/Object-relational_impedance_mismatch

The JDBC river plugin allows you to define SQL statements so you can easily construct JSON out if it, for indexing into ES.

If you can map identifiers from your RDBMS to JSON doc IDs and allocate the _id field in the JDBC river plugin, you are lucky. In that case you can just overwrite existing docs in ES to keep up with the most recent version.

Synchronization also includes modifications and deletions to avoid stale docs, and transactional ACID properties. I have no general solution for this. The best approach is to provide timewindowed indices and drop indices that are too old, similar to what Logstash does.

Jörg

On Thu, Sep 11, 2014 at 3:39 PM, James mail@employ.com wrote:
Hi Jorg,

Thank you for the reply. Yes I meant the elasticsearch river. Simply put, I want to syncronize the entries in my SQL database with my elasticsearch, so I can use elasicsearch for searching and not doing fulltext search. I want to know that when a new item gets added or removed from that database that it also gets added / removed from elasicsearch.

My understand, which might be wrong, is I can either use the PHP elasticsearch library to push updates (adds / removes) to elasticsearch when new items are added to SQL:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html

Or I can use the river JDBC river plugin for elasticsearch to connect to my database directly and syncronize elasticsearch with the SQL database.

My two questions are:

  1. Is my understanding above correct
  2. Does one option have advantages over the other
  • James

On Wednesday, September 10, 2014 10:59:18 AM UTC+1, James wrote:
Hi,

I'm setting up a system where I have a main SQL database which is synced with elasticsearch. My plan is to use the main PHP library for elasticsearch.

I was going to have a cron run every thirty minuets to check for items in my database that not only have an "active" flag but that also do not have an "indexed" flag, that means I need to add them to the index. Then I was going to add that item to the index. Since I am using taking this path, it doesn't seem like I need the JDBC driver, as I can add items to elasticsearch using the PHP library.

So, my question is, can I get away without using the JDBC driver?

James

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1d5fe901-fd0e-4663-9c68-5f7cf8092cf1%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/0dzSMbARlks/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFoUMAv3R0b1NN%2B9B7eBNCcqpjg7tTMuNQPzCgGGupkQw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9F779070-9B92-4A0E-A09A-71848F551214%40employ.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/0dzSMbARlks/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHjVQXhX403og_LCMB8OMTR%3DhZG9iYtJdWeTd7nN3umHg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/414A3494-DEAA-4B37-883E-8BC3854F8838%40employ.com.
For more options, visit https://groups.google.com/d/optout.

Like this: http://stackoverflow.com/questions/13576703/indexing-documents-in-elasticsearch-with-php-curl

Sent from my iPhone

On 11 Sep 2014, at 19:52, "joergprante@gmail.com" joergprante@gmail.com wrote:

I do not know the PHP client in particular, but this is just another one of the official Elasticsearch clients, like there are Elasticsearch clients for other language, Perl, Python, Ruby, etc.

With an Elasticsearch client, you can use Elasticsearch, not an RDBMS database.

Jörg

On Thu, Sep 11, 2014 at 6:57 PM, Employ mail@employ.com wrote:
Thank you, that answers a lot of my questions. There is still the point of using the Php library for elastic search, where I can send documents directly to elastic search in JSON format without needing a JDBC driver. Is this not a good option?

Sent from my iPhone

On 11 Sep 2014, at 16:33, "joergprante@gmail.com" joergprante@gmail.com wrote:

Synchronization of data is a very broad question. This is just because the data organization in an RDBMS is very different from ES. You surely know that. See also object-relational impedance mismatch http://en.wikipedia.org/wiki/Object-relational_impedance_mismatch

The JDBC river plugin allows you to define SQL statements so you can easily construct JSON out if it, for indexing into ES.

If you can map identifiers from your RDBMS to JSON doc IDs and allocate the _id field in the JDBC river plugin, you are lucky. In that case you can just overwrite existing docs in ES to keep up with the most recent version.

Synchronization also includes modifications and deletions to avoid stale docs, and transactional ACID properties. I have no general solution for this. The best approach is to provide timewindowed indices and drop indices that are too old, similar to what Logstash does.

Jörg

On Thu, Sep 11, 2014 at 3:39 PM, James mail@employ.com wrote:
Hi Jorg,

Thank you for the reply. Yes I meant the elasticsearch river. Simply put, I want to syncronize the entries in my SQL database with my elasticsearch, so I can use elasicsearch for searching and not doing fulltext search. I want to know that when a new item gets added or removed from that database that it also gets added / removed from elasicsearch.

My understand, which might be wrong, is I can either use the PHP elasticsearch library to push updates (adds / removes) to elasticsearch when new items are added to SQL:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html

Or I can use the river JDBC river plugin for elasticsearch to connect to my database directly and syncronize elasticsearch with the SQL database.

My two questions are:

  1. Is my understanding above correct
  2. Does one option have advantages over the other
  • James

On Wednesday, September 10, 2014 10:59:18 AM UTC+1, James wrote:
Hi,

I'm setting up a system where I have a main SQL database which is synced with elasticsearch. My plan is to use the main PHP library for elasticsearch.

I was going to have a cron run every thirty minuets to check for items in my database that not only have an "active" flag but that also do not have an "indexed" flag, that means I need to add them to the index. Then I was going to add that item to the index. Since I am using taking this path, it doesn't seem like I need the JDBC driver, as I can add items to elasticsearch using the PHP library.

So, my question is, can I get away without using the JDBC driver?

James

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1d5fe901-fd0e-4663-9c68-5f7cf8092cf1%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/0dzSMbARlks/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFoUMAv3R0b1NN%2B9B7eBNCcqpjg7tTMuNQPzCgGGupkQw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9F779070-9B92-4A0E-A09A-71848F551214%40employ.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/0dzSMbARlks/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHjVQXhX403og_LCMB8OMTR%3DhZG9iYtJdWeTd7nN3umHg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8BF3B8CA-2775-401F-A912-6AC0DFA50D80%40employ.com.
For more options, visit https://groups.google.com/d/optout.

I want to close this issue but I still do not understand if I should be
pushing documents from my database using the PHP client or using the JDBC
river to pull them into elasticsearch from the SQL database.

They can both achieve the same thing, but what is the usecase which defines
when is the right time to use each implementation.

On Wednesday, September 10, 2014 10:59:18 AM UTC+1, James wrote:

Hi,

I'm setting up a system where I have a main SQL database which is synced
with elasticsearch. My plan is to use the main PHP library for
elasticsearch.

I was going to have a cron run every thirty minuets to check for items in
my database that not only have an "active" flag but that also do not have
an "indexed" flag, that means I need to add them to the index. Then I was
going to add that item to the index. Since I am using taking this path, it
doesn't seem like I need the JDBC driver, as I can add items to
elasticsearch using the PHP library.

So, my question is, can I get away without using the JDBC driver?

James

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6c244e00-1f89-447d-8eb5-114f0b5efcbd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

You can use either style, it is a matter of taste, or convenience.

With the JDBC plugin, you can also push data instead of pull.

Jörg

On Fri, Sep 12, 2014 at 12:11 PM, James mail@employ.com wrote:

I want to close this issue but I still do not understand if I should be
pushing documents from my database using the PHP client or using the JDBC
river to pull them into elasticsearch from the SQL database.

They can both achieve the same thing, but what is the usecase which
defines when is the right time to use each implementation.

On Wednesday, September 10, 2014 10:59:18 AM UTC+1, James wrote:

Hi,

I'm setting up a system where I have a main SQL database which is synced
with elasticsearch. My plan is to use the main PHP library for
elasticsearch.

I was going to have a cron run every thirty minuets to check for items in
my database that not only have an "active" flag but that also do not have
an "indexed" flag, that means I need to add them to the index. Then I was
going to add that item to the index. Since I am using taking this path, it
doesn't seem like I need the JDBC driver, as I can add items to
elasticsearch using the PHP library.

So, my question is, can I get away without using the JDBC driver?

James

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/6c244e00-1f89-447d-8eb5-114f0b5efcbd%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/6c244e00-1f89-447d-8eb5-114f0b5efcbd%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHT2DcMHJwMjxBZ0RsV4_eKJyB2KjBCiqB2ZTac8fzkTg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

I would strongly prefer to maintain control of the indexing side and not in
Elasticsearch. In fact, the Elasticsearch team has talked about deprecating
river plugins. I do not have any numbers, but I would suspect that the
majority of users do not use a river plugin. And yes, the correct term is
the JDBC plugin, not driver. The wrong term confused many. :slight_smile:

--
Ivan

On Fri, Sep 12, 2014 at 3:24 AM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

You can use either style, it is a matter of taste, or convenience.

With the JDBC plugin, you can also push data instead of pull.

Jörg

On Fri, Sep 12, 2014 at 12:11 PM, James mail@employ.com wrote:

I want to close this issue but I still do not understand if I should be
pushing documents from my database using the PHP client or using the JDBC
river to pull them into elasticsearch from the SQL database.

They can both achieve the same thing, but what is the usecase which
defines when is the right time to use each implementation.

On Wednesday, September 10, 2014 10:59:18 AM UTC+1, James wrote:

Hi,

I'm setting up a system where I have a main SQL database which is synced
with elasticsearch. My plan is to use the main PHP library for
elasticsearch.

I was going to have a cron run every thirty minuets to check for items
in my database that not only have an "active" flag but that also do not
have an "indexed" flag, that means I need to add them to the index. Then I
was going to add that item to the index. Since I am using taking this path,
it doesn't seem like I need the JDBC driver, as I can add items to
elasticsearch using the PHP library.

So, my question is, can I get away without using the JDBC driver?

James

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/6c244e00-1f89-447d-8eb5-114f0b5efcbd%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/6c244e00-1f89-447d-8eb5-114f0b5efcbd%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHT2DcMHJwMjxBZ0RsV4_eKJyB2KjBCiqB2ZTac8fzkTg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHT2DcMHJwMjxBZ0RsV4_eKJyB2KjBCiqB2ZTac8fzkTg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBvDga8q--Au8yWaX2RMGgcDTpYMhLu243tB9w7z0W0_A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Ah that's really interesting, it's good to get some comparison. In that case you are saying to use the official php library for the document indexing?

James

On 12 Sep 2014, at 18:26, Ivan Brusic ivan@brusic.com wrote:

I would strongly prefer to maintain control of the indexing side and not in Elasticsearch. In fact, the Elasticsearch team has talked about deprecating river plugins. I do not have any numbers, but I would suspect that the majority of users do not use a river plugin. And yes, the correct term is the JDBC plugin, not driver. The wrong term confused many. :slight_smile:

--
Ivan

On Fri, Sep 12, 2014 at 3:24 AM, joergprante@gmail.com joergprante@gmail.com wrote:
You can use either style, it is a matter of taste, or convenience.

With the JDBC plugin, you can also push data instead of pull.

Jörg

On Fri, Sep 12, 2014 at 12:11 PM, James mail@employ.com wrote:
I want to close this issue but I still do not understand if I should be pushing documents from my database using the PHP client or using the JDBC river to pull them into elasticsearch from the SQL database.

They can both achieve the same thing, but what is the usecase which defines when is the right time to use each implementation.

On Wednesday, September 10, 2014 10:59:18 AM UTC+1, James wrote:
Hi,

I'm setting up a system where I have a main SQL database which is synced with elasticsearch. My plan is to use the main PHP library for elasticsearch.

I was going to have a cron run every thirty minuets to check for items in my database that not only have an "active" flag but that also do not have an "indexed" flag, that means I need to add them to the index. Then I was going to add that item to the index. Since I am using taking this path, it doesn't seem like I need the JDBC driver, as I can add items to elasticsearch using the PHP library.

So, my question is, can I get away without using the JDBC driver?

James

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6c244e00-1f89-447d-8eb5-114f0b5efcbd%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHT2DcMHJwMjxBZ0RsV4_eKJyB2KjBCiqB2ZTac8fzkTg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/0dzSMbARlks/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBvDga8q--Au8yWaX2RMGgcDTpYMhLu243tB9w7z0W0_A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8AB631FC-020F-4EF4-B639-F2D84D5AFC0C%40employ.com.
For more options, visit https://groups.google.com/d/optout.

I must admit I'm new to this so I find some of the information hard to understand. So sorry if I am asking stupid questions.

On 12 Sep 2014, at 18:26, Ivan Brusic ivan@brusic.com wrote:

I would strongly prefer to maintain control of the indexing side and not in Elasticsearch. In fact, the Elasticsearch team has talked about deprecating river plugins. I do not have any numbers, but I would suspect that the majority of users do not use a river plugin. And yes, the correct term is the JDBC plugin, not driver. The wrong term confused many. :slight_smile:

--
Ivan

On Fri, Sep 12, 2014 at 3:24 AM, joergprante@gmail.com joergprante@gmail.com wrote:
You can use either style, it is a matter of taste, or convenience.

With the JDBC plugin, you can also push data instead of pull.

Jörg

On Fri, Sep 12, 2014 at 12:11 PM, James mail@employ.com wrote:
I want to close this issue but I still do not understand if I should be pushing documents from my database using the PHP client or using the JDBC river to pull them into elasticsearch from the SQL database.

They can both achieve the same thing, but what is the usecase which defines when is the right time to use each implementation.

On Wednesday, September 10, 2014 10:59:18 AM UTC+1, James wrote:
Hi,

I'm setting up a system where I have a main SQL database which is synced with elasticsearch. My plan is to use the main PHP library for elasticsearch.

I was going to have a cron run every thirty minuets to check for items in my database that not only have an "active" flag but that also do not have an "indexed" flag, that means I need to add them to the index. Then I was going to add that item to the index. Since I am using taking this path, it doesn't seem like I need the JDBC driver, as I can add items to elasticsearch using the PHP library.

So, my question is, can I get away without using the JDBC driver?

James

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6c244e00-1f89-447d-8eb5-114f0b5efcbd%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHT2DcMHJwMjxBZ0RsV4_eKJyB2KjBCiqB2ZTac8fzkTg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/0dzSMbARlks/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBvDga8q--Au8yWaX2RMGgcDTpYMhLu243tB9w7z0W0_A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9EBFE31B-8DEA-4E05-8342-9E0013BC450B%40employ.com.
For more options, visit https://groups.google.com/d/optout.