A plugin to change the result set before sending it back to the http client


(Mario Mueller) #1

Hey folks,

I kindly ask for a hint to achieve the following thing:

The goal is to deliver only a json array of source objects to the client.
The php app that sits on the other side uses JMS\Serializer to deserialize
the response into entities. At the moment the app needs to take an overhead
to derserialize it, extract the source and serialize it again. Then the
serialized stuff is passed to the entity deserializer. That's really
painful.

I've found a thread that suggests a proxy in between to handle this type of
problem, but this is not possible in our env.

The real question is: Is this achievable by writing a plugin? And if so,
what type of plugin? I've looked at the RestFilter, but I don't know if
this is the right way to go...

Any hints are welcome!

Regards,
Mario

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6844c290-e7f8-46d7-b5b3-27cab18fb706%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Ivan Brusic) #2

If you are only modifying the REST API calls and not the Java API, such a
plugin should be easy. You are not creating a new type of action, merely
using the current search one, but changing the output format.

Here are two tutorials on simple REST plugins:
http://jprante.github.io/lessons/2012/03/27/Writing-a-simple-plugin-for-Elasticsearch.html

Hopefully the content is not too old. Base the plugin around the
existing RestSearchAction, but in the handleRequest method, instead of
returning the results directly, you can modify them before.

Cheers

Ivan

On Thu, Jun 5, 2014 at 10:45 AM, Mario Mueller mario@xenji.com wrote:

Hey folks,

I kindly ask for a hint to achieve the following thing:

The goal is to deliver only a json array of source objects to the client.
The php app that sits on the other side uses JMS\Serializer to deserialize
the response into entities. At the moment the app needs to take an overhead
to derserialize it, extract the source and serialize it again. Then the
serialized stuff is passed to the entity deserializer. That's really
painful.

I've found a thread that suggests a proxy in between to handle this type
of problem, but this is not possible in our env.

The real question is: Is this achievable by writing a plugin? And if so,
what type of plugin? I've looked at the RestFilter, but I don't know if
this is the right way to go...

Any hints are welcome!

Regards,
Mario

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/6844c290-e7f8-46d7-b5b3-27cab18fb706%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/6844c290-e7f8-46d7-b5b3-27cab18fb706%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDcoz8ap%3DYK-DHogkksbjecLnQGcaejOccgneWdutf4PQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Mario Mueller) #3

So, if I understood your approach in the right way ... I should build a new
Rest Action like "_search_and_return_source" that proxies the original
_search one?

I've already read those two articles and I've set up my development
environment with the help of those :wink:

Am Donnerstag, 5. Juni 2014 19:55:06 UTC+2 schrieb Ivan Brusic:

If you are only modifying the REST API calls and not the Java API, such a
plugin should be easy. You are not creating a new type of action, merely
using the current search one, but changing the output format.

Here are two tutorials on simple REST plugins:

http://jprante.github.io/lessons/2012/03/27/Writing-a-simple-plugin-for-Elasticsearch.html
http://blog.brusic.com/2011/09/create-pluggable-rest-endpoints-in.html

Hopefully the content is not too old. Base the plugin around the
existing RestSearchAction, but in the handleRequest method, instead of
returning the results directly, you can modify them before.

Cheers

Ivan

On Thu, Jun 5, 2014 at 10:45 AM, Mario Mueller <ma...@xenji.com
<javascript:>> wrote:

Hey folks,

I kindly ask for a hint to achieve the following thing:

The goal is to deliver only a json array of source objects to the client.
The php app that sits on the other side uses JMS\Serializer to deserialize
the response into entities. At the moment the app needs to take an overhead
to derserialize it, extract the source and serialize it again. Then the
serialized stuff is passed to the entity deserializer. That's really
painful.

I've found a thread that suggests a proxy in between to handle this type
of problem, but this is not possible in our env.

The real question is: Is this achievable by writing a plugin? And if so,
what type of plugin? I've looked at the RestFilter, but I don't know if
this is the right way to go...

Any hints are welcome!

Regards,
Mario

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/6844c290-e7f8-46d7-b5b3-27cab18fb706%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/6844c290-e7f8-46d7-b5b3-27cab18fb706%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8158ba2e-55e1-4c1b-b918-ee33000800a0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #4

Just a quick question, do you just want to extract a field from the json
source?

There are field filters and parameters for shaping such a JSON result,
maybe they can already help?

Or can you give an example of the problem?

Jörg

On Thu, Jun 5, 2014 at 7:45 PM, Mario Mueller mario@xenji.com wrote:

Hey folks,

I kindly ask for a hint to achieve the following thing:

The goal is to deliver only a json array of source objects to the client.
The php app that sits on the other side uses JMS\Serializer to deserialize
the response into entities. At the moment the app needs to take an overhead
to derserialize it, extract the source and serialize it again. Then the
serialized stuff is passed to the entity deserializer. That's really
painful.

I've found a thread that suggests a proxy in between to handle this type
of problem, but this is not possible in our env.

The real question is: Is this achievable by writing a plugin? And if so,
what type of plugin? I've looked at the RestFilter, but I don't know if
this is the right way to go...

Any hints are welcome!

Regards,
Mario

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/6844c290-e7f8-46d7-b5b3-27cab18fb706%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/6844c290-e7f8-46d7-b5b3-27cab18fb706%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHxptD%2BrE2YRSMerwtuxgGGReCNgP9wtJ2ppFWojbYMKw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Mario Mueller) #5

Hey Joerg,

I just need the whole content of the _source field like so:

[
{
"HotelName": "Plaka",
"ProductCode": "7050",
"objectId": "437-de",
"GroupId": 25223,
"readonly": false,
"lang": "de",
"City": "Athens"
},
{
"HotelName": "Hyatt at Fisherman's Wharf",
"ProductCode": "52397",
"objectId": "14479-de",
"GroupId": 25223,
"readonly": false,
"lang": "de",
"City": "San Francisco"
}
]

Those fields are not stored, but indexed. As I mentioned before, this
source matches exactly to an entity structure in our application and we
need the structure of the source document 1:1. If this is achievable by any
other methods than a plugin, then I am totally fine with it.

Am Donnerstag, 5. Juni 2014 21:36:31 UTC+2 schrieb Jörg Prante:

Just a quick question, do you just want to extract a field from the json
source?

There are field filters and parameters for shaping such a JSON result,
maybe they can already help?

Or can you give an example of the problem?

Jörg

On Thu, Jun 5, 2014 at 7:45 PM, Mario Mueller <ma...@xenji.com
<javascript:>> wrote:

Hey folks,

I kindly ask for a hint to achieve the following thing:

The goal is to deliver only a json array of source objects to the client.
The php app that sits on the other side uses JMS\Serializer to deserialize
the response into entities. At the moment the app needs to take an overhead
to derserialize it, extract the source and serialize it again. Then the
serialized stuff is passed to the entity deserializer. That's really
painful.

I've found a thread that suggests a proxy in between to handle this type
of problem, but this is not possible in our env.

The real question is: Is this achievable by writing a plugin? And if so,
what type of plugin? I've looked at the RestFilter, but I don't know if
this is the right way to go...

Any hints are welcome!

Regards,
Mario

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/6844c290-e7f8-46d7-b5b3-27cab18fb706%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/6844c290-e7f8-46d7-b5b3-27cab18fb706%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b9a6e43f-35ec-4275-9a29-417552e1d94f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Ivan Brusic) #6

There is no way to eliminate returning the search metadata. It has been
requested often.

--
Ivan

On Thu, Jun 5, 2014 at 12:40 PM, Mario Mueller mario@xenji.com wrote:

Hey Joerg,

I just need the whole content of the _source field like so:

[
{
"HotelName": "Plaka",
"ProductCode": "7050",
"objectId": "437-de",
"GroupId": 25223,
"readonly": false,
"lang": "de",
"City": "Athens"
},
{
"HotelName": "Hyatt at Fisherman's Wharf",
"ProductCode": "52397",
"objectId": "14479-de",
"GroupId": 25223,
"readonly": false,
"lang": "de",
"City": "San Francisco"
}
]

Those fields are not stored, but indexed. As I mentioned before, this
source matches exactly to an entity structure in our application and we
need the structure of the source document 1:1. If this is achievable by any
other methods than a plugin, then I am totally fine with it.

Am Donnerstag, 5. Juni 2014 21:36:31 UTC+2 schrieb Jörg Prante:

Just a quick question, do you just want to extract a field from the json
source?

There are field filters and parameters for shaping such a JSON result,
maybe they can already help?

Or can you give an example of the problem?

Jörg

On Thu, Jun 5, 2014 at 7:45 PM, Mario Mueller ma...@xenji.com wrote:

Hey folks,

I kindly ask for a hint to achieve the following thing:

The goal is to deliver only a json array of source objects to the
client. The php app that sits on the other side uses JMS\Serializer to
deserialize the response into entities. At the moment the app needs to take
an overhead to derserialize it, extract the source and serialize it again.
Then the serialized stuff is passed to the entity deserializer. That's
really painful.

I've found a thread that suggests a proxy in between to handle this type
of problem, but this is not possible in our env.

The real question is: Is this achievable by writing a plugin? And if so,
what type of plugin? I've looked at the RestFilter, but I don't know if
this is the right way to go...

Any hints are welcome!

Regards,
Mario

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/6844c290-e7f8-46d7-b5b3-27cab18fb706%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/6844c290-e7f8-46d7-b5b3-27cab18fb706%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b9a6e43f-35ec-4275-9a29-417552e1d94f%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b9a6e43f-35ec-4275-9a29-417552e1d94f%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBa%2BzqQ5b3baLpnXxnwR%3DUm6ANn92_XOSTvDgqBPiJ5ng%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #7

Ah, now I get it. Source without metadata, listed in the order of hits.

This should be easy to do in a plugin. REST filter is the way to go for
PHP. Just a minute...

Jörg

On Thu, Jun 5, 2014 at 9:49 PM, Ivan Brusic ivan@brusic.com wrote:

There is no way to eliminate returning the search metadata. It has been
requested often.

--
Ivan

On Thu, Jun 5, 2014 at 12:40 PM, Mario Mueller mario@xenji.com wrote:

Hey Joerg,

I just need the whole content of the _source field like so:

[
{
"HotelName": "Plaka",
"ProductCode": "7050",
"objectId": "437-de",
"GroupId": 25223,
"readonly": false,
"lang": "de",
"City": "Athens"
},
{
"HotelName": "Hyatt at Fisherman's Wharf",
"ProductCode": "52397",
"objectId": "14479-de",
"GroupId": 25223,
"readonly": false,
"lang": "de",
"City": "San Francisco"
}
]

Those fields are not stored, but indexed. As I mentioned before, this
source matches exactly to an entity structure in our application and we
need the structure of the source document 1:1. If this is achievable by any
other methods than a plugin, then I am totally fine with it.

Am Donnerstag, 5. Juni 2014 21:36:31 UTC+2 schrieb Jörg Prante:

Just a quick question, do you just want to extract a field from the json
source?

There are field filters and parameters for shaping such a JSON result,
maybe they can already help?

Or can you give an example of the problem?

Jörg

On Thu, Jun 5, 2014 at 7:45 PM, Mario Mueller ma...@xenji.com wrote:

Hey folks,

I kindly ask for a hint to achieve the following thing:

The goal is to deliver only a json array of source objects to the
client. The php app that sits on the other side uses JMS\Serializer to
deserialize the response into entities. At the moment the app needs to take
an overhead to derserialize it, extract the source and serialize it again.
Then the serialized stuff is passed to the entity deserializer. That's
really painful.

I've found a thread that suggests a proxy in between to handle this
type of problem, but this is not possible in our env.

The real question is: Is this achievable by writing a plugin? And if
so, what type of plugin? I've looked at the RestFilter, but I don't know if
this is the right way to go...

Any hints are welcome!

Regards,
Mario

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/6844c290-e7f8-46d7-b5b3-27cab18fb706%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/6844c290-e7f8-46d7-b5b3-27cab18fb706%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b9a6e43f-35ec-4275-9a29-417552e1d94f%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b9a6e43f-35ec-4275-9a29-417552e1d94f%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBa%2BzqQ5b3baLpnXxnwR%3DUm6ANn92_XOSTvDgqBPiJ5ng%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBa%2BzqQ5b3baLpnXxnwR%3DUm6ANn92_XOSTvDgqBPiJ5ng%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFmeHnhhcgu-5nYDLKqsvVBujbhLxmoNfF3PoZGAzORPA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Ivan Brusic) #8

I just looked it up and it should be as easy as creating your
own RestResponseListener that takes a SearchResponse and creates a
simplified version with no metadata.

Should be an interesting quick plugin, but it looks like Jorg is going to
beat me to it (I'm still at work for several more hours).

--
Ivan

On Thu, Jun 5, 2014 at 1:08 PM, joergprante@gmail.com <joergprante@gmail.com

wrote:

Ah, now I get it. Source without metadata, listed in the order of hits.

This should be easy to do in a plugin. REST filter is the way to go for
PHP. Just a minute...

Jörg

On Thu, Jun 5, 2014 at 9:49 PM, Ivan Brusic ivan@brusic.com wrote:

There is no way to eliminate returning the search metadata. It has been
requested often.

--
Ivan

On Thu, Jun 5, 2014 at 12:40 PM, Mario Mueller mario@xenji.com wrote:

Hey Joerg,

I just need the whole content of the _source field like so:

[
{
"HotelName": "Plaka",
"ProductCode": "7050",
"objectId": "437-de",
"GroupId": 25223,
"readonly": false,
"lang": "de",
"City": "Athens"
},
{
"HotelName": "Hyatt at Fisherman's Wharf",
"ProductCode": "52397",
"objectId": "14479-de",
"GroupId": 25223,
"readonly": false,
"lang": "de",
"City": "San Francisco"
}
]

Those fields are not stored, but indexed. As I mentioned before, this
source matches exactly to an entity structure in our application and we
need the structure of the source document 1:1. If this is achievable by any
other methods than a plugin, then I am totally fine with it.

Am Donnerstag, 5. Juni 2014 21:36:31 UTC+2 schrieb Jörg Prante:

Just a quick question, do you just want to extract a field from the
json source?

There are field filters and parameters for shaping such a JSON result,
maybe they can already help?

Or can you give an example of the problem?

Jörg

On Thu, Jun 5, 2014 at 7:45 PM, Mario Mueller ma...@xenji.com wrote:

Hey folks,

I kindly ask for a hint to achieve the following thing:

The goal is to deliver only a json array of source objects to the
client. The php app that sits on the other side uses JMS\Serializer to
deserialize the response into entities. At the moment the app needs to take
an overhead to derserialize it, extract the source and serialize it again.
Then the serialized stuff is passed to the entity deserializer. That's
really painful.

I've found a thread that suggests a proxy in between to handle this
type of problem, but this is not possible in our env.

The real question is: Is this achievable by writing a plugin? And if
so, what type of plugin? I've looked at the RestFilter, but I don't know if
this is the right way to go...

Any hints are welcome!

Regards,
Mario

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/6844c290-e7f8-46d7-b5b3-27cab18fb706%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/6844c290-e7f8-46d7-b5b3-27cab18fb706%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b9a6e43f-35ec-4275-9a29-417552e1d94f%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b9a6e43f-35ec-4275-9a29-417552e1d94f%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBa%2BzqQ5b3baLpnXxnwR%3DUm6ANn92_XOSTvDgqBPiJ5ng%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBa%2BzqQ5b3baLpnXxnwR%3DUm6ANn92_XOSTvDgqBPiJ5ng%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFmeHnhhcgu-5nYDLKqsvVBujbhLxmoNfF3PoZGAzORPA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFmeHnhhcgu-5nYDLKqsvVBujbhLxmoNfF3PoZGAzORPA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBZFwGD%3DwbHxx9aEUqcWTu-LoUu05ecj1jB3t0gZVrX4w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #9

OK, I think I made it. Good exercise to wrestle with Github before going to
sleep...

Best,

Jörg

On Thu, Jun 5, 2014 at 10:28 PM, Ivan Brusic ivan@brusic.com wrote:

I just looked it up and it should be as easy as creating your
own RestResponseListener that takes a SearchResponse and creates a
simplified version with no metadata.

Should be an interesting quick plugin, but it looks like Jorg is going to
beat me to it (I'm still at work for several more hours).

--
Ivan

On Thu, Jun 5, 2014 at 1:08 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Ah, now I get it. Source without metadata, listed in the order of hits.

This should be easy to do in a plugin. REST filter is the way to go for
PHP. Just a minute...

Jörg

On Thu, Jun 5, 2014 at 9:49 PM, Ivan Brusic ivan@brusic.com wrote:

There is no way to eliminate returning the search metadata. It has been
requested often.

--
Ivan

On Thu, Jun 5, 2014 at 12:40 PM, Mario Mueller mario@xenji.com wrote:

Hey Joerg,

I just need the whole content of the _source field like so:

[
{
"HotelName": "Plaka",
"ProductCode": "7050",
"objectId": "437-de",
"GroupId": 25223,
"readonly": false,
"lang": "de",
"City": "Athens"
},
{
"HotelName": "Hyatt at Fisherman's Wharf",
"ProductCode": "52397",
"objectId": "14479-de",
"GroupId": 25223,
"readonly": false,
"lang": "de",
"City": "San Francisco"
}
]

Those fields are not stored, but indexed. As I mentioned before, this
source matches exactly to an entity structure in our application and we
need the structure of the source document 1:1. If this is achievable by any
other methods than a plugin, then I am totally fine with it.

Am Donnerstag, 5. Juni 2014 21:36:31 UTC+2 schrieb Jörg Prante:

Just a quick question, do you just want to extract a field from the
json source?

There are field filters and parameters for shaping such a JSON result,
maybe they can already help?

Or can you give an example of the problem?

Jörg

On Thu, Jun 5, 2014 at 7:45 PM, Mario Mueller ma...@xenji.com wrote:

Hey folks,

I kindly ask for a hint to achieve the following thing:

The goal is to deliver only a json array of source objects to the
client. The php app that sits on the other side uses JMS\Serializer to
deserialize the response into entities. At the moment the app needs to take
an overhead to derserialize it, extract the source and serialize it again.
Then the serialized stuff is passed to the entity deserializer. That's
really painful.

I've found a thread that suggests a proxy in between to handle this
type of problem, but this is not possible in our env.

The real question is: Is this achievable by writing a plugin? And if
so, what type of plugin? I've looked at the RestFilter, but I don't know if
this is the right way to go...

Any hints are welcome!

Regards,
Mario

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/6844c290-e7f8-46d7-b5b3-27cab18fb706%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/6844c290-e7f8-46d7-b5b3-27cab18fb706%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b9a6e43f-35ec-4275-9a29-417552e1d94f%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b9a6e43f-35ec-4275-9a29-417552e1d94f%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBa%2BzqQ5b3baLpnXxnwR%3DUm6ANn92_XOSTvDgqBPiJ5ng%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBa%2BzqQ5b3baLpnXxnwR%3DUm6ANn92_XOSTvDgqBPiJ5ng%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFmeHnhhcgu-5nYDLKqsvVBujbhLxmoNfF3PoZGAzORPA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFmeHnhhcgu-5nYDLKqsvVBujbhLxmoNfF3PoZGAzORPA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBZFwGD%3DwbHxx9aEUqcWTu-LoUu05ecj1jB3t0gZVrX4w%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBZFwGD%3DwbHxx9aEUqcWTu-LoUu05ecj1jB3t0gZVrX4w%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGUOa4uFzfQYB%2BMpDdeEP3FdYc0fRA4OW7mkXLdzvkOJg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Ivan Brusic) #10

I see that we agree that a new RestResponseListener is the way to go.

I have not cloned your project yet, only looked at the code on github, but
I noticed that you provided your own parseSearchRequest, but still
call RestSearchAction.parseSearchRequest from inside handleRequest. Did I
misinterpret the code or is that a mistake?

--
Ivan

On Thu, Jun 5, 2014 at 2:37 PM, joergprante@gmail.com <joergprante@gmail.com

wrote:

OK, I think I made it. Good exercise to wrestle with Github before going
to sleep...

https://github.com/jprante/elasticsearch-arrayformat

Best,

Jörg

On Thu, Jun 5, 2014 at 10:28 PM, Ivan Brusic ivan@brusic.com wrote:

I just looked it up and it should be as easy as creating your
own RestResponseListener that takes a SearchResponse and creates a
simplified version with no metadata.

Should be an interesting quick plugin, but it looks like Jorg is going to
beat me to it (I'm still at work for several more hours).

--
Ivan

On Thu, Jun 5, 2014 at 1:08 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Ah, now I get it. Source without metadata, listed in the order of hits.

This should be easy to do in a plugin. REST filter is the way to go for
PHP. Just a minute...

Jörg

On Thu, Jun 5, 2014 at 9:49 PM, Ivan Brusic ivan@brusic.com wrote:

There is no way to eliminate returning the search metadata. It has been
requested often.

--
Ivan

On Thu, Jun 5, 2014 at 12:40 PM, Mario Mueller mario@xenji.com wrote:

Hey Joerg,

I just need the whole content of the _source field like so:

[
{
"HotelName": "Plaka",
"ProductCode": "7050",
"objectId": "437-de",
"GroupId": 25223,
"readonly": false,
"lang": "de",
"City": "Athens"
},
{
"HotelName": "Hyatt at Fisherman's Wharf",
"ProductCode": "52397",
"objectId": "14479-de",
"GroupId": 25223,
"readonly": false,
"lang": "de",
"City": "San Francisco"
}
]

Those fields are not stored, but indexed. As I mentioned before, this
source matches exactly to an entity structure in our application and we
need the structure of the source document 1:1. If this is achievable by any
other methods than a plugin, then I am totally fine with it.

Am Donnerstag, 5. Juni 2014 21:36:31 UTC+2 schrieb Jörg Prante:

Just a quick question, do you just want to extract a field from the
json source?

There are field filters and parameters for shaping such a JSON
result, maybe they can already help?

Or can you give an example of the problem?

Jörg

On Thu, Jun 5, 2014 at 7:45 PM, Mario Mueller ma...@xenji.com
wrote:

Hey folks,

I kindly ask for a hint to achieve the following thing:

The goal is to deliver only a json array of source objects to the
client. The php app that sits on the other side uses JMS\Serializer to
deserialize the response into entities. At the moment the app needs to take
an overhead to derserialize it, extract the source and serialize it again.
Then the serialized stuff is passed to the entity deserializer. That's
really painful.

I've found a thread that suggests a proxy in between to handle this
type of problem, but this is not possible in our env.

The real question is: Is this achievable by writing a plugin? And if
so, what type of plugin? I've looked at the RestFilter, but I don't know if
this is the right way to go...

Any hints are welcome!

Regards,
Mario

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/6844c290-
e7f8-46d7-b5b3-27cab18fb706%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/6844c290-e7f8-46d7-b5b3-27cab18fb706%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b9a6e43f-35ec-4275-9a29-417552e1d94f%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b9a6e43f-35ec-4275-9a29-417552e1d94f%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBa%2BzqQ5b3baLpnXxnwR%3DUm6ANn92_XOSTvDgqBPiJ5ng%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBa%2BzqQ5b3baLpnXxnwR%3DUm6ANn92_XOSTvDgqBPiJ5ng%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFmeHnhhcgu-5nYDLKqsvVBujbhLxmoNfF3PoZGAzORPA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFmeHnhhcgu-5nYDLKqsvVBujbhLxmoNfF3PoZGAzORPA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBZFwGD%3DwbHxx9aEUqcWTu-LoUu05ecj1jB3t0gZVrX4w%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBZFwGD%3DwbHxx9aEUqcWTu-LoUu05ecj1jB3t0gZVrX4w%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGUOa4uFzfQYB%2BMpDdeEP3FdYc0fRA4OW7mkXLdzvkOJg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGUOa4uFzfQYB%2BMpDdeEP3FdYc0fRA4OW7mkXLdzvkOJg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQC3osMZXmiOODFXk9cZP2XvE01F0WnBag%2B%3DHQNxzN6W%2Bw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #11

Ups, yes, a mistake... I bluntly copy/pasted the RestSearchAction. Thanks!

Jörg

On Fri, Jun 6, 2014 at 12:03 AM, Ivan Brusic ivan@brusic.com wrote:

I see that we agree that a new RestResponseListener is the way to go.

I have not cloned your project yet, only looked at the code on github, but
I noticed that you provided your own parseSearchRequest, but still
call RestSearchAction.parseSearchRequest from inside handleRequest. Did I
misinterpret the code or is that a mistake?

--
Ivan

On Thu, Jun 5, 2014 at 2:37 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

OK, I think I made it. Good exercise to wrestle with Github before going
to sleep...

https://github.com/jprante/elasticsearch-arrayformat

Best,

Jörg

On Thu, Jun 5, 2014 at 10:28 PM, Ivan Brusic ivan@brusic.com wrote:

I just looked it up and it should be as easy as creating your
own RestResponseListener that takes a SearchResponse and creates a
simplified version with no metadata.

Should be an interesting quick plugin, but it looks like Jorg is going
to beat me to it (I'm still at work for several more hours).

--
Ivan

On Thu, Jun 5, 2014 at 1:08 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Ah, now I get it. Source without metadata, listed in the order of hits.

This should be easy to do in a plugin. REST filter is the way to go for
PHP. Just a minute...

Jörg

On Thu, Jun 5, 2014 at 9:49 PM, Ivan Brusic ivan@brusic.com wrote:

There is no way to eliminate returning the search metadata. It has
been requested often.

--
Ivan

On Thu, Jun 5, 2014 at 12:40 PM, Mario Mueller mario@xenji.com
wrote:

Hey Joerg,

I just need the whole content of the _source field like so:

[
{
"HotelName": "Plaka",
"ProductCode": "7050",
"objectId": "437-de",
"GroupId": 25223,
"readonly": false,
"lang": "de",
"City": "Athens"
},
{
"HotelName": "Hyatt at Fisherman's Wharf",
"ProductCode": "52397",
"objectId": "14479-de",
"GroupId": 25223,
"readonly": false,
"lang": "de",
"City": "San Francisco"
}
]

Those fields are not stored, but indexed. As I mentioned before, this
source matches exactly to an entity structure in our application and we
need the structure of the source document 1:1. If this is achievable by any
other methods than a plugin, then I am totally fine with it.

Am Donnerstag, 5. Juni 2014 21:36:31 UTC+2 schrieb Jörg Prante:

Just a quick question, do you just want to extract a field from the
json source?

There are field filters and parameters for shaping such a JSON
result, maybe they can already help?

Or can you give an example of the problem?

Jörg

On Thu, Jun 5, 2014 at 7:45 PM, Mario Mueller ma...@xenji.com
wrote:

Hey folks,

I kindly ask for a hint to achieve the following thing:

The goal is to deliver only a json array of source objects to the
client. The php app that sits on the other side uses JMS\Serializer to
deserialize the response into entities. At the moment the app needs to take
an overhead to derserialize it, extract the source and serialize it again.
Then the serialized stuff is passed to the entity deserializer. That's
really painful.

I've found a thread that suggests a proxy in between to handle this
type of problem, but this is not possible in our env.

The real question is: Is this achievable by writing a plugin? And
if so, what type of plugin? I've looked at the RestFilter, but I don't know
if this is the right way to go...

Any hints are welcome!

Regards,
Mario

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/6844c290-
e7f8-46d7-b5b3-27cab18fb706%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/6844c290-e7f8-46d7-b5b3-27cab18fb706%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b9a6e43f-35ec-4275-9a29-417552e1d94f%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b9a6e43f-35ec-4275-9a29-417552e1d94f%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBa%2BzqQ5b3baLpnXxnwR%3DUm6ANn92_XOSTvDgqBPiJ5ng%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBa%2BzqQ5b3baLpnXxnwR%3DUm6ANn92_XOSTvDgqBPiJ5ng%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFmeHnhhcgu-5nYDLKqsvVBujbhLxmoNfF3PoZGAzORPA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFmeHnhhcgu-5nYDLKqsvVBujbhLxmoNfF3PoZGAzORPA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBZFwGD%3DwbHxx9aEUqcWTu-LoUu05ecj1jB3t0gZVrX4w%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBZFwGD%3DwbHxx9aEUqcWTu-LoUu05ecj1jB3t0gZVrX4w%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGUOa4uFzfQYB%2BMpDdeEP3FdYc0fRA4OW7mkXLdzvkOJg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGUOa4uFzfQYB%2BMpDdeEP3FdYc0fRA4OW7mkXLdzvkOJg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQC3osMZXmiOODFXk9cZP2XvE01F0WnBag%2B%3DHQNxzN6W%2Bw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQC3osMZXmiOODFXk9cZP2XvE01F0WnBag%2B%3DHQNxzN6W%2Bw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoF5qAr%3D8FTwYR687XUHRS2B2a%3DmzXhT12bypD-KB_7OZA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Brian Yoder) #12

This may or may not help, but the following worked well for me. Just as any
database-backed application, the business logic (such as what you
described) is best implemented outside of the database. Since ES is a
first-class Java citizen and its Java API is clean and superb
(documentation shortcomings are more than made up for by the excellent
advice from this newsgroup), this is what I did:

  1. Create my business logic, and use the TransportClient to contact
    Elasticsearch. It's easy to test it in a stand-alone driver.

  2. Create a server using Netty and the LMAX Disruptor. Netty is awesome,
    and I could craft the interface to accept the questions that made sense to
    the business, not to Elasticsearch. Then I instantiated an LMAX Disruptor
    in multi-producer mode, and each Netty thread simply created an event that
    contained 2 objects: The decoded HTTP request parameter, and the reference
    to the response output channel.

  3. The Disruptor contained a fixed number of Worker handler threads (to
    match those in Elasticsearch) that actually converted the request into one
    or more Elasticsearch requests, performing whatever functions the business
    logic required, and then writing the responses back to the response channel.

Even if my server got a huge burst of requests, they were quickly queued
which means that the Netty handler thread count stays low. And then the
back-end Disruptor thread max will ensure that Elasticsearch is never
overloaded.

This is kind of the hard way using Java, but it suited me fine. Other
solutions could be similarly implemented with comperable performance but
perhaps a lot less development time in Python with Django, or Node.js, or
Ruby on Rails. But Java worked well for me; Netty is fast and easy to work
with, and the LMAX Disruptor offers a near-zero latency nearly lockless
queuing, and together they keep the thread count low while still handling
many many more requests at one time.

This sounds like a lot of work, but the hard work is a one-off kind of
thing, and then the business logic can be easily tuned, fixed, and extended
without much effort. Even things that Elasticsearch does not do and will
never do (such as foreign keys and enforced consistency across types or
even indices) are relatively easily done within the custom business logic
of a front-end server.

Again, I don't offer this as a strong suggestion but rather as a case study
of what worked well for me.

Brian

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a5011573-970d-4d4c-839d-a274e4d4cc34%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Mario Mueller) #13

You guys are totally awesome! Thanks a lot! If you ever visit Duesseldorf
drop me a line, I owe you a beer.

@Brian:
Interesting approach, but wouldn't this go against the initial "no
additional proxy" statement, if I got you right ..

Am Donnerstag, 5. Juni 2014 19:45:33 UTC+2 schrieb Mario Mueller:

Hey folks,

I kindly ask for a hint to achieve the following thing:

The goal is to deliver only a json array of source objects to the client.
The php app that sits on the other side uses JMS\Serializer to deserialize
the response into entities. At the moment the app needs to take an overhead
to derserialize it, extract the source and serialize it again. Then the
serialized stuff is passed to the entity deserializer. That's really
painful.

I've found a thread that suggests a proxy in between to handle this type
of problem, but this is not possible in our env.

The real question is: Is this achievable by writing a plugin? And if so,
what type of plugin? I've looked at the RestFilter, but I don't know if
this is the right way to go...

Any hints are welcome!

Regards,
Mario

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b27db27a-b136-4a3b-81b5-16d60047540d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #14

I drink Kölsch only :slight_smile: ävver et hätt noh immer joot jejange

Greetings from Cologne!

Jörg

On Fri, Jun 6, 2014 at 7:14 AM, Mario Mueller mario@xenji.com wrote:

You guys are totally awesome! Thanks a lot! If you ever visit Duesseldorf
drop me a line, I owe you a beer.

@Brian:
Interesting approach, but wouldn't this go against the initial "no
additional proxy" statement, if I got you right ..

Am Donnerstag, 5. Juni 2014 19:45:33 UTC+2 schrieb Mario Mueller:

Hey folks,

I kindly ask for a hint to achieve the following thing:

The goal is to deliver only a json array of source objects to the client.
The php app that sits on the other side uses JMS\Serializer to deserialize
the response into entities. At the moment the app needs to take an overhead
to derserialize it, extract the source and serialize it again. Then the
serialized stuff is passed to the entity deserializer. That's really
painful.

I've found a thread that suggests a proxy in between to handle this type
of problem, but this is not possible in our env.

The real question is: Is this achievable by writing a plugin? And if so,
what type of plugin? I've looked at the RestFilter, but I don't know if
this is the right way to go...

Any hints are welcome!

Regards,
Mario

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b27db27a-b136-4a3b-81b5-16d60047540d%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b27db27a-b136-4a3b-81b5-16d60047540d%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHKWraQG7iMiUiV5ucR8VdLO2nxruVX5Vq_P%3D1zcB6WGQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Mario Mueller) #15

Et kütt wie et kütt :wink: Das mit dem Koelsch geht klar, gibts auch hier in
DUS.

Again thanks to all!

Am Freitag, 6. Juni 2014 09:09:39 UTC+2 schrieb Jörg Prante:

I drink Kölsch only :slight_smile: ävver et hätt noh immer joot jejange

Greetings from Cologne!

Jörg

On Fri, Jun 6, 2014 at 7:14 AM, Mario Mueller <ma...@xenji.com
<javascript:>> wrote:

You guys are totally awesome! Thanks a lot! If you ever visit Duesseldorf
drop me a line, I owe you a beer.

@Brian:
Interesting approach, but wouldn't this go against the initial "no
additional proxy" statement, if I got you right ..

Am Donnerstag, 5. Juni 2014 19:45:33 UTC+2 schrieb Mario Mueller:

Hey folks,

I kindly ask for a hint to achieve the following thing:

The goal is to deliver only a json array of source objects to the
client. The php app that sits on the other side uses JMS\Serializer to
deserialize the response into entities. At the moment the app needs to take
an overhead to derserialize it, extract the source and serialize it again.
Then the serialized stuff is passed to the entity deserializer. That's
really painful.

I've found a thread that suggests a proxy in between to handle this type
of problem, but this is not possible in our env.

The real question is: Is this achievable by writing a plugin? And if so,
what type of plugin? I've looked at the RestFilter, but I don't know if
this is the right way to go...

Any hints are welcome!

Regards,
Mario

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b27db27a-b136-4a3b-81b5-16d60047540d%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b27db27a-b136-4a3b-81b5-16d60047540d%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d8c20589-1f4b-4b87-b140-ec7126e5bbe2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Brian Yoder) #16

Hi, Mario. Yes, I suppose this kind of goes against the "no additional
proxy" requirement you have.

Hehehe. I'm a seeker of loopholes. In my scenario, it's still a plug-in
design, but ES is my plug-in and not the other way around. Still only one
HTTP interface in the mix, but it's mine and not ES's.

I also have avoided the plug-in approach because I've read that it's marked
for deprecation and eventual removal. Yet logstash and ES Head are still
offered as plug-ins as are a boat-load of other facilities, so I am not
really sure if that's still the case.

And of course, your own plug-in has a much better chance to be updated to
match exactly each new ES version to which you migrate. That's one of the
downsides of third-party plug-ins: They lock you into older ES versions
until the author gets a chance to update the plug-in.

Brian

On Friday, June 6, 2014 1:14:00 AM UTC-4, Mario Mueller wrote:

@Brian:
Interesting approach, but wouldn't this go against the initial "no
additional proxy" statement, if I got you right ..

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/287ee9f9-b6cd-47d1-937e-f38406a614a1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Ivan Brusic) #17

Plugins are essential to ES's success and are not going away any time soon.
The river plugins, aka cluster singletons, are the ones which are
discouraged from use. Good ahead and create more plugins!

--
Ivan

On Fri, Jun 6, 2014 at 7:23 AM, Brian brian.from.fl@gmail.com wrote:

I also have avoided the plug-in approach because I've read that it's
marked for deprecation and eventual removal. Yet logstash and ES Head are
still offered as plug-ins as are a boat-load of other facilities, so I am
not really sure if that's still the case.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBEXPBJ5HTqT8YK25w_qjioYzGuXebQ%3DVvrE5HHmUMNvw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Brian Yoder) #18

Thanks so much, Ivan. That's a very important distinction.

Brian

On Friday, June 6, 2014 12:28:56 PM UTC-4, Ivan Brusic wrote:

Plugins are essential to ES's success and are not going away any time
soon. The river plugins, aka cluster singletons, are the ones which are
discouraged from use. Good ahead and create more plugins!

--
Ivan

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fc5afbff-469c-4a02-8fa9-8190a82ce5fb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #19