Some efficient ways to export data other then JSON from Elasticsearch

javadevmtl · September 16, 2014, 2:04pm

Hi, building some sort of internal tool to export data from Elasticsearch
and I would liek to offer csv or XML.

Just wondering what options there are...

Bassically a user can login to a front end (No I cannot use what is out
there, it's only a small portion of a larger tool within the organization)
put their query in and then select the export format: JSON, XML, CSV.

When I have the SearchResponse from onResponse() what options do i have
here?

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2a4edfc9-ac2a-4466-ad01-07a79239400f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

javadevmtl · September 16, 2014, 2:13pm

Also it has to be done on the back end so JAVA it is...

On Tuesday, 16 September 2014 10:04:44 UTC-4, John Smith wrote:

Hi, building some sort of internal tool to export data from Elasticsearch
and I would liek to offer csv or XML.

Just wondering what options there are...

Bassically a user can login to a front end (No I cannot use what is out
there, it's only a small portion of a larger tool within the organization)
put their query in and then select the export format: JSON, XML, CSV.

When I have the SearchResponse from onResponse() what options do i have
here?

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/93236154-59a4-45b3-a2f7-aa93c928bcba%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

dadoonet · September 16, 2014, 2:22pm

You need to use the scan and scroll API for that.

See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-scroll.html#scroll-scan

This class could help you in Java: https://github.com/elasticsearch/elasticsearch/blob/master/src/test/java/org/elasticsearch/search/scroll/SearchScrollTests.java

HTH

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 16 septembre 2014 à 16:13:13, John Smith (java.dev.mtl@gmail.com) a écrit:

Also it has to be done on the back end so JAVA it is...

On Tuesday, 16 September 2014 10:04:44 UTC-4, John Smith wrote:
Hi, building some sort of internal tool to export data from Elasticsearch and I would liek to offer csv or XML.

Just wondering what options there are...

Bassically a user can login to a front end (No I cannot use what is out there, it's only a small portion of a larger tool within the organization) put their query in and then select the export format: JSON, XML, CSV.

When I have the SearchResponse from onResponse() what options do i have here?

Thanks

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/93236154-59a4-45b3-a2f7-aa93c928bcba%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.541847bd.275ac794.3c6%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/d/optout.

javadevmtl · September 16, 2014, 2:48pm

Yep, already doing that part actually...

Was just wondering I guess the best way to deserialize from json to xml for
instance.

I suppose it's slightly off topic but what are some good json to xml
converters.

On Tuesday, 16 September 2014 10:23:05 UTC-4, David Pilato wrote:

You need to use the scan and scroll API for that.

See
Elasticsearch Platform — Find real-time answers at scale | Elastic

This class could help you in Java:
https://github.com/elasticsearch/elasticsearch/blob/master/src/test/java/org/elasticsearch/search/scroll/SearchScrollTests.java

HTH

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfr
https://twitter.com/elasticsearchfr

Le 16 septembre 2014 à 16:13:13, John Smith (java.d...@gmail.com
<javascript:>) a écrit:

Also it has to be done on the back end so JAVA it is...

On Tuesday, 16 September 2014 10:04:44 UTC-4, John Smith wrote:

Hi, building some sort of internal tool to export data from Elasticsearch
and I would liek to offer csv or XML.

Just wondering what options there are...

Bassically a user can login to a front end (No I cannot use what is out
there, it's only a small portion of a larger tool within the organization)
put their query in and then select the export format: JSON, XML, CSV.

When I have the SearchResponse from onResponse() what options do i have
here?

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/93236154-59a4-45b3-a2f7-aa93c928bcba%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/93236154-59a4-45b3-a2f7-aa93c928bcba%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fe0ccad7-d34a-4c63-9e9a-cbd063896f56%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

costin · September 16, 2014, 3:07pm

When it comes to JSON, Jackson should be at the top of your list. It's an excellent library and it has plenty of support
for XML [1]

[1] GitHub - FasterXML/jackson-dataformat-xml: Extension for Jackson JSON processor that adds support for serializing POJOs as XML (and deserializing from XML) as an alternative to JSON

On 9/16/14 5:48 PM, John Smith wrote:

Yep, already doing that part actually...

Was just wondering I guess the best way to deserialize from json to xml for instance.

I suppose it's slightly off topic but what are some good json to xml converters.

On Tuesday, 16 September 2014 10:23:05 UTC-4, David Pilato wrote:
You need to use the scan and scroll API for that.

See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-scroll.html#scroll-scan
<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-scroll.html#scroll-scan>

This class could help you in Java:
https://github.com/elasticsearch/elasticsearch/blob/master/src/test/java/org/elasticsearch/search/scroll/SearchScrollTests.java
<https://github.com/elasticsearch/elasticsearch/blob/master/src/test/java/org/elasticsearch/search/scroll/SearchScrollTests.java>


HTH

--
*David Pilato* | /Technical Advocate/ | *Elasticsearch.com*
@dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr>


Le 16 septembre 2014 à 16:13:13, John Smith (java.d...@gmail.com <javascript:>) a écrit:
Also it has to be done on the back end so JAVA it is...

On Tuesday, 16 September 2014 10:04:44 UTC-4, John Smith wrote:

    Hi, building some sort of internal tool to export data from Elasticsearch and I would liek to offer csv or XML.

    Just wondering what options there are...


    Bassically a user can login to a front end (No I cannot use what is out there, it's only a small portion of a
    larger tool within the organization) put their query in and then select the export format: JSON, XML, CSV.

    When I have the SearchResponse from onResponse() what options do i have here?

    Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/93236154-59a4-45b3-a2f7-aa93c928bcba%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/93236154-59a4-45b3-a2f7-aa93c928bcba%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/fe0ccad7-d34a-4c63-9e9a-cbd063896f56%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/fe0ccad7-d34a-4c63-9e9a-cbd063896f56%40googlegroups.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5418522B.9080205%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

javadevmtl · September 16, 2014, 3:12pm

Hadn't looked at Jakson for a while but it seems to do both XML and CSV
(limited to json that represents tabular data)

On Tuesday, 16 September 2014 10:48:58 UTC-4, John Smith wrote:

Yep, already doing that part actually...

Was just wondering I guess the best way to deserialize from json to xml
for instance.

I suppose it's slightly off topic but what are some good json to xml
converters.

On Tuesday, 16 September 2014 10:23:05 UTC-4, David Pilato wrote:

You need to use the scan and scroll API for that.

See
Elasticsearch Platform — Find real-time answers at scale | Elastic

This class could help you in Java:
https://github.com/elasticsearch/elasticsearch/blob/master/src/test/java/org/elasticsearch/search/scroll/SearchScrollTests.java

HTH

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfr
https://twitter.com/elasticsearchfr

Le 16 septembre 2014 à 16:13:13, John Smith (java.d...@gmail.com) a
écrit:

Also it has to be done on the back end so JAVA it is...

On Tuesday, 16 September 2014 10:04:44 UTC-4, John Smith wrote:

Hi, building some sort of internal tool to export data from
Elasticsearch and I would liek to offer csv or XML.

Just wondering what options there are...

Bassically a user can login to a front end (No I cannot use what is out
there, it's only a small portion of a larger tool within the organization)
put their query in and then select the export format: JSON, XML, CSV.

When I have the SearchResponse from onResponse() what options do i have
here?

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/93236154-59a4-45b3-a2f7-aa93c928bcba%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/93236154-59a4-45b3-a2f7-aa93c928bcba%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0d7f834d-2ab8-4d5b-b3bb-04377efd5d03%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.