How to index a csv file with Elastic Search


(vb2008a) #1

I am a nebiew to elastic search. I have eleastic search and head running on
a VM. Can someone point me to the steps to index a csv file and search from
it.
Your asistance would be greatly appreciated.
Thanks,
Vidya

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/00ea65f8-2f4c-40d3-b5b9-bf56e8b70000%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Raffaele Sena) #2

What kind of queries would you want to be performing ?

If you just want to search for any record containing the search terms you
pass, then you can just index the CSV records as single strings. You still
need to wrap them in a JSON "document", but it can be
{"record":"csv-record-thingy"}.

If you want to be able to search or return specific fields, you'll have to
convert the CSV records into JSON objects like: {"field1": "value1",
"field2", "value2", ...} or whatever your field names are.

-- Raffaele

On Fri, Feb 28, 2014 at 5:07 PM, vb2008a@gmail.com wrote:

I am a nebiew to elastic search. I have eleastic search and head running
on a VM. Can someone point me to the steps to index a csv file and search
from it.
Your asistance would be greatly appreciated.
Thanks,
Vidya

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/00ea65f8-2f4c-40d3-b5b9-bf56e8b70000%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANKfucYsAnsG%3DGq32H5snVv4PXwHPut1-Be0Da9jA%2BOuOP2NRw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #3

You can also look at https://github.com/xxBedy/elasticsearch-river-csv

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 1 mars 2014 à 02:29, Raffaele Sena raff367@gmail.com a écrit :

What kind of queries would you want to be performing ?

If you just want to search for any record containing the search terms you pass, then you can just index the CSV records as single strings. You still need to wrap them in a JSON "document", but it can be {"record":"csv-record-thingy"}.

If you want to be able to search or return specific fields, you'll have to convert the CSV records into JSON objects like: {"field1": "value1", "field2", "value2", ...} or whatever your field names are.

-- Raffaele

On Fri, Feb 28, 2014 at 5:07 PM, vb2008a@gmail.com wrote:
I am a nebiew to elastic search. I have eleastic search and head running on a VM. Can someone point me to the steps to index a csv file and search from it.
Your asistance would be greatly appreciated.
Thanks,
Vidya

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/00ea65f8-2f4c-40d3-b5b9-bf56e8b70000%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANKfucYsAnsG%3DGq32H5snVv4PXwHPut1-Be0Da9jA%2BOuOP2NRw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5ADDA025-9927-4959-BA17-0D9337288381%40pilato.fr.
For more options, visit https://groups.google.com/groups/opt_out.


(David Zabner) #4

I don't quite understand how to use the river-csv tool.

I am passing it this:

curl -XPUT localhost:9200/_river/my_csv_river/_meta -d '

{

"type" : "csv",

"csv_file" : {

    "folder" : "/home/ec2-user/tweets",

    "first_line_is_header":"true"

},

"index" : {

    "index" : "categories",

    "type" : "tweet",

    "bulk_size" : 100,

    "bulk_threshold" : 10

}

}

and nothing is showing up.

Help!

On Friday, February 28, 2014 11:12:23 PM UTC-5, David Pilato wrote:

You can also look at https://github.com/xxBedy/elasticsearch-river-csv

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 1 mars 2014 à 02:29, Raffaele Sena <raf...@gmail.com <javascript:>> a
écrit :

What kind of queries would you want to be performing ?

If you just want to search for any record containing the search terms you
pass, then you can just index the CSV records as single strings. You still
need to wrap them in a JSON "document", but it can be
{"record":"csv-record-thingy"}.

If you want to be able to search or return specific fields, you'll have to
convert the CSV records into JSON objects like: {"field1": "value1",
"field2", "value2", ...} or whatever your field names are.

-- Raffaele

On Fri, Feb 28, 2014 at 5:07 PM, <vb2...@gmail.com <javascript:>> wrote:

I am a nebiew to elastic search. I have eleastic search and head running
on a VM. Can someone point me to the steps to index a csv file and search
from it.
Your asistance would be greatly appreciated.
Thanks,
Vidya

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/00ea65f8-2f4c-40d3-b5b9-bf56e8b70000%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANKfucYsAnsG%3DGq32H5snVv4PXwHPut1-Be0Da9jA%2BOuOP2NRw%40mail.gmail.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ca0bdd96-55a4-49f5-bd12-26a9e4a748a9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(David Pilato) #5

What gives /categories/_search

How many lines do you have in your csv file?
May be folder requires a ending /

Anything in logs?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 15 juil. 2014 à 17:55, David Zabner david.zabner@gmail.com a écrit :

I don't quite understand how to use the river-csv tool.

I am passing it this:
curl -XPUT localhost:9200/_river/my_csv_river/_meta -d '

{

"type" : "csv",

"csv_file" : {

    "folder" : "/home/ec2-user/tweets",

    "first_line_is_header":"true"

},

"index" : {

    "index" : "categories",

    "type" : "tweet",

    "bulk_size" : 100,

    "bulk_threshold" : 10

}

}

and nothing is showing up.

Help!

On Friday, February 28, 2014 11:12:23 PM UTC-5, David Pilato wrote:
You can also look at https://github.com/xxBedy/elasticsearch-river-csv

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 1 mars 2014 à 02:29, Raffaele Sena raf...@gmail.com a écrit :

What kind of queries would you want to be performing ?

If you just want to search for any record containing the search terms you pass, then you can just index the CSV records as single strings. You still need to wrap them in a JSON "document", but it can be {"record":"csv-record-thingy"}.

If you want to be able to search or return specific fields, you'll have to convert the CSV records into JSON objects like: {"field1": "value1", "field2", "value2", ...} or whatever your field names are.

-- Raffaele

On Fri, Feb 28, 2014 at 5:07 PM, vb2...@gmail.com wrote:
I am a nebiew to elastic search. I have eleastic search and head running on a VM. Can someone point me to the steps to index a csv file and search from it.
Your asistance would be greatly appreciated.
Thanks,
Vidya

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/00ea65f8-2f4c-40d3-b5b9-bf56e8b70000%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANKfucYsAnsG%3DGq32H5snVv4PXwHPut1-Be0Da9jA%2BOuOP2NRw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ca0bdd96-55a4-49f5-bd12-26a9e4a748a9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9A4E7AC9-6D06-4164-8666-65AD4C2E7910%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.


(David Zabner) #6

categories/_search returns:
{

  • error: "IndexMissingException[[categories] missing]",
  • status: 404

}

The logs do not seem to contain anything.

Am I supposed to input the data via curl as well?

On Tuesday, July 15, 2014 12:34:22 PM UTC-4, David Pilato wrote:

What gives /categories/_search

How many lines do you have in your csv file?
May be folder requires a ending /

Anything in logs?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 15 juil. 2014 à 17:55, David Zabner <david....@gmail.com <javascript:>>
a écrit :

I don't quite understand how to use the river-csv tool.

I am passing it this:

curl -XPUT localhost:9200/_river/my_csv_river/_meta -d '

{

"type" : "csv",

"csv_file" : {

    "folder" : "/home/ec2-user/tweets",

    "first_line_is_header":"true"

},

"index" : {

    "index" : "categories",

    "type" : "tweet",

    "bulk_size" : 100,

    "bulk_threshold" : 10

}

}

and nothing is showing up.

Help!

On Friday, February 28, 2014 11:12:23 PM UTC-5, David Pilato wrote:

You can also look at https://github.com/xxBedy/elasticsearch-river-csv

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 1 mars 2014 à 02:29, Raffaele Sena raf...@gmail.com a écrit :

What kind of queries would you want to be performing ?

If you just want to search for any record containing the search terms you
pass, then you can just index the CSV records as single strings. You still
need to wrap them in a JSON "document", but it can be
{"record":"csv-record-thingy"}.

If you want to be able to search or return specific fields, you'll have
to convert the CSV records into JSON objects like: {"field1": "value1",
"field2", "value2", ...} or whatever your field names are.

-- Raffaele

On Fri, Feb 28, 2014 at 5:07 PM, vb2...@gmail.com wrote:

I am a nebiew to elastic search. I have eleastic search and head running
on a VM. Can someone point me to the steps to index a csv file and search
from it.
Your asistance would be greatly appreciated.
Thanks,
Vidya

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/00ea65f8-2f4c-40d3-b5b9-bf56e8b70000%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANKfucYsAnsG%3DGq32H5snVv4PXwHPut1-Be0Da9jA%2BOuOP2NRw%40mail.gmail.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ca0bdd96-55a4-49f5-bd12-26a9e4a748a9%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/ca0bdd96-55a4-49f5-bd12-26a9e4a748a9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/be71dbc0-e78d-44f8-8f00-0b5dc55e0eb1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(David Pilato) #7

I think it should work.
May be you should create a ticket on the plugin repo.

Still, you should provide more details. For example, hits your logs, your CSV file, …
Also, check that your elasticsearch process has read access to your directory.

No more idea though

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 15 juillet 2014 à 20:05:44, David Zabner (david.zabner@gmail.com) a écrit:

categories/_search returns:
{
error: "IndexMissingException[[categories] missing]",
status: 404
}

The logs do not seem to contain anything.

Am I supposed to input the data via curl as well?

On Tuesday, July 15, 2014 12:34:22 PM UTC-4, David Pilato wrote:
What gives /categories/_search

How many lines do you have in your csv file?
May be folder requires a ending /

Anything in logs?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 15 juil. 2014 à 17:55, David Zabner david....@gmail.com a écrit :

I don't quite understand how to use the river-csv tool.

I am passing it this:
curl -XPUT localhost:9200/_river/my_csv_river/_meta -d '

{

"type" : "csv",

"csv_file" : {

    "folder" : "/home/ec2-user/tweets",

    "first_line_is_header":"true"

},

"index" : {

    "index" : "categories",

    "type" : "tweet",

    "bulk_size" : 100,

    "bulk_threshold" : 10

}

}

and nothing is showing up.

Help!

On Friday, February 28, 2014 11:12:23 PM UTC-5, David Pilato wrote:
You can also look at https://github.com/xxBedy/elasticsearch-river-csv

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 1 mars 2014 à 02:29, Raffaele Sena raf...@gmail.com a écrit :

What kind of queries would you want to be performing ?

If you just want to search for any record containing the search terms you pass, then you can just index the CSV records as single strings. You still need to wrap them in a JSON "document", but it can be {"record":"csv-record-thingy"}.

If you want to be able to search or return specific fields, you'll have to convert the CSV records into JSON objects like: {"field1": "value1", "field2", "value2", ...} or whatever your field names are.

-- Raffaele

On Fri, Feb 28, 2014 at 5:07 PM, vb2...@gmail.com wrote:
I am a nebiew to elastic search. I have eleastic search and head running on a VM. Can someone point me to the steps to index a csv file and search from it.
Your asistance would be greatly appreciated.
Thanks,
Vidya

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/00ea65f8-2f4c-40d3-b5b9-bf56e8b70000%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANKfucYsAnsG%3DGq32H5snVv4PXwHPut1-Be0Da9jA%2BOuOP2NRw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ca0bdd96-55a4-49f5-bd12-26a9e4a748a9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/be71dbc0-e78d-44f8-8f00-0b5dc55e0eb1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.53c581e7.6ceaf087.70e%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/d/optout.


(system) #8