Elastic search indexing documents

Deepikaa_Subramaniam · March 21, 2014, 11:22pm

Hi guys,

I am new to Elastic Search. Have setup my env use C# +Nest to access ES. I
am able to index txt files successfully. I downloaded the elastic search
mapper plugin to extract data from other document types. However, if i try
to search for some Keywords from within the doc the search doesn't return
any results. Please help.

public class Doc
{
public string file_id;
public string created;
[ElasticProperty(Type=Nest.FieldType.attachment, Store = true,
TermVector=Nest.termVectorOption.with_positions_offsets)]
public string content;
}
Doc doc = new Doc();
Doc.content = Convert.ToBase64String(File.ReadAllBytes(path));

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d8e90c02-09e2-4579-8091-d7521c27fc8b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

dadoonet · March 22, 2014, 4:05am

How can we help you?
You did not send what you are doing actually.

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 22 mars 2014 à 00:22, Deepikaa Subramaniam deeps.subramaniam@gmail.com a écrit :

Hi guys,

I am new to Elastic Search. Have setup my env use C# +Nest to access ES. I am able to index txt files successfully. I downloaded the elastic search mapper plugin to extract data from other document types. However, if i try to search for some Keywords from within the doc the search doesn't return any results. Please help.

public class Doc
{
public string file_id;
public string created;
[ElasticProperty(Type=Nest.FieldType.attachment, Store = true, TermVector=Nest.termVectorOption.with_positions_offsets)]
public string content;
}
Doc doc = new Doc();
Doc.content = Convert.ToBase64String(File.ReadAllBytes(path));

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d8e90c02-09e2-4579-8091-d7521c27fc8b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3581F728-E888-4230-9EE0-033DCF2DAAAF%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

Deepikaa_Subramaniam · March 24, 2014, 1:47am

I would like to index documents for full text search.

On Friday, March 21, 2014 4:22:44 PM UTC-7, Deepikaa Subramaniam wrote:

Hi guys,

I am new to Elastic Search. Have setup my env use C# +Nest to access ES. I
am able to index txt files successfully. I downloaded the Elasticsearch
mapper plugin to extract data from other document types. However, if i try
to search for some Keywords from within the doc the search doesn't return
any results. Please help.

public class Doc
{
public string file_id;
public string created;
[ElasticProperty(Type=Nest.FieldType.attachment, Store = true,
TermVector=Nest.termVectorOption.with_positions_offsets)]
public string content;
}
Doc doc = new Doc();
Doc.content = Convert.ToBase64String(File.ReadAllBytes(path));

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/07284ada-b85e-4c1c-8ba1-ef12c71c43cd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

dadoonet · March 24, 2014, 5:52am

What did you try so far?

Read this;: Elasticsearch Platform — Find real-time answers at scale | Elastic

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 24 mars 2014 à 02:47, Deepikaa Subramaniam deeps.subramaniam@gmail.com a écrit :

I would like to index documents for full text search.

On Friday, March 21, 2014 4:22:44 PM UTC-7, Deepikaa Subramaniam wrote:
Hi guys,

I am new to Elastic Search. Have setup my env use C# +Nest to access ES. I am able to index txt files successfully. I downloaded the Elasticsearch mapper plugin to extract data from other document types. However, if i try to search for some Keywords from within the doc the search doesn't return any results. Please help.

public class Doc
{
public string file_id;
public string created;
[ElasticProperty(Type=Nest.FieldType.attachment, Store = true, TermVector=Nest.termVectorOption.with_positions_offsets)]
public string content;
}
Doc doc = new Doc();
Doc.content = Convert.ToBase64String(File.ReadAllBytes(path));

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/07284ada-b85e-4c1c-8ba1-ef12c71c43cd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9EA0214D-D0E7-4386-A9AC-0BF784736F9D%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

Deepikaa_Subramaniam · March 24, 2014, 6:36am

I am using the attachment plugin. I have installed it under Elasticsearch
plugings.
I couldn't find any C# examples to use this plugin. Everything that I find
are curl samples.
Anyways, I have the content - base 64 encoded and then index it. I have set
the Elasticsearch property of this field as attachment.
but when I search for a text in the word doc that I tried to index, I don't
get any results back. if you can give me some samples, that would be great.

On Sunday, March 23, 2014 10:52:23 PM UTC-7, David Pilato wrote:

What did you try so far?

Read this;: Elasticsearch Platform — Find real-time answers at scale | Elastic

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 24 mars 2014 à 02:47, Deepikaa Subramaniam <deeps.su...@gmail.com<javascript:>>
a écrit :

I would like to index documents for full text search.

On Friday, March 21, 2014 4:22:44 PM UTC-7, Deepikaa Subramaniam wrote:

Hi guys,

I am new to Elastic Search. Have setup my env use C# +Nest to access ES.
I am able to index txt files successfully. I downloaded the Elasticsearch
mapper plugin to extract data from other document types. However, if i try
to search for some Keywords from within the doc the search doesn't return
any results. Please help.

public class Doc
{
public string file_id;
public string created;
[ElasticProperty(Type=Nest.FieldType.attachment, Store = true,
TermVector=Nest.termVectorOption.with_positions_offsets)]
public string content;
}
Doc doc = new Doc();
Doc.content = Convert.ToBase64String(File.ReadAllBytes(path));

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/07284ada-b85e-4c1c-8ba1-ef12c71c43cd%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/07284ada-b85e-4c1c-8ba1-ef12c71c43cd%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/33b0c0a0-cb66-4d1a-9e8b-4dd0acea9112%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Deepikaa_Subramaniam · March 24, 2014, 6:38am

Also, using Nest client with C#

On Sunday, March 23, 2014 11:36:11 PM UTC-7, Deepikaa Subramaniam wrote:

I am using the attachment plugin. I have installed it under Elasticsearch
plugings.
I couldn't find any C# examples to use this plugin. Everything that I find
are curl samples.
Anyways, I have the content - base 64 encoded and then index it. I have
set the Elasticsearch property of this field as attachment.
but when I search for a text in the word doc that I tried to index, I
don't get any results back. if you can give me some samples, that would be
great.

On Sunday, March 23, 2014 10:52:23 PM UTC-7, David Pilato wrote:

What did you try so far?

Read this;: Elasticsearch Platform — Find real-time answers at scale | Elastic

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 24 mars 2014 à 02:47, Deepikaa Subramaniam deeps.su...@gmail.com a
écrit :

I would like to index documents for full text search.

On Friday, March 21, 2014 4:22:44 PM UTC-7, Deepikaa Subramaniam wrote:

Hi guys,

I am new to Elastic Search. Have setup my env use C# +Nest to access ES.
I am able to index txt files successfully. I downloaded the Elasticsearch
mapper plugin to extract data from other document types. However, if i try
to search for some Keywords from within the doc the search doesn't return
any results. Please help.

public class Doc
{
public string file_id;
public string created;
[ElasticProperty(Type=Nest.FieldType.attachment, Store = true,
TermVector=Nest.termVectorOption.with_positions_offsets)]
public string content;
}
Doc doc = new Doc();
Doc.content = Convert.ToBase64String(File.ReadAllBytes(path));

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/07284ada-b85e-4c1c-8ba1-ef12c71c43cd%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/07284ada-b85e-4c1c-8ba1-ef12c71c43cd%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/86e8e14f-0d51-498e-a67e-57f91d1fcee7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

dadoonet · March 24, 2014, 7:27am

gist.github.com

https://gist.github.com/dadoonet/3907010

attachment.sh

#!/bin/sh

http://cloud.github.com/downloads/elasticsearch/elasticsearch/elasticsearch-0.19.10.zip

unzip elasticsearch-0.19.10.zip
elasticsearch-0.19.10/bin/plugin -install elasticsearch/elasticsearch-mapper-attachments/1.6.0

elasticsearch-0.19.10/bin/elasticsearch

sleep 10

This file has been truncated. show original

attachmentFile.json

{"file":"JVBERi0xLjQNJeLjz9MNCg==
MTU4IDAgb2JqIDw8L0xpbmVhcml6ZWQgMS9MIDM4MTE0NC9PIDE2My9FIDI0NzExL04gMTMvVCAz
Nzc5MzYvSCBbIDExNTYgNDY4XT4+DWVuZG9iag0gICAgICAgICAgICANCg==
eHJlZg0K
MTU4IDQzDQo=
MDAwMDAwMDAxNiAwMDAwMCBuDQo=
MDAwMDAwMTYyNCAwMDAwMCBuDQo=
MDAwMDAwMTc5NCAwMDAwMCBuDQo=
MDAwMDAwMTgyMCAwMDAwMCBuDQo=
MDAwMDAwMTg2NiAwMDAwMCBuDQo=

This file has been truncated. show original

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 24 mars 2014 à 07:38, Deepikaa Subramaniam deeps.subramaniam@gmail.com a écrit :

Also, using Nest client with C#

On Sunday, March 23, 2014 11:36:11 PM UTC-7, Deepikaa Subramaniam wrote:
I am using the attachment plugin. I have installed it under Elasticsearch plugings.
I couldn't find any C# examples to use this plugin. Everything that I find are curl samples.
Anyways, I have the content - base 64 encoded and then index it. I have set the Elasticsearch property of this field as attachment.
but when I search for a text in the word doc that I tried to index, I don't get any results back. if you can give me some samples, that would be great.

On Sunday, March 23, 2014 10:52:23 PM UTC-7, David Pilato wrote:
What did you try so far?

Read this;: Elasticsearch Platform — Find real-time answers at scale | Elastic

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 24 mars 2014 à 02:47, Deepikaa Subramaniam deeps.su...@gmail.com a écrit :

I would like to index documents for full text search.

On Friday, March 21, 2014 4:22:44 PM UTC-7, Deepikaa Subramaniam wrote:
Hi guys,

I am new to Elastic Search. Have setup my env use C# +Nest to access ES. I am able to index txt files successfully. I downloaded the Elasticsearch mapper plugin to extract data from other document types. However, if i try to search for some Keywords from within the doc the search doesn't return any results. Please help.

public class Doc
{
public string file_id;
public string created;
[ElasticProperty(Type=Nest.FieldType.attachment, Store = true, TermVector=Nest.termVectorOption.with_positions_offsets)]
public string content;
}
Doc doc = new Doc();
Doc.content = Convert.ToBase64String(File.ReadAllBytes(path));

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/07284ada-b85e-4c1c-8ba1-ef12c71c43cd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/86e8e14f-0d51-498e-a67e-57f91d1fcee7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/D882E325-AF8D-4E96-958E-1216D2A516D5%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

ravikanth · April 4, 2014, 7:00am

Hi David.

    Iam new to elasticsearch and using elasticsearch-1.0.0.RC1.
    I can indexing the files individually and able to search the

content.But
when iam indexing the bulk files(documents like txt,pdf etc) it
taking more time.
Apart from my application have one table with all files details
with url also.
loop the table data and get the content from the files with url by
using curl and convert into base64_encode and indexing.
(by following what u gave the url:
Test of attachments plugin · GitHub https://www.google.com/url?q=https%3A%2F%2Fgist.github.com%2Fdadoonet%2F3907010&sa=D&sntz=1&usg=AFQjCNEcyvaZBUX4tyZa8eBku9-J8KI16w
)
if you can give me some samples, that would be great.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5cbff19b-94ac-4d11-9073-c4cebc561c7d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Elastic search indexing documents

public class Doc { public string file_id; public string created; [ElasticProperty(Type=Nest.FieldType.attachment, Store = true, TermVector=Nest.termVectorOption.with_positions_offsets)] public string content; } Doc doc = new Doc(); Doc.content = Convert.ToBase64String(File.ReadAllBytes(path));

public class Doc
{
public string file_id;
public string created;
[ElasticProperty(Type=Nest.FieldType.attachment, Store = true, TermVector=Nest.termVectorOption.with_positions_offsets)]
public string content;
}
Doc doc = new Doc();
Doc.content = Convert.ToBase64String(File.ReadAllBytes(path));