Log server with Elasticsearch problem to index & search logfile

Elastic Stack Elasticsearch
1

Hi,

I have log server env 280Tera, i want to index the log file.
The files are *.TXT file that inch them can be more than 3GB.
I try with this command:

curl -XPUT 'http://localhost:9200/test/5' -d '{ "user" : "diag_auto" ,
"postDate" : "2013-05-01", "my_attachment" : { "_name" :
"/extra/projects/upc/2013-Apr/3C_62_00_60_5B_84/25-Apr-time-06-45-42/IPC_00_16_6B_D2_A2_A2_25-Apr-time-06-50-14/pars_log/levels.txt"}
}
1- this is not work

2- I want after to search text and receive back in which log this is
displayed and also the line in the log.
3- in the log i can have this series : "B" Line 0, "A" line 2, "C" line 5,
"B" Line 8, "A" line 10, and "C" line 15
i want to search the First A and only after B, C so i will have back : 1-
"A" line 2, 2- "B" Line 8, "C" line 15

So this is help that i need.
first how to index TXT file?
second how do simply search on the TXT file that they are index?
thirdly how do advanced search like i explain before?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Already answered:

Hi David,

  1. to use attachment plugin, you have to define a mapping first. Then encode your file in Base64, then send it to ES. Read https://github.com/elasticsearch/elasticsearch-mapper-attachments
    Look at the FSriver code source if you need an example. https://github.com/dadoonet/fsriver

a) ok.
b) with attachment plugin, everything will be flatten. So your lines will be indexed as "B A S C B C". Note that A is a stop word and won't be indexed.

BTW, have a look at logstash and Kibana project. It should help you here.

HTH

David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 5 mai 2013 à 05:52, David Barbe david@famillebarbe.com a écrit :

Hi,

I have log server env 280Tera, i want to index the log file.
The files are *.TXT file that inch them can be more than 3GB.
I try with this command:

curl -XPUT 'http://localhost:9200/test/5' -d '{ "user" : "diag_auto" , "postDate" : "2013-05-01", "my_attachment" : { "_name" : "/extra/projects/upc/2013-Apr/3C_62_00_60_5B_84/25-Apr-time-06-45-42/IPC_00_16_6B_D2_A2_A2_25-Apr-time-06-50-14/pars_log/levels.txt"} }
1- this is not work

2- I want after to search text and receive back in which log this is displayed and also the line in the log.
3- in the log i can have this series : "B" Line 0, "A" line 2, "C" line 5, "B" Line 8, "A" line 10, and "C" line 15
i want to search the First A and only after B, C so i will have back : 1- "A" line 2, 2- "B" Line 8, "C" line 15

So this is help that i need.
first how to index TXT file?
second how do simply search on the TXT file that they are index?
thirdly how do advanced search like i explain before?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.