Monitoring, Backup & Restore Tools for Elastic search

Craig,

By stating the phrase "100m files", do you mean "100 million individual
files" or do you mean "100-megabyte-sized files"?

I suspect the latter (e.g. a single file of 100 megabytes in size) and not
the former (100 million directory entries of type 'file').

Brian

On Friday, September 6, 2013 9:54:37 AM UTC-4, Craig Lebowitz wrote:

Jorg one question for you: if I extract the knapsack archive, will the
file system have a problem with 100m files?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re monitoring tools. The ones you listed are all free and do a good job.

Elasticsearch-head is especially good for seeing the cluster health and
shard allocation. It's browse mode is good for inspecting a particular
documents _source.

Bigdesk is great for real time performance data - which is collected for
each node, but only viewable one at a time (in a single window). You could
run multiple instances, at the cost of increasing the monitoring load upon
your cluster. The history is stored only in your browser, so it's
impossible to see what happened before.

Paramedic and ElasticHQ are both somewhat similar in their functionality to
Bigdesk, I haven't used them too much although both have nice GUI.

For serious production use I would highly recommend Sematext SPM. It is a
commercial product, where you run JMX agents and a 'sender' service on your
nodes, which uploads the metrics into the sematext cloud. This gives you
the advantage of having a proper history available. Their pricing is pretty
reasonable for the peace of mind you have. Check
out Elasticsearch Monitoring

On Wednesday, 31 July 2013 15:50:56 UTC+1, pranav amin wrote:

Hi There,

I'm a new user for Elastic search and plan to use it in Production
environment.

Can you please help me with some information -

Monitoring tools -
I looked at the following.
a) Elasticsearch head
b) Bigdesk
c) Paramedic
d) ElasticHQ

Not sure which of the above are free and recommend by others?

Backup & Restore -
a) Can i do backup and restore of a Node in a Cluster? Are there any
free/cost based tools (UI or Script based) that can do this and are
frequently recommended by others?
b) Are there any free/cost based tools (UI or Script based) that can do
backup & restore of ACTIVE AND NON-ACTIVE cluster?

Thanks for the help and advice.
Much appreciated.

Thanks
Pranav.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

On Fri, Sep 06, 2013 at 10:59:27PM +0200, joergprante@gmail.com wrote:

Jörg,

Good question, I think Linux ext4 can handle 100m dir entries quite well if
it was configured to do so - but, many file based tools will not work

Ext4 has no limit on directory entries. The limit for directory entries on ext3
is 32k.

properly with 100m dir entries, for example ls or rm -rf. Either your main
memory will blow up, or wait times will become excessively long.

This would be basically because the shell will expand the command if wildcards
are used. You can curcumvent this using other tools. Just have a look at find
and xargs if the shell is unable to handle the argument list.

Best regards, Adrian

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

You might get surprised that even xargs has arg list limits. Check this with

getconf ARG_MAX

Most Linux systems will return 2621440 which is far from being able to
handle 100m dir entries via arg lists.

Just my 2c.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

thanks for recommending.

when i try to use this it complains -

local@slsdev-int-09u:/mnt/sl/elastic/elasticsearch-0.90.1$ curl -XPOST
5.36.242.139:9200/test/test/_export
[2013-09-13 21:08:13,612][INFO ][cluster.service ] [Pranav]
detected_master
[Paul][PoBefbfWQ6K9XRJraJF0hA][inet[/5.36.242.138:9300]]{master=true},
added {[Tom][3SrFwYh8RmqyVkbhHfqyzQ][inet[/5.36.242.140:9300
]]{master=true},[Paul][PoBefbfWQ6K9XRJraJF0hA][inet[/5.36.242.138:9300]]{master=true},},
reason: zen-disco-receive(from master
[[Paul][PoBefbfWQ6K9XRJraJF0hA][inet[/5.36.242.138:9300]]{master=true}])
{"error":"RemoteTransportException[[Paul][inet[/5.36.242.138:9300]][index]];
nested: MapperParsingException[failed to parse, document is empty];
","status":400}local@slsdev-int-09u:/mnt/sl/elastic/elasticsearch-0.90.1$

Any idea what the above error means?

Thanks in advance.

On Wed, Sep 4, 2013 at 11:42 AM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

You can try my knapsack plugin which has a "size" URL parameter to control
the bulk export.

GitHub - jprante/elasticsearch-knapsack: Knapsack plugin is an import/export tool for Elasticsearch

Jörg

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Pf1tLdAtIp8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

fyi - i have data in the test indice so not sure why the error says
document is empty. There are two records i added.

local@slsdev-int-09u:/mnt/sl/elastic/elasticsearch-0.90.1$ curl -XGET
5.36.242.140:9200/test/test/_search
{"took":26,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":2,"max_score":1.0,"hits":[{"_index":"test","_type":"test","_id":"1","_score":1.0,
"_source" : {"key":"value
1"}},{"_index":"test","_type":"test","_id":"2","_score":1.0, "_source" :
{"key":"value 2"}}]}}local@slsdev-int-09

On Fri, Sep 13, 2013 at 5:09 PM, parul patel parulpatel25@gmail.com wrote:

thanks for recommending.

when i try to use this it complains -

local@slsdev-int-09u:/mnt/sl/elastic/elasticsearch-0.90.1$ curl -XPOST
5.36.242.139:9200/test/test/_export
[2013-09-13 21:08:13,612][INFO ][cluster.service ] [Pranav]
detected_master [Paul][PoBefbfWQ6K9XRJraJF0hA][inet[/5.36.242.138:9300]]{master=true},
added {[Tom][3SrFwYh8RmqyVkbhHfqyzQ][inet[/5.36.242.140:9300
]]{master=true},[Paul][PoBefbfWQ6K9XRJraJF0hA][inet[/5.36.242.138:9300]]{master=true},},
reason: zen-disco-receive(from master
[[Paul][PoBefbfWQ6K9XRJraJF0hA][inet[/5.36.242.138:9300]]{master=true}])
{"error":"RemoteTransportException[[Paul][inet[/5.36.242.138:9300]][index]];
nested: MapperParsingException[failed to parse, document is empty];
","status":400}local@slsdev-int-09u:/mnt/sl/elastic/elasticsearch-0.90.1$

Any idea what the above error means?

Thanks in advance.

On Wed, Sep 4, 2013 at 11:42 AM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

You can try my knapsack plugin which has a "size" URL parameter to
control the bulk export.

GitHub - jprante/elasticsearch-knapsack: Knapsack plugin is an import/export tool for Elasticsearch

Jörg

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Pf1tLdAtIp8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

I think the plugin is not loaded.
Did you restart es?
Can you see in logs at the begining that the plugin is loaded?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 13 sept. 2013 à 23:09, parul patel parulpatel25@gmail.com a écrit :

thanks for recommending.

when i try to use this it complains -

local@slsdev-int-09u:/mnt/sl/elastic/elasticsearch-0.90.1$ curl -XPOST 5.36.242.139:9200/test/test/_export
[2013-09-13 21:08:13,612][INFO ][cluster.service ] [Pranav] detected_master [Paul][PoBefbfWQ6K9XRJraJF0hA][inet[/5.36.242.138:9300]]{master=true}, added {[Tom][3SrFwYh8RmqyVkbhHfqyzQ][inet[/5.36.242.140:9300]]{master=true},[Paul][PoBefbfWQ6K9XRJraJF0hA][inet[/5.36.242.138:9300]]{master=true},}, reason: zen-disco-receive(from master [[Paul][PoBefbfWQ6K9XRJraJF0hA][inet[/5.36.242.138:9300]]{master=true}])
{"error":"RemoteTransportException[[Paul][inet[/5.36.242.138:9300]][index]]; nested: MapperParsingException[failed to parse, document is empty]; ","status":400}local@slsdev-int-09u:/mnt/sl/elastic/elasticsearch-0.90.1$

Any idea what the above error means?

Thanks in advance.

On Wed, Sep 4, 2013 at 11:42 AM, joergprante@gmail.com joergprante@gmail.com wrote:

You can try my knapsack plugin which has a "size" URL parameter to control the bulk export.

GitHub - jprante/elasticsearch-knapsack: Knapsack plugin is an import/export tool for Elasticsearch

Jörg

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/Pf1tLdAtIp8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

It is possible there are less docs in the index than shards are in the
index. Please open an issue on the knapsack github issues. I will have a
look.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

I did restart the ES. May be i'm missing something for _source field?
How do i check i have this field in the Index since knapsack works on the
document if it has _source field.

On Sat, Sep 14, 2013 at 2:11 AM, David Pilato david@pilato.fr wrote:

I think the plugin is not loaded.
Did you restart es?
Can you see in logs at the begining that the plugin is loaded?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 13 sept. 2013 à 23:09, parul patel parulpatel25@gmail.com a écrit :

thanks for recommending.

when i try to use this it complains -

local@slsdev-int-09u:/mnt/sl/elastic/elasticsearch-0.90.1$ curl -XPOST
5.36.242.139:9200/test/test/_export
[2013-09-13 21:08:13,612][INFO ][cluster.service ] [Pranav]
detected_master [Paul][PoBefbfWQ6K9XRJraJF0hA][inet[/5.36.242.138:9300]]{master=true},
added {[Tom][3SrFwYh8RmqyVkbhHfqyzQ][inet[/5.36.242.140:9300
]]{master=true},[Paul][PoBefbfWQ6K9XRJraJF0hA][inet[/5.36.242.138:9300]]{master=true},},
reason: zen-disco-receive(from master
[[Paul][PoBefbfWQ6K9XRJraJF0hA][inet[/5.36.242.138:9300]]{master=true}])
{"error":"RemoteTransportException[[Paul][inet[/5.36.242.138:9300]][index]];
nested: MapperParsingException[failed to parse, document is empty];
","status":400}local@slsdev-int-09u:/mnt/sl/elastic/elasticsearch-0.90.1$

Any idea what the above error means?

Thanks in advance.

On Wed, Sep 4, 2013 at 11:42 AM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

You can try my knapsack plugin which has a "size" URL parameter to
control the bulk export.

GitHub - jprante/elasticsearch-knapsack: Knapsack plugin is an import/export tool for Elasticsearch

Jörg

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Pf1tLdAtIp8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Pf1tLdAtIp8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

have i got the _source field populated for this indiced based on this data?
it looks to me like that, but wanted to confirm.

local@slsdev-int-09u:/mnt/sl/elastic/elasticpm/clintongormley-Elasticsearch.pm-d68cd02$
curl -XGET 5.36.242.139:9200/test/test/1
{"_index":"test","_type":"test","_id":"1","_version":1,"exists":true,
"_source" : {"key":"value 1"}}local@slsdev-int-09u
:/mnt/sl/elastic/elasticpm/clintongor

i will log a ticket if i have got everything right.

On Sat, Sep 14, 2013 at 8:19 AM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

It is possible there are less docs in the index than shards are in the
index. Please open an issue on the knapsack github issues. I will have a
look.

Issues · jprante/elasticsearch-knapsack · GitHub

Jörg

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Pf1tLdAtIp8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

now i see a different error for knapsack -

local@pldsvv-07:/var/elasticsearch/elasticsearch-0.90.3$ curl -XPUT
localhost:9200/test/test/1 -d '{"_source" : {"enabled" : true},
"key":"value 1"}'
{"ok":true,"_index":"test","_type":"test","_id":"1","_version":1}local@pldsvv-07
:/var/elasticsearch/elasticsearch-0.90.3$

local@pldsvv-07:/var/elasticsearch/elasticsearch-0.90.3$ curl -XGET
localhost:9200/test/test/1
{"_index":"test","_type":"test","_id":"1","_version":1,"exists":true,
"_source" : {"_source" : {"enabled" : true}, "key":"value
1"}}local@pldsvv-07:/var/elasticsearch/elasticsearch-0.90.3$

local@pldsvv-07:/var/elasticsearch/elasticsearch-0.90.3$
local@pldsvv-07:/var/elasticsearch/elasticsearch-0.90.3$ curl -XPOST
localhost:9200/test/test/_export
{"exports":,"totalExported":0,"_shards":{"total":5,"successful":0,"failed":5,"failures":[{"index":"test","shard":0,"reason":"BroadcastShardOperationFailedException[[test][0]
]; nested: SearchParseException[[test][0]: from[-1],size[-1]: Parse Failure
[Failed to parse source [na]]]; nested: SearchParseException[[test][0]:
from[-1],size[-1]: Parse Failure [No export fields defined]];
"},{"index":"test","shard":1,"reason":"BroadcastShardOperationFailedException[[test][1]
]; nested: SearchParseException[[test][1]: from[-1],size[-1]: Parse Failure
[Failed to parse source [na]]]; nested: SearchParseException[[test][1]:
from[-1],size[-1]: Parse Failure [No export fields defined]];
"},{"index":"test","shard":4,"reason":"BroadcastShardOperationFailedException[[test][4]
]; nested: SearchParseException[[test][4]: from[-1],size[-1]: Parse Failure
[Failed to parse source [na]]]; nested: SearchParseException[[test][4]:
from[-1],size[-1]: Parse Failure [No export fields defined]];
"},{"index":"test","shard":3,"reason":"BroadcastShardOperationFailedException[[test][3]
]; nested: SearchParseException[[test][3]: from[-1],size[-1]: Parse Failure
[Failed to parse source [na]]]; nested: SearchParseException[[test][3]:
from[-1],size[-1]: Parse Failure [No export fields defined]];
"},{"index":"test","shard":2,"reason":"BroadcastShardOperationFailedException[[test][2]
]; nested: SearchParseException[[test][2]: from[-1],size[-1]: Parse Failure
[Failed to parse source [na]]]; nested: SearchParseException[[test][2]:
from[-1],size[-1]: Parse Failure [No export fields defined]];
"}]}}local@pldsvv-07:/var/elasticsearch/elasticsearch-0.90.3$

any idea what i'm doing wrong here?

thanks

On Mon, Sep 16, 2013 at 11:19 AM, parul patel parulpatel25@gmail.comwrote:

have i got the _source field populated for this indiced based on this
data? it looks to me like that, but wanted to confirm.

local@slsdev-int-09u:/mnt/sl/elastic/elasticpm/clintongormley-Elasticsearch.pm-d68cd02$
curl -XGET 5.36.242.139:9200/test/test/1
{"_index":"test","_type":"test","_id":"1","_version":1,"exists":true,
"_source" : {"key":"value 1"}}local@slsdev-int-09u
:/mnt/sl/elastic/elasticpm/clintongor

i will log a ticket if i have got everything right.

On Sat, Sep 14, 2013 at 8:19 AM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

It is possible there are less docs in the index than shards are in the
index. Please open an issue on the knapsack github issues. I will have a
look.

Issues · jprante/elasticsearch-knapsack · GitHub

Jörg

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Pf1tLdAtIp8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi Pranav

Can you try 0.90.4 version of ES. It looks like its a 0.90.3 issue.
see Compatibility with 0.90.3 · Issue #35 · crate/elasticsearch-inout-plugin · GitHub
and BytesStreamOutput default size should be 2k instead of 32k · Issue #3638 · elastic/elasticsearch · GitHub

cheers, Bernd

On Friday, August 30, 2013 8:30:11 PM UTC+2, pranav amin wrote:

Hi,

I tried to use the elasticsearch-inout plug in for export. It worked fine.
Now when i try to import it - it fails with

lo...@pldsvv-07:/var/elasticsearch/elasticsearch-0.90.3/bin$ <javascript:>curl -X POST '
http://localhost:9200/abc_newindex/abc/_import' -d '{

"directory": "/tmp/es-data",

"compression": "gzip"

}

'

[2013-08-30 18:02:08,818][INFO ][cluster.metadata ] [Eric]
[abc_newindex] creating index, cause [auto(bulk api)], shards [5]/[1],
mappings

[2013-08-30 18:02:20,943][WARN ][index.engine.robin ] [Eric]
[abc_newindex][0] failed engine

[2013-08-30 18:02:21,227][DEBUG][action.bulk ] [Eric]
[abc_newindex][0], node[wHAaefkjSre_x6Lj8XS_6A], [P], s[STARTED]: Failed to
execute [org.elasticsearch.action.bulk.BulkShardRequest@591bc6d9]

java.lang.OutOfMemoryError: Java heap space

{"imports":,"failures":[{"node_id":"Iow0kmoGRwOLSqZ5jEivAQ","reason":"org.elasticsearch.action.FailedNodeException:
Failed node [Iow0kmoGRwOLSqZ5jEivAQ];
org.elasticsearch.transport.RemoteTransportException:
[Druig][inet[/10.3.176.23:9300]][el-crate-import/n];
org.elasticsearch.transport.ActionNotFoundTransportException: No handler
for action
[el-crate-import/n]"},{"node_id":"wHAaefkjSre_x6Lj8XS_6A","reason":"org.elasticsearch.action.FailedNodeException:
Failed node [wHAaefkjSre_x6Lj8XS_6A]; java.lang.OutOfMemoryError: Java heap
space"}]}lo...@pldsvv-07:/var/elasticsearch/elasticsearch-0.90.3/bin$<javascript:>Exception in thread "elasticsearch[Eric][bulk][T#2]"
java.lang.OutOfMemoryError: Java heap space

[2013-08-30 18:02:24,845][WARN ][index.engine.robin ] [Eric]
[abc_newindex][0] failed engine

Any idea what is wrong? I can increase the heap size but not sure of when
i will see the error again since i have limited RAM on the box.

On Fri, Aug 30, 2013 at 12:13 PM, parul patel <parulp...@gmail.com<javascript:>

wrote:

hey guys,

sorry false alarm. i needed to restart Elasticsearch after the plug in
was installed.
now it is working.

thanks a lot for the help. and sorry about my mistake.

On Fri, Aug 30, 2013 at 12:06 PM, parul patel <parulp...@gmail.com<javascript:>

wrote:

Thanks.
I installed it like this, it looked successful. but i don't know how to
make it detect by ES.

oc...@pldsvv-07:/var/elasticsearch/elasticsearch-0.90.3/bin/elasticsearch-inout-plugin-master$<javascript:>sudo ../plugin -i elasticsearch-inout-plugin -url
file:///$PWD/target/elasticsearch-inout-plugin-0.5.0.jar

-> Installing elasticsearch-inout-plugin...

Trying
file:////var/elasticsearch/elasticsearch-0.90.3/bin/elasticsearch-inout-plugin-master/target/elasticsearch-inout-plugin-0.5.0.jar
...

Downloading ..DONE

Installed elasticsearch-inout-plugin into
/var/elasticsearch/elasticsearch-0.90.3/plugins/elasticsearch-inout-plugin

lo...@pldsvv-07:/var/elasticsearch/elasticsearch-0.90.3/bin/elasticsearch-inout-plugin-master$<javascript:>

On Fri, Aug 30, 2013 at 11:58 AM, David Pilato <da...@pilato.fr<javascript:>

wrote:

Your plugin is not installed nor detected by elasticsearch.

HTH

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 30 août 2013 à 17:51, parul patel <parulp...@gmail.com <javascript:>>
a écrit :

Hi bernd,

when i try to use the plug in for export it gives me the following
issue, any idea what am i missing?

local@:/var/elasticsearch/elasticsearch-0.90.3/bin$
curl -X POST 'http://localhost:9200/_reindex'
{"error":"InvalidIndexNameException[[_reindex] Invalid index name
[reindex], must not start with '']","status":400}

local@:/var/elasticsearch/elasticsearch-0.90.3/bin$
curl -X POST 'http://localhost:9200/abc/_export' -d '{
"fields": ["_id", "_source", "_version", "_index", "_type"],
"output_file": "/tmp/es-data/dump-abc-${shard}.gz",
"compression": "gzip"
}
'
{"error":"InvalidTypeNameException[mapping type name [export] can't
start with '
']","status":400}

advance thanks for the help

On Thu, Aug 1, 2013 at 10:10 AM, pranav amin <parulp...@gmail.com<javascript:>

wrote:

Thank br Bernd.
Much appreciated. I will play around with this tool and let you know
if i come across any questions.
Thanks again.

If you have time and have used any Monitoring tools for ES which you
would recommend, please let me know. I had played around with some and
mentioned below.

Thanks again
Pranav.

On Thursday, August 1, 2013 12:57:23 AM UTC-4, dobe wrote:

Hi Pranav

You might want to take a look at Crate.io · GitHub**
elasticsearch-inout-pluginhttps://github.com/crate/elasticsearch-inout-pluginwe use it for backup restore. The plugin allows you to export/import json
data distributed - without having to download the data to the client.
However there is no UI for this plugin.

br Bernd

On Wednesday, July 31, 2013 4:50:56 PM UTC+2, pranav amin wrote:

Hi There,

I'm a new user for Elastic search and plan to use it in Production
environment.

Can you please help me with some information -

Monitoring tools -
I looked at the following.
a) Elasticsearch head
b) Bigdesk
c) Paramedic
d) ElasticHQ

Not sure which of the above are free and recommend by others?

Backup & Restore -
a) Can i do backup and restore of a Node in a Cluster? Are there any
free/cost based tools (UI or Script based) that can do this and are
frequently recommended by others?
b) Are there any free/cost based tools (UI or Script based) that can
do backup & restore of ACTIVE AND NON-ACTIVE cluster?

Thanks for the help and advice.
Much appreciated.

Thanks
Pranav.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Pf1tLdAtIp8/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com <javascript:>.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Pf1tLdAtIp8/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi,
I am having some problems with the knapsack plugin. I have managed to
export the data for an index into the tar.gz file using the _export
feature. However when trying to import the data into another index, i can
only see the index being created. No data appears to have been imported.

I can see the following message in the elasticsearch logs when i run the
export command.
starting import of test2
import of test2 completed

Any ideas on what might be happeing?

On Wednesday, September 4, 2013 4:42:38 PM UTC+1, Jörg Prante wrote:

You can try my knapsack plugin which has a "size" URL parameter to control
the bulk export.

GitHub - jprante/elasticsearch-knapsack: Knapsack plugin is an import/export tool for Elasticsearch

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Please can you open an issue on knapsack github?

And please, do not capture old threads.

Thanks

Jörg
Am 14.10.2013 16:52 schrieb "V" vmnadig@gmail.com:

Hi,
I am having some problems with the knapsack plugin. I have managed to
export the data for an index into the tar.gz file using the _export
feature. However when trying to import the data into another index, i can
only see the index being created. No data appears to have been imported.

I can see the following message in the elasticsearch logs when i run the
export command.
starting import of test2
import of test2 completed

Any ideas on what might be happeing?

On Wednesday, September 4, 2013 4:42:38 PM UTC+1, Jörg Prante wrote:

You can try my knapsack plugin which has a "size" URL parameter to
control the bulk export.

https://github.com/jprante/**elasticsearch-knapsackhttps://github.com/jprante/elasticsearch-knapsack

Jörg

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.