Hello folks
I'm running a mac osx 10.6.4 and we are using elastic search
{elasticsearch/0.9.0}[211]: started
We are programmatically building 2 indexes with (sub) types via groovy
client (and the server has been started separately)
Everything is hunky dory while the system is running
I can shut it the terminal window down with ctrl-c or passing the
shutdown all curl command and restart using elasticsearch -f
The problem is when I bounce my system and restart the server it
indexes are missing
I believe the gateway is set to FS, but the store is set to memory
After bounce I do see some stuff in my work directory
/usr/local/elasticsearch-0.9.0/work/Ticketfly-1/nodes/0/indices/
backstage/1/index
GIDEON-KAPLANs-MacBook-Pro:index gideon$ ls
_0.cfs _0_1.del segments.gen segments_2
But it's looks like it's missing the indices directory in the gateway
fs store /tmp/elasticsearch/data/cluster
after system bounce (the below dir is recreated when I rebuild the
indexes programatically)
/tmp/elasticsearch/data/cluster/Ticketfly-1/indices
This is before rebuilding
GIDEON-KAPLANs-MacBook-Pro:metadata gideon$ cat metadata-0
{
"meta-data" : {
"indices" : {
}
}
}GIDEON-KAPLANs-MacBook-Pro:metadata gideon$ pwd
/tmp/elasticsearch/data/cluster/Ticketfly-1/metadata
This is after
GIDEON-KAPLANs-MacBook-Pro:metadata gideon$ cat metadata-5 (metadata-5
is the only file there)
{
"meta-data" : {
"indices" : {
"inventory" : {
"settings" : {
"index.number_of_shards" : "5",
"index.number_of_replicas" : "4"
},
"mappings" : [ {
.....
I do see them on the file system though, maybe I'm just doing
something wrong or is there a way to tell it to rediscover the
existing indexes on startup or not delete them on shutdown (what I
think it's doing)
Does one have to set the store to fs in the yml file?
Sorry if this is a dumb question, I'm a bit new at using this tech
my yml file
name: SearchDevelopmentInstance
cluster:
name: Ticketfly-1
node:
data: true
http:
enabled: true
network:
#bind_host: 0.0.0.0
#publish_host: eth1
host: 127.0.0.1
gateway:
type: fs
fs:
location: /tmp/elasticsearch/data/cluster
index :
number_of_shards : 5
number_of_replicas : 4
analysis :
analyzer :
standard :
type : standard
store:
type: memory
memory:
cache_size: 100m
buffer_size: 10k
#transport:
tcp:
port: 9300
startup after bounce
GIDEON-KAPLANs-MacBook-Pro:bin gideon$ elasticsearch -f
[11:41:56,616][INFO ][node ]
[SearchDevelopmentInstance] {elasticsearch/0.9.0}[211]:
initializing ...
[11:41:56,635][INFO ][plugins ]
[SearchDevelopmentInstance] loaded []
[11:41:58,224][DEBUG][cache.memory ]
[SearchDevelopmentInstance] using bytebuffer cache with buffer_size
[100kb], cache_size [200mb], direct [true], warm_cache [false]
[11:41:58,272][DEBUG][threadpool.cached ]
[SearchDevelopmentInstance] Initializing cached thread pool with
keep_alive[1m], scheduled_size[20]
[11:41:58,317][DEBUG][discovery.zen.ping.multicast]
[SearchDevelopmentInstance] using group [224.2.2.4], with port
[54328], ttl [3], and address [null]
[11:41:58,323][DEBUG][discovery.zen.ping.unicast]
[SearchDevelopmentInstance] using initial hosts []
[11:41:58,333][DEBUG][discovery.zen ]
[SearchDevelopmentInstance] using initial_ping_timeout [3s]
[11:41:58,335][DEBUG][discovery.zen.fd ]
[SearchDevelopmentInstance] [master] uses ping_interval [1s],
ping_timeout [30s], ping_retries [3]
[11:41:58,342][DEBUG][discovery.zen.fd ]
[SearchDevelopmentInstance] [node ] uses ping_interval [1s],
ping_timeout [30s], ping_retries [3]
[11:41:58,410][DEBUG][env ]
[SearchDevelopmentInstance] using node location [/usr/local/
elasticsearch-0.9.0/work/Ticketfly-1/nodes/0]
[11:41:58,504][DEBUG][monitor.memory.alpha ]
[SearchDevelopmentInstance] interval [500ms], upper_memory_threshold
[0.95], lower_memory_threshold [0.8],
translog_number_of_operations_threshold [5000]
[11:41:58,536][DEBUG][monitor.network ]
[SearchDevelopmentInstance] net_info
host [GIDEON-KAPLANs-MacBook-Pro.local]
en1 display_name [en1]
address [/10.59.15.40] [/
fe80:0:0:0:cabc:c8ff:fed9:a56%5]
mtu [1500] multicast [true] ptp [false] loopback
[false] up [true] virtual [false]
en0 display_name [en0]
address [/10.56.10.139] [/
fe80:0:0:0:cabc:c8ff:fe96:9069%4]
mtu [1500] multicast [true] ptp [false] loopback
[false] up [true] virtual [false]
lo0 display_name [lo0]
address [/127.0.0.1] [/fe80:0:0:0:0:0:0:1%1] [/
0:0:0:0:0:0:0:1]
mtu [16384] multicast [true] ptp [false] loopback
[true] up [true] virtual [false]
[11:41:58,626][DEBUG][indices.recovery.throttler]
[SearchDevelopmentInstance] concurrent_recoveries [4],
concurrent_streams [4] interval [100ms]
[11:41:58,631][DEBUG][indices.memory ]
[SearchDevelopmentInstance] using index_buffer_size [406.2mb], with
min_shard_index_buffer_size [4mb]
[11:41:58,636][DEBUG][gateway.fs ]
[SearchDevelopmentInstance] Latest metadata found at index [-1]
[11:41:58,637][INFO ][node ]
[SearchDevelopmentInstance] {elasticsearch/0.9.0}[211]: initialized
[11:41:58,637][INFO ][node ]
[SearchDevelopmentInstance] {elasticsearch/0.9.0}[211]: starting ...
[11:41:58,698][DEBUG][netty.channel.socket.nio.NioProviderMetadata]
Using the autodetected NIO constraint level: 0
[11:41:58,740][DEBUG][transport.netty ]
[SearchDevelopmentInstance] Bound to address [/127.0.0.1:9300]
[11:41:58,741][INFO ][transport ]
[SearchDevelopmentInstance] bound_address {inet[/127.0.0.1:9300]},
publish_address {inet[/127.0.0.1:9300]}
[11:42:01,865][DEBUG][discovery.zen ]
[SearchDevelopmentInstance] ping responses: {none}
[11:42:01,870][DEBUG][cluster.service ]
[SearchDevelopmentInstance] processing [zen-disco-join
(elected_as_master)]: execute
[11:42:01,871][DEBUG][cluster.service ]
[SearchDevelopmentInstance] cluster state updated, version [1], source
[zen-disco-join (elected_as_master)]
[11:42:01,872][INFO ][cluster.service ]
[SearchDevelopmentInstance] new_master [SearchDevelopmentInstance]
[3d1d41e1-4846-4024-8738-230fbd6927b0][inet[/127.0.0.1:9300]], reason:
zen-disco-join (elected_as_master)
[11:42:02,014][DEBUG][transport.netty ]
[SearchDevelopmentInstance] Connected to node
[[SearchDevelopmentInstance][3d1d41e1-4846-4024-8738-230fbd6927b0]
[inet[/127.0.0.1:9300]]]
[11:42:02,016][DEBUG][cluster.service ]
[SearchDevelopmentInstance] processing [zen-disco-join
(elected_as_master)]: done applying updated cluster_state
[11:42:02,016][INFO ][discovery ]
[SearchDevelopmentInstance]
Ticketfly-1/3d1d41e1-4846-4024-8738-230fbd6927b0
[11:42:02,016][DEBUG][gateway ]
[SearchDevelopmentInstance] reading state from gateway
fs:///tmp/elasticsearch/data/cluster/[Ticketfly-1] ...
[11:42:02,017][DEBUG][gateway ]
[SearchDevelopmentInstance] read state from gateway
fs:///tmp/elasticsearch/data/cluster/[Ticketfly-1], took 0s
[11:42:02,017][DEBUG][gateway ]
[SearchDevelopmentInstance] no state read from gateway
[11:42:02,018][DEBUG][cluster.service ]
[SearchDevelopmentInstance] processing [gateway (marked as read,
reason=no state)]: execute
[11:42:02,019][DEBUG][cluster.service ]
[SearchDevelopmentInstance] cluster state updated, version [2], source
[gateway (marked as read, reason=no state)]
[11:42:02,020][DEBUG][gateway ]
[SearchDevelopmentInstance] writing to gateway
fs:///tmp/elasticsearch/data/cluster/[Ticketfly-1] ...
[11:42:02,020][DEBUG][cluster.service ]
[SearchDevelopmentInstance] processing [gateway (marked as read,
reason=no state)]: done applying updated cluster_state
[11:42:02,023][INFO ][http ]
[SearchDevelopmentInstance] bound_address {inet[/127.0.0.1:9200]},
publish_address {inet[/127.0.0.1:9200]}
[11:42:02,172][DEBUG][gateway ]
[SearchDevelopmentInstance] wrote to gateway fs:///tmp/elasticsearch/data/cluster/[Ticketfly-1],
took 152ms
[11:42:02,236][INFO ][jmx ]
[SearchDevelopmentInstance] bound_address {service:jmx:rmi:///jndi/
rmi://:9400/jmxrmi}, publish_address {service:jmx:rmi:///jndi/rmi://
127.0.0.1:9400/jmxrmi}
[11:42:02,236][INFO ][node ]
[SearchDevelopmentInstance] {elasticsearch/0.9.0}[211]: started
[11:42:12,016][DEBUG][cluster.service ]
[SearchDevelopmentInstance] processing [routing-table-updater]:
execute
[11:42:12,018][DEBUG][cluster.service ]
[SearchDevelopmentInstance] processing [routing-table-updater]: no
change in cluster_state
query to get a document I know was there before shutdown
GIDEON-KAPLANs-MacBook-Pro:~ gideon$ curl -XGET 'http://localhost:9200/
backstage/event/11637'
{"error":"IndexMissingException[[backstage] missing]"}GIDEON-KAPLANs-
MacBook-Pro: