Memory problem ( [WARN ][monitor.jvm] .... )?


(Tanguy Bernard) #1

Hello
I try to index data in ElasticSearch, everything find until this [WARN
][monitor.jvm] .... ?
This is the first time it happens to me.
Can you help me to solve my problem ?

PUT /my_index/
{
"mappings" : {
"source" : {
"properties" : {

        "id_source":{
            "type":"string"
        },
        "adresse_source":{
            "type":"string",
            "search_analyzer" : "str_search_analyzer",
            "index_analyzer" : "str_index_analyzer"
        },
         "type_source":{
            "type":"string"
        },
        
    "titre_source" : {
            "type" : "string",
            "search_analyzer" : "str_search_analyzer",
            "index_analyzer" : "str_index_analyzer"
            
    },
        "note_source":{
            "type":"string",
            "search_analyzer" : "str_search_analyzer",
            "index_analyzer" : "str_index_analyzer"
        },
        "desc_source":{
            "type" : "string",
          
            "search_analyzer" : "str_search_analyzer",
            "index_analyzer" : "str_index_analyzer"
            
            
        }

}
}
},

"settings" : {
"analysis" : {
"analyzer" : {
"str_search_analyzer" : {

      "tokenizer" : "standard",
      "filter" : ["lowercase", "asciifolding"]
    },

    "str_index_analyzer" : {
      "tokenizer" : "standard",
      "filter" : ["lowercase", "substring","asciifolding"]
    }
  },

  "filter" : {

    "substring" : {
        "token_chars" :[],
      "type" : "nGram",
      "min_gram" : 3,
      "max_gram"  : 250

}

  }
}

}
}

{
"acknowledged": true
}

PUT /_river/source/_meta
{
"type" : "jdbc",
"jdbc" : {

"url" : "jdbc:mysql:/192.168.50.62:9200/my_index",
"user" : "user",
"password" : "my-password",
"sql" : "select id_source, titre_source, desc_source, note_source, 

adresse_source, type_source from source",
"index" : "my_index",
"type" : "source",
"max_bulk_requests" : 5

}
}

[2014-05-21 10:21:27,134][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [1] of [99 items], 1 outstanding bulk requests
[2014-05-21 10:21:28,813][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [2] of [99 items], 2 outstanding bulk requests
[2014-05-21 10:21:31,518][WARN ][monitor.jvm ] [ik-test1]
[gc][young][179][4] duration [1.7s], collections [1]/[3.2s], total
[1.7s]/[5.5s], memory [78.5mb]->[26.7mb]/[1015.6mb], all_pools {[young]
[54.9mb]->[1.1mb]/[66.5mb]}{[survivor] [8.2mb]->[7.7mb]/[8.3mb]}{[old]
[15.4mb]->[17.8mb]/[940.8mb]}
[2014-05-21 10:21:31,511][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [3] of [4 items], 3 outstanding bulk requests
[2014-05-21 10:21:31,844][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [4] of [99 items], 4 outstanding bulk requests
[2014-05-21 10:21:32,159][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [5] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:33,853][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[1] success [99 items] [6707ms]
[2014-05-21 10:21:33,853][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [6] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:36,575][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[2] success [99 items] [7762ms]
[2014-05-21 10:21:36,584][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [7] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:36,707][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[3] success [4 items] [5172ms]
[2014-05-21 10:21:36,886][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [8] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:37,993][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[4] success [99 items] [6149ms]
[2014-05-21 10:21:37,999][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [9] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:38,576][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[5] success [99 items] [6416ms]
[2014-05-21 10:21:38,579][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [10] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:39,451][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[6] success [99 items] [5598ms]
[2014-05-21 10:21:39,452][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [11] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:41,390][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[7] success [99 items] [4805ms]
[2014-05-21 10:21:41,390][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [12] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:42,751][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[8] success [99 items] [5865ms]
[2014-05-21 10:21:42,755][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [13] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:43,617][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[9] success [99 items] [5618ms]
[2014-05-21 10:21:43,620][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [14] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:43,650][WARN ][discovery.zen.ping.multicast] [ik-test1]
received ping response ping_response{target
[[ik-test1][zS3-5g1vRme4Gl4beRJvFQ][inet[/192.168.50.62:9300]]], master
[[ik-test1][zS3-5g1vRme4Gl4beRJvFQ][inet[/192.168.50.62:9300]]],
cluster_name[elasticsearch]} with no matching id [1]
[2014-05-21 10:21:43,650][WARN ][discovery.zen.ping.multicast] [ik-test1]
received ping response ping_response{target
[[ik-test1][zS3-5g1vRme4Gl4beRJvFQ][inet[/192.168.50.62:9300]]], master
[[ik-test1][zS3-5g1vRme4Gl4beRJvFQ][inet[/192.168.50.62:9300]]],
cluster_name[elasticsearch]} with no matching id [1]
[2014-05-21 10:21:44,125][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[10] success [99 items] [5546ms]
[2014-05-21 10:21:44,130][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [15] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:45,360][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[11] success [99 items] [5907ms]
[2014-05-21 10:21:45,363][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [16] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:45,897][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[12] success [99 items] [4507ms]
[2014-05-21 10:21:45,897][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [17] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:58,592][WARN ][monitor.jvm ] [ik-test1]
[gc][young][204][32] duration [1.1s], collections [1]/[1.3s], total
[1.1s]/[10.7s], memory [363.8mb]->[336.1mb]/[1015.6mb], all_pools {[young]
[62.3mb]->[34.9kb]/[66.5mb]}{[survivor] [8.3mb]->[7mb]/[8.3mb]}{[old]
[293.1mb]->[329mb]/[940.8mb]}
[2014-05-21 10:22:08,913][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[13] success [99 items] [26158ms]
[2014-05-21 10:22:09,308][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [18] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:22:10,761][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[14] success [99 items] [27141ms]
[2014-05-21 10:22:10,762][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [19] of [99 items], 5 outstanding bulk requests

[2014-05-21 10:23:37,104][WARN ][monitor.jvm ] [ik-test1]
[gc][young][263][115] duration [14s], collections [1]/[14.9s], total
[14s]/[52.3s], memory [652mb]->[652.6mb]/[1015.6mb], all_pools {[young]
[65.2mb]->[232.1kb]/[66.5mb]}{[survivor] [6.4kb]->[8.2mb]/[8.3mb]}{[old]
[586.8mb]->[644.1mb]/[940.8mb]}
[2014-05-21 10:23:48,187][WARN ][monitor.jvm ] [ik-test1]
[gc][young][268][116] duration [6.8s], collections [1]/[7s], total
[6.8s]/[59.1s], memory [660.8mb]->[766mb]/[1015.6mb], all_pools {[young]
[8.3mb]->[52.5mb]/[66.5mb]}{[survivor] [8.2mb]->[8.2mb]/[8.3mb]}{[old]
[644.1mb]->[705.1mb]/[940.8mb]}
[2014-05-21 10:23:56,362][WARN ][monitor.jvm ] [ik-test1]
[gc][young][269][117] duration [7.9s], collections [1]/[8.1s], total
[7.9s]/[1.1m], memory [766mb]->[783.5mb]/[1015.6mb], all_pools {[young]
[52.5mb]->[17.5mb]/[66.5mb]}{[survivor] [8.2mb]->[2.5kb]/[8.3mb]}{[old]
[705.1mb]->[766mb]/[940.8mb]}
[2014-05-21 10:24:05,097][WARN ][monitor.jvm ] [ik-test1]
[gc][young][276][119] duration [2.4s], collections [1]/[2.7s], total
[2.4s]/[1.1m], memory [854.9mb]->[857mb]/[1015.6mb], all_pools {[young]
[64.4mb]->[34.1kb]/[66.5mb]}{[survivor] [6.9mb]->[2.5mb]/[8.3mb]}{[old]
[783.5mb]->[854.5mb]/[940.8mb]}

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ba369382-5551-4c49-8c99-3b10f1716cb9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Mark Walkom) #2

How much heap are you running with, from what I can tell it's around a gig?

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 21 May 2014 18:27, Tanguy Bernard bernardtanguy1pro@gmail.com wrote:

Hello
I try to index data in ElasticSearch, everything find until this [WARN
][monitor.jvm] .... ?
This is the first time it happens to me.
Can you help me to solve my problem ?

PUT /my_index/
{
"mappings" : {
"source" : {
"properties" : {

        "id_source":{
            "type":"string"
        },
        "adresse_source":{
            "type":"string",
            "search_analyzer" : "str_search_analyzer",
            "index_analyzer" : "str_index_analyzer"
        },
         "type_source":{
            "type":"string"
        },

    "titre_source" : {
            "type" : "string",
            "search_analyzer" : "str_search_analyzer",
            "index_analyzer" : "str_index_analyzer"

    },
        "note_source":{
            "type":"string",
            "search_analyzer" : "str_search_analyzer",
            "index_analyzer" : "str_index_analyzer"
        },
        "desc_source":{
            "type" : "string",

            "search_analyzer" : "str_search_analyzer",
            "index_analyzer" : "str_index_analyzer"


        }

}
}
},

"settings" : {
"analysis" : {
"analyzer" : {
"str_search_analyzer" : {

      "tokenizer" : "standard",
      "filter" : ["lowercase", "asciifolding"]
    },

    "str_index_analyzer" : {
      "tokenizer" : "standard",
      "filter" : ["lowercase", "substring","asciifolding"]
    }
  },

  "filter" : {

    "substring" : {
        "token_chars" :[],
      "type" : "nGram",
      "min_gram" : 3,
      "max_gram"  : 250

}

  }
}

}
}

{
"acknowledged": true
}

PUT /_river/source/_meta
{
"type" : "jdbc",
"jdbc" : {

"url" : "jdbc:mysql:/192.168.50.62:9200/my_index",
"user" : "user",
"password" : "my-password",
"sql" : "select id_source, titre_source, desc_source, note_source,

adresse_source, type_source from source",
"index" : "my_index",
"type" : "source",
"max_bulk_requests" : 5

}
}

[2014-05-21 10:21:27,134][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [1] of [99 items], 1 outstanding bulk requests
[2014-05-21 10:21:28,813][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [2] of [99 items], 2 outstanding bulk requests
[2014-05-21 10:21:31,518][WARN ][monitor.jvm ] [ik-test1]
[gc][young][179][4] duration [1.7s], collections [1]/[3.2s], total
[1.7s]/[5.5s], memory [78.5mb]->[26.7mb]/[1015.6mb], all_pools {[young]
[54.9mb]->[1.1mb]/[66.5mb]}{[survivor] [8.2mb]->[7.7mb]/[8.3mb]}{[old]
[15.4mb]->[17.8mb]/[940.8mb]}
[2014-05-21 10:21:31,511][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [3] of [4 items], 3 outstanding bulk requests
[2014-05-21 10:21:31,844][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [4] of [99 items], 4 outstanding bulk requests
[2014-05-21 10:21:32,159][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [5] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:33,853][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[1] success [99 items] [6707ms]
[2014-05-21 10:21:33,853][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [6] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:36,575][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[2] success [99 items] [7762ms]
[2014-05-21 10:21:36,584][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [7] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:36,707][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[3] success [4 items] [5172ms]
[2014-05-21 10:21:36,886][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [8] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:37,993][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[4] success [99 items] [6149ms]
[2014-05-21 10:21:37,999][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [9] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:38,576][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[5] success [99 items] [6416ms]
[2014-05-21 10:21:38,579][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [10] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:39,451][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[6] success [99 items] [5598ms]
[2014-05-21 10:21:39,452][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [11] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:41,390][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[7] success [99 items] [4805ms]
[2014-05-21 10:21:41,390][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [12] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:42,751][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[8] success [99 items] [5865ms]
[2014-05-21 10:21:42,755][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [13] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:43,617][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[9] success [99 items] [5618ms]
[2014-05-21 10:21:43,620][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [14] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:43,650][WARN ][discovery.zen.ping.multicast] [ik-test1]
received ping response ping_response{target
[[ik-test1][zS3-5g1vRme4Gl4beRJvFQ][inet[/192.168.50.62:9300]]], master
[[ik-test1][zS3-5g1vRme4Gl4beRJvFQ][inet[/192.168.50.62:9300]]],
cluster_name[elasticsearch]} with no matching id [1]
[2014-05-21 10:21:43,650][WARN ][discovery.zen.ping.multicast] [ik-test1]
received ping response ping_response{target
[[ik-test1][zS3-5g1vRme4Gl4beRJvFQ][inet[/192.168.50.62:9300]]], master
[[ik-test1][zS3-5g1vRme4Gl4beRJvFQ][inet[/192.168.50.62:9300]]],
cluster_name[elasticsearch]} with no matching id [1]
[2014-05-21 10:21:44,125][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[10] success [99 items] [5546ms]
[2014-05-21 10:21:44,130][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [15] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:45,360][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[11] success [99 items] [5907ms]
[2014-05-21 10:21:45,363][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [16] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:45,897][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[12] success [99 items] [4507ms]
[2014-05-21 10:21:45,897][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [17] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:58,592][WARN ][monitor.jvm ] [ik-test1]
[gc][young][204][32] duration [1.1s], collections [1]/[1.3s], total
[1.1s]/[10.7s], memory [363.8mb]->[336.1mb]/[1015.6mb], all_pools {[young]
[62.3mb]->[34.9kb]/[66.5mb]}{[survivor] [8.3mb]->[7mb]/[8.3mb]}{[old]
[293.1mb]->[329mb]/[940.8mb]}
[2014-05-21 10:22:08,913][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[13] success [99 items] [26158ms]
[2014-05-21 10:22:09,308][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [18] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:22:10,761][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[14] success [99 items] [27141ms]
[2014-05-21 10:22:10,762][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [19] of [99 items], 5 outstanding bulk requests

[2014-05-21 10:23:37,104][WARN ][monitor.jvm ] [ik-test1]
[gc][young][263][115] duration [14s], collections [1]/[14.9s], total
[14s]/[52.3s], memory [652mb]->[652.6mb]/[1015.6mb], all_pools {[young]
[65.2mb]->[232.1kb]/[66.5mb]}{[survivor] [6.4kb]->[8.2mb]/[8.3mb]}{[old]
[586.8mb]->[644.1mb]/[940.8mb]}
[2014-05-21 10:23:48,187][WARN ][monitor.jvm ] [ik-test1]
[gc][young][268][116] duration [6.8s], collections [1]/[7s], total
[6.8s]/[59.1s], memory [660.8mb]->[766mb]/[1015.6mb], all_pools {[young]
[8.3mb]->[52.5mb]/[66.5mb]}{[survivor] [8.2mb]->[8.2mb]/[8.3mb]}{[old]
[644.1mb]->[705.1mb]/[940.8mb]}
[2014-05-21 10:23:56,362][WARN ][monitor.jvm ] [ik-test1]
[gc][young][269][117] duration [7.9s], collections [1]/[8.1s], total
[7.9s]/[1.1m], memory [766mb]->[783.5mb]/[1015.6mb], all_pools {[young]
[52.5mb]->[17.5mb]/[66.5mb]}{[survivor] [8.2mb]->[2.5kb]/[8.3mb]}{[old]
[705.1mb]->[766mb]/[940.8mb]}
[2014-05-21 10:24:05,097][WARN ][monitor.jvm ] [ik-test1]
[gc][young][276][119] duration [2.4s], collections [1]/[2.7s], total
[2.4s]/[1.1m], memory [854.9mb]->[857mb]/[1015.6mb], all_pools {[young]
[64.4mb]->[34.1kb]/[66.5mb]}{[survivor] [6.9mb]->[2.5mb]/[8.3mb]}{[old]
[783.5mb]->[854.5mb]/[940.8mb]}

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ba369382-5551-4c49-8c99-3b10f1716cb9%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/ba369382-5551-4c49-8c99-3b10f1716cb9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624YyZ1Np8%3DExu%3DRYU_9L0fFoZYdBJrgeQPTDL-EqneS1zA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Tanguy Bernard) #3

I add about 100mo.
I try to set ES_HEAP_SIZE=8g and I have got an error java something like
not enough memory

Le mercredi 21 mai 2014 10:55:22 UTC+2, Mark Walkom a écrit :

How much heap are you running with, from what I can tell it's around a gig?

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com <javascript:>
web: www.campaignmonitor.com

On 21 May 2014 18:27, Tanguy Bernard <bernardt...@gmail.com <javascript:>>wrote:

Hello
I try to index data in ElasticSearch, everything find until this [WARN
][monitor.jvm] .... ?
This is the first time it happens to me.
Can you help me to solve my problem ?

PUT /my_index/
{
"mappings" : {
"source" : {
"properties" : {

        "id_source":{
            "type":"string"
        },
        "adresse_source":{
            "type":"string",
            "search_analyzer" : "str_search_analyzer",
            "index_analyzer" : "str_index_analyzer"
        },
         "type_source":{
            "type":"string"
        },
        
    "titre_source" : {
            "type" : "string",
            "search_analyzer" : "str_search_analyzer",
            "index_analyzer" : "str_index_analyzer"
            
    },
        "note_source":{
            "type":"string",
            "search_analyzer" : "str_search_analyzer",
            "index_analyzer" : "str_index_analyzer"
        },
        "desc_source":{
            "type" : "string",
          
            "search_analyzer" : "str_search_analyzer",
            "index_analyzer" : "str_index_analyzer"
            
            
        }

}
}
},

"settings" : {
"analysis" : {
"analyzer" : {
"str_search_analyzer" : {

      "tokenizer" : "standard",
      "filter" : ["lowercase", "asciifolding"]
    },

    "str_index_analyzer" : {
      "tokenizer" : "standard",
      "filter" : ["lowercase", "substring","asciifolding"]
    }
  },

  "filter" : {

    "substring" : {
        "token_chars" :[],
      "type" : "nGram",
      "min_gram" : 3,
      "max_gram"  : 250

}

  }
}

}
}

{
"acknowledged": true
}

PUT /_river/source/_meta
{
"type" : "jdbc",
"jdbc" : {

"url" : "jdbc:mysql:/192.168.50.62:9200/my_index",
"user" : "user",
"password" : "my-password",
"sql" : "select id_source, titre_source, desc_source, note_source, 

adresse_source, type_source from source",
"index" : "my_index",
"type" : "source",
"max_bulk_requests" : 5

}
}

[2014-05-21 10:21:27,134][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [1] of [99 items], 1 outstanding bulk requests
[2014-05-21 10:21:28,813][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [2] of [99 items], 2 outstanding bulk requests
[2014-05-21 10:21:31,518][WARN ][monitor.jvm ] [ik-test1]
[gc][young][179][4] duration [1.7s], collections [1]/[3.2s], total
[1.7s]/[5.5s], memory [78.5mb]->[26.7mb]/[1015.6mb], all_pools {[young]
[54.9mb]->[1.1mb]/[66.5mb]}{[survivor] [8.2mb]->[7.7mb]/[8.3mb]}{[old]
[15.4mb]->[17.8mb]/[940.8mb]}
[2014-05-21 10:21:31,511][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [3] of [4 items], 3 outstanding bulk requests
[2014-05-21 10:21:31,844][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [4] of [99 items], 4 outstanding bulk requests
[2014-05-21 10:21:32,159][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [5] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:33,853][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[1] success [99 items] [6707ms]
[2014-05-21 10:21:33,853][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [6] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:36,575][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[2] success [99 items] [7762ms]
[2014-05-21 10:21:36,584][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [7] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:36,707][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[3] success [4 items] [5172ms]
[2014-05-21 10:21:36,886][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [8] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:37,993][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[4] success [99 items] [6149ms]
[2014-05-21 10:21:37,999][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [9] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:38,576][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[5] success [99 items] [6416ms]
[2014-05-21 10:21:38,579][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [10] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:39,451][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[6] success [99 items] [5598ms]
[2014-05-21 10:21:39,452][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [11] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:41,390][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[7] success [99 items] [4805ms]
[2014-05-21 10:21:41,390][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [12] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:42,751][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[8] success [99 items] [5865ms]
[2014-05-21 10:21:42,755][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [13] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:43,617][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[9] success [99 items] [5618ms]
[2014-05-21 10:21:43,620][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [14] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:43,650][WARN ][discovery.zen.ping.multicast] [ik-test1]
received ping response ping_response{target
[[ik-test1][zS3-5g1vRme4Gl4beRJvFQ][inet[/192.168.50.62:9300]]], master
[[ik-test1][zS3-5g1vRme4Gl4beRJvFQ][inet[/192.168.50.62:9300]]],
cluster_name[elasticsearch]} with no matching id [1]
[2014-05-21 10:21:43,650][WARN ][discovery.zen.ping.multicast] [ik-test1]
received ping response ping_response{target
[[ik-test1][zS3-5g1vRme4Gl4beRJvFQ][inet[/192.168.50.62:9300]]], master
[[ik-test1][zS3-5g1vRme4Gl4beRJvFQ][inet[/192.168.50.62:9300]]],
cluster_name[elasticsearch]} with no matching id [1]
[2014-05-21 10:21:44,125][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[10] success [99 items] [5546ms]
[2014-05-21 10:21:44,130][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [15] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:45,360][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[11] success [99 items] [5907ms]
[2014-05-21 10:21:45,363][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [16] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:45,897][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[12] success [99 items] [4507ms]
[2014-05-21 10:21:45,897][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [17] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:21:58,592][WARN ][monitor.jvm ] [ik-test1]
[gc][young][204][32] duration [1.1s], collections [1]/[1.3s], total
[1.1s]/[10.7s], memory [363.8mb]->[336.1mb]/[1015.6mb], all_pools {[young]
[62.3mb]->[34.9kb]/[66.5mb]}{[survivor] [8.3mb]->[7mb]/[8.3mb]}{[old]
[293.1mb]->[329mb]/[940.8mb]}
[2014-05-21 10:22:08,913][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[13] success [99 items] [26158ms]
[2014-05-21 10:22:09,308][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [18] of [99 items], 5 outstanding bulk requests
[2014-05-21 10:22:10,761][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] bulk
[14] success [99 items] [27141ms]
[2014-05-21 10:22:10,762][INFO
][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverMouth] new
bulk [19] of [99 items], 5 outstanding bulk requests

[2014-05-21 10:23:37,104][WARN ][monitor.jvm ] [ik-test1]
[gc][young][263][115] duration [14s], collections [1]/[14.9s], total
[14s]/[52.3s], memory [652mb]->[652.6mb]/[1015.6mb], all_pools {[young]
[65.2mb]->[232.1kb]/[66.5mb]}{[survivor] [6.4kb]->[8.2mb]/[8.3mb]}{[old]
[586.8mb]->[644.1mb]/[940.8mb]}
[2014-05-21 10:23:48,187][WARN ][monitor.jvm ] [ik-test1]
[gc][young][268][116] duration [6.8s], collections [1]/[7s], total
[6.8s]/[59.1s], memory [660.8mb]->[766mb]/[1015.6mb], all_pools {[young]
[8.3mb]->[52.5mb]/[66.5mb]}{[survivor] [8.2mb]->[8.2mb]/[8.3mb]}{[old]
[644.1mb]->[705.1mb]/[940.8mb]}
[2014-05-21 10:23:56,362][WARN ][monitor.jvm ] [ik-test1]
[gc][young][269][117] duration [7.9s], collections [1]/[8.1s], total
[7.9s]/[1.1m], memory [766mb]->[783.5mb]/[1015.6mb], all_pools {[young]
[52.5mb]->[17.5mb]/[66.5mb]}{[survivor] [8.2mb]->[2.5kb]/[8.3mb]}{[old]
[705.1mb]->[766mb]/[940.8mb]}
[2014-05-21 10:24:05,097][WARN ][monitor.jvm ] [ik-test1]
[gc][young][276][119] duration [2.4s], collections [1]/[2.7s], total
[2.4s]/[1.1m], memory [854.9mb]->[857mb]/[1015.6mb], all_pools {[young]
[64.4mb]->[34.1kb]/[66.5mb]}{[survivor] [6.9mb]->[2.5mb]/[8.3mb]}{[old]
[783.5mb]->[854.5mb]/[940.8mb]}

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ba369382-5551-4c49-8c99-3b10f1716cb9%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/ba369382-5551-4c49-8c99-3b10f1716cb9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/75a0cb22-dae5-44ef-a561-9cbe78ddafc8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Tanguy Bernard) #4

I have try an other mapping, and everything fine, it's work.
Someone can explain me, why it's work when I remove "type_source" and
"note_source" of my mapping ?

My new mapping (miss type_source, note_source)
PUT /my_index
{
"mappings" : {
"source" : {
"properties" : {

        "id_source":{
            "type":"string"
        },
        "adresse_source":{
            "type":"string"
        },
        
    "titre_source" : {
            "type" : "string",
            "search_analyzer" : "str_search_analyzer",
            "index_analyzer" : "str_index_analyzer"
            
    },
        "desc_source":{
            "type" : "string",
          
            "search_analyzer" : "str_search_analyzer",
            "index_analyzer" : "str_index_analyzer"
            
            
        }

}
}
},

"settings" : {
"analysis" : {
"analyzer" : {
"str_search_analyzer" : {

      "tokenizer" : "standard",
      "filter" : ["lowercase", "asciifolding"]
    },

    "str_index_analyzer" : {
      "tokenizer" : "standard",
      "filter" : ["lowercase", "substring","asciifolding"]
    }
  },

  "filter" : {

    "substring" : {
        "token_chars" :[],
      "type" : "nGram",
      "min_gram" : 3,
      "max_gram"  : 250
    } 
         
          
          
  }
}

}
}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d3836b12-c063-4778-a666-3bd8ea1bf21c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Tanguy Bernard) #5

I found the solution of my problem.
My note_source contained base64 picture and my nGram is too large, so the
indexing takes a lot of time. Conclusion : I got [WARN ][monitor.jvm].

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/98c74acb-45b9-4a0c-9d57-2e694d8ea76e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #6