the same settings, mapping , query, and the data. all of things is the same, except es version.
the follow query cost 400+ms in es5, but cost 600+ms in es7.
we compare the profile, had found that:
- term query ( terms also) of numeric in es 7 is much slow than es5.
- match query in es 7 is better than es5.
how we should improve in es7 ??
some body suggest change numeric type to keyword, but it's not take effect.
Looking forward to your reply, thanks
settings (partially):
"analysis" : {
"filter" : {
"my_shingle_filter" : {
"max_shingle_size" : "2",
"min_shingle_size" : "2",
"output_unigrams" : "false",
"type" : "shingle"
}
},
"analyzer" : {
"my_shingle_analyzer" : {
"filter" : [
"my_shingle_filter",
"lowercase"
],
"type" : "custom",
"tokenizer" : "whitespace"
}
}
},
mapping:
"mappings" : {
"properties" : {
"ai_tags" : {
"type" : "long"
},
"album_meta" : {
"properties" : {
"episode_one" : {
"type" : "long"
},
"tags" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"title" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"type" : {
"type" : "long"
}
}
},
"chosen" : {
"type" : "boolean"
},
"comment_count" : {
"type" : "long"
},
"create_time" : {
"type" : "long"
},
"create_type" : {
"type" : "long"
},
"digg_count" : {
"type" : "long"
},
"favorite_count" : {
"type" : "long"
},
"go_detail_count" : {
"type" : "long"
},
"hashtags" : {
"type" : "keyword"
}
},
"heat" : {
"type" : "integer"
},
"human_tags" : {
"type" : "long"
},
"id" : {
"type" : "long"
},
"impr_count" : {
"type" : "long"
},
"language" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"level" : {
"type" : "long"
},
"link_level" : {
"type" : "integer"
},
"link_title_terms" : {
"type" : "text",
"fields" : {
"shingles" : {
"type" : "text",
"analyzer" : "my_shingle_analyzer"
}
},
"analyzer" : "whitespace"
},
"media_type" : {
"type" : "integer"
},
"post_source" : {
"type" : "long"
},
"region" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"share_count" : {
"type" : "long"
},
"sim_id" : {
"type" : "long"
},
"source" : {
"type" : "integer"
},
"status" : {
"type" : "integer"
},
"text_terms" : {
"type" : "text",
"fields" : {
"shingles" : {
"type" : "text",
"analyzer" : "my_shingle_analyzer"
}
},
"analyzer" : "whitespace"
},
"update_time" : {
"type" : "long"
},
"user_id" : {
"type" : "long"
},
"user_status" : {
"type" : "integer"
},
"username_terms" : {
"type" : "text",
"fields" : {
"shingles" : {
"type" : "text",
"analyzer" : "my_shingle_analyzer"
}
},
"analyzer" : "whitespace"
}
}
}
}
query:
time cost of each step: left is es7, right is es5;
total cost es7 600+ms, es5 400+ms
"profile": true,
"from": 0,
"query": {
"function_score": {
"boost_mode": "multiply",
"functions": [
{
"field_value_factor": {
"field": "favorite_count",
"missing": 1,
"modifier": "ln2p"
}
},
{
"field_value_factor": {
"field": "digg_count",
"missing": 1,
"modifier": "log2p"
}
}
],
"query": {
"bool": {
"filter": {
"terms": {
"media_type": [ //time cost: 143ms ->89ms
3,
5
]
}
},
"minimum_should_match": "0",
"must": [
{
"term": {
"status": 3 //time cost: 82ms -->45ms
}
},
{
"term": {
"user_status": 100 //time cost: 72ms -->35ms
}
},
{
"bool": {
"minimum_should_match": "1",
"should": [ //time cost: 148ms -->212ms
{
"match": {
"text_terms": {
"boost": 100000,
"query": "微信 动态 搞笑 图片 500张"
}
}
},
{
"match": {
"username_terms": {
"query": "微 信 动 态 搞 笑 图 片 5 0 0 张 "
}
}
},
{
"match": {
"hashtags": {
"query": "微信动态搞笑图片500张" //time cost: very little, can be ignore
}
}
},
{
"match": {
"hashtags.keyword": {
"query": "微信动态搞笑图片500张"
}
}
}
]
}
}
],
"must_not": [
{
"range": {
"link_level": { //161 ms -> 33ms
"from": null,
"include_lower": true,
"include_upper": false,
"to": 5
}
}
},
{
"terms": {
"create_type": [ //0.2 ms --> 0.6ms
100,
102,
103,
104
]
}
},
{
"term": {
"media_type": 5 //time cost: very little, can be ignore
}
}
]
}
},
"score_mode": "multiply"
}
},
"size": 3000
}
profile:
es5_profile
es 5 profile(cost 400+ms)
es7_profile