# Search by phone number **URL:** \ **Category:** Elasticsearch\ **Created:** [July 15, 2011, 6:09pm UTC](https://discuss.elastic.co/t/search-by-phone-number/4873 "2011-07-15T18:09:09Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![Ian\_Eure](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ian_eure/32/3157_2.png) [@Ian\_Eure](https://discuss.elastic.co/u/Ian_Eure)\ **Post date:** [July 15, 2011, 6:09pm UTC](https://discuss.elastic.co/t/search-by-phone-number/4873/1 "2011-07-15T18:09:09Z")
I'm having a hard time matching documents with a phone number, and I'm not sure what's going on. Some of my data has phone numbers separated with spaces: "phone": "+1 415 931 1182", Others have them with nothing but the numbers: "phone": "4159311182", This field is put into \_all, which has the default filters and analyzers. When I search for the former, I get no results, but searches for the latter match just fine. This is the query I'm using: ``` "query": { "sort": [ { "_score": "desc" } ], "from": 0, "fields": [ "_source" ], "explain": true, "query": { "dis_max": { "queries": [ { "term": { "_all": "415 931 1182" } } ] } }, "size": 25 }, ``` It returns zero matches. I suspect that the tokens for the numbers are getting filtered out, but I see nothing in the docs to indicate that this would take place. I haven't been able to figure out how to see what tokens are actually generated for a document, so I have no way to confirm this hypothesis. If anyone has any guidance, it would be very much appreciated. ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [July 15, 2011, 6:20pm UTC](https://discuss.elastic.co/t/search-by-phone-number/4873/2 "2011-07-15T18:20:50Z")
Hi Ian > Some of my data has phone numbers separated with spaces: > > "phone": "+1 415 931 1182", > > Others have them with nothing but the numbers: > > "phone": "4159311182", You don't mention what mapping the 'phone' field has. If you're relying on the defaults, then it will depend on which version was mapped first, If the first, then it will be string. If the second, it will be integer (which is, by default) not analyzed. Analysis would also remove the '+' > This field is put into \_all, which has the default filters and > analyzers. ...which is analyzed by default > When I search for the former, I get no results, but searches for the > latter match just fine. This is the query I'm using: > > ``` > "query": { > "sort": [ > { > "_score": "desc" > } > ], > "from": 0, > "fields": [ > "_source" > ], > "explain": true, > "query": { > "dis_max": { > "queries": [ > { > "term": { > "_all": "415 931 1182" > } > } > ] > } > }, > "size": 25 > }, > > ``` Note: no point in using a dis\_max query with only one query - doesn't make sense. > It returns zero matches. I suspect that the tokens for the numbers are > getting filtered out, but I see nothing in the docs to indicate that > this would take place. Mapping required before we can make any comment > I haven't been able to figure out how to see what tokens are actually > generated for a document, so I have no way to confirm this hypothesis. > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. > If anyone has any guidance, it would be very much appreciated. First, you need to decide how you want phone numbers to be searchable. If a user enters "+1 415 931 1182" but you want to be able to find "4159311182" then you're going to need to apply some rules to normalise your phone numbers clint ---
**Author:** ![Ian\_Eure](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ian_eure/32/3157_2.png) [@Ian\_Eure](https://discuss.elastic.co/u/Ian_Eure)\ **Post date:** [July 15, 2011, 6:34pm UTC](https://discuss.elastic.co/t/search-by-phone-number/4873/3 "2011-07-15T18:34:49Z")
On Jul 15, 2011, at 11:20 AM, Clinton Gormley wrote: > Hi Ian > > > Some of my data has phone numbers separated with spaces: > > > > "phone": "+1 415 931 1182", > > > > Others have them with nothing but the numbers: > > > > "phone": "4159311182", > > You don't mention what mapping the 'phone' field has. If you're relying > on the defaults, then it will depend on which version was mapped first, > If the first, then it will be string. If the second, it will be integer > (which is, by default) not analyzed. u'phone': {u'type': u'string'}, > Analysis would also remove the '+' That's fine. > > When I search for the former, I get no results, but searches for the > > latter match just fine. This is the query I'm using: > > > > "query": { > > "sort": [ > > { > > "\_score": "desc" > > } > > ], > > "from": 0, > > "fields": [ > > "\_source" > > ], > > "explain": true, > > "query": { > > "dis\_max": { > > "queries": [ > > { > > "term": { > > "\_all": "415 931 1182" > > } > > } > > ] > > } > > }, > > "size": 25 > > }, > > Note: no point in using a dis\_max query with only one query - doesn't > make sense. Hm, okay. I'm porting stuff from Solr, and it definitely made a difference there — I guess ES doesn't break down the query into words to build the DisMax query like Solr does. > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-analyze.html) Yes, I tried this — I believe you suggested it in IRC — but it only tells me how my input text is analyzed. I could use this to see how \_all is broken down, but I'm not sure how that is constructed from my input document, and it does not appear to be possible to fetch the \_all of a document. I updated my mapping to make \_all stored, and included it in the `fields' portion of my query, but it is not returned. > > If anyone has any guidance, it would be very much appreciated. > > First, you need to decide how you want phone numbers to be searchable. > If a user enters "+1 415 931 1182" but you want to be able to find > "4159311182" then you're going to need to apply some rules to normalise > your phone numbers The common case is that the number in the document is in "+1 aaa bbb cccc" format and this is also what the user enters. ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [July 15, 2011, 6:49pm UTC](https://discuss.elastic.co/t/search-by-phone-number/4873/4 "2011-07-15T18:49:37Z")
> > Note: no point in using a dis\_max query with only one query - > > doesn't > > make sense. > > Hm, okay. I'm porting stuff from Solr, and it definitely made a > difference there — I guess ES doesn't break down the query into words > to build the DisMax query like Solr does. No, the dis\_max query in ES is used to score separate queries. It chooses the best matching query, as opposed to bool query, which is additive. > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-analyze.html) > > > > > Yes, I tried this — I believe you suggested it in IRC — but it only > tells me how my input text is analyzed. I could use this to see how > \_all is broken down, but I'm not sure how that is constructed from my > input document, and it does not appear to be possible to fetch the > \_all of a document. I updated my mapping to make \_all stored, and > included it in the `fields' portion of my query, but it is not > returned. By default, the \_all field uses the 'default' analyzer. So it takes all of the fields, and runs the default analyzer on them. The analyzer that is defined on the field doesn't affect how \_all analyzes. > > > > The common case is that the number in the document is in "+1 aaa bbb > cccc" format and this is also what the user enters. with or without spaces? and how do you want to search on these? partial matching? full matching? If full matching, then I'd remove all spaces before and possibly prepend a +1 to make all your phone numbers uniform, then use a term query. Alternatively, you may want to separate country code, regional code and phone number into 3 separate entities. For partial matching, perhaps look at ngrams or edge ngrams clint ---
**Author:** ![Ian\_Eure](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ian_eure/32/3157_2.png) [@Ian\_Eure](https://discuss.elastic.co/u/Ian_Eure)\ **Post date:** [July 15, 2011, 9:06pm UTC](https://discuss.elastic.co/t/search-by-phone-number/4873/5 "2011-07-15T21:06:06Z")
On Jul 15, 2011, at 11:49 AM, Clinton Gormley wrote: > > > Note: no point in using a dis\_max query with only one query - > > > doesn't > > > make sense. > > > > Hm, okay. I'm porting stuff from Solr, and it definitely made a > > difference there — I guess ES doesn't break down the query into words > > to build the DisMax query like Solr does. > > No, the dis\_max query in ES is used to score separate queries. It > chooses the best matching query, as opposed to bool query, which is > additive. > > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-analyze.html) > > > > > > > > > Yes, I tried this — I believe you suggested it in IRC — but it only > > tells me how my input text is analyzed. I could use this to see how > > \_all is broken down, but I'm not sure how that is constructed from my > > input document, and it does not appear to be possible to fetch the > > \_all of a document. I updated my mapping to make \_all stored, and > > included it in the `fields' portion of my query, but it is not > > returned. > > By default, the \_all field uses the 'default' analyzer. So it takes all > of the fields, and runs the default analyzer on them. The analyzer that > is defined on the field doesn't affect how \_all analyzes. I understand this, but I don't think what I said has anything to do with it. My issue is that I cannot see the contents of \_all, either before or after they've been analyzed, so I don't know what information is or is not in there. > > > > > > > The common case is that the number in the document is in "+1 aaa bbb > > cccc" format and this is also what the user enters. > > with or without spaces? and how do you want to search on these? partial > matching? full matching? With spaces, exactly like I wrote. It probably makes more sense to store it as just the number and strip any non-numeric characters from the user's input, though. I'd want to match on either the full number with country & area code, as well as plain prefix/suffix; so +10005551212, +1 000 555 1212, 000 555 1212, 555 1212 should all match "+10005551212". > If full matching, then I'd remove all spaces before and possibly prepend > a +1 to make all your phone numbers uniform, then use a term query. I have documents with international numbers, too. I think a substring match is what I want. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [July 15, 2011, 10:04pm UTC](https://discuss.elastic.co/t/search-by-phone-number/4873/6 "2011-07-15T22:04:02Z")
Note, when doing term query, there is no analysis happening on the text provided to it. Use the text query for it. On Saturday, July 16, 2011 at 12:06 AM, Ian Eure wrote: > On Jul 15, 2011, at 11:49 AM, Clinton Gormley wrote: > > > > > Note: no point in using a dis\_max query with only one query - > > > > doesn't > > > > make sense. > > > > Hm, okay. I'm porting stuff from Solr, and it definitely made a > > > > difference there — I guess ES doesn't break down the query into words > > > > to build the DisMax query like Solr does. > > > > No, the dis\_max query in ES is used to score separate queries. It > > > > chooses the best matching query, as opposed to bool query, which is > > > > additive. > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-analyze.html) > > > Yes, I tried this — I believe you suggested it in IRC — but it only > > > tells me how my input text is analyzed. I could use this to see how > > > \_all is broken down, but I'm not sure how that is constructed from my > > > input document, and it does not appear to be possible to fetch the > > > \_all of a document. I updated my mapping to make \_all stored, and > > > included it in the `fields' portion of my query, but it is not > > > returned. > > > > By default, the \_all field uses the 'default' analyzer. So it takes all > > of the fields, and runs the default analyzer on them. The analyzer that > > is defined on the field doesn't affect how \_all analyzes. > > I understand this, but I don't think what I said has anything to do with it. My issue is that I cannot see the contents of \_all, either before or after they've been analyzed, so I don't know what information is or is not in there. > > > > The common case is that the number in the document is in "+1 aaa bbb > > > cccc" format and this is also what the user enters. > > > > with or without spaces? and how do you want to search on these? partial > > matching? full matching? > > With spaces, exactly like I wrote. It probably makes more sense to store it as just the number and strip any non-numeric characters from the user's input, though. I'd want to match on either the full number with country & area code, as well as plain prefix/suffix; so +10005551212, +1 000 555 1212, 000 555 1212, 555 1212 should all match "+10005551212". > > > If full matching, then I'd remove all spaces before and possibly prepend > > a +1 to make all your phone numbers uniform, then use a term query. > > I have documents with international numbers, too. I think a substring match is what I want. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [July 15, 2011, 11:27pm UTC](https://discuss.elastic.co/t/search-by-phone-number/4873/7 "2011-07-15T23:27:11Z")
I wonder if it makes sense to add a phone number type, which will normalize phone numbers to a consistent format. Maybe even something that can extract phone numbers from text? Possibly either as a type, or as an analysis step token filter.... On Saturday, July 16, 2011 at 1:04 AM, Shay Banon wrote: > Note, when doing term query, there is no analysis happening on the text provided to it. Use the text query for it. > > On Saturday, July 16, 2011 at 12:06 AM, Ian Eure wrote: > > > On Jul 15, 2011, at 11:49 AM, Clinton Gormley wrote: > > > > > > > Note: no point in using a dis\_max query with only one query - > > > > > doesn't > > > > > make sense. > > > > > Hm, okay. I'm porting stuff from Solr, and it definitely made a > > > > > difference there — I guess ES doesn't break down the query into words > > > > > to build the DisMax query like Solr does. > > > > > No, the dis\_max query in ES is used to score separate queries. It > > > > > chooses the best matching query, as opposed to bool query, which is > > > > > additive. > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-analyze.html) > > > > Yes, I tried this — I believe you suggested it in IRC — but it only > > > > tells me how my input text is analyzed. I could use this to see how > > > > \_all is broken down, but I'm not sure how that is constructed from my > > > > input document, and it does not appear to be possible to fetch the > > > > \_all of a document. I updated my mapping to make \_all stored, and > > > > included it in the `fields' portion of my query, but it is not > > > > returned. > > > > > > By default, the \_all field uses the 'default' analyzer. So it takes all > > > of the fields, and runs the default analyzer on them. The analyzer that > > > is defined on the field doesn't affect how \_all analyzes. > > > I understand this, but I don't think what I said has anything to do with it. My issue is that I cannot see the contents of \_all, either before or after they've been analyzed, so I don't know what information is or is not in there. > > > > > > The common case is that the number in the document is in "+1 aaa bbb > > > > cccc" format and this is also what the user enters. > > > > > > with or without spaces? and how do you want to search on these? partial > > > matching? full matching? > > > With spaces, exactly like I wrote. It probably makes more sense to store it as just the number and strip any non-numeric characters from the user's input, though. I'd want to match on either the full number with country & area code, as well as plain prefix/suffix; so +10005551212, +1 000 555 1212, 000 555 1212, 555 1212 should all match "+10005551212". > > > > > If full matching, then I'd remove all spaces before and possibly prepend > > > a +1 to make all your phone numbers uniform, then use a term query. > > > I have documents with international numbers, too. I think a substring match is what I want. ---
**Author:** ![Ian\_Eure](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ian_eure/32/3157_2.png) [@Ian\_Eure](https://discuss.elastic.co/u/Ian_Eure)\ **Post date:** [July 15, 2011, 11:32pm UTC](https://discuss.elastic.co/t/search-by-phone-number/4873/8 "2011-07-15T23:32:20Z")
On Jul 15, 2011, at 4:27 PM, Shay Banon wrote: > I wonder if it makes sense to add a phone number type, which will normalize phone numbers to a consistent format. Maybe even something that can extract phone numbers from text? Possibly either as a type, or as an analysis step token filter.... That would be pretty neat. I'm using Google's libphonenumber[1](http://code.google.com/p/libphonenumber/) for parsing, but it depends on knowing what country it's from, since local formats vary. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [July 15, 2011, 11:33pm UTC](https://discuss.elastic.co/t/search-by-phone-number/4873/9 "2011-07-15T23:33:45Z")
Yea, thats what I looked at as well, and wondered if it make sense to embed it in elasticsearch in one way or another... On Saturday, July 16, 2011 at 2:32 AM, Ian Eure wrote: > On Jul 15, 2011, at 4:27 PM, Shay Banon wrote: > > > I wonder if it makes sense to add a phone number type, which will normalize phone numbers to a consistent format. Maybe even something that can extract phone numbers from text? Possibly either as a type, or as an analysis step token filter.... > > That would be pretty neat. I'm using Google's libphonenumber[1](http://code.google.com/p/libphonenumber/) for parsing, but it depends on knowing what country it's from, since local formats vary. ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [July 16, 2011, 5:13am UTC](https://discuss.elastic.co/t/search-by-phone-number/4873/10 "2011-07-16T05:13:01Z")
Hi Ian > > > returned. > > > > By default, the \_all field uses the 'default' analyzer. So it takes > > all > > of the fields, and runs the default analyzer on them. The analyzer > > that > > is defined on the field doesn't affect how \_all analyzes. > > I understand this, but I don't think what I said has anything to do > with it. My issue is that I cannot see the contents of \_all, either > before or after they've been analyzed, so I don't know what > information is or is not in there. One thing you could do here is a terms facet on the \_all field - that way you can see what terms are stored there: curl -XGET '[http://127.0.0.1:9200/\_all/\_search?pretty=1](http://127.0.0.1:9200/_all/_search?pretty=1)' -d ' { "facets" : { "all\_field" : { "terms" : { "size" : 20, "field" : "\_all" } } }, "size" : 0 } ' clint ---
**Author:** ![karmi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/karmi/32/44951_2.png) [@karmi](https://discuss.elastic.co/u/karmi)\ **Post date:** [July 17, 2011, 6:19am UTC](https://discuss.elastic.co/t/search-by-phone-number/4873/11 "2011-07-17T06:19:56Z")
Is there any reason for not searching in the `phone` field, but in the `_all` field? On Jul 15, 11:06 pm, Ian Eure [i...@simplegeo.com](mailto:i...@simplegeo.com) wrote: > On Jul 15, 2011, at 11:49 AM, Clinton Gormley wrote: > > > > > Note: no point in using a dis\_max query with only one query - > > > > doesn't > > > > make sense. > > > > Hm, okay. I'm porting stuff from Solr, and it definitely made a > > > difference there — I guess ES doesn't break down the query into words > > > to build the DisMax query like Solr does. > > > No, the dis\_max query in ES is used to score separate queries. It > > chooses the best matching query, as opposed to bool query, which is > > additive. > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-analyz)... > > > > Yes, I tried this — I believe you suggested it in IRC — but it only > > > tells me how my input text is analyzed. I could use this to see how > > > \_all is broken down, but I'm not sure how that is constructed from my > > > input document, and it does not appear to be possible to fetch the > > > \_all of a document. I updated my mapping to make \_all stored, and > > > included it in the `fields' portion of my query, but it is not > > > returned. > > > By default, the \_all field uses the 'default' analyzer. So it takes all > > of the fields, and runs the default analyzer on them. The analyzer that > > is defined on the field doesn't affect how \_all analyzes. > > I understand this, but I don't think what I said has anything to do with it. My issue is that I cannot see the contents of \_all, either before or after they've been analyzed, so I don't know what information is or is not in there. > > > > The common case is that the number in the document is in "+1 aaa bbb > > > cccc" format and this is also what the user enters. > > > with or without spaces? and how do you want to search on these? partial > > matching? full matching? > > With spaces, exactly like I wrote. It probably makes more sense to store it as just the number and strip any non-numeric characters from the user's input, though. I'd want to match on either the full number with country & area code, as well as plain prefix/suffix; so +10005551212, +1 000 555 1212, 000 555 1212, 555 1212 should all match "+10005551212". > > > If full matching, then I'd remove all spaces before and possibly prepend > > a +1 to make all your phone numbers uniform, then use a term query. > > I have documents with international numbers, too. I think a substring match is what I want. ---
**Author:** ![Ian\_Eure](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ian_eure/32/3157_2.png) [@Ian\_Eure](https://discuss.elastic.co/u/Ian_Eure)\ **Post date:** [July 20, 2011, 5:57pm UTC](https://discuss.elastic.co/t/search-by-phone-number/4873/12 "2011-07-20T17:57:14Z")
On Jul 16, 2011, at 11:19 PM, Karel Minarik wrote: > Is there any reason for not searching in the `phone` field, but in the > `_all` field? A phone number is one of many different things that a user might enter to search. I probably need a heuristic to change the query behavior, e.g. if it's all numbers with an optional plus prefix, only query the phone number field. ---
**Author:** ![karmi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/karmi/32/44951_2.png) [@karmi](https://discuss.elastic.co/u/karmi)\ **Post date:** [July 21, 2011, 8:51am UTC](https://discuss.elastic.co/t/search-by-phone-number/4873/13 "2011-07-21T08:51:12Z")
> A phone number is one of many different things that a user might enter to search. I probably need a heuristic to change the query behavior, e.g. if it's all numbers with an optional plus prefix, only query the phone number field. Sure. It is my impression -- which may be wrong --, that you're doing it too complicated, and not using the precious Elasticsearch features. Why not search for multiple fields with the same user-entered query? Ie. `q=phone: OR name: OR whatever:`. (Of course, you can use a boolean query or whatever DSL syntax would work best for you here.) This way, the user-entered query is analyzed in the same way as the field in question, so phone numbers are normalized, names are lowercased, whatever is stemmed etc. ---
**Author:** ![drewdahlke](https://avatars.discourse-cdn.com/v4/letter/d/bcef8e/32.png) [@drewdahlke](https://discuss.elastic.co/u/drewdahlke)\ **Post date:** [August 7, 2015, 10:26am UTC](https://discuss.elastic.co/t/search-by-phone-number/4873/17 "2015-08-07T10:26:14Z")
Ancient post, but still relevant. I took a stab at writing a phone/sip analyzer plugin using google's libphone and it's working well for us. Figured I'd share [https://github.com/MyPureCloud/elasticsearch-phone](https://github.com/MyPureCloud/elasticsearch-phone) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 11:56pm UTC](https://discuss.elastic.co/t/search-by-phone-number/4873/18 "2017-07-05T23:56:49Z")