# Use mapping character filter from file **URL:** \ **Category:** Elasticsearch\ **Created:** [October 2, 2017, 8:34pm UTC](https://discuss.elastic.co/t/use-mapping-character-filter-from-file/102485 "2017-10-02T20:34:23Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![MPurcell](https://avatars.discourse-cdn.com/v4/letter/m/9fc348/32.png) [@MPurcell](https://discuss.elastic.co/u/MPurcell)\ **Post date:** [October 2, 2017, 8:34pm UTC](https://discuss.elastic.co/t/use-mapping-character-filter-from-file/102485/1 "2017-10-02T20:34:23Z")
Hello! I'm looking to use the mapping character filter over a large number of terms. The basic idea is that I'm trying to standardize street addresses based on the USPS list seen here: [https://pe.usps.com/text/pub28/28apc\_002.htm](https://pe.usps.com/text/pub28/28apc_002.htm). I'm fairly sure the best way to do this is with the mapping character filter. What I would like to do, and because this is such a large list of values being mapped, is put these in a separate file that the analyzer/token filter reads rather then directly map these values in the token filter itself as seen here: [https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-mapping-charfilter.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-mapping-charfilter.html). What are some resources/examples I could use/mimic in order to create this type of file, and how would I use the mapping character filter to read from this file? Thanks for any/all help! ---
**Author:** ![MPurcell](https://avatars.discourse-cdn.com/v4/letter/m/9fc348/32.png) [@MPurcell](https://discuss.elastic.co/u/MPurcell)\ **Post date:** [October 3, 2017, 8:30pm UTC](https://discuss.elastic.co/t/use-mapping-character-filter-from-file/102485/2 "2017-10-03T20:30:48Z")
BUMP ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [October 4, 2017, 4:25pm UTC](https://discuss.elastic.co/t/use-mapping-character-filter-from-file/102485/3 "2017-10-04T16:25:01Z")
I might have something to help you, but I am a few weeks from releasing it. I will try to remember to update you once it is done. ---
**Author:** ![MPurcell](https://avatars.discourse-cdn.com/v4/letter/m/9fc348/32.png) [@MPurcell](https://discuss.elastic.co/u/MPurcell)\ **Post date:** [October 4, 2017, 5:07pm UTC](https://discuss.elastic.co/t/use-mapping-character-filter-from-file/102485/4 "2017-10-04T17:07:30Z")
Thanks @Ivan, there is no real rush as this is a side project I'm working on. I remember seeing documentation about this in 2.x but I have not been able to find it. Thanks again. ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [October 4, 2017, 6:20pm UTC](https://discuss.elastic.co/t/use-mapping-character-filter-from-file/102485/5 "2017-10-04T18:20:14Z")
Judging by the code, path should be supported. Try using mappings\_path > > ---
**Author:** ![MPurcell](https://avatars.discourse-cdn.com/v4/letter/m/9fc348/32.png) [@MPurcell](https://discuss.elastic.co/u/MPurcell)\ **Post date:** [October 10, 2017, 4:28pm UTC](https://discuss.elastic.co/t/use-mapping-character-filter-from-file/102485/6 "2017-10-10T16:28:11Z")
Thanks @Ivan, I'm planning on getting this to work this week. I'll post may results when done. ---
**Author:** ![MPurcell](https://avatars.discourse-cdn.com/v4/letter/m/9fc348/32.png) [@MPurcell](https://discuss.elastic.co/u/MPurcell)\ **Post date:** [October 20, 2017, 3:07pm UTC](https://discuss.elastic.co/t/use-mapping-character-filter-from-file/102485/7 "2017-10-20T15:07:23Z")
Hey @Ivan I have a follow-up question I hope you may be able to answer. After reading your previous response, I created the key,value pairs for my mapping file which is located on my D drive. I also changed the char\_filter type mapping to "mappings\_path" and pointed to this file. When trying to apply the settings the settings apply, the index closes. When trying to open the index I get the following error message: { "error": { "root\_cause": [ { "type": "exception", "reason": "Failed to verify index [wf\_loan\_sample/wrse5BUyRs-wtVek3vkV\_g]" } ], "type": "exception", "reason": "Failed to verify index [wf\_loan\_sample/wrse5BUyRs-wtVek3vkV\_g]", "caused\_by": { "type": "illegal\_argument\_exception", "reason": "IOException while reading mappings\_path: /etc/elasticsearch/D:\usps\_mappings.txt", "caused\_by": { "type": "file\_not\_found\_exception", "reason": "/etc/elasticsearch/D:\usps\_mappings.txt (No such file or directory)" } } }, "status": 500 } My thinking is that this means the file has to be kept server side in /etc/elasticsearch. My hope is that I could keep the file on my ETL server instead. Is this thinking correct, or do you think its possible to keep the mappings\_path file on my ETL server? I'd much prefer to only maintain the file in my ETL environment, rather then having more to worry about server side. Thanks again. ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [October 22, 2017, 4:00pm UTC](https://discuss.elastic.co/t/use-mapping-character-filter-from-file/102485/8 "2017-10-22T16:00:32Z")
Yes, the file needs to exist on every node that uses it when the index is (re)opened. How were you expecting to connect Elasticsearch to your ETL environment? ---
**Author:** ![MPurcell](https://avatars.discourse-cdn.com/v4/letter/m/9fc348/32.png) [@MPurcell](https://discuss.elastic.co/u/MPurcell)\ **Post date:** [October 23, 2017, 2:01pm UTC](https://discuss.elastic.co/t/use-mapping-character-filter-from-file/102485/9 "2017-10-23T14:01:07Z")
@Ivan I connect to my Elasticsearch environments with a windows or ubuntu machine, using Python and the package 'elasticsearch'. I keep my mappings and analyzers in separate JSON files, usually in a shared location, but if not on my windows/ubuntu machine within the project directory. These are then read during ETL and 'PUT' on the index via the elasticsearch package, before the bulk indexing begins. Using this method all the settings and mappings are kept on my machine rather then the cluster itself. In relation to my previous question I could in fact have a separate JSON file with a new char\_filter 'mapping' field and just fill out all the mappings, kinda like the example below. But it would be a tremendously large char\_filter, and likely cause some headaches if anyone else attempts to take a look at the settings on that particular index. This is why I want to use the 'mappings\_path', instead of 'mapping'. As you just described, it seems the only way I'll be able to utilize this feature is if I keep the 'mappings\_path' file on each node in the cluster. I will do this if necessary, but I'm hesitant as all the other code has been kept on my ETL machines (or a shared location that they read from), and I would rather not start putting index related code directly on the cluster. kinda like the example below: ``` "cfPunctuation" : { "type" : "mapping", "description": "Company mapping list to remove punctuation from input string", "mappings" : [ "\\u0027=>", "\\u0022=>", "\\u002E=>\\u0020" ] } ``` ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [October 23, 2017, 8:17pm UTC](https://discuss.elastic.co/t/use-mapping-character-filter-from-file/102485/10 "2017-10-23T20:17:48Z")
Your alternative to using a path is to simply provide the complete list of terms, which is what you are looking to avoid in the first place! Can you setup an external mount that is shared drive on the cluster nodes? That way you only need to mount a drive once. Elasticsearch snapshots require a shared file system, so perhaps you might have something in place. I do have a plugin that pulls in the the terms from a database, but I only converted the synonym, stop and keepwork filters, not any of the char filters. I'll add that to my TODO list. ---
**Author:** ![MPurcell](https://avatars.discourse-cdn.com/v4/letter/m/9fc348/32.png) [@MPurcell](https://discuss.elastic.co/u/MPurcell)\ **Post date:** [October 25, 2017, 1:58pm UTC](https://discuss.elastic.co/t/use-mapping-character-filter-from-file/102485/11 "2017-10-25T13:58:01Z")
@Ivan I do have a shared file store (on AWS s3) that all clusters have access to as its where I store snapshots. If I put the .txt file in this shared space, would I just change the mappings\_path to access it there, or would I have to have each node in the cluster copy the file from this shared space and put it in /etc/elasticsearch? Thanks again for all your help! ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [October 26, 2017, 3:22am UTC](https://discuss.elastic.co/t/use-mapping-character-filter-from-file/102485/12 "2017-10-26T03:22:24Z")
There are utilities to mount an S3 bucket as a local drive. It all depends on your OS and your level of sysadmin abilities. I was thinking of adding s3 support to my plugin, but I need to figure out conditional dependencies first in gradle since I do not want unneeded libs polluting the code. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 23, 2017, 3:22am UTC](https://discuss.elastic.co/t/use-mapping-character-filter-from-file/102485/13 "2017-11-23T03:22:31Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.