Synonyms in synonyms.txt not recognized

Elastic Stack Elasticsearch
1

I have created a synonyms.txt file as in Synonym token filter | Elasticsearch Guide [8.11] | Elastic .

Synonyms.txt contains :
i-pod, i pod => ipod,
sea biscuit, sea biscit => seabiscuit

I then use the following code to create a filter of an analyzer to use this file :

curl -XPUT 'localhost:9200/test_index?pretty' -H 'Content-Type: application/json' -d'
{
"settings": {
"index" : {
"analysis" : {
"analyzer" : {
"search_synonyms" : {
"type" : "keyword",
"filter" : ["lowercase", "synonyms"]
}
},
"filter" : {
"synonyms" : {
"type" : "synonym",
"synonyms_path" : "analysis/synonyms.txt"
}
}
}
}
}
}'

And I get:

{
"acknowledged" : true,
"shards_acknowledged" : true,
"index" : "test_index"
}

But if I try the analyzer :

curl -XGET 'localhost:9200/test_index/_analyze?pretty' -H 'Content-Type: application/json' -d'
{
"analyzer": "search_synonyms",
"text": "i pod"
}'

I get :

{
"tokens" : [
{
"token" : "i pod",
"start_offset" : 0,
"end_offset" : 5,
"type" : "word",
"position" : 0
}
]
}

So the synonyms in the file are not recognized...
Any thoughts or suggestions to make it work?
Thank you in advance for your time.

2

You have a typo in your analyzer definition, which should be defined like this:

"search_synonyms" : {
  "type" : "custom",
  "tokenizer": "keyword",
  "filter" : ["lowercase", "synonyms"]
}
3

I corrected my analyzer definition as you suggested.
Deleted and re-created the index.
Now I have this error showing up upon creation of the index :

{
"error" : {
"root_cause" : [
{
"type" : "illegal_argument_exception",
"reason" : "IOException while reading synonyms_path_path: /usr/share/elasticsearch/config/analysis/synonyms.txt"
}
],
"type" : "illegal_argument_exception",
"reason" : "IOException while reading synonyms_path_path: /usr/share/elasticsearch/config/analysis/synonyms.txt",
"caused_by" : {
"type" : "no_such_file_exception",
"reason" : "/usr/share/elasticsearch/config/analysis/synonyms.txt"
}
},
"status" : 400
}

4

Is your synonyms file at that location?

5

Yes.

XXX@XXX:/usr/share/elasticsearch/config/analysis$ ll

total 16
drwxr-xr-x 2 root root 4096 déc. 28 10:14 ./
drwxr-xr-x 3 root root 4096 déc. 26 15:39 ../
-rw-r--r-- 1 root root 60 déc. 28 10:14 out.txt
-rwxrwxrwx 1 root root 60 déc. 27 11:22 synonyms.txt*

6

Maybe make it readable by the elasticsearch user

7

With the rights set as -rwxrwxrwx, doesn't everyone has the right to read and wright on the file?
If not, then how can I change it so as to make readable by the elasticsearch user (which is me, so "root" no?) ?
Actually, I am using a docker to run ES 6.1. Could that be a source of issues?

8

In that case, are you sure that the file is present in the docker container? Have you mapped the volume?

9

Indeed, I did not think about that. Not knowing much about docker, I am actually trying to figure where I need to put the file synonyms.txt or how to map the docker so that it can go get the file.

IF it can help, here is my docker-compose file :

version: '2.0'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:6.1.0
container_name: elasticsearch
environment:
- cluster.name=docker-cluster
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- "discovery.type=single-node"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./esdata1:/usr/share/elasticsearch/data
ports:
- 9200:9200
- 9300:9300
kibana:
image: docker.elastic.co/kibana/kibana:6.1.0
container_name: kibana
ports:
- 5601:5601

I tried without docker and I indeed have no problem getting the synonyms.txt file.
So clearly a docker issue. And I am not finding how to fix it so far...

10

You simply need to add the following volume:

volumes:
- ./esconfig:/usr/share/elasticsearch/config

Where esconfig is your local config folder

11

I modified docker-compose.yml as you suggested.
I then did the following commands :

sudo docker-compose stop
sudo docker-compose up -d --build
sudo docker ps -a

The response was :

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f2cfd2187bea docker.elastic.co/elasticsearch/elasticsearch:6.1.0 "/usr/local/bin/do..." 3 minutes ago Exited (78) 2 seconds ago elasticsearch
9b12332577eb docker.elastic.co/kibana/kibana:6.1.0 "/bin/bash /usr/lo..." 13 days ago Up 3 seconds 0.0.0.0:5601->5601/tcp kibana

ES is not running anymore with this docker. Am I missing something?

12

What does the following command yield?

docker logs -f f2cfd2187bea --tail 1000
13

2017-12-28 14:30:46,962 main ERROR Could not register mbeans java.security.AccessControlException: access denied ("javax.management.MBeanTrustPermission" "register")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:585)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanTrustPermission(DefaultMBeanServerInterceptor.java:1848)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:322)
at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
at org.apache.logging.log4j.core.jmx.Server.register(Server.java:389)
at org.apache.logging.log4j.core.jmx.Server.reregisterMBeansAfterReconfigure(Server.java:167)
at org.apache.logging.log4j.core.jmx.Server.reregisterMBeansAfterReconfigure(Server.java:140)
at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:556)
at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:261)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:206)
at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:220)
at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:197)
at org.elasticsearch.common.logging.LogConfigurator.configureStatusLogger(LogConfigurator.java:171)
at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:140)
at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:119)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:290)
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121)
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112)
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
at org.elasticsearch.cli.Command.main(Command.java:90)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85)

14

And this as well :
ERROR: no log4j2.properties found; tried [/usr/share/elasticsearch/config] and its subdirectories

With a list of subdirectories.

15

Can you paste your current docker-compose file?

16

version: '2.0'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:6.1.0
container_name: elasticsearch
environment:
- cluster.name=docker-cluster
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- "discovery.type=single-node"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./esdata1:/usr/share/elasticsearch/data
- ./esconfig:/usr/share/elasticsearch/config
ports:
- 9200:9200
- 9300:9300
kibana:
image: docker.elastic.co/kibana/kibana:6.1.0
container_name: kibana
ports:
- 5601:5601

17

And what does the local esconfig folder contain?

18

I tried with it empty, and then after copying and pasting the content of the /usr/share/elasticsearch/config directory into it. Neither situation worked.

19

You need to make sure that your synonyms.txt file ends up where it should within your container, the same way you did with the data folder.

20

So, synonyms.txt needs to be in :
/usr/share/elasticsearch/config/analysis/

or in :
/esconfig/analysis/

?