Synonyms in synonyms.txt not recognized


(Aurelien Quillet) #1

I have created a synonyms.txt file as in https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-synonym-tokenfilter.html .

Synonyms.txt contains :
i-pod, i pod => ipod,
sea biscuit, sea biscit => seabiscuit

I then use the following code to create a filter of an analyzer to use this file :

curl -XPUT 'localhost:9200/test_index?pretty' -H 'Content-Type: application/json' -d'
{
"settings": {
"index" : {
"analysis" : {
"analyzer" : {
"search_synonyms" : {
"type" : "keyword",
"filter" : ["lowercase", "synonyms"]
}
},
"filter" : {
"synonyms" : {
"type" : "synonym",
"synonyms_path" : "analysis/synonyms.txt"
}
}
}
}
}
}'

And I get:

{
"acknowledged" : true,
"shards_acknowledged" : true,
"index" : "test_index"
}

But if I try the analyzer :

curl -XGET 'localhost:9200/test_index/_analyze?pretty' -H 'Content-Type: application/json' -d'
{
"analyzer": "search_synonyms",
"text": "i pod"
}'

I get :

{
"tokens" : [
{
"token" : "i pod",
"start_offset" : 0,
"end_offset" : 5,
"type" : "word",
"position" : 0
}
]
}

So the synonyms in the file are not recognized...
Any thoughts or suggestions to make it work?
Thank you in advance for your time.


(Val Crettaz) #2

You have a typo in your analyzer definition, which should be defined like this:

"search_synonyms" : {
  "type" : "custom",
  "tokenizer": "keyword",
  "filter" : ["lowercase", "synonyms"]
}

(Aurelien Quillet) #3

I corrected my analyzer definition as you suggested.
Deleted and re-created the index.
Now I have this error showing up upon creation of the index :

{
"error" : {
"root_cause" : [
{
"type" : "illegal_argument_exception",
"reason" : "IOException while reading synonyms_path_path: /usr/share/elasticsearch/config/analysis/synonyms.txt"
}
],
"type" : "illegal_argument_exception",
"reason" : "IOException while reading synonyms_path_path: /usr/share/elasticsearch/config/analysis/synonyms.txt",
"caused_by" : {
"type" : "no_such_file_exception",
"reason" : "/usr/share/elasticsearch/config/analysis/synonyms.txt"
}
},
"status" : 400
}


(Val Crettaz) #4

Is your synonyms file at that location?


(Aurelien Quillet) #5

Yes.

XXX@XXX:/usr/share/elasticsearch/config/analysis$ ll

total 16
drwxr-xr-x 2 root root 4096 déc. 28 10:14 ./
drwxr-xr-x 3 root root 4096 déc. 26 15:39 ../
-rw-r--r-- 1 root root 60 déc. 28 10:14 out.txt
-rwxrwxrwx 1 root root 60 déc. 27 11:22 synonyms.txt*


(Val Crettaz) #6

Maybe make it readable by the elasticsearch user


(Aurelien Quillet) #7

With the rights set as -rwxrwxrwx, doesn't everyone has the right to read and wright on the file?
If not, then how can I change it so as to make readable by the elasticsearch user (which is me, so "root" no?) ?
Actually, I am using a docker to run ES 6.1. Could that be a source of issues?


(Val Crettaz) #8

In that case, are you sure that the file is present in the docker container? Have you mapped the volume?


(Aurelien Quillet) #9

Indeed, I did not think about that. Not knowing much about docker, I am actually trying to figure where I need to put the file synonyms.txt or how to map the docker so that it can go get the file.

IF it can help, here is my docker-compose file :

version: '2.0'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:6.1.0
container_name: elasticsearch
environment:
- cluster.name=docker-cluster
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- "discovery.type=single-node"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./esdata1:/usr/share/elasticsearch/data
ports:
- 9200:9200
- 9300:9300
kibana:
image: docker.elastic.co/kibana/kibana:6.1.0
container_name: kibana
ports:
- 5601:5601

I tried without docker and I indeed have no problem getting the synonyms.txt file.
So clearly a docker issue. And I am not finding how to fix it so far...


(Val Crettaz) #10

You simply need to add the following volume:

volumes:
- ./esconfig:/usr/share/elasticsearch/config

Where esconfig is your local config folder


(Aurelien Quillet) #11

I modified docker-compose.yml as you suggested.
I then did the following commands :

sudo docker-compose stop
sudo docker-compose up -d --build
sudo docker ps -a

The response was :

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f2cfd2187bea docker.elastic.co/elasticsearch/elasticsearch:6.1.0 "/usr/local/bin/do..." 3 minutes ago Exited (78) 2 seconds ago elasticsearch
9b12332577eb docker.elastic.co/kibana/kibana:6.1.0 "/bin/bash /usr/lo..." 13 days ago Up 3 seconds 0.0.0.0:5601->5601/tcp kibana

ES is not running anymore with this docker. Am I missing something?


(Val Crettaz) #12

What does the following command yield?

docker logs -f f2cfd2187bea --tail 1000

(Aurelien Quillet) #13

2017-12-28 14:30:46,962 main ERROR Could not register mbeans java.security.AccessControlException: access denied ("javax.management.MBeanTrustPermission" "register")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:585)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanTrustPermission(DefaultMBeanServerInterceptor.java:1848)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:322)
at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
at org.apache.logging.log4j.core.jmx.Server.register(Server.java:389)
at org.apache.logging.log4j.core.jmx.Server.reregisterMBeansAfterReconfigure(Server.java:167)
at org.apache.logging.log4j.core.jmx.Server.reregisterMBeansAfterReconfigure(Server.java:140)
at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:556)
at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:261)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:206)
at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:220)
at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:197)
at org.elasticsearch.common.logging.LogConfigurator.configureStatusLogger(LogConfigurator.java:171)
at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:140)
at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:119)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:290)
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121)
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112)
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
at org.elasticsearch.cli.Command.main(Command.java:90)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85)


(Aurelien Quillet) #14

And this as well :
ERROR: no log4j2.properties found; tried [/usr/share/elasticsearch/config] and its subdirectories

With a list of subdirectories.


(Val Crettaz) #15

Can you paste your current docker-compose file?


(Aurelien Quillet) #16

version: '2.0'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:6.1.0
container_name: elasticsearch
environment:
- cluster.name=docker-cluster
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- "discovery.type=single-node"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./esdata1:/usr/share/elasticsearch/data
- ./esconfig:/usr/share/elasticsearch/config
ports:
- 9200:9200
- 9300:9300
kibana:
image: docker.elastic.co/kibana/kibana:6.1.0
container_name: kibana
ports:
- 5601:5601


(Val Crettaz) #17

And what does the local esconfig folder contain?


(Aurelien Quillet) #18

I tried with it empty, and then after copying and pasting the content of the /usr/share/elasticsearch/config directory into it. Neither situation worked.


(Val Crettaz) #19

You need to make sure that your synonyms.txt file ends up where it should within your container, the same way you did with the data folder.


(Aurelien Quillet) #20

So, synonyms.txt needs to be in :
/usr/share/elasticsearch/config/analysis/

or in :
/esconfig/analysis/

?