Help using the Trim function

LoZio · January 5, 2019, 11:08am

I thought it was a simple task, but I'm having a problem trying to trim a field.
I have the City field in my index, it contains "MyCity", "blank_spaceMyCity", "Mycityblank_space" so I tried to apply the trim funcion in a painless scripted field, like

doc['City.keyword'].value.trim()

This returns just nothing in the field test and throws errors in Discover. Removing the trim() part I get the city names as expected, but with surrounding spaces of course.
I found no useful doc around or examples.
How do I use the trim function?
Using 6.5.
Thanks

Marius_Dragomir · January 7, 2019, 10:41am

Are you getting any errors in your ES logs? Also, can you check that the field types are set OK? as trim will only work on strings.
That is the correct syntax for the trim function and I just tried it on 6.6.0 and it was fine.

LoZio · January 7, 2019, 11:04am

I'm pretty sure the city is a string field. See this example. I created "trim_field" without the trim function applied, the test is:

You can see the countries on the right, this is fine. I add the trim function and get:

No visible errors here.

LoZio · January 7, 2019, 11:11am

In the log I found:

[2019-01-07T12:03:32,957][DEBUG][o.e.a.s.TransportSearchAction] [oiZDz7D] [609645] Failed to execute fetch phase
org.elasticsearch.transport.RemoteTransportException: [oiZDz7D][127.0.0.1:9300][indices:data/read/search[phase/fetch/id]]
Caused by: org.elasticsearch.script.ScriptException: runtime error
at org.elasticsearch.painless.PainlessScript.convertToScriptException(PainlessScript.java:94) ~[?:?]
at org.elasticsearch.painless.PainlessScript$Script.execute(doc['Country.keyword'].value.trim() ...:29) ~[?:?]
at org.elasticsearch.painless.ScriptImpl.run(ScriptImpl.java:104) ~[?:?]
at org.elasticsearch.search.fetch.subphase.ScriptFieldsFetchSubPhase.hitsExecute(ScriptFieldsFetchSubPhase.java:67) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:167) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.search.SearchService.lambda$executeFetchPhase$3(SearchService.java:556) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.search.SearchService$3.doRun(SearchService.java:361) [elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:723) [elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) [elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.5.4.jar:6.5.4]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_191]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_191]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]
Caused by: java.lang.IllegalArgumentException: dynamic getter [java.lang.String, trim] not found
at org.elasticsearch.painless.Def.lookupGetter(Def.java:395) ~[?:?]
at org.elasticsearch.painless.DefBootstrap$PIC.lookup(DefBootstrap.java:154) ~[?:?]
at org.elasticsearch.painless.DefBootstrap$PIC.fallback(DefBootstrap.java:208) ~[?:?]
at org.elasticsearch.painless.PainlessScript$Script.execute(doc['Country.keyword'].value.trim ...:29) ~[?:?]
at org.elasticsearch.painless.ScriptImpl.run(ScriptImpl.java:104) ~[?:?]
at org.elasticsearch.search.fetch.subphase.ScriptFieldsFetchSubPhase.hitsExecute(ScriptFieldsFetchSubPhase.java:67) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:167) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.search.SearchService.lambda$executeFetchPhase$3(SearchService.java:556) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.search.SearchService$3.doRun(SearchService.java:361) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:723) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.5.4.jar:6.5.4]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_191]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_191]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_191]

Running
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)
on Ubuntu 16

Marius_Dragomir · January 7, 2019, 11:22am

This looks weird. I presume you are setting the script type as painless, as default. That should be OK.
I'll have to refer somebody from ES team to this issue.

LoZio · January 7, 2019, 12:02pm

Yes, painless scripting and string as in your pic above. I just added "Title case" transform at the end, since I have mixed "City", "city", "CITY" and so on. (Same for country and other details... bad data collection)

Jack_Conradson · January 7, 2019, 8:40pm

@LoZio The error that is occurring would happen if the script was run as the following: doc['Country.keyword'].value.trim

Is it possible the script was run this way previously? (without parens for the trim method)

LoZio · January 8, 2019, 7:42am

Hi Jack, the images are from the actual run that generated the logs so the parenthesis are there

system · February 5, 2019, 7:42am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.