# Scripted field: Read field data from a document using doc\['some\_filed'\].value statement is not working for bigger field data

**URL:** <https://discuss.elastic.co/t/scripted-field-read-field-data-from-a-document-using-doc-some-filed-value-statement-is-not-working-for-bigger-field-data/113775>\
**Category:** Elasticsearch\
**Created:** [January 2, 2018, 1:16pm UTC](https://discuss.elastic.co/t/scripted-field-read-field-data-from-a-document-using-doc-some-filed-value-statement-is-not-working-for-bigger-field-data/113775 "2018-01-02T13:16:05Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![vinjamr](https://avatars.discourse-cdn.com/v4/letter/v/919ad9/32.png) [@vinjamr](https://discuss.elastic.co/u/vinjamr)\
**Post date:** [January 2, 2018, 1:16pm UTC](https://discuss.elastic.co/t/scripted-field-read-field-data-from-a-document-using-doc-some-filed-value-statement-is-not-working-for-bigger-field-data/113775/1 "2018-01-02T13:16:05Z")

</div>

Hi Everyone,  
I am new to kibana. Need your help to solve the following problem which am facing from a day.  
I am working on kibana scripted field approach to transform a field value by accessing it using "doc[some\_field].value" statement and perform some computation using groovy script then finally return a new field.

##### Problem:

"doc[some\_field].value" is returning null in-case if the field value is large.

```auto
Kibana version is : 6.0.1
field name: message.keyword => doc[message.keyword].value
language: painless
Type: String
Format: default(String)
New field name: channel

```

```auto
example script: if (doc['message.keyword'].value != null) {
return "data-present";
} else {
return "data-not-present";
}

```

for one set of events the above script is returning "channel" value as "data-present" and for rest of the events script is returning "data-not-present".  
Especially "data-not-present" value is getting returned incase if the accessing field value is larger in length.  
is this the expected behavior? if not - please could someone help me what is the mistake am doing here.

##### Sample long message =\> doc[message.keyword].value:

```auto
INFO 2017-02-01 17:01:12,628 c.i.c.f.SimpleJmsSender QW-1-my_Name Annamiyyya Sending to dstntn=abcd_bdc_kch prsst=true crrltn: 'null' prpty[J=1] pyld=<?xml version='1.0' encoding='UTF-8' standalone='yes'?><Order><TimeStamps><SubmittedAt>0</SubmittedAt><ReceivedAt>0</ReceivedAt><CompletedAt>0</CompletedAt><OSReceivedAt>1514826072209</OSReceivedAt><ManualStartAt>0</ManualStartAt><ManualProcessAt>0</ManualProcessAt><ConfirmedAt>0</ConfirmedAt><FinishedAt>0</FinishedAt></TimeStamps><RequestDetails><ExecutionInstruction>ABC</ExecutionInstruction><AccountCurrencyCode>#.</AccountCurrencyCode><AccountId>MY_Name</AccountId><AdvancedOrderType></AdvancedOrderType><AttachedOrderIndicator>330</AttachedOrderIndicator><BookingSystem>fooooooo</BookingSystem><BusinessArea>busy</BusinessArea><Channel>fjdnfj</Channel><ChannelId>6374j8485</ChannelId><ClientCategory></ClientCategory><ClientMarketGroup></ClientMarketGroup><CurrentLevel>0</CurrentLevel><DealId>hfjfnjkgbgdb</DealId><DealLevel>0</DealLevel><DealReference>hdgdjekifindj</DealReference><Deposit>0</Deposit><DepositAccountCurrency>0</DepositAccountCurrency><DigitalSignature></DigitalSignature><Direction>+</Direction><DisplayOrderLevel>748595</DisplayOrderLevel><DisplayOrderSize>1</DisplayOrderSize><DisplayProfitLoss></DisplayProfitLoss><ExpiryTime></ExpiryTime><ForceOpen>N</ForceOpen><ForeignOfficeAccount></ForeignOfficeAccount><IsControlledRisk>N</IsControlledRisk><IsSwapTrade>N</IsSwapTrade><IsReinject>N</IsReinject><Locale>en_GB</Locale><LimitDistance>0</LimitDistance><LimitLevel>0</LimitLevel><Market><MktCommod></MktCommod><EpicCode>hcdasjsdfsjfdsajfj</EpicCode><Period> jdsajfdk989</Period><DisplayPeriod> dfdkjkjvf89</DisplayPeriod><MarketName>ewjk/sdhjdsd</MarketName><DisplayMarketName>fjkf/kjds</DisplayMarketName><MarketStatus></MarketStatus><Level1></Level1><Level2></Level2><Level3></Level3><LotSize>10</LotSize><InstrumentLotSize>10</InstrumentLotSize><L2LookupCode>cd/dd</L2LookupCode><ContractId>dsvvddvd</ContractId><ContractVersion>2</ContractVersion><TradableId>dcddcdd</TradableId><TradableVersion>1</TradableVersion></Market><MarketFlipped>0</MarketFlipped><MessageId></MessageId><OrderCurrencyCode>#.</OrderCurrencyCode><PermissionGroup></PermissionGroup><PriceType>S</PriceType><RequestType>C</RequestType><SessionId></SessionId><SiteType>C</SiteType><SourceType>C</SourceType><Spread>0.0</Spread><Size>10</Size><StopDistance>0</StopDistance><StopLevel>0</StopLevel><TimeTillCancelled></TimeTillCancelled><TradingSystem>IGG</TradingSystem><TriggerLevel>8890</TriggerLevel><Type>1</Type><UKExpiryTime></UKExpiryTime><Unit>C</Unit><WebSiteId>igm</WebSiteId><PartialClose>N</PartialClose><PriceImprovement>N</PriceImprovement><RequestedDealLevel>0</RequestedDealLevel><POAName></POAName><TrailingStopDistance>0</TrailingStopDistance><TrailingStopIncrement>0</TrailingStopIncrement><TrailingTriggerDistance>0</TrailingTriggerDistance><TrailingTriggerIncrement>0</TrailingTriggerIncrement><ScalingFactor>10000</ScalingFactor><OrderType>D</OrderType><TmInForce>3</TmInForce></RequestDetails><Result><ActionType>D</ActionType><Action>REJECT</Action><FailedCheck>MARKET_OFFLINE</FailedCheck><Check><CheckName>MARKET_OFFLINE</CheckName><CheckValue></CheckValue><CheckUnit></CheckUnit></Check><UserCode></UserCode><ManualResult></ManualResult><DMAOrderId>dsds:50364~27</DMAOrderId><IGBid>9303.2</IGBid><IGAsk>8890</IGAsk></Result><Source><Internet><IPAddress>3340439434344</IPAddress></Internet></Source><Summary></Summary><ResultCode>normal</ResultCode><ResultDescription>MARKET_OFFLINE</ResultDescription><DisplayResult></DisplayResult><DisplayChannel>PureDMA</DisplayChannel><DisplayTime></DisplayTime><UpdateType>Order</UpdateType><DealResults/><ChangeInfo><Channel>PureDMA</Channel></ChangeInfo><CorpAction></CorpAction></Order>

```

```auto
Note: message.keyword is a searchable and aggregatable field on kibana.

```

---

<div class="post-metadata">

**Author:** ![Joe\_Fleming](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joe_fleming/32/3561_2.png) [@Joe\_Fleming](https://discuss.elastic.co/u/Joe_Fleming)\
**Post date:** [January 2, 2018, 7:13pm UTC](https://discuss.elastic.co/t/scripted-field-read-field-data-from-a-document-using-doc-some-filed-value-statement-is-not-working-for-bigger-field-data/113775/2 "2018-01-02T19:13:14Z")

</div>

> Especially "data-not-present" value is getting returned incase if the accessing field value is larger in length. Is this the expected behavior?

I wouldn't expect that it is, but I can imagine a configuration in Elasticsearch that would prevent the value from being parsed if it's too large. I don't know if that's actually the case though, and I can't find anything about it when I search for it. I'd recommend opening a thread on the Elasticsearch forums, the folks monitoring that would know better.

You could also try using `if (doc['message.keyword'].empty)` instead of comparing it to null. Maybe that would be more reliable? I'm not really familiar with Groovy scripting, but I found a lot of answers that used that when searching for behavior related to field length.

---

<div class="post-metadata">

**Author:** ![vinjamr](https://avatars.discourse-cdn.com/v4/letter/v/919ad9/32.png) [@vinjamr](https://discuss.elastic.co/u/vinjamr)\
**Post date:** [January 3, 2018, 7:24am UTC](https://discuss.elastic.co/t/scripted-field-read-field-data-from-a-document-using-doc-some-filed-value-statement-is-not-working-for-bigger-field-data/113775/3 "2018-01-03T07:24:45Z")

</div>

Thank you for the reply Joe.  
I have tried with `if(doc['message.keyword'].empty)`, but no change in the result.  
I will open a thread in Elasticsearch forum.

---

<div class="post-metadata">

**Author:** ![vinjamr](https://avatars.discourse-cdn.com/v4/letter/v/919ad9/32.png) [@vinjamr](https://discuss.elastic.co/u/vinjamr)\
**Post date:** [January 3, 2018, 12:41pm UTC](https://discuss.elastic.co/t/scripted-field-read-field-data-from-a-document-using-doc-some-filed-value-statement-is-not-working-for-bigger-field-data/113775/4 "2018-01-03T12:41:27Z")

</div>

Any help from anyone?

---

<div class="post-metadata">

**Author:** ![Arsavel](https://avatars.discourse-cdn.com/v4/letter/a/6f9a4e/32.png) [@Arsavel](https://discuss.elastic.co/u/Arsavel)\
**Post date:** [January 17, 2018, 1:34pm UTC](https://discuss.elastic.co/t/scripted-field-read-field-data-from-a-document-using-doc-some-filed-value-statement-is-not-working-for-bigger-field-data/113775/5 "2018-01-17T13:34:16Z")

</div>

Think this would help  
[https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-script-fields.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-script-fields.html)  
Seems that you try to access json-like field, so you need to use `params['_source']['my_field']` in this case

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 14, 2018, 1:34pm UTC](https://discuss.elastic.co/t/scripted-field-read-field-data-from-a-document-using-doc-some-filed-value-statement-is-not-working-for-bigger-field-data/113775/6 "2018-02-14T13:34:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.