# Scripted field: Read field data from a document using doc\[‘some\_field’\].value return null in some cases

**URL:** <https://discuss.elastic.co/t/scripted-field-read-field-data-from-a-document-using-doc-some-field-value-return-null-in-some-cases/187385>\
**Category:** Kibana\
**Created:** [June 25, 2019, 4:30pm UTC](https://discuss.elastic.co/t/scripted-field-read-field-data-from-a-document-using-doc-some-field-value-return-null-in-some-cases/187385 "2019-06-25T16:30:17Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![dreamer1](https://avatars.discourse-cdn.com/v4/letter/d/b5e925/32.png) [@dreamer1](https://discuss.elastic.co/u/dreamer1)\
**Post date:** [June 25, 2019, 4:30pm UTC](https://discuss.elastic.co/t/scripted-field-read-field-data-from-a-document-using-doc-some-field-value-return-null-in-some-cases/187385/1 "2019-06-25T16:30:17Z")

</div>

hi,  
on my document I have a field always populated. It's a string field and it can contains data like json or XML.  
I created this scripted field to find problem.

def myField = doc['myfield'].value;  
if (payload != null){  
return payload;  
}  
else{  
return "payload is null";  
}

Now, in some cases I can see the payload data, but when I have an XML inside "myField", the scripted fields return "payload is null".  
I don't know the reason, it's unbelievable.  
Maybe the length of the string in XML, could it be?

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/9/b98a5a061bdb70a28370fbe082028a6105287c13.png)

but the payload is not null, this is the field content

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/b/0b837c0bf86bb327e07ee86d86f3aebf2a6898ca.png)

any ideas?

---

<div class="post-metadata">

**Author:** ![nickpeihl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nickpeihl/32/112622_2.png) [@nickpeihl](https://discuss.elastic.co/u/nickpeihl)\
**Post date:** [June 25, 2019, 10:43pm UTC](https://discuss.elastic.co/t/scripted-field-read-field-data-from-a-document-using-doc-some-field-value-return-null-in-some-cases/187385/2 "2019-06-25T22:43:12Z")

</div>

Hi, I'm not able to replicate the issue.

- Which version of Elasticsearch and Kibana are you running?
- What is the mapping of your field? i.e. `text` or `keyword`?

If the field is mapped as `text` you might need to use `params['_source']['myfield']` to get the value of the field. [See the last paragraph on this page](https://www.elastic.co/guide/en/elasticsearch/reference/7.1/search-request-script-fields.html#search-request-script-fields).

---

<div class="post-metadata">

**Author:** ![dreamer1](https://avatars.discourse-cdn.com/v4/letter/d/b5e925/32.png) [@dreamer1](https://discuss.elastic.co/u/dreamer1)\
**Post date:** [June 27, 2019, 8:43am UTC](https://discuss.elastic.co/t/scripted-field-read-field-data-from-a-document-using-doc-some-field-value-return-null-in-some-cases/187385/3 "2019-06-27T08:43:59Z")

</div>

Tnx nick,  
the versions are 6.3 for both.  
I tried your  
**doc['event.payload].value** --\> works only in some cases, as described...  
**params['\_source']['event.payload']** --\> it doesn't work (I haven t errors but my fields seems to be always empty).

**ELK:**

 ![elk-conf](https://us1.discourse-cdn.com/elastic/original/3X/2/8/28488b96efa4d531b15034239ecbc33fe8f42808.gif)

**KIBANA**

 ![kib](https://us1.discourse-cdn.com/elastic/original/3X/4/9/49959c848e1eaab218d97dfb3e3f7fcd552e6dc2.gif)

**In this case, doc['event.payload].value works (with this content):**

 ![noproblem](https://us1.discourse-cdn.com/elastic/original/3X/5/f/5f8e875c53ab88e6c549f616d9fa63e470cd20d6.gif)

**In this case, doc['event.payload] is null (but it's not null!)**

 ![problem](https://us1.discourse-cdn.com/elastic/original/3X/9/5/95381370c9177a0fa9b91f6ad501f6bd6253f43c.gif)

According to me the problem is related to the size(length) of field (event.payload); in the first example with XML, there are a lot of characters, could it be?  
Tnx

---

<div class="post-metadata">

**Author:** ![nickpeihl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nickpeihl/32/112622_2.png) [@nickpeihl](https://discuss.elastic.co/u/nickpeihl)\
**Post date:** [July 1, 2019, 11:49pm UTC](https://discuss.elastic.co/t/scripted-field-read-field-data-from-a-document-using-doc-some-field-value-return-null-in-some-cases/187385/4 "2019-07-01T23:49:22Z")

</div>

Can you share the entire XML string that is failing? This may be a bug in the 6.3.0 version and I'd like to see if it has been fixed in subsequent releases.

---

<div class="post-metadata">

**Author:** ![dreamer1](https://avatars.discourse-cdn.com/v4/letter/d/b5e925/32.png) [@dreamer1](https://discuss.elastic.co/u/dreamer1)\
**Post date:** [July 2, 2019, 9:43am UTC](https://discuss.elastic.co/t/scripted-field-read-field-data-from-a-document-using-doc-some-field-value-return-null-in-some-cases/187385/5 "2019-07-02T09:43:43Z")

</div>

Thanks nick,  
I'm sharing the XML string that is failing, but the problem is related to **all** XML explicit content... Could the problem be related to XML declaration "\<?xml version="1.0"...."

```
<?xml version='1.0' encoding='UTF-8'?>
<ZINT_SFDC_COMM>
<import>
<XT_SFDC>
<row id="0">
<ZID_RECORD>a0u1X000000gQ1p</ZID_RECORD>
<ZTIPOOPER>SWIN</ZTIPOOPER>
<DATA_EVENTO>2019-06-01</DATA_EVENTO>
<BP_CRM>0000004</BP_CRM>
<CC_CRM>BAN-0000001</CC_CRM>
<IMPIANTO_CRM>ABC-000001</IMPIANTO_CRM>
<CONTR_CRM>ASN-00000001</CONTR_CRM>
<BP_SAP></BP_SAP>
<CC_SAP></CC_SAP>
<POD_SAP>IT001E0000001</POD_SAP>
<CONTR_SAP></CONTR_SAP>
<IMPIANTO_SAP></IMPIANTO_SAP>
<BP_SAP_OLD></BP_SAP_OLD>
<CC_SAP_OLD></CC_SAP_OLD>
<CONTR_SAP_OLD></CONTR_SAP_OLD>
<TYPE>2</TYPE>
<NAME_FIRST></NAME_FIRST>
<NAME_LAST></NAME_LAST>
<NAME_ORG1>ENERGIA MOCK S.R.L.</NAME_ORG1>
<LEGAL_ENTITY>02</LEGAL_ENTITY>
<AUGRP>IT01</AUGRP>
<BPKIND>Z007</BPKIND>
<CF_TAXNUM></CF_TAXNUM>
<PI_TAXNUM>02000000001</PI_TAXNUM>
<CITY1_ESAZ>Milan</CITY1_ESAZ>
<COUNTRY_ESAZ>IT</COUNTRY_ESAZ>
<HOUSE_NUM1_ESAZ>12</HOUSE_NUM1_ESAZ>
<NAME_CO_ESAZ></NAME_CO_ESAZ>
<FLOOR_ESAZ></FLOOR_ESAZ>
<POST_CODE1_ESAZ>21052</POST_CODE1_ESAZ>
<REGION_ESAZ>VA</REGION_ESAZ>
<STREET_ESAZ>12 VIA SILVIO</STREET_ESAZ>
<ZBP_ISTATLOC>012026</ZBP_ISTATLOC>
<ROOMNUMBER_ESAZ></ROOMNUMBER_ESAZ>
<CITY1_CLIENTE>Milano</CITY1_CLIENTE>
<COUNTRY_CLIENTE>IT</COUNTRY_CLIENTE>
<HOUSE_NUM1_CLIENTE>12</HOUSE_NUM1_CLIENTE>
<NAME_CO_CLIENTE></NAME_CO_CLIENTE>
<FLOOR_CLIENTE></FLOOR_CLIENTE>
<POST_CODE1_CLIENTE>21052</POST_CODE1_CLIENTE>
<REGION_CLIENTE>VA</REGION_CLIENTE>
<STREET_CLIENTE>VIA SILVIO</STREET_CLIENTE>
<ROOMNUMBER_CLIENTE></ROOMNUMBER_CLIENTE>
<ZBP_ISTATLOC_CLIENTE>012026</ZBP_ISTATLOC_CLIENTE>
<TEL_NUMBER1></TEL_NUMBER1>
<SMTP_ADDR>testmail@gmail.com</SMTP_ADDR>
<PEC_SMTP_ADDR>testmail@gmail.com</PEC_SMTP_ADDR>
<PEC_SOLLECITI></PEC_SOLLECITI>
<CCNUM></CCNUM>
<DATA_INIZIO_CCNUM>0000-00-00</DATA_INIZIO_CCNUM>
<DATA_FINE_CCNUM>0000-00-00</DATA_FINE_CCNUM>
<BANKS></BANKS>
<BANKL></BANKL>
<BANKN></BANKN>
<BKONT></BKONT>
<IBAN></IBAN>
<Z_MANDATO></Z_MANDATO>
<STAT_MAND></STAT_MAND>
<COD_CUCSIA></COD_CUCSIA>
<CREDITORID></CREDITORID>
<DATASOTT>0000-00-00</DATASOTT>
<CODFISCFIRMA></CODFISCFIRMA>
<NOMEFIRMA></NOMEFIRMA>
<COGNOMEFIRMA></COGNOMEFIRMA>
<AB_MANDATO>0000-00-00</AB_MANDATO>
<BIS_MANDATO>0000-00-00</BIS_MANDATO>
<ZPUBB>N</ZPUBB>
<LGZUSATZ_BP></LGZUSATZ_BP>
<VBUND></VBUND>
<Z_COBRAND>IT01</Z_COBRAND>
<IKEY>Z2</IKEY>
<ZAHLKOND>DM30</ZAHLKOND>
<Z_MODINV>ZD</Z_MODINV>
<Z_SINTDETT>S</Z_SINTDETT>
<Z_DUALCODE>N</Z_DUALCODE>
<IPA_CODE></IPA_CODE>
<IPA_BEGDA>0000-00-00</IPA_BEGDA>
<Z_OU></Z_OU>
<Z_IOU></Z_IOU>
<Z_CUP></Z_CUP>
<Z_CIG></Z_CIG>
<Z_CONSIP></Z_CONSIP>
<Z_RIFAMM></Z_RIFAMM>
<EZAWE>B</EZAWE>
<ZCOD_ASS></ZCOD_ASS>
<ZCA_RES>Y</ZCA_RES>
<ZCA_TOP></ZCA_TOP>
<ZCOD_RES>AGESP</ZCOD_RES>
<SPARTE>E</SPARTE>
<GEMFAKT>2</GEMFAKT>
<KOFIZ>01</KOFIZ>
<ZCANACQ>AG</ZCANACQ>
<Z_KAM>Default</Z_KAM>
<Z_MERCRIE>SL</Z_MERCRIE>
<Z_PRODOTTO>Axpo99912</Z_PRODOTTO>
<Z_PRODOTTO_DESC>Listino Fisso 3Fasce</Z_PRODOTTO_DESC>
<Z_INIZIO>2019-06-01</Z_INIZIO>
<Z_LISTINO>RESEL_FI01386_1907FIX</Z_LISTINO>
<Z_MERCATO>ML</Z_MERCATO>
<Z_DEPOSITO>N</Z_DEPOSITO>
<ZTIP_OFF>B2B</ZTIP_OFF>
<Z_OFF_VERDE>N</Z_OFF_VERDE>
<Z_IMPDEP></Z_IMPDEP>
<ZAGENZIE></ZAGENZIE>
<ZGRUPPO></ZGRUPPO>
<ZCAPOGR></ZCAPOGR>
<ZCOD_OFFERTA_TOP></ZCOD_OFFERTA_TOP>
<ZFREQ>MENS</ZFREQ>
<ZTENS>220</ZTENS>
<Z_DISVENDITORE>Y</Z_DISVENDITORE>
<SERVICE>ZDIS</SERVICE>
<BRANCHE>35.23.00</BRANCHE>
<ER_TIPOUT_TF>DO</ER_TIPOUT_TF>
<ED_POTDIS>3.0</ED_POTDIS>
<ED_POTCON>3.0</ED_POTCON>
<ER_RESI_TF>Y</ER_RESI_TF>
<COANC0>1800</COANC0>
<COAND0></COAND0>
<EQ_COAND1></EQ_COAND1>
<EQ_COAND2></EQ_COAND2>
<EQ_COAND3></EQ_COAND3>
<DISDIS></DISDIS>
<EG_ENERG>N</EG_ENERG>
<EG_AUSIL>N</EG_AUSIL>
<EG_DOMD1>N</EG_DOMD1>
<EG_MIS2G></EG_MIS2G>
<EU_CLESEN></EU_CLESEN>
<DIS_CODE>D001</DIS_CODE>
<GU_CLIMP></GU_CLIMP>
<ER_ACC>ACCRES</ER_ACC>
<ED_ACCAUES></ED_ACCAUES>
<EQ_ACCFXES></EQ_ACCFXES>
<EU_STATOF>Attiva</EU_STATOF>
<AB_2G>0000-00-00</AB_2G>
<GF_CG>0.0000000</GF_CG>
<CITY1_FORN>PINEROLO</CITY1_FORN>
<COUNTRY_FORN>IT</COUNTRY_FORN>
<HOUSE_NUM1_FORN>45</HOUSE_NUM1_FORN>
<POST_CODE1_FORN>10064</POST_CODE1_FORN>
<REGION_FORN>TO</REGION_FORN>
<STREET_FORN>VIGONE</STREET_FORN>
<ROOMNUMBER></ROOMNUMBER>
<FLOOR></FLOOR>
<ZOGAL_ISTATLOC>000001</ZOGAL_ISTATLOC>
<LGZUSATZ></LGZUSATZ>
<Z_DAT_REMI></Z_DAT_REMI>
<ZINT_OPERAND>
<row id="0">
<ZCRM_CODE>E_VFXF</ZCRM_CODE>
<Z_DATA_INIZIO>2019-06-01</Z_DATA_INIZIO>
<Z_DATA_FINE>2852-08-31</Z_DATA_FINE>
<ZCRM_INPUT>EQP_VFXM</ZCRM_INPUT>
<Z_INPUT></Z_INPUT></row>
<row id="1">
<ZCRM_CODE>E_VFXF</ZCRM_CODE>
<Z_DATA_INIZIO>2019-06-01</Z_DATA_INIZIO>
<Z_DATA_FINE>2852-08-31</Z_DATA_FINE>
<ZCRM_INPUT>EQP_VFXF1</ZCRM_INPUT>
<Z_INPUT></Z_INPUT></row>
<row id="2">
<ZCRM_CODE>E_VFXF</ZCRM_CODE>
<Z_DATA_INIZIO>2019-06-01</Z_DATA_INIZIO>
<Z_DATA_FINE>2852-08-31</Z_DATA_FINE>
<ZCRM_INPUT>EQP_VFXF3</ZCRM_INPUT>
<Z_INPUT></Z_INPUT></row>
<row id="3">
<ZCRM_CODE>E_VFXF</ZCRM_CODE>
<Z_DATA_INIZIO>2019-06-01</Z_DATA_INIZIO>
<Z_DATA_FINE>2852-08-31</Z_DATA_FINE>
<ZCRM_INPUT>EQP_VFXF2</ZCRM_INPUT>
<Z_INPUT></Z_INPUT></row></ZINT_OPERAND>
<ZINT_ACCISE/></row></XT_SFDC>
<X_HEAD>
<ZID_FLUSSO>165420000000001</ZID_FLUSSO>
<ZCOD_FLUSSO>IN1</ZCOD_FLUSSO>
<ZPROCESS>ATT</ZPROCESS>
<ZDATA_RIC>2019-07-01</ZDATA_RIC>
</X_HEAD>
<X_N_RECORDS>1</X_N_RECORDS>
</import>
<export>
<YT_ESITI>
<row id="0">
<ZID_LETTURA>A0U1X000000GQ1P</ZID_LETTURA>
<ZCOD_ERR>INT_004</ZCOD_ERR>
<ZDESCR>ERRORE LISTA VALORI</ZDESCR>
<ZMOTIVAZIONE>Valore 35.23.00 non ammesso per campo BRANCHE</ZMOTIVAZIONE>
<ZID_LETT_INT>000000000000001</ZID_LETT_INT></row></YT_ESITI>
<Y_ESITO_HEAD>
<ZID_FLUSSO></ZID_FLUSSO>
<ZESITO>OKP</ZESITO>
<ZCOD_ERR></ZCOD_ERR>
<ZDESCR></ZDESCR>
<ZMOTIVAZIONE></ZMOTIVAZIONE>
</Y_ESITO_HEAD>
</export>
</ZINT_SFDC_COMM>

```

thanks

---

<div class="post-metadata">

**Author:** ![dreamer1](https://avatars.discourse-cdn.com/v4/letter/d/b5e925/32.png) [@dreamer1](https://discuss.elastic.co/u/dreamer1)\
**Post date:** [July 16, 2019, 9:31am UTC](https://discuss.elastic.co/t/scripted-field-read-field-data-from-a-document-using-doc-some-field-value-return-null-in-some-cases/187385/6 "2019-07-16T09:31:13Z")

</div>

Hi nickpeihl,  
is it allowed to upgrade only Kibana with a newver version?  
(I don't wnt to change my elasticsearch version v. 6.3)  
Many thanks

---

<div class="post-metadata">

**Author:** ![dreamer1](https://avatars.discourse-cdn.com/v4/letter/d/b5e925/32.png) [@dreamer1](https://discuss.elastic.co/u/dreamer1)\
**Post date:** [July 30, 2019, 9:00am UTC](https://discuss.elastic.co/t/scripted-field-read-field-data-from-a-document-using-doc-some-field-value-return-null-in-some-cases/187385/7 "2019-07-30T09:00:26Z")

</div>

Just to update, this problem seems to exist in the 7.2 version  
too.

> [@Scripted field not working properly in ELK 7.2](https://discuss.elastic.co/t/scripted-field-not-working-properly-in-elk-7-2/188599/4):
>
> Hi @mark.quilates, I have the same problem with ver. 6.3.0 They replied that it was a bug in 6.3.0 , but apparently it came back again in later versions(regression?). I'd like to know the "stable" version, and maybe before installing and configuring everything...

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 27, 2019, 9:00am UTC](https://discuss.elastic.co/t/scripted-field-read-field-data-from-a-document-using-doc-some-field-value-return-null-in-some-cases/187385/8 "2019-08-27T09:00:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.