Search query format

Elastic Stack Elasticsearch
1

Hi ,

I am trying to run search from elastic based on output from another application which dumps data in JSON format. Here is the format :

{"currentRow":100,"fields":[{"name":"dDocName"},{"name":"dDocTitle"},{"name":"dDocType"},{"name":"dSecurityGroup"},{"name":"dInDate"},{"name":"xColor"},{"name":"xPersonType"},{"name":"xRegionDefinition"},{"name":"xLibraryGUID"},{"name":"dDocLastModifiedDate"},{"name":"xIdentityNum"},{"name":"xLonTriggeyu"},{"name":"xIntField"},{"name":"dRevClassID"},{"name":"xFFTest"},{"name":"xWCWorkflowAssignment"},{"name":"dDocClass"},{"name":"xWebsiteObjectType"},{"name":"xCustomerCode"},{"name":"xInvoiceNum"},{"name":"AlternateFormat"},{"name":"dDocAuthor"},{"name":"xfruit"},{"name":"xSupplierNum"},{"name":"xEBSParam"},{"name":"xTestTree"},{"name":"xVideoRenditions"},{"name":"xStorageRule"},{"name":"xstatecitymemo"},{"name":"xPOHeaderId"},{"name":"xTREELOCATION"},{"name":"xDamConversionType"},{"name":"xInvoiceAmount"},{"name":"xDiscussionType"},{"name":"dDocFunction"},{"name":"xModifiedBy"},{"name":"xCustomerTaxPayerId"},{"name":"dOutDate"},{"name":"xIPMSYS_BATCH_SEQ"},{"name":"dDocLastModifier"},{"name":"dFormat"},{"name":"dRendition2"},{"name":"dRendition1"},{"name":"xCustomerName"},{"name":"xHideThread"},{"name":"xGender"},{"name":"xWCTags"},{"name":"xExtURL"},{"name":"xTestFolder1"},{"name":"xPackagedConversions"},{"name":"xClbraRoleList"},{"name":"xFFTest1"},{"name":"xInvoiceCurrency"},{"name":"dDocCreatedDate"},{"name":"xWebsites"},{"name":"xTestFiddler"},{"name":"xDontShowInListsForWebsites"},{"name":"dDocAccount"},{"name":"URL"},{"name":"xClbraUserList"},{"name":"xAvaya_Region"},{"name":"dCreateDate"},{"name":"dID"},{"name":"xSri2"},{"name":"dExtension"},{"name":"xSri1"},{"name":"xfwm_cat_Mercados"},{"name":"dWebExtension"},{"name":"xcateg1"},{"name":"xChecksum"},{"name":"xPONum"},{"name":"dDocCreator"},{"name":"VaultFileSize"},{"name":"dRevLabel"},{"name":"xFirstName"},{"name":"xCMUTest"},{"name":"xDiscussionCount"},{"name":"xClbraAliasList"},{"name":"xPartitionId"},{"name":"dGif"},{"name":"xIPMSYS_APP_ID"},{"name":"dFullTextFormat"},{"name":"xTest1"},{"name":"xFamilyName"},{"name":"xInvoice"},{"name":"xInvoiceDate"},{"name":"dRevisionID"},{"name":"xWebsiteSection"},{"name":"xWCWorkflowApproverUserList"},{"name":"WebFileSize"},{"name":"xComments"},{"name":"xWebFlag"},{"name":"xNewtest"},{"name":"xOptionListIssue"},{"name":"xtest"},{"name":"xIPMSYS_BATCH_ID1"},{"name":"xIdcProfile"},{"name":"dOriginalName"},{"name":"dDocOwner"},{"name":"dPublishType"},{"name":"otsFormat"},{"name":"otsCharset"},{"name":"otsLanguage"},{"name":"SCORE"},{"name":"srfDocSnippet"}],"rows":[["WCCPS7_024401","test1","EBSAttachment","AOK-Public","5/3/16 7:18 AM","","","IDCNULL","","5/3/16 7:19 AM","","","0","24401","","","","","","","","wccuser","","","","0","","DispByContentId","","","","","","N/A","","","","","","wccuser","Application/unknown","","","","","","","","","","","","","5/3/16 7:19 AM","","","","","/cs/groups/aok-public/documents/ebsattachment/czdf/mdi0/~edisp/wccps7_024401","","","5/3/16 7:19 AM","24801","","","","","","","2cd4124073fed81c624af0101ba28bda16db650fee35cbc8fd629904dead1b09/SHA-256","","wccuser","377","1","","","0","","","archiv.gif","","","","","","","1","","","377","","","","","","0","EBSProfile","Untitled Document","wccuser","","","","","3",""],["WCCPS7_024202","DLEASE_RAW_response","Document","AOK-Public","4/19/16 11:11 AM","","","IDCNULL","","4/19/16 11:11 AM","","","0","24202","","","","","","","","weblogic","","","","0","","DispByContentId","","","","","","N/A","","","","","","weblogic","text/plain","","","","","","","","","","","","","4/19/16 11:11 AM","","","","","/cs/groups/aok-public/documents/document/czdf/mdi0/~edisp/wccps7_024202.txt","","","4/19/16 11:11 AM","24402","","txt","","","txt","","2e2a98a3af833032d4f2b5ec3a8c62b80edeb13ac417d472744c713e4cae27e5/SHA-256","","weblogic","594","1","","","0","","","ucm_document.png","","txt","","","","","1","","","594","","","","","","0","","DLEASE_RAW_response.txt","weblogic","","","","","3",""]

Have dumped this data to a json file and uploaded it to elastic.

I am unable to create a query which would list items / data based on specific values for each of the fields.
For eg :

In this data dump , how should I set up a query which will return all items where dDocAuthor is weblogic .

Any inputs will be very helpful .

Thanks,
Srinath

2

Hi,

Can some one kindly provide some pointers for this query ?

Thanks,
Srinath

3

Elastic works with JSON that is of this form:

{
	"fieldName": "value1",
	"fieldName2": "value2"
}

Your JSON is unconventional because the field names and values are not paired with each other. You essentially have 2 arrays - one of field names and one of values and elasticsearch is not built to understand that these should be associated in any way.
You need to provide JSON as in my example where field names are used as the left hand side of a key:value pairing.

1 Like
4

Thanks Mark for the confirmation . I thought it was more so related to the format of JSON being created by the external application .

Would elastic have some form of manipulation logic where this json could be interpreted correctly ? Or any other ways that you can think of which can be applied to get this requirement working ?

-Srinath

5

Some basic Python?

oldDoc ={
	"fields":["foo", "bar"],
	"values":[1,2]
}
newDoc={}
for idx, field in enumerate( oldDoc["fields"] ):
	newDoc[field] = oldDoc["values"][idx]