How to make query output data format in 1 string


(Joyce Feng) #1

In ES2.2, the output source is 1 single line/string:
"hits" : [ {
"_index" : "ise",
"_type" : "contextrepo",
"_id" : "da:d8:54:41:ee:23",
"_score" : 1.0,
"_source":{"AuthenticationMethod":"Wired","AuthorizationPolicyMatchedRule":"Default","SelectedAuthorizationProfiles":"Dot1X","AuthenticationIdentityStore":"InternalUser","Description":"description7499628","Email":"emailid153048@rediff.com","EndPointPolicy":"XeroxWorkCentre7556","FirstName":"fname9494","IdentityGroup":"Engineering","ip":"250.64.136.241","ipv6":"9feb:7ec8:3e6c:2a97::9","LastName":"lname75739","LogicalProfile":"GamingDevices","MACAddress":"da:d8:54:41:ee:23","MDMCompliant":"noncompliant","MDMDiskEncrypted":"true","MDMJailBroken":"false","MDMPinLockSet":"true","mdmRegistrationStatus":"false","Location":"locations#alllocations#ntn#ntn10","OUI":"da:d8:54","PostureStatus":"unknown","status":"Connected","PortalUser":"portaluser56880","userName":"user103124","NetworkDeviceName":"ntn10NAD456908","StaticAssignment":"Yes","StaticGroupAssignment":"Yes","FailureReason":"Reason Code: 15455","User-Fetch-Department":"ABC-3","User-Fetch-Telephone":"19","User-Fetch-Job-Title":"ENG-3","User-Fetch-Organizational-Unit":"SAMPG-8","User-Fetch-CountryName":"USA-12","User-Fetch-LocalityName":"LOC-4","User-Fetch-StateOrProvinceName":"CA-2","User-Fetch-StreetAddress":"STR-8","UserType":"Guest-Sponsor","PortalUserGuestSponsor":"Desk Admin 1","PortalUserGuestType":"Contractor","PhoneID":"UDID3697091"}}

But in 5.2, the output is multiple line:
{
"_index" : "ise",
"_type" : "contextrepo",
"_id" : "f5:6e:ca:11:ed:f5",
"_score" : 1.0,
"_source" : {
"AuthenticationMethod" : "Wireless",
"AuthorizationPolicyMatchedRule" : "Default",
"SelectedAuthorizationProfiles" : "Dot1X",
"AuthenticationIdentityStore" : "Internal User",
"Description" : "description9496862",
"Email" : "emailid104970@rediff.com",
"EndPointPolicy" : "Cisco-AIR-AP-1250",
"FirstName" : "fname19454",
"IdentityGroup" : "HR",
"ip" : "43.20.3.32",
"ipv6" : "44ae:1430:27a5:10bc::9",
"LastName" : "lname97497",
"LogicalProfile" : "Medical Devices",
"MACAddress" : "f5:6e:ca:11:ed:f5",
"MDMDiskEncrypted" : "true",
"MDMJailBroken" : "true",
"MDMPinLockSet" : "true",
"mdmRegistrationStatus" : "false",
"Location" : "locations#all locations#bxb#bxb2",
"OUI" : "f5:6e:ca",
"PostureStatus" : "non-compliant",
"status" : "Connected",

How to make 5.5.2 output same as 2.4 data to be 1 string?

the command I used:
curl -k -XGET "http://localhost:9200/ise/tableindex/_search?pretty"


(David Pilato) #2

Please format your code using </> icon as explained in this guide. It will make your post more readable.

Or use markdown style like:

```
CODE
```

Remove ?pretty and you should be ok.


(Joyce Feng) #3

Remove pretty will combine all the output in one line, in ES2.4 each output is still a separate item, just the source part is one single line. I'd still like each output to be separated, but the source part to be one string.


(David Pilato) #4

Normally we don’t prettify the _source and we just present it as it was sent to elasticsearch. Apparently we do now.

You can’t have finer control I’m afraid.

pretty=true is made for humans
pretty=false is made for computers

What is your use case that would require a mix of that?


(Joyce Feng) #5

I have some perl scripts parsing the curl output data into Json, then use the Json file to do bulk import into new database.

The 5.5.2 export format broke the perl script, so I am wondering if there is some parameters to set to keep the 2.4 curl output format, or else I will fix the perl script.


(David Pilato) #6

I just double checked and indeed when you use pretty in 5.6 we are prettifying also the _source field.

With:

DELETE test
PUT test/doc/1
{"foo":"bar"}
PUT test/doc/2
{
  
  "foo": "bar"

  
}
$ curl -XGET "http://localhost:9200/test/_search?pretty"

Gives

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 2,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "test",
        "_type" : "doc",
        "_id" : "2",
        "_score" : 1.0,
        "_source" : {
          "foo" : "bar"
        }
      },
      {
        "_index" : "test",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "foo" : "bar"
        }
      }
    ]
  }
}

And

$ curl -XGET "http://localhost:9200/test/_search"

Gives

{"took":1,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":2,"max_score":1.0,"hits":[{"_index":"test","_type":"doc","_id":"2","_score":1.0,"_source":{
  
  "foo": "bar"

  
}
},{"_index":"test","_type":"doc","_id":"1","_score":1.0,"_source":{"foo":"bar"}
}]}}

I'd not use pretty at all when the content is supposed to be parsed by a script.


(Joyce Feng) #7

Thanks!

Is there any option to not prettify all the others but not prettify the source field?
ES2.4 behaves like that.


(Joyce Feng) #8

Is there any option to prettify all the others but not prettify the source field?
ES2.4 behaves like that.


(David Pilato) #9

I don't think there is.

Again, pretty is meant for humans. Not pretty is meant for computers.


(system) #10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.