How to export csv when _source is disabled

[t98907]

Hi there,

I looked into the reason why kibana can't export csv.
It was because I had _source disabled in my index, so I couldn't export csv with kibana.

But I'm storing the fields I need, and discover is outputting the values.

Is there a way to export csv even if _source is disabled?

Thank you,

[bhavyarm]

Hello,

This is interesting. I disabled source on a sample index and printed out the CSV. Granted it was of not much use because my ingested data was pretty dummy.

Can you please let me know how are you disabling the source and do you have permissions to print CSVs?

Thanks,
Bhavya

[t98907]

Hi,

Thanks for the reply. The kibana log was as follows:

{"type":"log","@timestamp":"2020-08-04T04:23:52Z","tags":["info","reporting","savedobject-csv","csv_from_savedobject","execute-job","immediate"],"pid":7,"message":"Executing job from Immediate API using request context"}
{"type":"log","@timestamp":"2020-08-04T04:23:52Z","tags":["error","reporting","savedobject-csv"],"pid":7,"message":"{ Error: [illegal_argument_exception] unable to fetch fields from _source field: _source is disabled in the mappings for index [job-202007201811]\n    at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:349:15)\n    at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:306:7)\n    at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)\n    at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4929:19)\n    at IncomingMessage.emit (events.js:203:15)\n    at endReadableNT (_stream_readable.js:1145:12)\n    at process._tickCallback (internal/process/next_tick.js:63:19)\n  status: 400,\n  displayName: 'BadRequest',\n  message:\n   '[illegal_argument_exception] unable to fetch fields from _source field: _source is disabled in the mappings for index [job-202007201811]',\n  path: '/job/_search',\n  query: { scroll: '30s', size: 500 },\n  body:\n   { error:\n      { root_cause: [Array],\n        type: 'search_phase_execution_exception',\n        reason: 'all shards failed',\n        phase: 'fetch',\n        grouped: true,\n        failed_shards: [Array],\n        caused_by: [Object] },\n     status: 400 },\n  statusCode: 400,\n  response:\n   '{\"error\":{\"root_cause\":[{\"type\":\"illegal_argument_exception\",\"reason\":\"unable to fetch fields from _source field: _source is disabled in the mappings for index [job-202007201811]\"}],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"fetch\",\"grouped\":true,\"failed_shards\":[{\"shard\":0,\"index\":\"job-202007201811\",\"node\":\"6-g-PYqpQgeJ8P_SuApQ9Q\",\"reason\":{\"type\":\"illegal_argument_exception\",\"reason\":\"unable to fetch fields from _source field: _source is disabled in the mappings for index [job-202007201811]\"}}],\"caused_by\":{\"type\":\"illegal_argument_exception\",\"reason\":\"unable to fetch fields from _source field: _source is disabled in the mappings for index [job-202007201811]\",\"caused_by\":{\"type\":\"illegal_argument_exception\",\"reason\":\"unable to fetch fields from _source field: _source is disabled in the mappings for index [job-202007201811]\"}}},\"status\":400}',\n  toString: [Function],\n  toJSON: [Function] }"}
{"type":"log","@timestamp":"2020-08-04T04:23:52Z","tags":["error","reporting","savedobject-csv","csv_from_savedobject","execute-job","immediate"],"pid":7,"message":"Generate CSV Error! [illegal_argument_exception] unable to fetch fields from _source field: _source is disabled in the mappings for index [job-202007201811] :: {\"path\":\"/job/_search\",\"query\":{\"scroll\":\"30s\",\"size\":500},\"body\":\"{\\\"_source\\\":{\\\"includes\\\":[\\\"orderDate\\\",\\\"jutakuNo\\\",\\\"company\\\",\\\"head\\\"]},\\\"docvalue_fields\\\":[\\\"orderDate\\\",{\\\"field\\\":\\\"orderDate\\\",\\\"format\\\":\\\"date_time\\\"}],\\\"query\\\":{\\\"bool\\\":{\\\"must\\\":[{\\\"match_all\\\":{}}],\\\"filter\\\":[{\\\"range\\\":{\\\"orderDate\\\":{\\\"format\\\":\\\"strict_date_time\\\",\\\"gte\\\":\\\"2020-07-28T13:23:51+09:00\\\",\\\"lte\\\":\\\"2020-08-04T13:23:51+09:00\\\"}}},{\\\"bool\\\":{\\\"must\\\":[{\\\"match_all\\\":{}},{\\\"match_all\\\":{}}],\\\"filter\\\":[{\\\"range\\\":{\\\"orderDate\\\":{\\\"gte\\\":\\\"2020-07-28T04:23:51.910Z\\\",\\\"lte\\\":\\\"2020-08-04T04:23:51.911Z\\\",\\\"format\\\":\\\"strict_date_optional_time\\\"}}}],\\\"should\\\":[],\\\"must_not\\\":[]}}],\\\"should\\\":[],\\\"must_not\\\":[]}},\\\"script_fields\\\":{},\\\"sort\\\":[{\\\"orderDate\\\":{\\\"order\\\":\\\"desc\\\",\\\"unmapped_type\\\":\\\"boolean\\\"}},{\\\"orderDate\\\":{\\\"order\\\":\\\"desc\\\"}}]}\",\"statusCode\":400,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"illegal_argument_exception\\\",\\\"reason\\\":\\\"unable to fetch fields from _source field: _source is disabled in the mappings for index [job-202007201811]\\\"}],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"fetch\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[{\\\"shard\\\":0,\\\"index\\\":\\\"job-202007201811\\\",\\\"node\\\":\\\"6-g-PYqpQgeJ8P_SuApQ9Q\\\",\\\"reason\\\":{\\\"type\\\":\\\"illegal_argument_exception\\\",\\\"reason\\\":\\\"unable to fetch fields from _source field: _source is disabled in the mappings for index [job-202007201811]\\\"}}],\\\"caused_by\\\":{\\\"type\\\":\\\"illegal_argument_exception\\\",\\\"reason\\\":\\\"unable to fetch fields from _source field: _source is disabled in the mappings for index [job-202007201811]\\\",\\\"caused_by\\\":{\\\"type\\\":\\\"illegal_argument_exception\\\",\\\"reason\\\":\\\"unable to fetch fields from _source field: _source is disabled in the mappings for index [job-202007201811]\\\"}}},\\\"status\\\":400}\"}"}
{"type":"error","@timestamp":"2020-08-04T04:23:51Z","tags":[],"pid":7,"level":"error","error":{"message":"[illegal_argument_exception] unable to fetch fields from _source field: _source is disabled in the mappings for index [job-202007201811]","name":"Error","stack":"Error: [illegal_argument_exception] unable to fetch fields from _source field: _source is disabled in the mappings for index [job-202007201811]\n    at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:349:15)\n    at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:306:7)\n    at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)\n    at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4929:19)\n    at IncomingMessage.emit (events.js:203:15)\n    at endReadableNT (_stream_readable.js:1145:12)\n    at process._tickCallback (internal/process/next_tick.js:63:19)"},"url":{"protocol":null,"slashes":null,"auth":null,"host":null,"port":null,"hostname":null,"hash":null,"search":null,"query":{},"pathname":"/api/reporting/v1/generate/immediate/csv/saved-object/search:901b67a0-25ba-11e8-8735-1dd0f5e7cc80","path":"/api/reporting/v1/generate/immediate/csv/saved-object/search:901b67a0-25ba-11e8-8735-1dd0f5e7cc80","href":"/api/reporting/v1/generate/immediate/csv/saved-object/search:901b67a0-25ba-11e8-8735-1dd0f5e7cc80"},"message":"[illegal_argument_exception] unable to fetch fields from _source field: _source is disabled in the mappings for index [job-202007201811]"}

In the job-202007201811 index schema, _source is defined as follows:

    ...
      "mappings": {
        "_doc": {
          "dynamic": false,
          "_source": {
            "enabled": false
          },
    ...

The roles of dashboard-only and reporting_user are assigned to kibana users.

Thank you,