I'm exporting data from a view created from the discover module. In the data there is a field that is filled only in some records, less than 10% of records have this field filled. When I filter or export the entire listing, the field is not filled in the csv file. If I export the list making a filter to bring only records that have the field filled in, then the csv list is exported correctly with the field filled in.
Analyzing the logs of kibana and Elasticsearch no errors are presented that could be related to this problem.
I'm using version 7.16.1 of Elasticsearch and kibana.
Are there any errors in the Kibana server logs?
Can you provide a dataset with mappings that reproduces this problem?
The error below occurred 69 times. And this is the number of missing values.
{"type":"log","@timestamp":"2021-12-17T08:48:06-03:00","tags":["error","plugins","reporting","csv_searchsource_immediate","csv_searchsource_immediate","execute-job"],"pid":2049,"message":"TypeError: Cannot read properties of undefined (reading 'convert')\n at /usr/share/kibana/x-pack/plugins/reporting/server/export_types/csv_searchsource/generate_csv/generate_csv.js:169:42\n at CsvGenerator.generateRows (/usr/share/kibana/x-pack/plugins/reporting/server/export_types/csv_searchsource/generate_csv/generate_csv.js:256:35)\n at CsvGenerator.generateData (/usr/share/kibana/x-pack/plugins/reporting/server/export_types/csv_searchsource/generate_csv/generate_csv.js:390:9)\n at runTask (/usr/share/kibana/x-pack/plugins/reporting/server/export_types/csv_searchsource_immediate/execute_job.js:47:20)\n at /usr/share/kibana/x-pack/plugins/reporting/server/routes/generate/csv_searchsource_immediate.js:84:11\n at Router.handle (/usr/share/kibana/src/core/server/http/router/router.js:163:30)\n at handler (/usr/share/kibana/src/core/server/http/router/router.js:124:50)\n at exports.Manager.execute (/usr/share/kibana/node_modules/@hapi/hapi/lib/toolkit.js:60:28)\n at Object.internals.handler (/usr/share/kibana/node_modules/@hapi/hapi/lib/handler.js:46:20)\n at exports.execute (/usr/share/kibana/node_modules/@hapi/hapi/lib/handler.js:31:20)\n at Request._lifecycle (/usr/share/kibana/node_modules/@hapi/hapi/lib/request.js:371:32)\n at Request._execute (/usr/share/kibana/node_modules/@hapi/hapi/lib/request.js:281:9)"}
The error below occurred one time.
{"type":"log","@timestamp":"2021-12-17T08:51:42-03:00","tags":["warning","plugins","reporting","csv_searchsource_immediate","csv_searchsource_immediate","execute-job"],"pid":2049,"message":"CSV may contain formulas whose values have been escaped"}
I'm looking for a way to share the data.
Thank you for the log messages. It confirms that this open bug is the issue: [Reporting] CSV generation logs show a thrown TypeError · Issue #120278 · elastic/kibana · GitHub
Thank you very much. We just need a minimal dataset that will reproduce the problem.
Thanks for the feedback!
The problem seems to be related to data conversion, I am preparing a small dataset to share, and what I realized is that if the data is imported with the field in which the problem occurs with the data type text (originally the field is mapped with the date type), then the problem in exporting the data does not occur.
I keep working to share the data.
Hi, I want to make sure I understand.
- A field exists in 10% of the records.
- The export of the data has no filters
- When it is mapped as
date, then the bug occurs
- When the field is mapped as
textthe bug does not occur
Exactly, your description is correct.
I'm working to be able to share the data, it involves sensitive customer information, we're obfuscating the data.
Hi @Richard_Schroeder I was able to reproduce the TypeError using data with a null timestamp field for the first 500 documents, and the latter documents have the timestamp field existing in 10% of the data.
Hi @tsullivan this is good news. I saw that you updated the issue, do you still need the database to reproduce the problem?
Thank you very much for your commitment to solving the problem.
Hi Richard, I do not need you to supply any data about this issue. At this point, the fix is ready for the team to review. Thank you for your patience with this bug!
Hi tsullivan, are there any expectations for the release date of this fix?
It will be available when 8.1 is released. There is not a firm date on when that release will be.
Is there any possibility to release the fix in 7.16.3 or 7.17?
Because I believe these versions will be released before 8.1.
Thanks for the feedback.
Hi Richard, this is a reasonable request and the fix is low-risk to backport to these versions. I believe the changes for 7.16.3 and 7.17 may already be locked in and undergoing testing. I am backporting the fix to the 8.0, 7.16 and 7.17 branches.
The 7.16.3 branch may already be locked for release - if it didn't get in early enough for 7.16.3, it will be in one of the later releases.
Thank you tsullivan!
I have just one more request, when you finished the backport, let me know which version you did the backport, so I can follow along and know when it is released.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.