# I tried to upload a log file and a CSV file through the Kibana UI upload file option. But in both cases getting "File structure cannot be determined" error

**URL:** <https://discuss.elastic.co/t/i-tried-to-upload-a-log-file-and-a-csv-file-through-the-kibana-ui-upload-file-option-but-in-both-cases-getting-file-structure-cannot-be-determined-error/314878>\
**Category:** Kibana\
**Created:** [September 21, 2022, 2:39pm UTC](https://discuss.elastic.co/t/i-tried-to-upload-a-log-file-and-a-csv-file-through-the-kibana-ui-upload-file-option-but-in-both-cases-getting-file-structure-cannot-be-determined-error/314878 "2022-09-21T14:39:30Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![cadrija](https://avatars.discourse-cdn.com/v4/letter/c/dc4da7/32.png) [@cadrija](https://discuss.elastic.co/u/cadrija)\
**Post date:** [September 21, 2022, 2:39pm UTC](https://discuss.elastic.co/t/i-tried-to-upload-a-log-file-and-a-csv-file-through-the-kibana-ui-upload-file-option-but-in-both-cases-getting-file-structure-cannot-be-determined-error/314878/1 "2022-09-21T14:39:30Z")

</div>

I am new to Elastic.  
I have setup ELK in Ubuntu machine.  
I tried to upload a log file and a CSV file through the Kibana UI upload file option. But in both cases getting "File structure cannot be determined" error.  
I have tried to override settings to take 10 lines only, still getting same issue.  
Please help.

Error snapshot

 ![Kibana upload error](https://us1.discourse-cdn.com/elastic/original/3X/7/0/701c41924295e4c5e7be4616a770cbad8e866c44.jpeg)

My CSV file looks like this - just a simple CSV file

 ![CSV file](https://us1.discourse-cdn.com/elastic/original/3X/5/c/5c792f9c02b8beb662ff89d713e44a0a1e2321e3.jpeg)

My log file looks like this

 ![Log file](https://us1.discourse-cdn.com/elastic/original/3X/d/c/dcfb933c1786bb09f57e990c8a9668e4fc7f8e05.jpeg)

---

<div class="post-metadata">

**Author:** ![nickpeihl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nickpeihl/32/112622_2.png) [@nickpeihl](https://discuss.elastic.co/u/nickpeihl)\
**Post date:** [September 22, 2022, 2:15pm UTC](https://discuss.elastic.co/t/i-tried-to-upload-a-log-file-and-a-csv-file-through-the-kibana-ui-upload-file-option-but-in-both-cases-getting-file-structure-cannot-be-determined-error/314878/2 "2022-09-22T14:15:12Z")

</div>

Hi @cadrija. Can you share some more information?

- What version of Kibana are you on?
- How large are your CSV and log files?
- Do you see any relevant errors in the Kibana server logs (likely located at `/var/log/kibana`)?

---

<div class="post-metadata">

**Author:** ![cadrija](https://avatars.discourse-cdn.com/v4/letter/c/dc4da7/32.png) [@cadrija](https://discuss.elastic.co/u/cadrija)\
**Post date:** [September 23, 2022, 5:19am UTC](https://discuss.elastic.co/t/i-tried-to-upload-a-log-file-and-a-csv-file-through-the-kibana-ui-upload-file-option-but-in-both-cases-getting-file-structure-cannot-be-determined-error/314878/3 "2022-09-23T05:19:13Z")

</div>

Hi @nickpeihl  
Please consider below answers.

- What version of Kibana are you on? - 7.17.6
- How large are your CSV and log files? - Log file is 4.76 MB and CSV file is 2.38 MB
- Do you see any relevant errors in the Kibana server logs (likely located at `/var/log/kibana`)? - Okay I will check this.  
Update - Checked logs, no error stack trace found.

---

<div class="post-metadata">

**Author:** ![nickpeihl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nickpeihl/32/112622_2.png) [@nickpeihl](https://discuss.elastic.co/u/nickpeihl)\
**Post date:** [September 26, 2022, 2:08pm UTC](https://discuss.elastic.co/t/i-tried-to-upload-a-log-file-and-a-csv-file-through-the-kibana-ui-upload-file-option-but-in-both-cases-getting-file-structure-cannot-be-determined-error/314878/4 "2022-09-26T14:08:36Z")

</div>

Hi @cadrija. What version of Kibana are you running? I wonder if your issue is related to [this bug](https://github.com/elastic/kibana/issues/96425)?

---

<div class="post-metadata">

**Author:** ![cadrija](https://avatars.discourse-cdn.com/v4/letter/c/dc4da7/32.png) [@cadrija](https://discuss.elastic.co/u/cadrija)\
**Post date:** [September 28, 2022, 5:25am UTC](https://discuss.elastic.co/t/i-tried-to-upload-a-log-file-and-a-csv-file-through-the-kibana-ui-upload-file-option-but-in-both-cases-getting-file-structure-cannot-be-determined-error/314878/5 "2022-09-28T05:25:55Z")

</div>

Same version as Elasticsearch i.e. 7.17.6

---

<div class="post-metadata">

**Author:** ![cadrija](https://avatars.discourse-cdn.com/v4/letter/c/dc4da7/32.png) [@cadrija](https://discuss.elastic.co/u/cadrija)\
**Post date:** [September 28, 2022, 5:29am UTC](https://discuss.elastic.co/t/i-tried-to-upload-a-log-file-and-a-csv-file-through-the-kibana-ui-upload-file-option-but-in-both-cases-getting-file-structure-cannot-be-determined-error/314878/6 "2022-09-28T05:29:06Z")

</div>

I checked with different CSV file, it seems if the file has timestamp and it is matching with the listed time format then it is getting imported.  
However all of my log files are throwing the above mentioned error even if they are having timestamp and time format is matching the listed options.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [September 28, 2022, 6:05am UTC](https://discuss.elastic.co/t/i-tried-to-upload-a-log-file-and-a-csv-file-through-the-kibana-ui-upload-file-option-but-in-both-cases-getting-file-structure-cannot-be-determined-error/314878/7 "2022-09-28T06:05:06Z")

</div>

Hi @cadrija Welcome to the community!

Apologies you are having issues Can you provide a couple lines of the raw CSV in text (not screen shot)?

Raw lines exactly how they are in the CSV you can anonymize anything you need to we just want to try with your data.

I suspect there is still a format issue, and you can change the format / data type under the advanced setting give us a couple lines and we may be able to help.

---

<div class="post-metadata">

**Author:** ![cadrija](https://avatars.discourse-cdn.com/v4/letter/c/dc4da7/32.png) [@cadrija](https://discuss.elastic.co/u/cadrija)\
**Post date:** [September 28, 2022, 12:17pm UTC](https://discuss.elastic.co/t/i-tried-to-upload-a-log-file-and-a-csv-file-through-the-kibana-ui-upload-file-option-but-in-both-cases-getting-file-structure-cannot-be-determined-error/314878/8 "2022-09-28T12:17:52Z")

</div>

Hello @stephenb thank you!

**Sample logs 1**

```auto
2022-09-21T04:39:42,473 [main] ERROR: Test log: Inside LoanIQLoggingManager
2022-09-21T04:39:44,685 [main] INFO: Started loading datatables
2022-09-21T04:39:44,701 [main] INFO: Found resource DataTable.Deal.xml, jar:file:/C:/LoanIQ/Server/lib/liq_datatables-7.6.0.0.jar!/DataTable.Deal.xml
2022-09-21T04:39:44,732 [main] INFO: adding key:datatable.abstractsgchangetransaction.xml, value:{DataTable.AbstractSGChangeTransaction.xml, jar:file:/C:/LoanIQ/Server/lib/liq_datatables-7.6.0.0.jar!/DataTable.AbstractSGChangeTransaction.xml}
2022-09-21T04:39:44,732 [main] INFO: adding key:datatable.abstractschedule.xml, value:{DataTable.AbstractSchedule.xml, jar:file:/C:/LoanIQ/Server/lib/liq_datatables-7.6.0.0.jar!/DataTable.AbstractSchedule.xml}
2022-09-21T04:39:44,732 [main] INFO: adding key:datatable.abstractscheduleitem.xml, value:{DataTable.AbstractScheduleItem.xml, jar:file:/C:/LoanIQ/Server/lib/liq_datatables-7.6.0.0.jar!/DataTable.AbstractScheduleItem.xml}
2022-09-21T04:39:44,732 [main] INFO: adding key:datatable.abstracttemplatefield.xml, value:{DataTable.AbstractTemplateField.xml, jar:file:/C:/LoanIQ/Server/lib/liq_datatables-7.6.0.0.jar!/DataTable.AbstractTemplateField.xml}
2022-09-21T04:39:44,732 [main] INFO: adding key:datatable.accrualcyclepayment.xml, value:{DataTable.AccrualCyclePayment.xml, jar:file:/C:/LoanIQ/Server/lib/liq_datatables-7.6.0.0.jar!/DataTable.AccrualCyclePayment.xml}
2022-09-21T04:39:44,732 [main] INFO: adding key:datatable.accruallineitem.xml, value:{DataTable.AccrualLineItem.xml, jar:file:/C:/LoanIQ/Server/lib/liq_datatables-7.6.0.0.jar!/DataTable.AccrualLineItem.xml}
2022-09-21T04:39:44,732 [main] INFO: adding key:datatable.accrualmatchfundedcostoffundspayableaggregation.xml, value:{DataTable.AccrualMatchFundedCostOfFundsPayableAggregation.xml, jar:file:/C:/LoanIQ/Server/lib/liq_datatables-7.6.0.0.jar!/DataTable.AccrualMatchFundedCostOfFundsPayableAggregation.xml}

```

**Sample logs 2**

```auto
2022-09-01 09:08:05.369 INFO 119618 --- [main] c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at : http://localhost:8887
2022-09-01 09:08:05.966 INFO 119618 --- [main] c.c.c.ConfigServicePropertySourceLocator : Located environment: name=userManagement-service, profiles=[development], label=null, version=null, state=null
2022-09-01 09:08:05.968 INFO 119618 --- [main] b.c.PropertySourceBootstrapConfiguration : Located property source: [BootstrapPropertySource {name='bootstrapProperties-file:./userManagement-service.yaml'}, BootstrapPropertySource {name='bootstrapProperties-classpath:/config/userManagement-service.yaml'}]
2022-09-01 09:08:05.975 INFO 119618 --- [main] c.u.microservice.LaunchUserApplication : The following profiles are active: development
2022-09-01 09:08:08.025 INFO 119618 --- [main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFERRED mode.
2022-09-01 09:08:08.637 INFO 119618 --- [main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 597ms. Found 34 JPA repository interfaces.
2022-09-01 09:08:08.818 WARN 119618 --- [main] o.s.boot.actuate.endpoint.EndpointId : Endpoint ID 'service-registry' contains invalid characters, please migrate to a valid format.
2022-09-01 09:08:08.842 WARN 119618 --- [main] o.s.boot.actuate.endpoint.EndpointId : Endpoint ID 'hystrix.stream' contains invalid characters, please migrate to a valid format.
2022-09-01 09:08:09.138 INFO 119618 --- [main] o.s.cloud.context.scope.GenericScope : BeanFactory id=267123e5-afdb-3749-aa42-6d221c5a80ab
2022-09-01 09:08:09.730 INFO 119618 --- [main] trationDelegate$BeanPostProcessorChecker : Bean 'zuulConfiguration' of type [com.usermanagement.microservice.ZuulConfiguration$$EnhancerBySpringCGLIB$$82804667] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)

```

**Sample CSV**

| carat | cut | color | clarity | depth | table | price | x | y | z |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 0.23 | Ideal | E | SI2 | 61.5 | 55 | 326 | 3.95 | 3.98 | 2.43 |
| 0.21 | Premium | E | SI1 | 59.8 | 61 | 326 | 3.89 | 3.84 | 2.31 |
| 0.23 | Good | E | VS1 | 56.9 | 65 | 327 | 4.05 | 4.07 | 2.31 |
| 0.29 | Premium | I | VS2 | 62.4 | 58 | 334 | 4.2 | 4.23 | 2.63 |
| 0.31 | Good | J | SI2 | 63.3 | 58 | 335 | 4.34 | 4.35 | 2.75 |
| 0.24 | Very Good | J | VVS2 | 62.8 | 57 | 336 | 3.94 | 3.96 | 2.48 |
| 0.24 | Very Good | I | VVS1 | 62.3 | 57 | 336 | 3.95 | 3.98 | 2.47 |
| 0.26 | Very Good | H | SI1 | 61.9 | 55 | 337 | 4.07 | 4.11 | 2.53 |
| 0.22 | Fair | E | VS2 | 65.1 | 61 | 337 | 3.87 | 3.78 | 2.49 |
| 0.23 | Very Good | H | VS1 | 59.4 | 61 | 338 | 4 | 4.05 | 2.39 |

Please let me know if this helps. I would be very grateful if any of the log files were able to get imported.  
Thanks in advance!

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [September 28, 2022, 2:54pm UTC](https://discuss.elastic.co/t/i-tried-to-upload-a-log-file-and-a-csv-file-through-the-kibana-ui-upload-file-option-but-in-both-cases-getting-file-structure-cannot-be-determined-error/314878/9 "2022-09-28T14:54:33Z")

</div>

Hi @cadrija

I am using 7.17.6, Basic License

In short the samples you provided just loaded up for me, no extra work. etc. I simply used all the default and clicked Import..

I wonder if you have some corrupt characters, or something else in the files...These samples loaded fine for me....

A little what I noticed

First your Sample1 and 2 only partially meet the criteria

- Delimited text files, such as CSV and TSV
- Newline-delimited JSON
- Log files with a common format for the timestamp

Sample 1 and Sample 2

Have a common time format but are not CSV, TSV delimited or ndjson the the automatic parsing will be minimal in other words it will not parse "the message" portion

Both of these imported the first time for me. ...

 ![Screen Shot 2022-09-28 at 7.37.49 AM](https://us1.discourse-cdn.com/elastic/original/3X/4/5/458bb4b0f3d204662d2db7fbc689c74529c98cb2.jpeg)

 ![Screen Shot 2022-09-28 at 7.39.58 AM](https://us1.discourse-cdn.com/elastic/original/3X/b/e/beb20da7193df47f7428ae8a7c1a02ab15763c3f.png)

I just Clicked Through and and the Log Was parsed and Loaded

BUT it is not going to parse the whole log message because it is CSV, TSV or ndjson it is Unstructured Text

**You will need to build your own custom parsing to parse these logs if you want additional parsing**

The results look like

```auto
GET discuss-sample-1/_search
...
     {
        "_index" : "discuss-sample-1",
        "_type" : "_doc",
        "_id" : "xQ6JhIMB2qNNMYeXRN1I",
        "_score" : 1.0,
        "_source" : {
          "@timestamp" : "2022-09-21T04:39:42.473-07:00",
          "loglevel" : "ERROR",
          "message" : "2022-09-21T04:39:42,473 [main] ERROR: Test log: Inside LoanIQLoggingManager"
        }
      },
      {
        "_index" : "discuss-sample-1",
        "_type" : "_doc",
        "_id" : "xg6JhIMB2qNNMYeXRN1I",
        "_score" : 1.0,
        "_source" : {
          "@timestamp" : "2022-09-21T04:39:44.685-07:00",
          "loglevel" : "INFO",
          "message" : "2022-09-21T04:39:44,685 [main] INFO: Started loading datatables"
        }
      }, {
        "_index" : "discuss-sample-1",
        "_type" : "_doc",
        "_id" : "xQ6JhIMB2qNNMYeXRN1I",
        "_score" : 1.0,
        "_source" : {
          "@timestamp" : "2022-09-21T04:39:42.473-07:00",
          "loglevel" : "ERROR",
          "message" : "2022-09-21T04:39:42,473 [main] ERROR: Test log: Inside LoanIQLoggingManager"
        }
      },
      {
        "_index" : "discuss-sample-1",
        "_type" : "_doc",
        "_id" : "xg6JhIMB2qNNMYeXRN1I",
        "_score" : 1.0,
        "_source" : {
          "@timestamp" : "2022-09-21T04:39:44.685-07:00",
          "loglevel" : "INFO",
          "message" : "2022-09-21T04:39:44,685 [main] INFO: Started loading datatables"
        }
      },

```

In Discover

 ![Screen Shot 2022-09-28 at 7.46.36 AM](https://us1.discourse-cdn.com/elastic/original/3X/d/2/d28ab7dbc45150e6ecd0d4feac3e3fd0a6ab27ed.png)

This is Sample2 in

 ![Screen Shot 2022-09-28 at 7.47.49 AM](https://us1.discourse-cdn.com/elastic/original/3X/7/1/710961b877c8cde267cfe3b61bfbc99a001ab542.png)

Sample 3 you did not provide the actual CSV txt ... I would expect that to load as well..

---

<div class="post-metadata">

**Author:** ![cadrija](https://avatars.discourse-cdn.com/v4/letter/c/dc4da7/32.png) [@cadrija](https://discuss.elastic.co/u/cadrija)\
**Post date:** [September 29, 2022, 6:59am UTC](https://discuss.elastic.co/t/i-tried-to-upload-a-log-file-and-a-csv-file-through-the-kibana-ui-upload-file-option-but-in-both-cases-getting-file-structure-cannot-be-determined-error/314878/10 "2022-09-29T06:59:09Z")

</div>

@stephenb Thanks a lot!  
I tried with partial file it is importing first 1000 lines only. Let me analyze what is wrong with the whole file.  
Could you kindly help me with another query.  
What do I need to do in order to pull this logs live from another machine?  
My ELK is on x.x.x.x host and the application log file is on y.y.y.y host.  
I am learning about the integrations but I am confused about which one to use in this case.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [September 29, 2022, 2:30pm UTC](https://discuss.elastic.co/t/i-tried-to-upload-a-log-file-and-a-csv-file-through-the-kibana-ui-upload-file-option-but-in-both-cases-getting-file-structure-cannot-be-determined-error/314878/11 "2022-09-29T14:30:30Z")

</div>

> [@cadrija](#):
>
> What do I need to do in order to pull this logs live from another machine?  
> My ELK is on x.x.x.x host and the application log file is on y.y.y.y host.

Please open a new thread with the new question with all the details.

You'll need to use filebeat or the elastic agent. I would suggest that you read about those. That's pretty common way to ship logs.

If you're very new to all these concepts, I might just start with filebeat.

---

<div class="post-metadata">

**Author:** ![cadrija](https://avatars.discourse-cdn.com/v4/letter/c/dc4da7/32.png) [@cadrija](https://discuss.elastic.co/u/cadrija)\
**Post date:** [September 30, 2022, 5:23am UTC](https://discuss.elastic.co/t/i-tried-to-upload-a-log-file-and-a-csv-file-through-the-kibana-ui-upload-file-option-but-in-both-cases-getting-file-structure-cannot-be-determined-error/314878/12 "2022-09-30T05:23:52Z")

</div>

Thank you so much @stephenb

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 28, 2022, 5:23am UTC](https://discuss.elastic.co/t/i-tried-to-upload-a-log-file-and-a-csv-file-through-the-kibana-ui-upload-file-option-but-in-both-cases-getting-file-structure-cannot-be-determined-error/314878/13 "2022-10-28T05:23:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.