# Why doesn’t dense\_vector field show up in Spark schema when using Elasticsearch-Hadoop?

**URL:** https://discuss.elastic.co/t/why-doesn-t-dense-vector-field-show-up-in-spark-schema-when-using-elasticsearch-hadoop/381715
**Category:** Elasticsearch
**Tags:** es-hadoop
**Created:** [September 7, 2025, 6:36am UTC](https://discuss.elastic.co/t/why-doesn-t-dense-vector-field-show-up-in-spark-schema-when-using-elasticsearch-hadoop/381715 "2025-09-07T06:36:22Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![dany\_fard](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dany_fard/32/142249_2.png) [@dany\_fard](https://discuss.elastic.co/u/dany_fard)
#### Post date: [September 7, 2025, 6:36am UTC](https://discuss.elastic.co/t/why-doesn-t-dense-vector-field-show-up-in-spark-schema-when-using-elasticsearch-hadoop/381715/1 "2025-09-07T06:36:22Z")

</div>

Hi everyone,

I created an Elasticsearch index with a `dense_vector` field, along with some text fields. The mapping looks like this (simplified):

```auto
{"mappings": {"properties": {"embedding": {"type": "dense_vector","dims": 3,"index": true,"index_options": { "type": "int8_hnsw" }},"title": { "type": "text" },"text": { "type": "text" }}}}

```

When I read this index in Spark using the **Elasticsearch for Apache Hadoop** connector:

```auto
df = spark.read.format("es").option("es.nodes", "172.22.10.20").option("es.port", "9200").option("es.nodes.wan.only", "true").load("bbb")

```

```auto
df.printSchema()

```

```auto
df = spark.read.format("es").option("es.nodes", "172.22.10.20").option("es.port", "9200").option("es.nodes.wan.only", "true").load("bbb")

```

```auto
df.printSchema()

```

the output only shows:

`root`  
`|-- text: string (nullable = true)`  
`|-- title: string (nullable = true)`

The `embedding` (`dense_vector`) field is completely missing.

### **My Questions**

- Is `dense_vector` officially unsupported in the ES-Hadoop connector?

- The [documentation on supported field mappings](https://www.elastic.co/guide/en/elasticsearch/hadoop/current/mapping.html) doesn’t mention vector types. Does that mean they are silently ignored?

- Is there any workaround to read these fields into Spark (e.g., as arrays of floats), or is duplicating the field into a regular `float` array the only option?

Thanks in advance for clarifying!

---

<div class="post-metadata">

### Author: ![Keith\_Massey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/keith_massey/32/83666_2.png) [@Keith\_Massey](https://discuss.elastic.co/u/Keith_Massey)
#### Post date: [September 8, 2025, 1:13pm UTC](https://discuss.elastic.co/t/why-doesn-t-dense-vector-field-show-up-in-spark-schema-when-using-elasticsearch-hadoop/381715/2 "2025-09-08T13:13:33Z")

</div>

Unfortunately `dense_vector` is one of the many unsupported field types: [Support for all Elasticsearch field types · Issue #1813 · elastic/elasticsearch-hadoop · GitHub](https://github.com/elastic/elasticsearch-hadoop/issues/1813) . Your best option might be to set `es.output.json` to true to dump out the raw json.

---

<div class="post-metadata">

### Author: ![dany\_fard](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dany_fard/32/142249_2.png) [@dany\_fard](https://discuss.elastic.co/u/dany_fard)
#### Post date: [September 9, 2025, 1:03pm UTC](https://discuss.elastic.co/t/why-doesn-t-dense-vector-field-show-up-in-spark-schema-when-using-elasticsearch-hadoop/381715/4 "2025-09-09T13:03:46Z")

</div>

Thank you very much for taking the time to answer my question.
