# Resolved attribute is missing ... but is not!

**URL:** <https://discuss.elastic.co/t/resolved-attribute-is-missing-but-is-not/277510>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [July 1, 2021, 5:59am UTC](https://discuss.elastic.co/t/resolved-attribute-is-missing-but-is-not/277510 "2021-07-01T05:59:32Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![priamai](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/priamai/32/80269_2.png) [@priamai](https://discuss.elastic.co/u/priamai)\
**Post date:** [July 1, 2021, 5:59am UTC](https://discuss.elastic.co/t/resolved-attribute-is-missing-but-is-not/277510/1 "2021-07-01T05:59:32Z")

</div>

I am on ELK 7.13.0 with the Hadoop 7.13.0 library.  
I have the following schema:

root  
|-- @timestamp: timestamp (nullable = true)  
|-- AuthenticationPackage: string (nullable = true)  
|-- Destination: string (nullable = true)  
|-- DomainName: string (nullable = true)  
|-- EventID: long (nullable = true)  
|-- FailureReason: string (nullable = true)  
|-- LogHost: string (nullable = true)  
|-- LogonID: string (nullable = true)  
|-- LogonType: long (nullable = true)  
|-- LogonTypeDescription: string (nullable = true)  
|-- ParentProcessID: string (nullable = true)  
|-- ParentProcessName: string (nullable = true)  
|-- ProcessID: string (nullable = true)  
|-- ProcessName: string (nullable = true)  
|-- ServiceName: string (nullable = true)  
|-- Source: string (nullable = true)  
|-- Status: string (nullable = true)  
|-- SubjectDomainName: string (nullable = true)  
|-- SubjectLogonID: string (nullable = true)  
|-- SubjectUserName: string (nullable = true)  
|-- Time: long (nullable = true)  
|-- UserName: string (nullable = true)

When I try a simple filter operation:

```auto
ss = (
    SparkSession.builder.master('local[24]').appName("ES")
    .config("spark.driver.memory", "8g")
    .getOrCreate()
)

es_reader = (ss.read
    .format("org.elasticsearch.spark.sql").option("inferSchema", "false")
    .option("es.read.field.as.array.include", "tags").option("es.nodes","elasticsearch:9200")
    .option("es.net.http.auth.user","elastic").option("es.net.http.auth.pass","123")
             .option("es.net.ssl","true").option("es.net.ssl.cert.allow.self.signed","true"))

small_df = es_reader.load("priam_unified_host-{0}/unified-host".format(date))
small.filter(small_df.EventID == 4688).explain(extended=True)

```

I am getting the following error as if EventID is not existing but all documents in that index have in fact EventID field populated.

```auto
AnalysisException: Resolved attribute(s) EventID#559L missing from @timestamp#203,AuthenticationPackage#204,Destination#205,DomainName#206,EventID#207L,FailureReason#208,LogHost#209,LogonID#210,LogonType#211L,LogonTypeDescription#212,ParentProcessID#213,ParentProcessName#214,ProcessID#215,ProcessName#216,ServiceName#217,Source#218,Status#219,SubjectDomainName#220,SubjectLogonID#221,SubjectUserName#222,Time#223L,UserName#224 in operator !Filter (EventID#559L = cast(4688 as bigint)). Attribute(s) with the same name appear in the operation: EventID. Please check if the right attribute(s) are used.;
!Filter (EventID#559L = cast(4688 as bigint))

```

Any idea why this is happening?

---

<div class="post-metadata">

**Author:** ![priamai](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/priamai/32/80269_2.png) [@priamai](https://discuss.elastic.co/u/priamai)\
**Post date:** [July 1, 2021, 6:07am UTC](https://discuss.elastic.co/t/resolved-attribute-is-missing-but-is-not/277510/2 "2021-07-01T06:07:53Z")

</div>

Interesting it seems that it doesn't like that syntax, this works:

`small_df.where('EventID == 4688').limit(10).explain(extended=True)`

Maybe worth adding in the documentation examples.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 29, 2021, 6:08am UTC](https://discuss.elastic.co/t/resolved-attribute-is-missing-but-is-not/277510/3 "2021-07-29T06:08:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.