Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Calling to_yaml on pyspark dataframemodel leads to 'Column' object has no attribute 'unique' #1854

Open
pascalwhoop opened this issue Nov 13, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@pascalwhoop
Copy link

Describe the bug
When calling

KGEdgeSchema.to_json() or to_yaml() on

class KGEdgeSchema(DataFrameModel):
    """Schema for a knowledge graph edges as exposed by the Data API."""

    # fmt: off
    subject:                     StringType()            = pa.Field(nullable = False)
    predicate:                   StringType()            = pa.Field(nullable = False)
    object:                      StringType()            = pa.Field(nullable = False)
    knowledge_level:             StringType()            = pa.Field(nullable = True)
    primary_knowledge_source:    StringType()            = pa.Field(nullable = True)
    aggregator_knowledge_source: ArrayType(StringType()) = pa.Field(nullable = True)
    publications:                ArrayType(StringType()) = pa.Field(nullable = True)
    subject_aspect_qualifier:    StringType()            = pa.Field(nullable = True)
    subject_direction_qualifier: StringType()            = pa.Field(nullable = True)
    object_aspect_qualifier:     StringType()            = pa.Field(nullable = True)
    object_direction_qualifier:  StringType()            = pa.Field(nullable = True)
    # We manually set this for every KG we ingest
    upstream_data_source:          ArrayType(StringType()) = pa.Field(nullable = False)
    # fmt: on

    class Config:  # noqa: D106
        coerce = True
        strict = False
AttributeError: 'Column' object has no attribute 'unique'
@pascalwhoop pascalwhoop added the bug Something isn't working label Nov 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant