How to evaluate the extracted entity, relationship, and community reports?

[yangxue-1]

Is there an existing issue for this?

• I have searched the existing issues
• I have checked #657 to validate if my issue is covered by community support

Describe the issue

How to evaluate the extracted entity, relationship, and community reports?

The result of the global search is always like this：I am sorry but I am unable to answer this question given the provided data.

Steps to reproduce

No response

GraphRAG Config Used

# Paste your config here

Logs and screenshots

No response

Additional Information

• GraphRAG Version:
• Operating System:
• Python Version:
• Related Issues:

[natoverse]

There are some comments here about the technical aspects of the process: #849

You can inspect the final output parquet files create_final_entities.parquet, create_final_relationships.parquet and create_final_community_reports.parquet to see if they are filled in and look reasonable. If you are seeing the "unable to answer question" message that most likely means the LLM was not satisfied that the reports were able to answer your question, and it filtered out all of them, and thus didn't have any reports to summarize.

[yangxue-1]

There are some comments here about the technical aspects of the process: #849

You can inspect the final output parquet files create_final_entities.parquet, create_final_relationships.parquet and create_final_community_reports.parquet to see if they are filled in and look reasonable. If you are seeing the "unable to answer question" message that most likely means the LLM was not satisfied that the reports were able to answer your question, and it filtered out all of them, and thus didn't have any reports to summarize.

The create_summarized_entities.parquet file shows that Nothing to show. Does this mean that the summarize_descriptions.txt file needs to be modified?

[yangxue-1]

There are some comments here about the technical aspects of the process: #849

You can inspect the final output parquet files create_final_entities.parquet, create_final_relationships.parquet and create_final_community_reports.parquet to see if they are filled in and look reasonable. If you are seeing the "unable to answer question" message that most likely means the LLM was not satisfied that the reports were able to answer your question, and it filtered out all of them, and thus didn't have any reports to summarize.

The description of entities in the create_final_entities.parquet file is incorrect. Some entities contain garbled characters, and some descriptions are inconsistent with the entities.