[Bug]: Fail to run with a local LLM (Ollama) #1186

horaceho · 2024-09-23T06:13:12Z

Do you need to file an issue?

I have searched the existing issues and this bug is not already filed.
My model is hosted on OpenAI or Azure. If not, please look at the "model providers" issue and don't file a new one here.
I believe this is a legitimate bug, not just a question. If this is a question, please use the Discussions area.

Describe the bug

Fail to run with locally installed Ollama ...

Steps to reproduce

Ollama

ollama list
NAME                         ID              SIZE      MODIFIED       
llama3.1:latest              42182419e950    4.7 GB    31 seconds ago 
nomic-embed-text:latest      0a109f422b47    274 MB    3 days ago

Input

mkdir -p ./t2/input
cp the-heart-sutra.txt ./t2/input/

the-heart-sutra.txt

Init

python -m graphrag.index --init --root t2

Initializing project at t2

Modify settings.yaml to use local llama2:latest and nomic-embed-text:

llm:
  model: llama3.1:latest
  model_supports_json: true # recommended if this is available for your model.
  api_base: http://localhost:11434/v1

embeddings:
  llm:
    model: nomic-embed-text
    api_base: http://localhost:11434/api

Index

python -m graphrag.index --root t2

Expected Behavior

GraphRAG should index the text file ...

GraphRAG Config Used

encoding_model: cl100k_base
skip_workflows: []
llm:
  api_key: ${GRAPHRAG_API_KEY}
  type: openai_chat # or azure_openai_chat
  model: llama3.1:latest
  model_supports_json: true # recommended if this is available for your model.
  # max_tokens: 4000
  # request_timeout: 180.0
  api_base: http://localhost:11434/v1
  # api_version: 2024-02-15-preview
  # organization: <organization_id>
  # deployment_name: <azure_model_deployment_name>
  # tokens_per_minute: 150_000 # set a leaky bucket throttle
  # requests_per_minute: 10_000 # set a leaky bucket throttle
  # max_retries: 10
  # max_retry_wait: 10.0
  # sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
  # concurrent_requests: 25 # the number of parallel inflight requests that may be made
  # temperature: 0 # temperature for sampling
  # top_p: 1 # top-p sampling
  # n: 1 # Number of completions to generate

parallelization:
  stagger: 0.3
  # num_threads: 50 # the number of threads to use for parallel processing

async_mode: threaded # or asyncio

embeddings:
  ## parallelization: override the global parallelization settings for embeddings
  async_mode: threaded # or asyncio
  # target: required # or all
  # batch_size: 16 # the number of documents to send in a single request
  # batch_max_tokens: 8191 # the maximum number of tokens to send in a single request
  llm:
    api_key: ${GRAPHRAG_API_KEY}
    type: openai_embedding # or azure_openai_embedding
    model: nomic-embed-text
    api_base: http://localhost:11434/api
    # api_version: 2024-02-15-preview
    # organization: <organization_id>
    # deployment_name: <azure_model_deployment_name>
    # tokens_per_minute: 150_000 # set a leaky bucket throttle
    # requests_per_minute: 10_000 # set a leaky bucket throttle
    # max_retries: 10
    # max_retry_wait: 10.0
    # sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
    # concurrent_requests: 25 # the number of parallel inflight requests that may be made
    
  


chunks:
  size: 1200
  overlap: 100
  group_by_columns: [id] # by default, we don't allow chunks to cross documents
    
input:
  type: file # or blob
  file_type: text # or csv
  base_dir: "input"
  file_encoding: utf-8
  file_pattern: ".*\\.txt$"

cache:
  type: file # or blob
  base_dir: "cache"
  # connection_string: <azure_blob_storage_connection_string>
  # container_name: <azure_blob_storage_container_name>

storage:
  type: file # or blob
  base_dir: "output/${timestamp}/artifacts"
  # connection_string: <azure_blob_storage_connection_string>
  # container_name: <azure_blob_storage_container_name>

reporting:
  type: file # or console, blob
  base_dir: "output/${timestamp}/reports"
  # connection_string: <azure_blob_storage_connection_string>
  # container_name: <azure_blob_storage_container_name>

entity_extraction:
  ## strategy: fully override the entity extraction strategy.
  ##   type: one of graph_intelligence, graph_intelligence_json and nltk
  ## llm: override the global llm settings for this task
  ## parallelization: override the global parallelization settings for this task
  ## async_mode: override the global async_mode settings for this task
  prompt: "prompts/entity_extraction.txt"
  entity_types: [organization,person,geo,event]
  max_gleanings: 1

summarize_descriptions:
  ## llm: override the global llm settings for this task
  ## parallelization: override the global parallelization settings for this task
  ## async_mode: override the global async_mode settings for this task
  prompt: "prompts/summarize_descriptions.txt"
  max_length: 500

claim_extraction:
  ## llm: override the global llm settings for this task
  ## parallelization: override the global parallelization settings for this task
  ## async_mode: override the global async_mode settings for this task
  # enabled: true
  prompt: "prompts/claim_extraction.txt"
  description: "Any claims or facts that could be relevant to information discovery."
  max_gleanings: 1

community_reports:
  ## llm: override the global llm settings for this task
  ## parallelization: override the global parallelization settings for this task
  ## async_mode: override the global async_mode settings for this task
  prompt: "prompts/community_report.txt"
  max_length: 2000
  max_input_length: 8000

cluster_graph:
  max_cluster_size: 10

embed_graph:
  enabled: false # if true, will generate node2vec embeddings for nodes
  # num_walks: 10
  # walk_length: 40
  # window_size: 2
  # iterations: 3
  # random_seed: 597832

umap:
  enabled: false # if true, will generate UMAP embeddings for nodes

snapshots:
  graphml: false
  raw_entities: false
  top_level_nodes: false

local_search:
  # text_unit_prop: 0.5
  # community_prop: 0.1
  # conversation_history_max_turns: 5
  # top_k_mapped_entities: 10
  # top_k_relationships: 10
  # llm_temperature: 0 # temperature for sampling
  # llm_top_p: 1 # top-p sampling
  # llm_n: 1 # Number of completions to generate
  # max_tokens: 12000

global_search:
  # llm_temperature: 0 # temperature for sampling
  # llm_top_p: 1 # top-p sampling
  # llm_n: 1 # Number of completions to generate
  # max_tokens: 12000
  # data_max_tokens: 12000
  # map_max_tokens: 1000
  # reduce_max_tokens: 2000
  # concurrency: 32

Logs and screenshots

Logging enabled at /home/ohho/codes/python/try/graphrag/t2/output/indexing-engine.log
/home/ohho/conda/envs/graphrag/lib/python3.10/site-packages/numpy/core/fromnumeric.py:59: FutureWarning: 
'DataFrame.swapaxes' is deprecated and will be removed in a future version. Please use 'DataFrame.transpose' instead.
  return bound(*args, **kwds)
🚀 create_base_text_units
                                 id  ... n_tokens
0  6d2823d11f18b1dbb3e9c4d3a1a7b3bf  ...      401

[1 rows x 5 columns]
🚀 create_base_extracted_entities
                                        entity_graph
0  <graphml xmlns="http://graphml.graphdrawing.or...
🚀 create_summarized_entities
                                        entity_graph
0  <graphml xmlns="http://graphml.graphdrawing.or...
❌ create_base_entity_graph
None
⠇ GraphRAG Indexer 
├── Loading Input (text) - 1 files loaded (0 filtered) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00 0:00:00
├── create_base_text_units
├── create_base_extracted_entities
├── create_summarized_entities
└── create_base_entity_graph
❌ Errors occurred during the pipeline run, see logs for more details.

Additional Information

GraphRAG Version: v0.3.6
Operating System: Ubuntu 22.04.4 LTS
Python Version: 3.10
Related Issues:

The text was updated successfully, but these errors were encountered:

horaceho · 2024-09-23T06:30:10Z

"Error Invoking LLM"

horaceho · 2024-09-23T06:48:57Z

Embeddings API

curl http://localhost:11434/api/embeddings -d '{
>  "model": "nomic-embed-text",
>  "prompt": "Good morning"
> }'

{"embedding":[-0.9532470703125,-0.457611083984375,-3.6005859375,-0.163177490234375,-0.8304443359375,0.8436279296875,0.058013916015625,-0.48779296875,-0.018356800079345703,-1.24462890625,1.359619140625,1.21435546875,1.44140625,1.5159912109375,0.13267135620117188,0.0018310546875,0.4750518798828125,-0.78692626953125,-0.859893798828125,0.2451019287109375,-0.203155517578125,-0.639617919921875,-0.5706787109375,-0.975341796875,2.3072509765625,0.2158203125,-0.3594932556152344,1.658447265625,-0.6072998046875,0.0201263427734375,1.76953125,-0.388092041015625,-0.047771453857421875,-1.15771484375,-0.362335205078125,-0.082794189453125,-0.5257110595703125,-0.2566070556640625,0.230804443359375,-0.51434326171875,1.450927734375,-0.13272857666015625,0.38324737548828125,-0.6046142578125,0.47967529296875,0.0544891357421875,-0.19964599609375,0.7012939453125,1.388916015625,-1.66015625,-0.407958984375,-1.40576171875,0.1427450180053711,-1.119873046875,0.919525146484375,0.8438720703125,2.2939453125,0.01105499267578125,-0.0741424560546875,0.39068603515625,2.30322265625,0.45452880859375,-0.5588779449462891,1.159912109375,0.37579345703125,-0.43837738037109375,0.300567626953125,1.1546630859375,-0.21026611328125,1.07537841796875,1.288818359375,-0.04261016845703125,-0.405517578125,0.039306640625,0.047607421875,0.24953460693359375,0.0952911376953125,-0.57061767578125,-0.583740234375,-0.73150634765625,0.85992431640625,-0.638427734375,1.0047607421875,-0.3076629638671875,0.88616943359375,-0.3418426513671875,0.357086181640625,-0.35013866424560547,-0.408721923828125,-0.09002685546875,1.9013671875,-0.043849945068359375,0.14818572998046875,0.440948486328125,-0.85498046875,-0.02426910400390625,0.22553253173828125,1.5166015625,-0.9599609375,-1.1292724609375,1.220458984375,-0.5972900390625,-0.664215087890625,0.23602294921875,1.3953857421875,0.85760498046875,-1.29296875,-0.50775146484375,-1.97509765625,-0.72607421875,-1.34619140625,-0.31353759765625,-0.15783309936523438,-1.147705078125,0.07466888427734375,-0.51458740234375,2.057861328125,-1.994384765625,1.1192626953125,0.42047119140625,-0.9381103515625,-0.417510986328125,0.282257080078125,-0.1992340087890625,0.64166259765625,0.65594482421875,-1.835693359375,-1.1243896484375,-0.118255615234375,0.526153564453125,1.30615234375,-1.1182861328125,-0.32598876953125,0.3839569091796875,-0.7340087890625,1.238525390625,0.78912353515625,-0.3046722412109375,1.561279296875,0.4564208984375,0.79547119140625,0.978515625,-0.19110107421875,-1.196533203125,-0.0543060302734375,-0.211273193359375,0.9215087890625,-1.163330078125,-0.22772216796875,0.54144287109375,-0.539581298828125,0.367156982421875,-0.2320098876953125,0.84912109375,0.48486328125,-1.0509033203125,-0.4037933349609375,-0.13262939453125,1.3302001953125,0.40692138671875,1.545654296875,-0.30770111083984375,-1.563232421875,0.3138427734375,0.066162109375,-1.21490478515625,-1.26806640625,0.32611083984375,0.06850433349609375,1.70068359375,-0.43174171447753906,-0.07520866394042969,-1.11328125,-1.3681640625,0.8704833984375,-0.48931884765625,-0.57098388671875,-0.7716064453125,0.7298583984375,-0.59808349609375,-0.33929443359375,-0.414581298828125,0.1025390625,1.2266845703125,0.42816162109375,-0.807861328125,0.361602783203125,-0.8507080078125,-2.0341796875,-0.703857421875,0.12890625,0.47442626953125,-1.19384765625,-0.588134765625,0.01586151123046875,0.256378173828125,0.36114501953125,0.2126922607421875,2.204833984375,0.22434234619140625,0.09332275390625,-0.6929931640625,0.1107177734375,0.598876953125,0.54254150390625,2.134521484375,0.1041412353515625,1.48681640625,-0.1275634765625,0.292236328125,0.7822265625,0.2671966552734375,0.09737205505371094,-0.46954345703125,0.1925201416015625,0.02885150909423828,0.045196533203125,0.33062744140625,-1.032958984375,0.9552001953125,0.102752685546875,0.927734375,0.0724029541015625,-0.7486572265625,1.1007080078125,-0.3365478515625,-0.419708251953125,-1.0091552734375,-1.1881103515625,0.3725128173828125,-0.55859375,-0.7355880737304688,0.3479156494140625,0.101043701171875,0.66864013671875,0.633544921875,1.222900390625,1.385986328125,0.6439208984375,-0.8426513671875,-0.30462646484375,0.97802734375,0.3501625061035156,-0.454193115234375,-1.751708984375,0.363800048828125,0.08476495742797852,-0.432220458984375,-0.9049072265625,0.341552734375,0.42474365234375,-0.383758544921875,0.1580333709716797,-0.30384063720703125,0.488037109375,-1.6123046875,-0.09548091888427734,1.3516845703125,0.05536651611328125,-0.27874755859375,0.1202392578125,-2.123779296875,-0.3704833984375,-0.88446044921875,1.1922607421875,0.38851165771484375,0.23211669921875,0.4566650390625,-0.078399658203125,-1.5205078125,0.53070068359375,0.0497589111328125,1.372802734375,0.9677734375,-1.189208984375,-0.6856689453125,-1.2099609375,0.261993408203125,-1.925537109375,0.79150390625,-0.405059814453125,0.17988014221191406,-0.8179931640625,-0.14848709106445312,-0.506866455078125,0.9002685546875,0.9677734375,0.05352783203125,0.07342720031738281,0.22979736328125,1.627685546875,1.1768798828125,0.544219970703125,0.9749755859375,-0.406402587890625,-0.03668212890625,1.7470703125,0.003173828125,0.46917724609375,-0.552886962890625,-0.2722930908203125,0.82598876953125,1.02117919921875,-0.50787353515625,-0.3562650680541992,0.5240449905395508,0.4923095703125,0.383941650390625,-0.025167465209960938,-0.5028076171875,-0.6953353881835938,0.5027313232421875,-0.8018798828125,0.2240619659423828,-0.63861083984375,1.348876953125,0.35028076171875,0.68743896484375,1.239501953125,0.794921875,-0.1614227294921875,-0.371307373046875,-0.1471710205078125,-1.0521240234375,-0.31060791015625,0.06662368774414062,0.1534900665283203,1.951416015625,1.0489501953125,-0.404144287109375,0.0746612548828125,0.50347900390625,-0.639251708984375,-0.03094482421875,-0.86328125,0.802001953125,0.32769203186035156,-0.71337890625,-1.05181884765625,0.988037109375,-0.546051025390625,-0.65692138671875,0.4254798889160156,-1.20751953125,-1.111083984375,-0.21834754943847656,-0.4756431579589844,-0.6209716796875,0.2401123046875,-0.285430908203125,-1.0523681640625,0.9193115234375,-0.221282958984375,-0.11413383483886719,-0.08739852905273438,-0.175201416015625,0.113983154296875,1.1822509765625,0.9268798828125,-0.55474853515625,-0.49102783203125,-0.6048583984375,0.051971435546875,0.09192276000976562,-0.8843994140625,0.2191162109375,0.170654296875,0.38893890380859375,0.11091995239257812,0.0906524658203125,0.92236328125,0.246551513671875,0.26116943359375,0.570587158203125,1.381103515625,0.936767578125,-0.8555908203125,0.032196044921875,-0.4095458984375,0.2176055908203125,-0.8021240234375,-0.62060546875,0.268707275390625,-0.21563720703125,-1.05340576171875,1.1646728515625,0.433837890625,0.1472930908203125,-1.6845703125,-0.3991875648498535,0.517425537109375,-1.2269287109375,-1.0855712890625,0.659423828125,0.21179580688476562,-0.2398681640625,0.934661865234375,-0.3683624267578125,-0.755615234375,0.355560302734375,-0.3685150146484375,-0.38604736328125,0.04891301691532135,-0.72802734375,-0.62030029296875,0.1968536376953125,-1.19775390625,0.34820556640625,1.59375,-0.7813720703125,0.647125244140625,-0.22021484375,0.388458251953125,-0.9110107421875,-0.4603118896484375,0.7662353515625,0.360321044921875,-0.2811431884765625,-0.084381103515625,0.74566650390625,0.126922607421875,1.301513671875,-0.0454864501953125,0.13364797830581665,0.17798423767089844,1.440673828125,1.289306640625,-0.52252197265625,-1.49658203125,-2.004638671875,0.10943603515625,0.543701171875,0.40240478515625,-0.04791259765625,-0.556060791015625,0.06923675537109375,-0.083465576171875,0.8189697265625,-0.2060699462890625,0.57965087890625,1.29345703125,-0.1748046875,-0.69873046875,0.208160400390625,0.99072265625,3.18505859375,1.74853515625,-0.8223876953125,-1.000244140625,0.8284912109375,-0.053436279296875,0.81634521484375,0.4374542236328125,-0.059441566467285156,1.89599609375,-0.2098541259765625,0.48358154296875,0.91162109375,1.1109619140625,0.687744140625,1.1767578125,0.31285858154296875,-0.511505126953125,0.126983642578125,-0.818603515625,-1.1549072265625,-1.33642578125,0.8125,0.8345947265625,1.671142578125,-1.0694580078125,0.422515869140625,-0.61669921875,-0.854736328125,-0.392303466796875,-0.21673583984375,-0.490386962890625,-0.2745819091796875,0.338348388671875,1.769775390625,0.7159423828125,0.1293487548828125,-0.32505035400390625,-0.96978759765625,0.48663330078125,1.55810546875,1.42333984375,1.26806640625,-0.10715484619140625,-1.707275390625,0.732666015625,0.4626274108886719,0.8072509765625,-0.18320846557617188,0.900634765625,0.205902099609375,-1.2447509765625,0.5501375198364258,-0.067626953125,-0.30220794677734375,-1.546630859375,-0.3255805969238281,-0.28021240234375,-0.236236572265625,0.10071563720703125,-0.062450408935546875,-1.40185546875,-0.48760986328125,-0.7615966796875,0.1539459228515625,-0.463836669921875,-0.39532470703125,0.3518218994140625,1.224609375,0.9971923828125,-0.95458984375,0.25048828125,0.412689208984375,-0.8297214508056641,0.9842529296875,0.6094970703125,0.315765380859375,-0.3290557861328125,0.22796630859375,-0.34637451171875,0.7667236328125,-1.164306640625,-1.226318359375,0.36822509765625,-1.044677734375,0.197113037109375,0.621978759765625,0.59222412109375,0.2056427001953125,0.86346435546875,-0.60626220703125,-0.4264373779296875,0.67510986328125,0.09618759155273438,0.20380306243896484,-0.39398193359375,0.339813232421875,0.20688247680664062,0.17503929138183594,-0.3899383544921875,0.5027618408203125,-2.31494140625,0.0726776123046875,0.025909423828125,-0.476226806640625,0.15985107421875,-2.36962890625,0.056758880615234375,0.23986053466796875,-0.0033742189407348633,-0.983154296875,-0.5360374450683594,-0.4186553955078125,0.16510009765625,-0.3431129455566406,0.37799072265625,-0.05168914794921875,0.1660003662109375,0.1323089599609375,0.98681640625,0.21869373321533203,0.14003372192382812,-0.153839111328125,-0.7457275390625,-0.603759765625,-1.2054443359375,-0.0327606201171875,0.57861328125,0.71240234375,-0.8106689453125,-0.43511199951171875,-0.5113525390625,0.47777557373046875,-1.657470703125,-0.55712890625,0.9425048828125,-0.756103515625,0.9146728515625,0.98291015625,-0.29189300537109375,-0.7532958984375,-0.99072265625,-0.638763427734375,1.184814453125,0.184173583984375,0.39361572265625,1.0731201171875,0.047576904296875,-0.9503173828125,0.0052642822265625,0.224365234375,-0.20470428466796875,-1.67767333984375,-1.4921875,-1.287109375,-0.3265228271484375,-0.6295166015625,1.491455078125,-0.75732421875,-0.559326171875,1.3270263671875,0.350982666015625,0.53851318359375,-0.054534912109375,0.172088623046875,-0.40252685546875,1.511474609375,0.226959228515625,0.3145751953125,0.9840087890625,-0.9000244140625,0.71026611328125,0.0608062744140625,0.3066864013671875,-0.8043212890625,-0.41606903076171875,-0.3103790283203125,1.13836669921875,0.1932373046875,0.55682373046875,0.7596435546875,-0.6771240234375,-0.8277587890625,-0.77105712890625,0.957275390625,0.1772594451904297,-1.32958984375,-1.6732177734375,-0.9344482421875,0.1116943359375,-0.0357666015625,0.9981689453125,0.000148773193359375,-1.005615234375,0.66485595703125,0.49478912353515625,-0.3583221435546875,-0.16078567504882812,0.420166015625,0.277557373046875,-0.373138427734375,1.063720703125,0.20745849609375,1.492919921875,-0.130462646484375,1.55908203125,-0.173583984375,0.849365234375,0.0257568359375,-0.13447237014770508,-1.0125732421875,0.685302734375,-0.65789794921875,-0.1214752197265625,-2.06298828125,0.13034439086914062,-1.44873046875,-0.92822265625,0.026153564453125,0.3846435546875,-0.3443145751953125,-0.7200927734375,1.0894775390625,-1.86572265625,-0.173095703125,0.3957347869873047,0.778076171875,-0.5191192626953125,0.237274169921875,-0.492462158203125,0.44078826904296875,-0.162872314453125,-0.2239990234375,0.394622802734375,-0.7061767578125,-0.04705810546875,0.297607421875,-0.5181884765625,0.57061767578125,0.11334228515625,-0.00984954833984375,0.06342697143554688,0.1662750244140625,-0.292388916015625,-1.330078125,0.43804931640625,-0.547882080078125,-0.3859710693359375,-0.46136474609375,-0.742431640625,0.304351806640625,0.222320556640625,-0.219635009765625,-0.57855224609375,0.197998046875,-0.807861328125,0.1013031005859375,-0.43292236328125,0.8463134765625,-1.099365234375,-0.3472747802734375,0.1977081298828125,-0.5777587890625,-0.1681976318359375,-0.2085113525390625,-0.59197998046875,-0.3203125,0.70428466796875,0.474609375,1.1611328125,-0.890380859375,-0.55572509765625,0.3564605712890625,-0.12945556640625,1.803466796875,2.36376953125,0.398681640625,-1.327392578125,0.1839599609375,-0.612030029296875,-0.6049652099609375,-0.17806053161621094,-1.82421875,0.56988525390625,-1.214111328125,0.77215576171875,-1.1741943359375,0.057903289794921875,2.065185546875,0.21588134765625,0.31290435791015625,-1.765625,0.16321945190429688,-0.61279296875,0.15477752685546875,-0.53448486328125,-1.2510986328125,0.0748138427734375,0.1174468994140625,0.4141845703125,-0.22984695434570312,0.7135009765625,1.0921630859375,-0.14450454711914062,-1.000732421875,-0.235260009765625,0.74200439453125,-1.0675048828125,-0.763214111328125,-0.713134765625,0.8084716796875,0.72296142578125,0.315704345703125,1.3758544921875,-0.157318115234375,1.0859375,-0.30115509033203125,0.72015380859375,0.2470703125,-0.65179443359375,-0.1112213134765625,-1.846923828125,0.64300537109375]}

balezeauquentin · 2024-09-23T06:53:00Z

Hello,

Can you provide us your logs ?

Anyway, you can check those threads it helped me a lot when I had problems with Ollama #619, #650.

#619 is for query embedding and #650 is for embedding when you are creating the graph.

horaceho · 2024-09-23T12:03:29Z

Logs

indexing-engine.log
logs.json

balezeauquentin · 2024-09-23T12:27:56Z

Verify the name of your models, seems like llama3.1 isn't pulled on your Ollama instance.

user@VRH035941:~/.venv/graphrag$ ollama list 
NAME                            ID              SIZE    MODIFIED    
mxbai-embed-large:latest        468836162de7    669 MB  6 days ago 
nomic-embed-text:latest         0a109f422b47    274 MB  10 days ago
mistral:latest                  f974a74358d6    4.1 GB  10 days ago
llama3.1:latest                 42182419e950    4.7 GB  10 days ago

horaceho · 2024-09-23T12:45:21Z

@balezeauquentin sorry for the outdated info (I switched to a different model):

$ ollama list 
NAME                         ID              SIZE      MODIFIED      
llama3.1:latest              42182419e950    4.7 GB    3 minutes ago    
llama3.1-32k-q8:latest       ec0697ee6936    8.5 GB    6 hours ago      
mistral:latest               f974a74358d6    4.1 GB    9 hours ago      
llama2:latest                78e26419b446    3.8 GB    10 hours ago     
nomic-embed-text:latest      0a109f422b47    274 MB    4 days ago       
llama3.1:8b-instruct-q8_0    b158ded76fa0    8.5 GB    4 days ago

$ python -m graphrag.index --root t2
Logging enabled at /home/ohho/codes/python/try/graphrag/t2/output/indexing-engine.log
/home/ohho/conda/envs/graphrag/lib/python3.10/site-packages/numpy/core/fromnumeric.py:59: FutureWarning: 
'DataFrame.swapaxes' is deprecated and will be removed in a future version. Please use 'DataFrame.transpose' instead.
  return bound(*args, **kwds)
🚀 create_base_text_units
                                 id  ... n_tokens
0  6d2823d11f18b1dbb3e9c4d3a1a7b3bf  ...      401

[1 rows x 5 columns]
🚀 create_base_extracted_entities
                                        entity_graph
0  <graphml xmlns="http://graphml.graphdrawing.or...
🚀 create_summarized_entities
                                        entity_graph
0  <graphml xmlns="http://graphml.graphdrawing.or...
❌ create_base_entity_graph
None
⠇ GraphRAG Indexer 
├── Loading Input (text) - 1 files loaded (0 filtered) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00 0:00:00
├── create_base_text_units
├── create_base_extracted_entities
├── create_summarized_entities
└── create_base_entity_graph
❌ Errors occurred during the pipeline run, see logs for more details.

Updated logs

indexing-engine.log
stats.json
logs.json (zero byte empty file, please ignore)

balezeauquentin · 2024-09-23T12:56:26Z

I can't open your logs file, I don't get why can you send it again please.

<Error>
 <Code>InvalidRange</Code>
 <Message>The requested range is not satisfiable</Message>
 <RangeRequested>bytes=0-10485759</RangeRequested>
 <ActualObjectSize>0</ActualObjectSize>
 <RequestId>5Q4X6TPBJE7NYK7R</RequestId>
 <HostId>
   PUR02jTh8ibmw+Bg/Nj45Z/OERZyIHP9UzEBR9IcF2N4GYXqv/SEc41zKz1lZuU9gM6T4dbmuL4=
 </HostId>
</Error>

horaceho · 2024-09-23T12:58:34Z

The logs.json is an empty file:

$ ls -al t2/output/
total 52
drwxrwxr-x 2 ohho ohho  4096 Sep 23 20:40 .
drwxrwxr-x 6 ohho ohho  4096 Sep 23 20:40 ..
-rw-rw-r-- 1 ohho ohho  3433 Sep 23 20:40 create_base_extracted_entities.parquet
-rw-rw-r-- 1 ohho ohho 14725 Sep 23 20:40 create_base_text_units.parquet
-rw-rw-r-- 1 ohho ohho  3433 Sep 23 20:40 create_summarized_entities.parquet
-rw-rw-r-- 1 ohho ohho 16347 Sep 23 20:40 indexing-engine.log
-rw-rw-r-- 1 ohho ohho     0 Sep 23 20:40 logs.json
-rw-rw-r-- 1 ohho ohho  1076 Sep 23 20:40 stats.json

balezeauquentin · 2024-09-23T13:03:11Z

And your RAG pipeline isn't working ?

wy371900521 · 2024-09-23T14:12:34Z

这个问题特别恶心,困扰我好几天了

9prodhi · 2024-09-23T21:03:13Z

This should resolve the embedding issue you encountered. I faced a similar problem due to the different embedding format used by OpenAI.
After installing the required dependencies, you can run the file as follows:

# repo: https://github.com/9prodhi/EmbedAdapter/blob/main/ollama_serv.py
python ollama_serv.py

Also do not forget to change embedding llm api_base setting.

horaceho · 2024-09-24T06:55:20Z

After reducing the chunks size:

chunks:
  size: 300

it pass create_base_entity_graph and stopped at create_final_entities:

$ python -m graphrag.index --root t4
Logging enabled at /home/ohho/codes/python/try/graphrag/t4/output/indexing-engine.log
/home/ohho/conda/envs/graphrag/lib/python3.10/site-packages/numpy/core/fromnumeric.py:59: FutureWarning: 
'DataFrame.swapaxes' is deprecated and will be removed in a future version. Please use 'DataFrame.transpose' instead.
  return bound(*args, **kwds)
🚀 create_base_text_units
                                 id  ... n_tokens
0  ac67bbf50ea59187cea5947b4e482e79  ...      300
1  0650fd69f66ac0d668f05d218942ac62  ...      201
2  7d902ccc1d6328cdf06ab78c6a43b5f8  ...        1

[3 rows x 5 columns]
🚀 create_base_extracted_entities
                                        entity_graph
0  <graphml xmlns="http://graphml.graphdrawing.or...
🚀 create_summarized_entities
                                        entity_graph
0  <graphml xmlns="http://graphml.graphdrawing.or...
🚀 create_base_entity_graph
   level                                    clustered_graph
0      0  <graphml xmlns="http://graphml.graphdrawing.or...
/home/ohho/conda/envs/graphrag/lib/python3.10/site-packages/numpy/core/fromnumeric.py:59: FutureWarning: 
'DataFrame.swapaxes' is deprecated and will be removed in a future version. Please use 'DataFrame.transpose' instead.
  return bound(*args, **kwds)
❌ create_final_entities
None
⠙ GraphRAG Indexer 
├── Loading Input (text) - 1 files loaded (0 filtered) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00 0:00:00
├── create_base_text_units
├── create_base_extracted_entities
├── create_summarized_entities
├── create_base_entity_graph
└── create_final_entities
❌ Errors occurred during the pipeline run, see logs for more details.

settings.yaml.txt
indexing-engine.log
logs.json
stats.json

jackiezhangcn · 2024-09-24T08:26:24Z

File "/home/zhangyj/anaconda3/envs/graphrag/lib/python3.12/site-packages/graphrag/llm/openai/openai_chat_llm.py", line 56, in _execute_llm
return completion.choices[0].message.content
~~~~~~~~~~~~~~~~~~^^^
TypeError: 'NoneType' object is not subscriptable

seems no response generated from LLM (Ollama model), but don't know how to debug

worstkid92 · 2024-09-24T12:06:17Z

My index process has completed,but query shows
`See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
entity_df["community"] = entity_df["community"].astype(int)
creating llm client with {'api_key': 'REDACTED,len=19', 'type': "openai_chat", 'model': 'llama3.1', 'max_tokens': 4000, 'temperature': 0.0, 'top_p': 1.0, 'request_timeout': 180.0, 'api_base': 'http://localhost:11434/v1', 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': None, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25}
Error parsing search response json
Traceback (most recent call last):
File "/mnt/codes/graphrag/graphrag-local-ollama/graphrag/query/structured_search/global_search/search.py", line 194, in _map_response_single_batch
processed_response = self.parse_search_response(search_response)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/codes/graphrag/graphrag-local-ollama/graphrag/query/structured_search/global_search/search.py", line 232, in parse_search_response
parsed_elements = json.loads(search_response)["points"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/anaconda3_install/envs/graphrag-ollama-local/lib/python3.11/json/init.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/anaconda3_install/envs/graphrag-ollama-local/lib/python3.11/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/anaconda3_install/envs/graphrag-ollama-local/lib/python3.11/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

SUCCESS: Global Search Response: I am sorry but I am unable to answer this question given the provided data.
`

wy371900521 · 2024-09-28T12:27:07Z

最近问题都是这个问题,就是官方的bug吧

natoverse · 2024-10-01T21:49:13Z

Routing to #657

horaceho added bug Something isn't working triage Default label assignment, indicates new issue needs reviewed by a maintainer labels Sep 23, 2024

natoverse closed this as not planned Won't fix, can't repro, duplicate, stale Oct 1, 2024

natoverse added community_support Issue handled by community members and removed bug Something isn't working triage Default label assignment, indicates new issue needs reviewed by a maintainer labels Oct 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: Fail to run with a local LLM (Ollama) #1186

[Bug]: Fail to run with a local LLM (Ollama) #1186

horaceho commented Sep 23, 2024 •

edited

Loading

horaceho commented Sep 23, 2024

horaceho commented Sep 23, 2024

balezeauquentin commented Sep 23, 2024 •

edited

Loading

horaceho commented Sep 23, 2024

balezeauquentin commented Sep 23, 2024

horaceho commented Sep 23, 2024 •

edited

Loading

balezeauquentin commented Sep 23, 2024

horaceho commented Sep 23, 2024

balezeauquentin commented Sep 23, 2024

wy371900521 commented Sep 23, 2024

9prodhi commented Sep 23, 2024

horaceho commented Sep 24, 2024

jackiezhangcn commented Sep 24, 2024

worstkid92 commented Sep 24, 2024

wy371900521 commented Sep 28, 2024

natoverse commented Oct 1, 2024

[Bug]: Fail to run with a local LLM (Ollama) #1186

[Bug]: Fail to run with a local LLM (Ollama) #1186

Comments

horaceho commented Sep 23, 2024 • edited Loading

Do you need to file an issue?

Describe the bug

Steps to reproduce

Expected Behavior

GraphRAG Config Used

Logs and screenshots

Additional Information

horaceho commented Sep 23, 2024

horaceho commented Sep 23, 2024

balezeauquentin commented Sep 23, 2024 • edited Loading

horaceho commented Sep 23, 2024

balezeauquentin commented Sep 23, 2024

horaceho commented Sep 23, 2024 • edited Loading

balezeauquentin commented Sep 23, 2024

horaceho commented Sep 23, 2024

balezeauquentin commented Sep 23, 2024

wy371900521 commented Sep 23, 2024

9prodhi commented Sep 23, 2024

horaceho commented Sep 24, 2024

jackiezhangcn commented Sep 24, 2024

worstkid92 commented Sep 24, 2024

wy371900521 commented Sep 28, 2024

natoverse commented Oct 1, 2024

horaceho commented Sep 23, 2024 •

edited

Loading

balezeauquentin commented Sep 23, 2024 •

edited

Loading

horaceho commented Sep 23, 2024 •

edited

Loading