Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Qemu VM infer error #601

Open
JocelynPanPan opened this issue Jan 10, 2025 · 1 comment
Open

Qemu VM infer error #601

JocelynPanPan opened this issue Jan 10, 2025 · 1 comment

Comments

@JocelynPanPan
Copy link

Device: Qemu RISC-V virtual machine, with only CPU.
I get an error when I execute the following command: DEBUG=9 exo run llama-3.1-8b
error:
Error processing prompt: /tmp/tmp0a312mvx: undefined symbol: __gnu_f2h_ieee
Traceback (most recent call last):
File "/opt/exo/exo/main.py", line 203, in run_model_cli
await node.process_prompt(shard, prompt, request_id=request_id)
File "/opt/exo/exo/orchestration/node.py", line 163, in process_prompt
resp = await self._process_prompt(base_shard, prompt, request_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/exo/exo/orchestration/node.py", line 197, in _process_prompt
result = await self.inference_engine.infer_prompt(request_id, shard, prompt)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/exo/exo/inference/inference_engine.py", line 45, in infer_prompt
output_data = await self.infer_tensor(request_id, shard, x)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/exo/exo/inference/tinygrad/inference.py", line 116, in infer_tensor
output_data = await asyncio.get_running_loop().run_in_executor(self.executor, wrap_infer)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/exo_py12/lib/python3.12/concurrent/futures/thread.py", line 59, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/exo/exo/inference/tinygrad/inference.py", line 113, in wrap_infer
out = self.model.forward(h, **state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/exo/exo/inference/tinygrad/models/llama.py", line 271, in forward
return self.forward_base(x, start_pos, cache=cache)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/exo/exo/inference/tinygrad/models/llama.py", line 263, in forward_base
x = layer(x, start_pos, freqs_cis, mask, cache=c)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/exo/exo/inference/tinygrad/models/llama.py", line 115, in call
h = x + self.attention(self.attention_norm(x), start_pos, freqs_cis, mask, cache=cache)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/exo/exo/inference/tinygrad/models/llama.py", line 82, in call
cache.shrink((None, None, (start_pos, start_pos + seqlen), None, None)).assign(Tensor.stack(xk, xv)).realize()
File "/root/exo_py312/lib/python3.12/site-packages/tinygrad/tensor.py", line 3754, in _wrapper
ret = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/root/exo_py312/lib/python3.12/site-packages/tinygrad/tensor.py", line 219, in realize
run_schedule(*self.schedule_with_vars(*lst), do_update_stats=do_update_stats)
File "/root/exo_py312/lib/python3.12/site-packages/tinygrad/engine/realize.py", line 215, in run_schedule
for ei in lower_schedule(schedule):
^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/exo_py312/lib/python3.12/site-packages/tinygrad/engine/realize.py", line 208, in lower_schedule
raise e
File "/root/exo_py312/lib/python3.12/site-packages/tinygrad/engine/realize.py", line 202, in lower_schedule
try: yield lower_schedule_item(si)
^^^^^^^^^^^^^^^^^^^^^^^
File "/root/exo_py312/lib/python3.12/site-packages/tinygrad/engine/realize.py", line 187, in lower_schedule_item
runner = get_runner(si.outputs[0].device, si.ast)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/exo_py312/lib/python3.12/site-packages/tinygrad/engine/realize.py", line 155, in get_runner
method_cache[ckey] = method_cache[bkey] = ret = CompiledRunner(replace(prg, device=device))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/exo_py312/lib/python3.12/site-packages/tinygrad/engine/realize.py", line 83, in init
self._prg = Device[p.device].runtime(p.function_name, self.lib)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/exo_py312/lib/python3.12/site-packages/tinygrad/runtime/ops_clang.py", line 28, in init
self.fxn = ctypes.CDLL(str(cached_file_path.name))[name]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/exo_py12/lib/python3.12/ctypes/init.py", line 379, in init
self._handle = _dlopen(self._name, mode)
^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: /tmp/tmp0a312mvx: undefined symbol: __gnu_f2h_ieee
update_peers: added=[] removed=[] updated=[] unchanged=[] to_disconnect=[] to_connect=[]
did_peers_change=False
Received exit signal SIGTERM...
Thank you for using exo.

How can I solve this error?

@JocelynPanPan
Copy link
Author

Sorry, my virtual machine is of the x86 architecture, not RISC-V.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant