Getting "'NoneType' object has no attribute 'size'" error when trying to run model
#3
by Bandeze - opened
I'm trying to run the provided example of using the model, but I keep getting this error:
Traceback (most recent call last):
File "/home/bandeze/Ethical_AI_Stuff/./crystal_chat.py", line 12, in <module>
gen_tokens = model.generate(input_ids, do_sample=True, max_length=400)
File "/home/bandeze/Ethical_AI_Stuff/ai_venv/lib/python3.13/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
return func(*args, **kwargs)
File "/home/bandeze/Ethical_AI_Stuff/ai_venv/lib/python3.13/site-packages/transformers/generation/utils.py", line 2566, in generate
result = decoding_method(
self,
...<5 lines>...
**model_kwargs,
)
File "/home/bandeze/Ethical_AI_Stuff/ai_venv/lib/python3.13/site-packages/transformers/generation/utils.py", line 2786, in _sample
outputs = self(**model_inputs, return_dict=True)
File "/home/bandeze/Ethical_AI_Stuff/ai_venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/bandeze/Ethical_AI_Stuff/ai_venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
return forward_call(*args, **kwargs)
File "/home/bandeze/.cache/huggingface/modules/transformers_modules/LLM360/CrystalChat/18091ea1d89f8c6c18d3a90eb86c8324bb09a6f6/modeling_crystalcoder.py", line 1271, in forward
transformer_outputs = self.transformer(
input_ids,
...<11 lines>...
return_dict=return_dict,
)
File "/home/bandeze/Ethical_AI_Stuff/ai_venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/bandeze/Ethical_AI_Stuff/ai_venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
return forward_call(*args, **kwargs)
File "/home/bandeze/.cache/huggingface/modules/transformers_modules/LLM360/CrystalChat/18091ea1d89f8c6c18d3a90eb86c8324bb09a6f6/modeling_crystalcoder.py", line 981, in forward
past_length = past_key_values[0][0].size(-2)
^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'size'
I'm very new to this, so I don't know how to fix this.
I'm using Python 3.13.7, and the only change I made to the script is forcing it to use the CPU, since my NVIDIA card doesn't have enough VRAM (I have a good amount of DRAM, thankfully).
What do I do?