You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
when i run pretrain_llama2_distributed.sh occurs:
node0: [rank0]: File "/home/nccl/Megatron-DeepSpeed/megatron/utils.py", line 39, in update_rotary_pos_emb
node0: [rank0]: rotary_pos_emb = RotaryEmbedding(rotary_dim, theta=args.rope_theta)(seq_length).to(
node0: [rank0]: AttributeError: 'list' object has no attribute 'to'
RotaryEmbedding(rotary_dim, theta=args.rope_theta)(seq_length) return a list, but the code in utils.py treat it as tensor
The text was updated successfully, but these errors were encountered:
when i run pretrain_llama2_distributed.sh occurs:
node0: [rank0]: File "/home/nccl/Megatron-DeepSpeed/megatron/utils.py", line 39, in update_rotary_pos_emb
node0: [rank0]: rotary_pos_emb = RotaryEmbedding(rotary_dim, theta=args.rope_theta)(seq_length).to(
node0: [rank0]: AttributeError: 'list' object has no attribute 'to'
RotaryEmbedding(rotary_dim, theta=args.rope_theta)(seq_length) return a list, but the code in utils.py treat it as tensor
The text was updated successfully, but these errors were encountered: