Code

Add RWKV-4#22797

Code

Merged

sgugger merged 31 commits intomainhuggingface/transformers:mainfrom add_rwkvhuggingface/transformers:add_rwkvCopy head branch name to clipboardMay 9, 2023

+2,284-3

Commits

Commits on Apr 27, 2023

• First draft of RWKV-4

  sgugger

  committed
  3e0855b

• Add support for generate

  sgugger

  committed
  db50d8b

• Style post-rebase

  sgugger

  committed
  b6273e9

• Properly use state

  sgugger

  committed
  7b81930

Commits on Apr 28, 2023

• Write doc

  sgugger

  committed
  400147e

• Fix doc

  sgugger

  committed
  8603532

• More math

  sgugger

  committed
  bc0cd7c

• Add model to README, dummies and clean config

  sgugger

  committed
  7736584

• Fix init

  sgugger

  committed
  a6aa932

Commits on May 2, 2023

• multiple fixes:

  Show description for a50d49c
  younesbelkada

  committed
  a50d49c

• Merge branch 'main' into add_rwkv

  younesbelkada

  authored
  1ca42ad

• correct tokenizer

  younesbelkada

  committed
  848ccf8

• Merge branch 'add_rwkv' of https://github.com/huggingface/transformers into add_rwkv

  younesbelkada

  committed
  a33c5aa

Commits on May 3, 2023

• some tweaks

  Show description for b679baa
  younesbelkada

  committed
  b679baa

• Merge remote-tracking branch 'upstream/main' into add_rwkv

  younesbelkada

  committed
  6ce80d2

• fix CI tests

  Show description for 10b5b81
  younesbelkada

  committed
  10b5b81

Commits on May 4, 2023

• fix conversion script

  Show description for ffb55f9
  younesbelkada

  committed
  ffb55f9

• add slow tests + more fixes on conversion script

  younesbelkada

  committed
  7fd5702

• add another test

  younesbelkada

  committed
  e52be94

• final fixes

  younesbelkada

  committed
  3c02506

• change single name variable

  younesbelkada

  committed
  69774da

Commits on May 5, 2023

• add mock attention mask for pipeline to work

  younesbelkada

  committed
  d52d6db

• correct eos token id

  younesbelkada

  committed
  0d758e1

• fix nits

  younesbelkada

  committed
  df44a60

• add checkpoints

  younesbelkada

  committed
  9e3efc5

• Apply suggestions from code review

  Show description for 0b095c7
  

  younesbelkada

  and

  amyeroberts

  authored
  0b095c7

• add tie_word_embeddings in docstring

  younesbelkada

  committed
  f912a0d

• change tensor name

  younesbelkada

  committed
  2a6dd61

Commits on May 9, 2023

• fix final nits

  younesbelkada

  committed
  4fdbeef

• Merge remote-tracking branch 'upstream/main' into add_rwkv

  younesbelkada

  committed
  dc22169

• Trigger CI

  sgugger

  committed
  d1c2b15