Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GPT-J-6B #13022

Merged
merged 133 commits into from
Aug 31, 2021
Merged

GPT-J-6B #13022

merged 133 commits into from
Aug 31, 2021

Commits on Aug 4, 2021

  1. Test GPTJ implementation

    @StellaAthena
    StellaAthena committed Aug 4, 2021
    Configuration menu
    Copy the full SHA
    f01d47f View commit details
    Browse the repository at this point in the history

  2. fix conflicts

    @StellaAthena
    StellaAthena committed Aug 4, 2021
    Configuration menu
    Copy the full SHA
    44ccee7 View commit details
    Browse the repository at this point in the history

  3. Fixed conflicts

    @StellaAthena
    StellaAthena authored Aug 4, 2021
    Configuration menu
    Copy the full SHA
    fe991bf View commit details
    Browse the repository at this point in the history

  4. Update __init__.py

    @StellaAthena
    StellaAthena authored Aug 4, 2021
    Configuration menu
    Copy the full SHA
    54f2b33 View commit details
    Browse the repository at this point in the history

  5. Update __init__.py

    @StellaAthena
    StellaAthena authored Aug 4, 2021
    Configuration menu
    Copy the full SHA
    e2ce2a3 View commit details
    Browse the repository at this point in the history

  6. change GPT_J to GPTJ

    @kurumuz
    kurumuz committed Aug 4, 2021
    Configuration menu
    Copy the full SHA
    e59e579 View commit details
    Browse the repository at this point in the history

  7. fix missing imports and typos

    @kurumuz
    kurumuz committed Aug 4, 2021
    Configuration menu
    Copy the full SHA
    e2329b4 View commit details
    Browse the repository at this point in the history

  8. use einops for now 

    (need to change to torch ops later)
    @kurumuz
    kurumuz committed Aug 4, 2021
    Configuration menu
    Copy the full SHA
    1bee4ee View commit details
    Browse the repository at this point in the history

  9. Use torch ops instead of einsum

    @kurumuz
    kurumuz committed Aug 4, 2021
    Configuration menu
    Copy the full SHA
    03b7278 View commit details
    Browse the repository at this point in the history

  10. remove einops deps

    @kurumuz
    kurumuz committed Aug 4, 2021
    Configuration menu
    Copy the full SHA
    8034f2c View commit details
    Browse the repository at this point in the history

  11. Merge pull request #1 from kurumuz/gptj_fixes 

    GPT-J fixes
    @StellaAthena
    StellaAthena authored Aug 4, 2021
    Configuration menu
    Copy the full SHA
    f86b47b View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2021

  1. Merge branch 'huggingface:master' into master

    @StellaAthena
    StellaAthena authored Aug 5, 2021
    Configuration menu
    Copy the full SHA
    0a344ba View commit details
    Browse the repository at this point in the history

  2. Update configuration_auto.py

    @StellaAthena
    StellaAthena authored Aug 5, 2021
    Configuration menu
    Copy the full SHA
    194d024 View commit details
    Browse the repository at this point in the history

  3. Added GPT J

    @StellaAthena
    StellaAthena authored Aug 5, 2021
    Configuration menu
    Copy the full SHA
    06f07da View commit details
    Browse the repository at this point in the history

  4. Update gptj.rst

    @StellaAthena
    StellaAthena authored Aug 5, 2021
    Configuration menu
    Copy the full SHA
    979bff8 View commit details
    Browse the repository at this point in the history

  5. Update __init__.py

    @StellaAthena
    StellaAthena authored Aug 5, 2021
    Configuration menu
    Copy the full SHA
    30635c1 View commit details
    Browse the repository at this point in the history

  6. Update test_modeling_gptj.py

    @StellaAthena
    StellaAthena authored Aug 5, 2021
    Configuration menu
    Copy the full SHA
    bae5e27 View commit details
    Browse the repository at this point in the history

  7. Added GPT J

    @StellaAthena
    StellaAthena authored Aug 5, 2021
    Configuration menu
    Copy the full SHA
    1bcf933 View commit details
    Browse the repository at this point in the history

  8. Changed configs to match GPT2 instead of GPT Neo

    @StellaAthena
    StellaAthena authored Aug 5, 2021
    Configuration menu
    Copy the full SHA
    12a12a7 View commit details
    Browse the repository at this point in the history

  9. Removed non-existent sequence model

    @StellaAthena
    StellaAthena authored Aug 5, 2021
    Configuration menu
    Copy the full SHA
    4efbbec View commit details
    Browse the repository at this point in the history

  10. Update configuration_auto.py

    @StellaAthena
    StellaAthena authored Aug 5, 2021
    Configuration menu
    Copy the full SHA
    6877889 View commit details
    Browse the repository at this point in the history

  11. Update configuration_auto.py

    @StellaAthena
    StellaAthena authored Aug 5, 2021
    Configuration menu
    Copy the full SHA
    cfaaae4 View commit details
    Browse the repository at this point in the history

  12. Update configuration_auto.py

    @StellaAthena
    StellaAthena authored Aug 5, 2021
    Configuration menu
    Copy the full SHA
    e9860e9 View commit details
    Browse the repository at this point in the history

  13. Update modeling_gptj.py

    @StellaAthena
    StellaAthena authored Aug 5, 2021
    Configuration menu
    Copy the full SHA
    e8a2333 View commit details
    Browse the repository at this point in the history

  14. Update modeling_gptj.py

    @StellaAthena
    StellaAthena authored Aug 5, 2021
    Configuration menu
    Copy the full SHA
    3bd2879 View commit details
    Browse the repository at this point in the history

Commits on Aug 6, 2021

  1. Progress on updating configs to agree with GPT2

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    8c524f7 View commit details
    Browse the repository at this point in the history

  2. Update modeling_gptj.py

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    f0c0a31 View commit details
    Browse the repository at this point in the history

  3. num_layers -> n_layer

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    1ad512b View commit details
    Browse the repository at this point in the history

  4. layer_norm_eps -> layer_norm_epsilon

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    89b8724 View commit details
    Browse the repository at this point in the history

  5. attention_layers -> num_hidden_layers

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    76fc4e1 View commit details
    Browse the repository at this point in the history

  6. Update modeling_gptj.py

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    6284c7e View commit details
    Browse the repository at this point in the history

  7. attention_pdrop -> attn_pdrop

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    2d5cc30 View commit details
    Browse the repository at this point in the history

  8. hidden_act -> activation_function

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    1ddbb63 View commit details
    Browse the repository at this point in the history

  9. Update configuration_gptj.py

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    b46551d View commit details
    Browse the repository at this point in the history

  10. Update configuration_gptj.py

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    60daf97 View commit details
    Browse the repository at this point in the history

  11. Update configuration_gptj.py

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    1c9ba25 View commit details
    Browse the repository at this point in the history

  12. Update configuration_gptj.py

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    7f52c42 View commit details
    Browse the repository at this point in the history

  13. Update configuration_gptj.py

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    33380ca View commit details
    Browse the repository at this point in the history

  14. Update modeling_gptj.py

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    05b2b3b View commit details
    Browse the repository at this point in the history

  15. Update modeling_gptj.py

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    d6b86f8 View commit details
    Browse the repository at this point in the history

  16. Update modeling_gptj.py

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    2b633be View commit details
    Browse the repository at this point in the history

  17. Update modeling_gptj.py

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    dde732d View commit details
    Browse the repository at this point in the history

  18. Update modeling_gptj.py

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    89818be View commit details
    Browse the repository at this point in the history

  19. Update modeling_gptj.py

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    0bc0c25 View commit details
    Browse the repository at this point in the history

  20. fix layernorm and lm_head size 

    delete attn_type
    @kurumuz
    kurumuz committed Aug 6, 2021
    Configuration menu
    Copy the full SHA
    e9bc670 View commit details
    Browse the repository at this point in the history

  21. Merge branch 'huggingface:master' into master

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    0ac5c64 View commit details
    Browse the repository at this point in the history

  22. Update docs/source/model_doc/gptj.rst 

    Co-authored-by: Suraj Patil <surajp815@gmail.com>
    @StellaAthena @patil-suraj
    StellaAthena and patil-suraj authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    3553d8e View commit details
    Browse the repository at this point in the history

  23. removed claim that GPT J uses local attention

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    6b6e41d View commit details
    Browse the repository at this point in the history

  24. Removed GPTJForSequenceClassification

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    5fc60db View commit details
    Browse the repository at this point in the history

  25. Update src/transformers/models/gptj/configuration_gptj.py 

    Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
    @StellaAthena @LysandreJik
    StellaAthena and LysandreJik authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    d70f1f8 View commit details
    Browse the repository at this point in the history

  26. Removed unsupported boilerplate

    @StellaAthena
    StellaAthena authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    878518a View commit details
    Browse the repository at this point in the history

  27. Update tests/test_modeling_gptj.py 

    Co-authored-by: Suraj Patil <surajp815@gmail.com>
    @StellaAthena @patil-suraj
    StellaAthena and patil-suraj authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    a48ee07 View commit details
    Browse the repository at this point in the history

  28. Update src/transformers/models/gptj/modeling_gptj.py 

    Co-authored-by: Suraj Patil <surajp815@gmail.com>
    @StellaAthena @patil-suraj
    StellaAthena and patil-suraj authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    f189b6d View commit details
    Browse the repository at this point in the history

  29. Update src/transformers/models/gptj/modeling_gptj.py 

    Co-authored-by: Suraj Patil <surajp815@gmail.com>
    @StellaAthena @patil-suraj
    StellaAthena and patil-suraj authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    6059ef6 View commit details
    Browse the repository at this point in the history

  30. Update src/transformers/models/gptj/modeling_gptj.py 

    Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
    @StellaAthena @LysandreJik
    StellaAthena and LysandreJik authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    8793237 View commit details
    Browse the repository at this point in the history

  31. Update tests/test_modeling_gptj.py 

    Co-authored-by: Eric Hallahan <eric@hallahans.name>
    @StellaAthena @EricHallahan
    StellaAthena and EricHallahan authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    d5d758e View commit details
    Browse the repository at this point in the history

  32. Update tests/test_modeling_gptj.py 

    Co-authored-by: Eric Hallahan <eric@hallahans.name>
    @StellaAthena @EricHallahan
    StellaAthena and EricHallahan authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    58b77c1 View commit details
    Browse the repository at this point in the history

  33. Update tests/test_modeling_gptj.py 

    Co-authored-by: Eric Hallahan <eric@hallahans.name>
    @StellaAthena @EricHallahan
    StellaAthena and EricHallahan authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    513fa3e View commit details
    Browse the repository at this point in the history

  34. Update src/transformers/models/gptj/modeling_gptj.py 

    Co-authored-by: Suraj Patil <surajp815@gmail.com>
    @StellaAthena @patil-suraj
    StellaAthena and patil-suraj authored Aug 6, 2021
    Configuration menu
    Copy the full SHA
    9876057 View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2021

  1. Update __init__.py

    @StellaAthena
    StellaAthena authored Aug 7, 2021
    Configuration menu
    Copy the full SHA
    de6c5af View commit details
    Browse the repository at this point in the history

  2. Update configuration_gptj.py

    @StellaAthena
    StellaAthena authored Aug 7, 2021
    Configuration menu
    Copy the full SHA
    3f21e98 View commit details
    Browse the repository at this point in the history

  3. Update modeling_gptj.py

    @StellaAthena
    StellaAthena authored Aug 7, 2021
    Configuration menu
    Copy the full SHA
    316cc95 View commit details
    Browse the repository at this point in the history

  4. Corrected indentation

    @StellaAthena
    StellaAthena authored Aug 7, 2021
    Configuration menu
    Copy the full SHA
    4be1484 View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2021

  1. Remove stray backslash

    @EricHallahan
    EricHallahan committed Aug 8, 2021
    Configuration menu
    Copy the full SHA
    80f4658 View commit details
    Browse the repository at this point in the history

  2. Delete .DS_Store

    @leogao2
    leogao2 authored Aug 8, 2021
    Configuration menu
    Copy the full SHA
    f4d70d2 View commit details
    Browse the repository at this point in the history

  3. Delete .DS_Store

    @leogao2
    leogao2 authored Aug 8, 2021
    Configuration menu
    Copy the full SHA
    28caefb View commit details
    Browse the repository at this point in the history

  4. Delete .DS_Store

    @leogao2
    leogao2 authored Aug 8, 2021
    Configuration menu
    Copy the full SHA
    6633eba View commit details
    Browse the repository at this point in the history

  5. Delete .DS_Store

    @leogao2
    leogao2 authored Aug 8, 2021
    Configuration menu
    Copy the full SHA
    2f92631 View commit details
    Browse the repository at this point in the history

  6. Delete .DS_Store

    @leogao2
    leogao2 authored Aug 8, 2021
    Configuration menu
    Copy the full SHA
    23356a0 View commit details
    Browse the repository at this point in the history

  7. Update docs to match

    @leogao2
    leogao2 authored Aug 8, 2021
    Configuration menu
    Copy the full SHA
    dda2643 View commit details
    Browse the repository at this point in the history

  8. Remove tf loading

    @leogao2
    leogao2 authored Aug 8, 2021
    Configuration menu
    Copy the full SHA
    a31f11a View commit details
    Browse the repository at this point in the history

  9. Remove config.jax

    @leogao2
    leogao2 authored Aug 8, 2021
    Configuration menu
    Copy the full SHA
    cbf8dd1 View commit details
    Browse the repository at this point in the history

  10. Remove stray else: statement

    @EricHallahan
    EricHallahan committed Aug 8, 2021
    Configuration menu
    Copy the full SHA
    fed0955 View commit details
    Browse the repository at this point in the history

  11. Remove references to load_tf_weights_in_gptj

    @EricHallahan
    EricHallahan committed Aug 8, 2021
    Configuration menu
    Copy the full SHA
    0ae4be5 View commit details
    Browse the repository at this point in the history

  12. Adapt tests to match output from GPT-J 6B

    @EricHallahan
    EricHallahan committed Aug 8, 2021
    Configuration menu
    Copy the full SHA
    3c6161d View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2021

  1. Apply suggestions from code review 

    Co-authored-by: Suraj Patil <surajp815@gmail.com>
    @StellaAthena @patil-suraj
    StellaAthena and patil-suraj authored Aug 9, 2021
    Configuration menu
    Copy the full SHA
    dd4f02d View commit details
    Browse the repository at this point in the history

  2. Default activation_function to gelu_new 

    - Specify the approximate formulation of GELU to ensure parity with the default setting of `jax.nn.gelu()`
    @EricHallahan
    EricHallahan authored Aug 9, 2021
    Configuration menu
    Copy the full SHA
    752595f View commit details
    Browse the repository at this point in the history

  3. Fix part of the config documentation

    @EricHallahan
    EricHallahan authored Aug 9, 2021
    Configuration menu
    Copy the full SHA
    7a032e5 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2021

  1. Revert "Update configuration_auto.py" 

    This reverts commit e9860e9.
    @EricHallahan
    EricHallahan committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    455c311 View commit details
    Browse the repository at this point in the history

  2. Revert "Update configuration_auto.py" 

    This reverts commit cfaaae4.
    @EricHallahan
    EricHallahan committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    49ba5cc View commit details
    Browse the repository at this point in the history

  3. Revert "Update configuration_auto.py" 

    This reverts commit 6877889.
    @EricHallahan
    EricHallahan committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    3ebf87b View commit details
    Browse the repository at this point in the history

  4. Revert "Update configuration_auto.py" 

    This reverts commit 194d024.
    @EricHallahan
    EricHallahan committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    c844dcb View commit details
    Browse the repository at this point in the history

  5. Hyphenate GPT-J

    @EricHallahan
    EricHallahan committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    9af5cff View commit details
    Browse the repository at this point in the history

  6. Undid sorting of the models alphabetically

    @StellaAthena
    StellaAthena authored Aug 10, 2021
    Configuration menu
    Copy the full SHA
    74a9777 View commit details
    Browse the repository at this point in the history

  7. Reverting previous commit

    @StellaAthena
    StellaAthena authored Aug 10, 2021
    Configuration menu
    Copy the full SHA
    4a40d00 View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2021

  1. Merge branch 'master' into master

    @patil-suraj
    patil-suraj authored Aug 13, 2021
    Configuration menu
    Copy the full SHA
    176ec56 View commit details
    Browse the repository at this point in the history

  2. fix style and quality issues

    @patil-suraj
    patil-suraj committed Aug 13, 2021
    Configuration menu
    Copy the full SHA
    24ac25a View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2021

  1. Update docs/source/model_doc/gptj.rst 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @StellaAthena @sgugger
    StellaAthena and sgugger authored Aug 14, 2021
    Configuration menu
    Copy the full SHA
    d7ac30f View commit details
    Browse the repository at this point in the history

  2. Update src/transformers/__init__.py 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @StellaAthena @sgugger
    StellaAthena and sgugger authored Aug 14, 2021
    Configuration menu
    Copy the full SHA
    6857e93 View commit details
    Browse the repository at this point in the history

  3. Update tests/test_modeling_gptj.py 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @StellaAthena @sgugger
    StellaAthena and sgugger authored Aug 14, 2021
    Configuration menu
    Copy the full SHA
    94db694 View commit details
    Browse the repository at this point in the history

  4. Update src/transformers/models/gptj/modeling_gptj.py 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @StellaAthena @sgugger
    StellaAthena and sgugger authored Aug 14, 2021
    Configuration menu
    Copy the full SHA
    5fa31e0 View commit details
    Browse the repository at this point in the history

  5. Update src/transformers/__init__.py 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @StellaAthena @sgugger
    StellaAthena and sgugger authored Aug 14, 2021
    Configuration menu
    Copy the full SHA
    f38e019 View commit details
    Browse the repository at this point in the history

  6. Update src/transformers/models/gptj/modeling_gptj.py 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @StellaAthena @sgugger
    StellaAthena and sgugger authored Aug 14, 2021
    Configuration menu
    Copy the full SHA
    e4a5f5a View commit details
    Browse the repository at this point in the history

  7. Update src/transformers/models/gptj/modeling_gptj.py 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @StellaAthena @sgugger
    StellaAthena and sgugger authored Aug 14, 2021
    Configuration menu
    Copy the full SHA
    2d0a2a0 View commit details
    Browse the repository at this point in the history

  8. Update src/transformers/models/gptj/configuration_gptj.py 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @StellaAthena @sgugger
    StellaAthena and sgugger authored Aug 14, 2021
    Configuration menu
    Copy the full SHA
    7443fcb View commit details
    Browse the repository at this point in the history

  9. Update src/transformers/models/gptj/configuration_gptj.py 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @StellaAthena @sgugger
    StellaAthena and sgugger authored Aug 14, 2021
    Configuration menu
    Copy the full SHA
    b3c1a20 View commit details
    Browse the repository at this point in the history

  10. Update src/transformers/models/gptj/configuration_gptj.py 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @StellaAthena @sgugger
    StellaAthena and sgugger authored Aug 14, 2021
    Configuration menu
    Copy the full SHA
    0bedd33 View commit details
    Browse the repository at this point in the history

  11. Update src/transformers/models/gptj/modeling_gptj.py 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @StellaAthena @sgugger
    StellaAthena and sgugger authored Aug 14, 2021
    Configuration menu
    Copy the full SHA
    2507592 View commit details
    Browse the repository at this point in the history

  12. Update src/transformers/models/gptj/modeling_gptj.py 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @StellaAthena @sgugger
    StellaAthena and sgugger authored Aug 14, 2021
    Configuration menu
    Copy the full SHA
    cd4713f View commit details
    Browse the repository at this point in the history

  13. Update src/transformers/models/gptj/modeling_gptj.py 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @StellaAthena @sgugger
    StellaAthena and sgugger authored Aug 14, 2021
    Configuration menu
    Copy the full SHA
    f0a3c0a View commit details
    Browse the repository at this point in the history

  14. Update src/transformers/models/gptj/modeling_gptj.py 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @StellaAthena @sgugger
    StellaAthena and sgugger authored Aug 14, 2021
    Configuration menu
    Copy the full SHA
    f046728 View commit details
    Browse the repository at this point in the history

Commits on Aug 17, 2021

  1. Update src/transformers/models/gptj/modeling_gptj.py 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @StellaAthena @sgugger
    StellaAthena and sgugger authored Aug 17, 2021
    Configuration menu
    Copy the full SHA
    3ae0298 View commit details
    Browse the repository at this point in the history

  2. Replaced GPTJ-specific code with generic code

    @StellaAthena
    StellaAthena authored Aug 17, 2021
    Configuration menu
    Copy the full SHA
    27aeb5f View commit details
    Browse the repository at this point in the history

  3. Update src/transformers/models/gptj/modeling_gptj.py 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @StellaAthena @sgugger
    StellaAthena and sgugger authored Aug 17, 2021
    Configuration menu
    Copy the full SHA
    8c8ee6b View commit details
    Browse the repository at this point in the history

  4. Made the code always use rotary positional encodings

    @StellaAthena
    StellaAthena authored Aug 17, 2021
    Configuration menu
    Copy the full SHA
    53b801a View commit details
    Browse the repository at this point in the history

  5. Update index.rst

    @StellaAthena
    StellaAthena authored Aug 17, 2021
    Configuration menu
    Copy the full SHA
    ae18ff5 View commit details
    Browse the repository at this point in the history

  6. Fix documentation

    @StellaAthena
    StellaAthena committed Aug 17, 2021
    Configuration menu
    Copy the full SHA
    c4d11ca View commit details
    Browse the repository at this point in the history

  7. Combine attention classes 

    - Condense all attention operations into `GPTJAttention`
- Replicate GPT-2 and improve code clarity by renaming `GPTJAttention.attn_pdrop` and `GPTJAttention.resid_pdrop` to `GPTJAttention.attn_dropout` and `GPTJAttention.resid_dropout`
    @EricHallahan
    EricHallahan committed Aug 17, 2021
    Configuration menu
    Copy the full SHA
    4a37a78 View commit details
    Browse the repository at this point in the history

  8. Removed config.rotary_dim from tests

    @StellaAthena
    StellaAthena authored Aug 17, 2021
    Configuration menu
    Copy the full SHA
    d5e5f84 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'huggingface:master' into master

    @StellaAthena
    StellaAthena authored Aug 17, 2021
    Configuration menu
    Copy the full SHA
    3522e07 View commit details
    Browse the repository at this point in the history

  10. Update test_modeling_gptj.py

    @StellaAthena
    StellaAthena authored Aug 17, 2021
    Configuration menu
    Copy the full SHA
    c27e587 View commit details
    Browse the repository at this point in the history

  11. Update test_modeling_gptj.py

    @StellaAthena
    StellaAthena authored Aug 17, 2021
    Configuration menu
    Copy the full SHA
    9eebb6f View commit details
    Browse the repository at this point in the history

  12. Fix formatting

    @EricHallahan
    EricHallahan committed Aug 17, 2021
    Configuration menu
    Copy the full SHA
    ff301d3 View commit details
    Browse the repository at this point in the history

Commits on Aug 18, 2021

  1. Removed depreciated argument layer_id to GPTJAttention

    @StellaAthena
    StellaAthena authored Aug 18, 2021
    Configuration menu
    Copy the full SHA
    ff1eb1d View commit details
    Browse the repository at this point in the history

  2. Update modeling_gptj.py

    @StellaAthena
    StellaAthena authored Aug 18, 2021
    Configuration menu
    Copy the full SHA
    4c86bbc View commit details
    Browse the repository at this point in the history

  3. Update modeling_gptj.py

    @StellaAthena
    StellaAthena authored Aug 18, 2021
    Configuration menu
    Copy the full SHA
    1f99941 View commit details
    Browse the repository at this point in the history

  4. Fix code quality

    @EricHallahan
    EricHallahan committed Aug 18, 2021
    Configuration menu
    Copy the full SHA
    b6021cf View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2021

  1. Restore model functionality

    @EricHallahan
    EricHallahan committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    d2c85a2 View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2021

  1. Save lm_head.weight in checkpoints

    @EricHallahan
    EricHallahan authored Aug 22, 2021
    Configuration menu
    Copy the full SHA
    223bda1 View commit details
    Browse the repository at this point in the history

  2. Fix crashes when loading with reduced precision

    @EricHallahan
    EricHallahan authored Aug 22, 2021
    Configuration menu
    Copy the full SHA
    ad567a9 View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2021

  1. refactor self._attn(...)` and rename layer weights"

    your_github_username committed Aug 23, 2021
    Configuration menu
    Copy the full SHA
    af0c01d View commit details
    Browse the repository at this point in the history

  2. make sure logits are in fp32 for sampling

    @patrickvonplaten
    patrickvonplaten committed Aug 23, 2021
    Configuration menu
    Copy the full SHA
    c90eb3b View commit details
    Browse the repository at this point in the history

  3. improve docs

    @patrickvonplaten
    patrickvonplaten committed Aug 23, 2021
    Configuration menu
    Copy the full SHA
    3897823 View commit details
    Browse the repository at this point in the history

Commits on Aug 25, 2021

  1. Add GPTJForCausalLM to TextGenerationPipeline whitelist

    @EricHallahan
    EricHallahan authored Aug 25, 2021
    Configuration menu
    Copy the full SHA
    504b339 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'huggingface:master' into master

    @StellaAthena
    StellaAthena authored Aug 25, 2021
    Configuration menu
    Copy the full SHA
    7dcc7c5 View commit details
    Browse the repository at this point in the history

  3. Added GPT-J to the README

    @StellaAthena
    StellaAthena authored Aug 25, 2021
    Configuration menu
    Copy the full SHA
    8cbaa1f View commit details
    Browse the repository at this point in the history

  4. Fix doc/readme consistency

    @EricHallahan
    EricHallahan committed Aug 25, 2021
    Configuration menu
    Copy the full SHA
    bca938d View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2021

  1. Merge branch 'master' into master

    @StellaAthena
    StellaAthena authored Aug 27, 2021
    Configuration menu
    Copy the full SHA
    71d3300 View commit details
    Browse the repository at this point in the history

  2. Add rough parallelization support 

    - Remove unused imports and variables
- Clean up docstrings
- Port experimental parallelization code from GPT-2 into GPT-J
    @EricHallahan
    EricHallahan committed Aug 27, 2021
    Configuration menu
    Copy the full SHA
    22f8131 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2021

  1. Clean up loose ends

    @EricHallahan
    EricHallahan authored Aug 30, 2021
    Configuration menu
    Copy the full SHA
    1d69e42 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into master

    @EricHallahan
    EricHallahan authored Aug 30, 2021
    Configuration menu
    Copy the full SHA
    bce04cb View commit details
    Browse the repository at this point in the history

  3. Fix index.rst

    @EricHallahan
    EricHallahan authored Aug 30, 2021
    Configuration menu
    Copy the full SHA
    4784eae View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2021

  1. fix merge conflicts

    @patrickvonplaten
    patrickvonplaten committed Aug 31, 2021
    Configuration menu
    Copy the full SHA
    3466cd0 View commit details
    Browse the repository at this point in the history