WIP: [MPT] Support MPT-7b-instruct model#460

Closed

vvchernov wants to merge 114 commits intomlc-ai:mainfrom Deelvin:vc/mpt-7b-instruct

+1,827-25

Commits on Jul 5, 2023

add file for mpt-7b-instruct model
Valery Chernov
committed
update build.py for mpt
Valery Chernov
committed
update utils
Valery Chernov
committed
add MPTConfig
Valery Chernov
committed
add get_model function like implementation for t5
Valery Chernov
committed
add MPTMLP, get Linear from Llama for a moment
Valery Chernov
committed
add low-precision layer norm, need to correct it further
Valery Chernov
committed
MPTBlock was implemented
Valery Chernov
committed
update MPTConfig by dtype
Valery Chernov
committed
draft for attentions layers of MPT. some updates
Valery Chernov
committed
_reset_is_causal and attn_bias_shape methods were added. __init__ of MPTModel was refactored
Valery Chernov
committed
MPTForCausalLM was implemented on Relax, some TODOs were added
Valery Chernov
committed
rearrange from einops was replaced by relax ops
Valery Chernov
committed
reimplement scaled_multihead_dot_product_attention by relax
Valery Chernov
committed
replace torch.finfo
Valery Chernov
committed
finish scaled_multihead_dot_product_attention, some TODOs are still there
Valery Chernov
committed
replace torch from flash_attn_fn
Valery Chernov
committed
replace torch from triton_flash_attn_fn
Valery Chernov
committed
update MPTModel forward, replace all torch operations
Valery Chernov
committed
implement masked_fill by relax, replace torch masked_fill by it. remove corresponding TODOs. other torch replacements
Valery Chernov
committed
fix max on dynamic values
Valery Chernov
committed
implement build_attn_bias with dependencies
Valery Chernov
committed
transfer of code for the sake of convenience
Valery Chernov
committed
_attn_bias of MPTModel was implemented
Valery Chernov
committed
_apply_prefix_mask of MPTModel was implemented on relax
Valery Chernov
committed
_apply_sequence_id of MPTModel was implemented on relax
Valery Chernov
committed
fix layer norm
Valery Chernov
committed
remove device
Valery Chernov
committed
unroll flash_attn implementation using sources
Valery Chernov
committed
slicings were reimplemented from python style to relax. corresponding TODOs were removed
Valery Chernov
committed
add draft for create_decoding_func. Fix two TODOs related to rank
Valery Chernov
committed
replace handmade masked_filled by relax op implemented in mlc-relax
Valery Chernov
committed
replace torch logical_or by relax op implemented in mlc-relax
Valery Chernov
committed
replace torch logical_not by relax op implemented in mlc-relax
Valery Chernov
committed
fix TODO with index_select
Valery Chernov
committed
small fixes
Valery Chernov
committed
remove backwards
Valery Chernov
committed
zone different types of flash attention implementation
Valery Chernov
committed
commented flash attention implementations with types flash and triton
Valery Chernov
committed
some fixes
Valery Chernov
committed
fix dtype in Linear layers
Valery Chernov
committed
fix dtype in layer norm
Valery Chernov
committed
fix config using
Valery Chernov
committed
small fixes
Valery Chernov
committed
tir.Cast was replaced by relax.op.astype
Valery Chernov
committed
update downcast workaround for lplayernorm
Valery Chernov
committed
more torch group were replaced by relax ops
Valery Chernov
committed
correct rearrange
Valery Chernov
committed
switch on model_path in get_model method. need to redo due to update in other models code
Valery Chernov
committed
small fixes
Valery Chernov
committed
replace matmul by linear for weight transposition from the box
Valery Chernov
committed
fixes
Valery Chernov
committed
check decode only for mpt models
Valery Chernov
committed
add desc for mpt-7b-instruct
Valery Chernov
committed
upstream weights mapping
Valery Chernov
committed
fix assert check
Valery Chernov
committed
add custom f_convert_pname_fwd
Valery Chernov
committed
once more update of f_convert_pname_fwd
Valery Chernov
committed
skip bias from weights
Valery Chernov
committed
add f_convert_param_bkwd
Valery Chernov
committed
try to fix bfloat16
Valery Chernov
committed
file structure for mpt model was refactored
Valery Chernov
committed
add script to convert model from bfloat16 to float16
Valery Chernov
committed
fix f_convert_param_bkwd
Valery Chernov
committed
clean code for conversion script, add desc
Valery Chernov
committed
workaround for lookup func
Valery Chernov
committed
add dummy create_kv_cache_func method to support mlc_chat_cli
Valery Chernov
committed
debug log
Valery Chernov
committed
add create_softmax_func for MPT
Valery Chernov
committed
update transform
Valery Chernov
committed
remove debug log
Valery Chernov
committed
add conversation for MPT
Valery Chernov
committed
remove kv_cache_func
Valery Chernov
committed
remove kv_cache from the list of funcs for mpt
Valery Chernov
committed
cast logits to float32 before softmax with temperature
Valery Chernov
committed
debug log: check contiguous weight
Valery Chernov
committed
debug log for logits
Valery Chernov
committed
update readme and prepare_inputs_for_generation based on mpt model specific
Valery Chernov
committed
flash attn implementation was transferred to outside mpt model implementation
Valery Chernov
committed
correct import after rename file
Valery Chernov
committed
set temperature to zero by default for mpt model
Valery Chernov
committed
transfer attention_mask preprocessing to decoder forward. update prepare_inputs_for_generation
Valery Chernov
committed
prepare_inputs_for_generation was fully transferred to forward pass
Valery Chernov
committed
remove excess methods
Valery Chernov
committed
remove excess methods once more
Valery Chernov
committed
init kv_cache only if need
Valery Chernov
committed
do not use kv_cache during Forward if it is empty
Valery Chernov
committed
remove input_data from Decode_step due to it is not used and recalculated in Forward
Valery Chernov
committed
test fixes
Valery Chernov
committed
unroll method from generate in README
Valery Chernov
committed
remove test logs. add PrintLogits method
Valery Chernov
committed
print logits after copy to cpu
Valery Chernov
committed

Commits on Jul 6, 2023

print shape together with logits
Valery Chernov
committed
test log
Valery Chernov
committed
print decode input
Valery Chernov
committed
test log in prefill step
Valery Chernov
committed
print intermediate tensors in topology to catch nan generation
Valery Chernov
committed
print intermediate tensors in topology to catch nan generation: attn_bias
Valery Chernov
committed
revert some debug logs
Valery Chernov
committed
artificial error
Valery Chernov
committed
reimplement all remaining tir funcs to relax
Valery Chernov
committed
print only 10 values from logits
Valery Chernov
committed
revert test logits transform
Valery Chernov
committed
remove debug logs and workaround
Valery Chernov
committed
return correct output
Valery Chernov
committed
continue debug
Valery Chernov
committed

Commits on Jul 12, 2023

remove unneccessary parts from mpt topology. calculate query-key matmul in float32 to avoid inf generation
Valery Chernov
committed

Commits on Jul 24, 2023

create comparator
Valery Chernov
committed
update README
Valery Chernov
committed
update mpt model file: fix layernorm, remove some TODOs, remove excess code, comment unneccessary code parts, upstream layer names for correct mapping
Valery Chernov
committed
remove debug prints
Valery Chernov
committed
update PrintNDArray method
Valery Chernov
committed

Commits on Aug 25, 2023

support mlc-llm chat using with or without kv cache
Valery Chernov
committed
strong refactor based on vc/dev of mpt-like relax model to support using with/without kv cache
Valery Chernov
committed