Skip to content

Commit

Permalink
Update SQ/WOQ status (#1869)
Browse files Browse the repository at this point in the history 
Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>
  • Loading branch information
@XuehaoSun @chensuyue
XuehaoSun and chensuyue authored Jun 14, 2024
1 parent b401b02 commit 498af74
Showing 1 changed file with 132 additions and 134 deletions.
266 changes: 132 additions & 134 deletions docs/source/llm_recipes.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ This document aims to publish the specific recipes we achieved for the popular L
| databricks/dolly-v2-12b | ✖ | ✔ | ✖ |
| EleutherAI/gpt-neox-20b | ✖ | ✔ | ✔ |
| mistralai/Mistral-7B-v0.1 | ✖ | ✔ | ✔ |
| THUDM/chatglm2-6b | WIP | ✔ | ✔ |
| THUDM/chatglm3-6b | WIP | ✔ | ✔ |
| THUDM/chatglm2-6b | ✔ | ✔ | ✔ |
| THUDM/chatglm3-6b | WIP | ✔ | WIP |

**Detail recipes can be found [HERE](https://github.com/intel/intel-extension-for-transformers/blob/main/examples/huggingface/pytorch/text-generation/quantization/llm_quantization_recipes.md).**

Expand All @@ -40,8 +40,8 @@ This document aims to publish the specific recipes we achieved for the popular L
> - The WIP recipes will be published soon.
## Large Language Models Accuracy
<table>
<thead>

<table><thead>
<tr>
<th rowspan="3">Model</th>
<th colspan="9">lambada_openai</th>
Expand All @@ -63,212 +63,210 @@ This document aims to publish the specific recipes we achieved for the popular L
<th>Ratio</th>
<th>ACC</th>
<th>Ratio</th>
</tr>
</thead>
</tr></thead>
<tbody>
<tr>
<td>baichuan-inc/Baichuan-13B-Chat</td>
<td>67.57%</td>
<td>68.23%</td>
<td>1.0098</td>
<td>67.57%</td>
<td>1.0000</td>
<td>67.84%</td>
<td>1.0040</td>
<td>NA</td>
<td>NA</td>
<td>69.07%</td>
<td>1.0222</td>
<td>67.55%</td>
<td>0.9997</td>
<td>68.12%</td>
<td>1.0081</td>
<td>66.93%</td>
<td>0.9905</td>
</tr>
<tr>
<td>baichuan-inc/Baichuan2-13B-Chat</td>
<td>71.51%</td>
<td>70.89%</td>
<td>0.9913</td>
<td>71.53%</td>
<td>1.0003</td>
<td>71.76%</td>
<td>1.0035</td>
<td>NA</td>
<td>NA</td>
<td>75.57%</td>
<td>1.0568</td>
<td>71.57%</td>
<td>1.0008</td>
<td>70.81%</td>
<td>0.9902</td>
<td>N/A</td>
<td>N/A</td>
</tr>
<tr>
<td>baichuan-inc/Baichuan2-7B-Chat</td>
<td>67.67%</td>
<td>67.96%</td>
<td>1.0043</td>
<td>67.59%</td>
<td>0.9988</td>
<td>67.24%</td>
<td>0.9936</td>
<td>67.42%</td>
<td>0.9963</td>
<td>68.06%</td>
<td>1.0058</td>
<td>67.61%</td>
<td>0.9991</td>
<td>67.90%</td>
<td>1.0034</td>
<td>N/A</td>
<td>N/A</td>
</tr>
<tr>
<td>bigscience/bloom-1b7</td>
<td>46.34%</td>
<td>47.99%</td>
<td>1.0356</td>
<td>46.38%</td>
<td>1.0009</td>
<td>46.19%</td>
<td>0.9968</td>
<td>NA</td>
<td>NA</td>
<td>46.21%</td>
<td>0.9972</td>
<td>46.90%</td>
<td>1.0121</td>
<td>N/A</td>
<td>N/A</td>
</tr>
<tr>
<td>databricks/dolly-v2-12b</td>
<td>64.35%</td>
<td>NA</td>
<td>NA</td>
<td>64.10%</td>
<td>0.9961</td>
<td>NA</td>
<td>NA</td>
<td>NA</td>
<td>NA</td>
<td>N/A</td>
<td>N/A</td>
<td>63.92%</td>
<td>0.9933</td>
<td>N/A</td>
<td>N/A</td>
<td>N/A</td>
<td>N/A</td>
</tr>
<tr>
<td>EleutherAI/gpt-j-6b</td>
<td>68.31%</td>
<td>68.33%</td>
<td>1.0003</td>
<td>68.23%</td>
<td>0.9988</td>
<td>68.79%</td>
<td>1.0070</td>
<td>68.43%</td>
<td>1.0018</td>
<td>68.27%</td>
<td>0.9994</td>
<td>68.27%</td>
<td>0.9994</td>
<td>68.35%</td>
<td>1.0006</td>
<td>68.02%</td>
<td>0.9958</td>
</tr>
<tr>
<td>EleutherAI/gpt-neox-20b</td>
<td>72.33%</td>
<td>NA</td>
<td>NA</td>
<td>72.25%</td>
<td>0.9989</td>
<td>71.96%</td>
<td>0.9949</td>
<td>NA</td>
<td>NA</td>
<td>N/A</td>
<td>N/A</td>
<td>72.29%</td>
<td>0.9994</td>
<td>71.74%</td>
<td>0.9918</td>
<td>N/A</td>
<td>N/A</td>
</tr>
<tr>
<td>facebook/opt-1.3b</td>
<td>57.89%</td>
<td>57.54%</td>
<td>0.9940</td>
<td>58.08%</td>
<td>1.0033</td>
<td>58.57%</td>
<td>1.0117</td>
<td>NA</td>
<td>NA</td>
<td>57.68%</td>
<td>0.9964</td>
<td>58.12%</td>
<td>1.0040</td>
<td>58.26%</td>
<td>1.0064</td>
<td>N/A</td>
<td>N/A</td>
</tr>
<tr>
<td>facebook/opt-30b</td>
<td>71.49%</td>
<td>71.51%</td>
<td>1.0003</td>
<td>71.51%</td>
<td>1.0003</td>
<td>71.82%</td>
<td>1.0046</td>
<td>72.11%</td>
<td>1.0087</td>
<td>71.78%</td>
<td>1.0041</td>
<td>71.53%</td>
<td>1.0006</td>
<td>71.59%</td>
<td>1.0014</td>
<td>71.80%</td>
<td>1.0043</td>
</tr>
<tr>
<td>meta-llama/Llama-2-13b-hf</td>
<td>76.77%</td>
<td>76.25%</td>
<td>0.9932</td>
<td>76.75%</td>
<td>0.9997</td>
<td>77.43%</td>
<td>1.0086</td>
<td>76.75%</td>
<td>0.9997</td>
<td>76.89%</td>
<td>1.0016</td>
<td>77.66%</td>
<td>1.0116</td>
<td>76.60%</td>
<td>0.9978</td>
</tr>
<tr>
<td>meta-llama/Llama-2-70b-hf</td>
<td>79.64%</td>
<td>79.55%</td>
<td>0.9989</td>
<td>79.57%</td>
<td>0.9991</td>
<td>79.14%</td>
<td>0.9937</td>
<td>79.62%</td>
<td>0.9997</td>
<td>80.09%</td>
<td>1.0057</td>
<td>79.97%</td>
<td>1.0041</td>
<td>79.68%</td>
<td>1.0005</td>
</tr>
<tr>
<td>meta-llama/Llama-2-7b-hf</td>
<td>73.92%</td>
<td>73.45%</td>
<td>0.9936</td>
<td>73.96%</td>
<td>1.0005</td>
<td>73.45%</td>
<td>0.9936</td>
<td>73.49%</td>
<td>0.9942</td>
<td>73.90%</td>
<td>0.9997</td>
<td>73.84%</td>
<td>0.9989</td>
<td>N/A</td>
<td>N/A</td>
</tr>
<tr>
<td>mistralai/Mistral-7B-v0.1</td>
<td>75.90%</td>
<td>NA</td>
<td>NA</td>
<td>N/A</td>
<td>N/A</td>
<td>75.80%</td>
<td>0.9987</td>
<td>76.13%</td>
<td>1.0030</td>
<td>75.61%</td>
<td>0.9962</td>
<td>76.25%</td>
<td>1.0046</td>
<td>75.74%</td>
<td>0.9979</td>
</tr>
<tr>
<td>THUDM/chatglm2-6b</td>
<td>53.23%</td>
<td>NA</td>
<td>NA</td>
<td>53.19%</td>
<td>0.9992</td>
<td>52.77%</td>
<td>0.9914</td>
<td>53.35%</td>
<td>1.0023</td>
<td>52.86%</td>
<td>0.9930</td>
<td>53.00%</td>
<td>0.9957</td>
<td>52.90%</td>
<td>0.9938</td>
<td>52.92%</td>
<td>0.9942</td>
</tr>
<tr>
<td>THUDM/chatglm3-6b</td>
<td>59.09%</td>
<td>NA</td>
<td>NA</td>
<td>59.01%</td>
<td>0.9986</td>
<td>NA</td>
<td>NA</td>
<td>58.61%</td>
<td>0.9919</td>
<td>N/A</td>
<td>N/A</td>
<td>59.03%</td>
<td>0.9990</td>
<td>N/A</td>
<td>N/A</td>
<td>N/A</td>
<td>N/A</td>
</tr>
<tr>
<td>tiiuae/falcon-40b</td>
<td>77.22%</td>
<td>77.04%</td>
<td>0.9977</td>
<td>77.22%</td>
<td>1.0000</td>
<td>77.94%</td>
<td>1.0093</td>
<td>78.79%</td>
<td>1.0203</td>
<td>76.95%</td>
<td>0.9965</td>
<td>77.18%</td>
<td>0.9995</td>
<td>77.55%</td>
<td>1.0043</td>
<td>77.82%</td>
<td>1.0078</td>
</tr>
<tr>
<td>tiiuae/falcon-7b</td>
<td>74.67%</td>
<td>76.44%</td>
<td>1.0237</td>
<td>74.77%</td>
<td>1.0013</td>
<td>75.00%</td>
<td>1.0044</td>
<td>NA</td>
<td>NA</td>
<td>76.63%</td>
<td>1.0262</td>
<td>74.73%</td>
<td>1.0008</td>
<td>75.06%</td>
<td>1.0052</td>
<td>74.00%</td>
<td>0.9910</td>
</tr>
</tbody>
</table>
</tbody></table>

0 comments on commit 498af74

Please sign in to comment.