Add save/load for pt2e example #1927

Kaihui-intel · 2024-07-16T07:04:13Z

Type of Change

feature

Description

detail description

Expected Behavior & Potential Risk

the expected behavior that triggered by this PR

How has this PR been tested?

how to reproduce the test (including hardware information)

Dependency Change?

any library dependency introduced or removed

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

github-actions · 2024-07-16T07:04:39Z

⚡ Required checks status: All passing 🟢

No groups match the files changed in this PR.

Thank you for your contribution! 💜

Note
This comment is automatically generated and will be updates every 180 seconds within the next 6 hours. If you have any other questions, contact chensuyue or XuehaoSun for help.

yiliu30

Please trigger some extension tests. Others are LGTM.

...i/pytorch/nlp/huggingface_models/language-modeling/quantization/weight_only/run_benchmark.sh

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

...rch/nlp/huggingface_models/language-modeling/quantization/static_quant/pt2e/run_benchmark.sh

examples/3.x_api/pytorch/cv/static_quant/run_quant.sh

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

for more information, see https://pre-commit.ci

…eural-compressor into kaihui/pt2e_eg_save_load

examples/3.x_api/pytorch/cv/static_quant/main.py

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

commit 2bb257e Author: Kaihui-intel <kaihui.tang@intel.com> Date: Thu Oct 10 19:27:11 2024 +0800 Add woq examples (#1982) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> Co-authored-by: Sun, Xuehao <xuehao.sun@intel.com> commit 586eb88 Author: Huang, Tai <tai.huang@intel.com> Date: Wed Oct 9 09:22:39 2024 +0800 add transformers-like api link in readme (#2022) Signed-off-by: Huang, Tai <tai.huang@intel.com> commit 4e9c764 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Oct 8 13:13:45 2024 +0800 Remove itrex dependency for 3x example (#2016) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Co-authored-by: Sun, Xuehao <xuehao.sun@intel.com> commit a0066d4 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Mon Sep 30 18:17:32 2024 +0800 Fix transformers rtn layer-wise quant (#2008) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> commit 802a5af Author: Huang, Tai <tai.huang@intel.com> Date: Mon Sep 30 17:02:52 2024 +0800 add autoround EMNLP24 to pub list (#2014) Signed-off-by: Huang, Tai <tai.huang@intel.com> commit 44795a1 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Mon Sep 30 16:55:22 2024 +0800 Adapt transformers 4.45.1 (#2019) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Co-authored-by: changwangss <chang1.wang@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> commit d4662ad Author: Kaihui-intel <kaihui.tang@intel.com> Date: Mon Sep 30 15:52:17 2024 +0800 Add transformers-like api doc (#2018) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 72398b6 Author: Wang, Chang <chang1.wang@intel.com> Date: Fri Sep 27 15:11:04 2024 +0800 fix xpu device set weight and bias (#2010) Signed-off-by: changwangss <chang1.wang@intel.com> Co-authored-by: Sun, Xuehao <xuehao.sun@intel.com> commit 9d27743 Author: Sun, Xuehao <xuehao.sun@intel.com> Date: Fri Sep 27 14:17:24 2024 +0800 Update model accuracy (#2006) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> commit 7bbc473 Author: xinhe <xin3.he@intel.com> Date: Fri Sep 27 11:47:00 2024 +0800 add pad_to_buckets in evaluation for hpu performance (#2011) * add pad_to_buckets in evaluation for hpu performance --------- Signed-off-by: xin3he <xin3.he@intel.com> commit b6b7d7c Author: Kaihui-intel <kaihui.tang@intel.com> Date: Thu Sep 26 17:21:54 2024 +0800 Update auto_round requirements for transformers example (#2013) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit ee600ba Author: Wang, Chang <chang1.wang@intel.com> Date: Fri Sep 20 13:54:06 2024 +0800 add repack_awq_to_optimum_format function (#1998) Signed-off-by: changwangss <chang1.wang@intel.com> commit 4ee6861 Author: Sun, Xuehao <xuehao.sun@intel.com> Date: Thu Sep 19 22:27:05 2024 +0800 remove accelerate version in unit test (#2007) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> commit 2445811 Author: WeiweiZhang1 <weiwei1.zhang@intel.com> Date: Sat Sep 14 18:13:30 2024 +0800 enable auto_round format export (#2002) Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> commit 906333a Author: Kaihui-intel <kaihui.tang@intel.com> Date: Sat Sep 14 16:17:46 2024 +0800 Replace FORCE_DEVICE with INC_TARGET_DEVICE [transformers] (#2005) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 443d007 Author: xinhe <xin3.he@intel.com> Date: Fri Sep 13 21:35:32 2024 +0800 add INC_FORCE_DEVICE introduction (#1988) * add INC_FORCE_DEVICE introduction Signed-off-by: xin3he <xin3.he@intel.com> * Update PyTorch.md * Update PyTorch.md * Update docs/source/3x/PyTorch.md Co-authored-by: Yi Liu <yi4.liu@intel.com> * rename to INC_TARGET_DEVICE Signed-off-by: xin3he <xin3.he@intel.com> --------- Signed-off-by: xin3he <xin3.he@intel.com> Co-authored-by: Yi Liu <yi4.liu@intel.com> commit 5de9a4f Author: Kaihui-intel <kaihui.tang@intel.com> Date: Fri Sep 13 20:48:22 2024 +0800 Support transformers-like api for woq quantization (#1987) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Wang, Chang <chang1.wang@intel.com> commit 9c39b42 Author: chen, suyue <suyue.chen@intel.com> Date: Thu Sep 12 14:34:49 2024 +0800 update docker image prune rules (#2003) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 09d4f2d Author: Huang, Tai <tai.huang@intel.com> Date: Mon Sep 9 09:24:35 2024 +0800 Add recent publications (#1995) * add recent publications Signed-off-by: Huang, Tai <tai.huang@intel.com> * update total count Signed-off-by: Huang, Tai <tai.huang@intel.com> --------- Signed-off-by: Huang, Tai <tai.huang@intel.com> commit 399cd44 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Sep 3 16:37:09 2024 +0800 Remove the save of gptq config (#1993) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 05272c4 Author: Yi Liu <yi4.liu@intel.com> Date: Tue Sep 3 10:21:51 2024 +0800 add per_channel_minmax (#1990) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 82d8c06 Author: chen, suyue <suyue.chen@intel.com> Date: Fri Aug 30 21:21:00 2024 +0800 update 3x pt binary build (#1992) Signed-off-by: chensuyue <suyue.chen@intel.com> commit e9f06af Author: Huang, Tai <tai.huang@intel.com> Date: Fri Aug 30 17:49:48 2024 +0800 Update installation_guide.md (#1989) Correct typo in installation doc commit 093c966 Author: Wang, Chang <chang1.wang@intel.com> Date: Fri Aug 30 17:45:54 2024 +0800 add quantize, save, load function for transformers-like api (#1986) Signed-off-by: changwangss <chang1.wang@intel.com> commit 4dd49a4 Author: xinhe <xin3.he@intel.com> Date: Thu Aug 29 17:23:18 2024 +0800 add hasattr check for torch fp8 dtype (#1985) Signed-off-by: xin3he <xin3.he@intel.com> commit f2c454f Author: chen, suyue <suyue.chen@intel.com> Date: Thu Aug 29 13:45:39 2024 +0800 update installation and ci test for 3x api (#1991) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 7ba9fdc Author: Kaihui-intel <kaihui.tang@intel.com> Date: Mon Aug 19 14:50:50 2024 +0800 support gptq `true_sequential` and `quant_lm_head` (#1977) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 68b1f8b Author: Sun, Xuehao <xuehao.sun@intel.com> Date: Fri Aug 16 09:43:46 2024 +0800 Fix UT env and upgrade torch to 2.4.0 (#1978) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> commit f9dfd54 Author: Yi Liu <yi4.liu@intel.com> Date: Thu Aug 15 14:13:26 2024 +0800 Skip some tests for torch 2.4 (#1981) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 46d9192 Author: xinhe <xin3.he@intel.com> Date: Thu Aug 15 09:57:22 2024 +0800 update readme for fp8 (#1979) Signed-off-by: xinhe3 <xinhe3@habana.ai> commit 842b715 Author: chen, suyue <suyue.chen@intel.com> Date: Tue Aug 13 12:09:25 2024 +0800 bump main version into v3.1 (#1974) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 3845cdc Author: Neo Zhang Jianyu <jianyu.zhang@intel.com> Date: Tue Aug 13 12:09:09 2024 +0800 fix online doc search issue (#1975) Co-authored-by: ZhangJianyu <zhang.jianyu@outlook.com> commit 7056720 Author: chen, suyue <suyue.chen@intel.com> Date: Sun Aug 11 20:58:34 2024 +0800 update main page (#1973) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 95197d1 Author: xinhe <xin3.he@intel.com> Date: Sat Aug 10 23:28:43 2024 +0800 Cherry pick v1.17.0 (#1964) * [SW-184941] INC CI, CD and Promotion Change-Id: I60c420f9776e1bdab7bb9e02e5bcbdb6891bfe52 * [SW-183320]updated setup.py Change-Id: I592af89486cb1d9e0b5197521c428920197a9103 * [SW-177474] add HQT FP8 porting code Change-Id: I4676f13a5ed43c444f2ec68675cc41335e7234dd Signed-off-by: Zhou Yuwen <zyuwen@habana.ai> * [SW-189361] Fix white list extend Change-Id: Ic2021c248798fce37710d28014a6d59259c868a3 * [SW-191317] Raise exception according to hqt config object Change-Id: I06ba8fa912c811c88912987c11e5c12ef328348a * [SW-184714] Port HQT code into INC HQT lib content was copied as is under fp8_quant Tests were copied to 3.x torch location Change-Id: Iec6e1fa7ac4bf1df1c95b429524c40e32bc13ac9 * [SW-184714] Add internal folder to fp8 quant This is a folder used for experiments, not to be used by users Change-Id: I9e221ae582794e304e95392c0f37638f7bce69bc * [SW-177468] Removed unused code + cleanup Change-Id: I4d27c067e87c1a30eb1da9df16a16c46d092c638 * Fix errors in regression_detection Change-Id: Iee5318bd5593ba349812516eb5641958ece3c438 * [SW-187731] Save orig module as member of patched module This allows direct usage of the original module methods, which solves torch compile issue Change-Id: I464d8bd1bacdfc3cd1f128a67114e1e43f092632 * [SW-190899] Install packages according to configuration Change-Id: I570b490658f5d2c5399ba1db93f8f52f56449525 * [SW-184689] use finalize_calibration intrenaly for one step flow Change-Id: Ie0b8b426c951cf57ed7e6e678c86813fb2d05c89 * [SW-191945] align requirement_pt.txt in gerrit INC with Github INC Change-Id: If5c0dbf21bf989af37a8e29246e4f8760cd215ef Signed-off-by: xinhe3 <xinhe3@hababa.ai> * [SW-192358] Remove HQT reference in INC Change-Id: Ic25f9323486596fa2dc6d909cd568a37ab84dd5e * [SW-191415] update fp8 maxAbs observer using torch.copy_ Change-Id: I3923c832f9a8a2b14e392f3f4719d233a457702f * [SW-184943] Enhance INC WOQ model loading - Support loading huggingface WOQ model - Abstract WeightOnlyLinear base class. Add INCWeightOnlyLinear and HPUWeighOnlyLinear subclasses - Load woq linear weight module by module - Save hpu format tensor to reuse it once load it again Change-Id: I679a42759b49e1f45f52bbb0bdae8580a23d0bcf * [SW-190303] Implement HPUWeightOnlyLinear class in INC Change-Id: Ie05c8787e708e2c3559dce24ef0758d6c498ac41 * [SW-192809] fix json_file bug when instantiating FP8Config class Change-Id: I4a715d0a706efe20ccdb49033755cabbc729ccdc Signed-off-by: Zhou Yuwen <zyuwen@habana.ai> * [SW-192931] align setup.py with github INC and remove fp8_convert Change-Id: Ibbc157646cfcfad64b323ecfd96b9bbda5ba9e2f Signed-off-by: xinhe3 <xinhe3@hababa.ai> * [SW-192917] Update all HQT logic files with pre-commit check Change-Id: I119dc8578cb10932fd1a8a674a8bdbf61f978e42 Signed-off-by: xinhe3 <xinhe3@hababa.ai> * update docstring Signed-off-by: yuwenzho <yuwen.zhou@intel.com> * add fp8 example and document (#1639) Signed-off-by: xinhe3 <xinhe3@hababa.ai> * Update settings to be compatible with gerrit * enhance ut Signed-off-by: yuwenzho <yuwen.zhou@intel.com> * move fp8 sample to helloworld folder Signed-off-by: yuwenzho <yuwen.zhou@intel.com> * update torch version of habana docker Signed-off-by: xinhe3 <xinhe3@hababa.ai> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update readme demo Signed-off-by: xinhe3 <xinhe3@hababa.ai> * update WeightOnlyLinear to INCWeightOnlyLinear Signed-off-by: xinhe3 <xinhe3@hababa.ai> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add docstring for FP8Config Signed-off-by: xinhe3 <xinhe3@hababa.ai> * fix pylint Signed-off-by: xinhe3 <xinhe3@hababa.ai> * update fp8 test scripts Signed-off-by: chensuyue <suyue.chen@intel.com> * delete deps Signed-off-by: chensuyue <suyue.chen@intel.com> * update container into v1.17.0 Signed-off-by: chensuyue <suyue.chen@intel.com> * update docker version Signed-off-by: xinhe3 <xinhe3@hababa.ai> * update pt ut Signed-off-by: chensuyue <suyue.chen@intel.com> * add lib path Signed-off-by: chensuyue <suyue.chen@intel.com> * fix dir issue Signed-off-by: xinhe3 <xinhe3@hababa.ai> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update fp8 test scope Signed-off-by: chensuyue <suyue.chen@intel.com> * fix typo Signed-off-by: xinhe3 <xinhe3@hababa.ai> * update fp8 test scope Signed-off-by: chensuyue <suyue.chen@intel.com> * update pre-commit-ci Signed-off-by: chensuyue <suyue.chen@intel.com> * work around for hpu Signed-off-by: xinhe3 <xinhe3@hababa.ai> * fix UT Signed-off-by: xinhe3 <xinhe3@hababa.ai> * fix parameter Signed-off-by: chensuyue <suyue.chen@intel.com> * omit some test Signed-off-by: chensuyue <suyue.chen@intel.com> * update main page example to llm loading Signed-off-by: xinhe3 <xinhe3@hababa.ai> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix autotune Signed-off-by: xinhe3 <xinhe3@hababa.ai> --------- Signed-off-by: Zhou Yuwen <zyuwen@habana.ai> Signed-off-by: xinhe3 <xinhe3@hababa.ai> Signed-off-by: yuwenzho <yuwen.zhou@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: yan tomsinsky <ytomsinsky@habana.ai> Co-authored-by: Ron Ben Moshe <rbenmoshe@habana.ai> Co-authored-by: Uri Livne <ulivne@habana.ai> Co-authored-by: Danny Semiat <dsemiat@habana.ai> Co-authored-by: smarkovichgolan <smarkovich@habana.ai> Co-authored-by: Dudi Lester <dlester@habana.ai> commit de0fa21 Author: Huang, Tai <tai.huang@intel.com> Date: Fri Aug 9 22:32:37 2024 +0800 Fix broken link in docs (#1969) Signed-off-by: Huang, Tai <tai.huang@intel.com> commit 385da7c Author: Sun, Xuehao <xuehao.sun@intel.com> Date: Fri Aug 9 21:53:51 2024 +0800 Add 3.x readme (#1971) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> commit acd8f4f Author: Huang, Tai <tai.huang@intel.com> Date: Fri Aug 9 15:24:14 2024 +0800 Add version mapping between INC and Gaudi SW Stack (#1967) Signed-off-by: Huang, Tai <tai.huang@intel.com> commit 74a4641 Author: Sun, Xuehao <xuehao.sun@intel.com> Date: Fri Aug 9 10:23:59 2024 +0800 remove unnecessary CI (#1966) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> commit b99abae Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Aug 6 16:02:03 2024 +0800 Fix `opt_125m_woq_gptq_int4_dq_ggml` issue (#1965) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit b35ff8f Author: Zixuan Cheng <110808245+violetch24@users.noreply.github.com> Date: Fri Aug 2 09:06:35 2024 +0800 example update for 3.x ipex sq (#1902) Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> commit 000946f Author: Zixuan Cheng <110808245+violetch24@users.noreply.github.com> Date: Thu Aug 1 10:19:32 2024 +0800 add SDXL model example to INC 3.x (#1887) * add SDXL model example to INC 3.x Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> * add evaluation script Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> * add test script Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> * minor fix Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> * Update run_quant.sh * add iter limit Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> * modify test script Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> * update json Signed-off-by: chensuyue <suyue.chen@intel.com> * add requirements Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> * Update run_benchmark.sh * Update sdxl_smooth_quant.py * minor fix Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> --------- Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> Co-authored-by: chensuyue <suyue.chen@intel.com> commit aa42e5e Author: xinhe <xin3.he@intel.com> Date: Wed Jul 31 15:36:06 2024 +0800 replenish docstring (#1955) * replenish docstring Signed-off-by: xin3he <xin3.he@intel.com> * update Quantizer API docstring Signed-off-by: xin3he <xin3.he@intel.com> * Add docstring for auto accelerator (#1956) Signed-off-by: yiliu30 <yi4.liu@intel.com> * temporary remove torch/quantization and add it back after fp8 code is updated. * Update config.py --------- Signed-off-by: xin3he <xin3.he@intel.com> Signed-off-by: yiliu30 <yi4.liu@intel.com> Co-authored-by: Yi Liu <106061964+yiliu30@users.noreply.github.com> commit 81a076d Author: Neo Zhang Jianyu <jianyu.zhang@intel.com> Date: Wed Jul 31 13:51:33 2024 +0800 fix welcome.html link issue (#1962) Co-authored-by: ZhangJianyu <zhang.jianyu@outlook.com> commit 87f02c1 Author: chen, suyue <suyue.chen@intel.com> Date: Wed Jul 31 10:09:47 2024 +0800 fix docs link (#1959) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 03813e2 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed Jul 31 10:09:29 2024 +0800 Bump tensorflow version (#1961) Signed-off-by: dependabot[bot] <support@github.com> commit 3b5dbf6 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Jul 30 17:27:21 2024 +0800 Set low_gpu_mem_usage=False for AutoRound Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 41244d3 Author: chen, suyue <suyue.chen@intel.com> Date: Mon Jul 29 23:05:36 2024 +0800 new previous results could not find all raise issues in CI model test (#1958) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 190e6b2 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Mon Jul 29 19:39:57 2024 +0800 Fix itrex qbits nf4/int8 training core dumped issue (#1954) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> commit 0e724a4 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Mon Jul 29 16:22:13 2024 +0800 Add save/load for pt2e example (#1927) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 50eb6fb Author: chen, suyue <suyue.chen@intel.com> Date: Mon Jul 29 13:40:36 2024 +0800 update 3x torch installation (#1957) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 6e1b1da Author: Zixuan Cheng <110808245+violetch24@users.noreply.github.com> Date: Fri Jul 26 15:58:00 2024 +0800 add ipex xpu example to 3x API (#1948) Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> commit 19024b3 Author: zehao-intel <zehao.huang@intel.com> Date: Fri Jul 26 14:52:01 2024 +0800 Enable yolov5 Example for TF 3x API (#1943) Signed-off-by: zehao-intel <zehao.huang@intel.com> commit d84a93f Author: zehao-intel <zehao.huang@intel.com> Date: Thu Jul 25 14:45:19 2024 +0800 Complement UT of calibration function for TF 3x API (#1945) Signed-off-by: zehao-intel <zehao.huang@intel.com> commit fb85779 Author: zehao-intel <zehao.huang@intel.com> Date: Thu Jul 25 14:04:25 2024 +0800 Update Examples for TF 3x API (#1901) Signed-off-by: zehao-intel <zehao.huang@intel.com> commit 6b30207 Author: zehao-intel <zehao.huang@intel.com> Date: Thu Jul 25 13:39:06 2024 +0800 Add Docstring for TF 3x API and Torch 3x Mixed Precision (#1944) Signed-off-by: zehao-intel <zehao.huang@intel.com> commit d254d50 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Wed Jul 24 21:50:44 2024 +0800 Update doc for client-usage and LWQ (#1947) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit f253d35 Author: Neo Zhang Jianyu <jianyu.zhang@intel.com> Date: Wed Jul 24 17:48:05 2024 +0800 Update publish.yml (#1950) commit 6cda338 Author: Neo Zhang Jianyu <jianyu.zhang@intel.com> Date: Wed Jul 24 17:31:19 2024 +0800 Update publish.yml (#1949) * Update publish.yml * Update publish.yml commit c80b68a Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Jul 23 21:26:53 2024 +0800 Update AutoRound commit version (#1941) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 9077b38 Author: zehao-intel <zehao.huang@intel.com> Date: Tue Jul 23 17:04:37 2024 +0800 Refine Pytorch 3x Mixed Precision Example (#1946) Signed-off-by: zehao-intel <zehao.huang@intel.com> commit efcb293 Author: Neo Zhang Jianyu <jianyu.zhang@intel.com> Date: Tue Jul 23 10:15:41 2024 +0800 Update for API 3.0 online doc (#1940) Co-authored-by: ZhangJianyu <zhang.jianyu@outlook.com> commit b787940 Author: Wang, Mengni <mengni.wang@intel.com> Date: Tue Jul 23 10:12:34 2024 +0800 add docstring for mx quant (#1932) Signed-off-by: Mengni Wang <mengni.wang@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: xinhe <xin3.he@intel.com> commit 0c52e12 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Jul 23 09:59:17 2024 +0800 Add docstring for WOQ&LayerWise (#1938) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: xinhe <xin3.he@intel.com> commit 08914d6 Author: Huang, Tai <tai.huang@intel.com> Date: Mon Jul 22 11:14:44 2024 +0800 add read permission token (#1942) Signed-off-by: Huang, Tai <tai.huang@intel.com> commit e106dea Author: zehao-intel <zehao.huang@intel.com> Date: Sun Jul 21 21:48:51 2024 +0800 Update Example for Pytorch 3x Mixed Precision (#1882) Signed-off-by: zehao-intel <zehao.huang@intel.com> commit 1ebf698 Author: Zixuan Cheng <110808245+violetch24@users.noreply.github.com> Date: Fri Jul 19 15:56:09 2024 +0800 add docstring for static quant and smooth quant (#1936) * add docstring for static quant and smooth quant Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> * format fix Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> * update scan path Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> * Update utility.py --------- Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> Co-authored-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> commit 296c5d4 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Fri Jul 19 15:08:05 2024 +0800 Add docstring for PT2E and HQQ (#1937) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 437c8e7 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Thu Jul 18 10:00:41 2024 +0800 Fix unused pkgs import (#1931) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit ff37401 Author: chen, suyue <suyue.chen@intel.com> Date: Wed Jul 17 23:11:15 2024 +0800 3.X API installation update (#1935) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 6c27c19 Author: zehao-intel <zehao.huang@intel.com> Date: Wed Jul 17 20:35:42 2024 +0800 Support calib_func on TF 3x API (#1934) Signed-off-by: zehao-intel <zehao.huang@intel.com> commit 53e6ee6 Author: Zixuan Cheng <110808245+violetch24@users.noreply.github.com> Date: Wed Jul 17 20:35:03 2024 +0800 Support xpu for ipex static quant (#1916) Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> commit a1cc618 Author: chen, suyue <suyue.chen@intel.com> Date: Wed Jul 17 17:29:49 2024 +0800 remove peft version limit (#1933) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 3058388 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Wed Jul 17 15:31:38 2024 +0800 Add doc for client usage (#1914) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 29471df Author: Kaihui-intel <kaihui.tang@intel.com> Date: Wed Jul 17 12:12:40 2024 +0800 Enhance load_empty_model import (#1930) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit fd96851 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Wed Jul 17 12:05:32 2024 +0800 Integrate AutoRound v0.3 to 2x (#1926) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit bfa27e4 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Wed Jul 17 09:33:13 2024 +0800 Integrate AutoRound v0.3 (#1925) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 5767aed Author: xinhe <xin3.he@intel.com> Date: Wed Jul 17 09:16:37 2024 +0800 add docstring for torch.quantization and torch.utils (#1928) Signed-off-by: xin3he <xin3.he@intel.com> commit f909bca Author: chen, suyue <suyue.chen@intel.com> Date: Tue Jul 16 21:12:54 2024 +0800 update itrex ut test (#1929) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 649e6b1 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Jul 16 21:05:55 2024 +0800 Support LayerWise for RTN/GPTQ (#1883) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Co-authored-by: chensuyue <suyue.chen@intel.com> commit de43d85 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Jul 16 17:18:12 2024 +0800 Support absorb dict for awq (#1920) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit e976595 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Jul 16 17:17:56 2024 +0800 Support woq Autotune (#1921) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit d56075c Author: Huang, Tai <tai.huang@intel.com> Date: Tue Jul 16 15:21:06 2024 +0800 fix typo in architecture diagram (#1924) Signed-off-by: Huang, Tai <tai.huang@intel.com> commit 0a54239 Author: chen, suyue <suyue.chen@intel.com> Date: Tue Jul 16 15:12:43 2024 +0800 update documentation for 3x API (#1923) Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: xin3he <xin3.he@intel.com> Signed-off-by: yiliu30 <yi4.liu@intel.com> commit be42d03 Author: xinhe <xin3.he@intel.com> Date: Tue Jul 16 09:48:48 2024 +0800 implement TorchBaseConfig (#1911) Signed-off-by: xin3he <xin3.he@intel.com> commit 7a4715c Author: Kaihui-intel <kaihui.tang@intel.com> Date: Mon Jul 15 14:59:03 2024 +0800 Support PT2E save and load (#1918) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 34f0a9f Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Mon Jul 15 09:10:14 2024 +0800 Add `save`/`load` support for HQQ (#1913) Signed-off-by: yiliu30 <yi4.liu@intel.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> commit d320460 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Fri Jul 12 14:48:12 2024 +0800 remove 1x docs (#1900) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 6c547f7 Author: chen, suyue <suyue.chen@intel.com> Date: Fri Jul 12 14:42:04 2024 +0800 fix CI docker container clean up issue (#1917) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 1703658 Author: chen, suyue <suyue.chen@intel.com> Date: Fri Jul 12 11:14:48 2024 +0800 Remove deprecated modules (#1872) Signed-off-by: chensuyue <suyue.chen@intel.com> commit f698c96 Author: chen, suyue <suyue.chen@intel.com> Date: Thu Jul 11 18:00:28 2024 +0800 update Gaudi CI baseline artifacts name (#1912) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 4a45093 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Thu Jul 11 17:47:47 2024 +0800 Add export support for TEQ (#1910) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 16a7b11 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Thu Jul 11 17:13:24 2024 +0800 Get default config based on the auto-detect CPU type (#1904) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 2fc7255 Author: xinhe <xin3.he@intel.com> Date: Thu Jul 11 13:22:52 2024 +0800 implement `incbench` command for ease-of-use benchmark (#1884) implement incbench command as entrypoint for ease-of-use benchmark automatically check numa/socket info and dump it with table for ease-of-understand supports both Linux and Windows platform add benchmark documents dump benchmark summary add benchmark UTs incbench main.py: run 1 instance on NUMA:0. incbench --num_i 2 main.py: run 2 instances on NUMA:0. incbench --num_c 2 main.py: run multi-instances with 2 cores per instance on NUMA:0. incbench -C 24-47 main.py: run 1 instance on COREs:24-47. incbench -C 24-47 --num_c 4 main.py: run multi-instances with 4 COREs per instance on COREs:24-47. --------- Signed-off-by: xin3he <xin3.he@intel.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> commit de8577e Author: chen, suyue <suyue.chen@intel.com> Date: Wed Jul 10 17:21:45 2024 +0800 bump version into 3.0 (#1908) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 01f16c4 Author: chen, suyue <suyue.chen@intel.com> Date: Wed Jul 10 17:19:57 2024 +0800 support habana fp8 UT test in CI (#1909) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 28578b9 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Wed Jul 10 13:19:27 2024 +0800 Add docstring for `common` module (#1905) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 5fde50f Author: Wang, Chang <chang1.wang@intel.com> Date: Wed Jul 10 10:34:46 2024 +0800 update fp4_e2m1 mapping list (#1906) * update fp4_e2m1 mapping list * Update utility.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> commit 3fe2fd9 Author: xinhe <xin3.he@intel.com> Date: Tue Jul 9 15:01:25 2024 +0800 fix bf16 symbolic_trace bug (#1892) Description: fix bf16 symbolic_trace bug, - cause abnormal recursive calling. - missing necessary attributes - By moving BF16 fallback ahead of quantization and removing bf16_symbolic_trace, we fix it. --------- Signed-off-by: xin3he <xin3.he@intel.com> Co-authored-by: Sun, Xuehao <xuehao.sun@intel.com> commit e080e06 Author: Sun, Xuehao <xuehao.sun@intel.com> Date: Tue Jul 9 11:04:30 2024 +0800 remove neural insight CI (#1903) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> commit f28fcee Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Fri Jul 5 15:47:37 2024 +0800 Remove 1x API (#1865) Signed-off-by: yiliu30 <yi4.liu@intel.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> commit 1386ac5 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Thu Jul 4 12:18:03 2024 +0800 Port auto-detect absorb layers for TEQ (#1895) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 856118e Author: Wang, Chang <chang1.wang@intel.com> Date: Wed Jul 3 13:50:00 2024 +0800 remove import pdb (#1897) Signed-off-by: changwangss <chang1.wang@intel.com> commit f75ff40 Author: xinhe <xin3.he@intel.com> Date: Wed Jul 3 13:07:48 2024 +0800 support auto_host2device on RTN and GPTQ(#1894) Signed-off-by: He, Xin3 <xin3.he@intel.com> commit b9e73f5 Author: chen, suyue <suyue.chen@intel.com> Date: Wed Jul 3 11:10:45 2024 +0800 tmp fix nas deps issue (#1896) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 63b2912 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Tue Jul 2 14:46:02 2024 +0800 Refine HQQ UTs (#1888) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 5592acc Author: zehao-intel <zehao.huang@intel.com> Date: Tue Jul 2 14:18:51 2024 +0800 Remove Gelu Fusion for TF Newapi (#1886) Signed-off-by: zehao-intel <zehao.huang@intel.com> commit 4372a76 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Fri Jun 28 14:55:10 2024 +0800 Fix sql injection for Neural Solution gRPC (#1879) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 4ae2e87 Author: xinhe <xin3.he@intel.com> Date: Thu Jun 27 09:56:52 2024 +0800 support quant_lm_head arg in all WOQ configs (#1881) Signed-off-by: xin3he <xin3.he@intel.com> commit cc763f5 Author: Dina Suehiro Jones <dina.s.jones@intel.com> Date: Wed Jun 26 18:29:06 2024 -0700 Update the Gaudi container example in the README (#1885) commit 1f58f02 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Thu Jun 20 22:03:45 2024 +0800 Add `set_local` support for static quant with pt2e (#1870) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 0341295 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Wed Jun 19 09:40:11 2024 +0800 rm cov (#1878) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 503d9ef Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Jun 18 17:12:12 2024 +0800 Add op statistics dump for woq (#1876) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 5a0374e Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Tue Jun 18 16:21:05 2024 +0800 Enhance autotune to return the best `q_model` directly (#1875) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 90fb431 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Jun 18 16:06:04 2024 +0800 fix layer match (#1873) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Co-authored-by: Sun, Xuehao <xuehao.sun@intel.com> commit f4eb660 Author: Sun, Xuehao <xuehao.sun@intel.com> Date: Mon Jun 17 16:12:06 2024 +0800 Limit numpy versions (#1874) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> commit 2928d85 Author: chen, suyue <suyue.chen@intel.com> Date: Fri Jun 14 21:51:13 2024 +0800 update v2.6 release readme (#1871) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 48c5e3a Author: Kaihui-intel <kaihui.tang@intel.com> Date: Fri Jun 14 21:10:14 2024 +0800 Modify WOQ examples structure (#1866) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> commit 498af74 Author: Sun, Xuehao <xuehao.sun@intel.com> Date: Fri Jun 14 21:09:36 2024 +0800 Update SQ/WOQ status (#1869) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> commit b401b02 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Fri Jun 14 17:48:03 2024 +0800 Add PT2E cv&llm example (#1853) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit e470f6c Author: xinhe <xin3.he@intel.com> Date: Fri Jun 14 17:34:26 2024 +0800 [3x] add recommendation examples (#1844) Signed-off-by: xin3he <xin3.he@intel.com> commit a141512 Author: zehao-intel <zehao.huang@intel.com> Date: Fri Jun 14 14:56:30 2024 +0800 Improve UT Branch Coverage for TF 3x (#1867) Signed-off-by: zehao-intel <zehao.huang@intel.com> commit b99a79d Author: Zixuan Cheng <110808245+violetch24@users.noreply.github.com> Date: Fri Jun 14 14:10:49 2024 +0800 modify 3.x ipex example structure (#1858) * modify 3.x ipex example structure Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> * add json path Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> * fix for sq Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> * minor fix Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> * Update run_clm_no_trainer.py * Update run_clm_no_trainer.py * Update run_clm_no_trainer.py * minor fix Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> * remove old files Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> * fix act_algo Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> --------- Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> Co-authored-by: xinhe <xin3.he@intel.com> commit 922b247 Author: zehao-intel <zehao.huang@intel.com> Date: Fri Jun 14 12:33:39 2024 +0800 Add TF 3x Examples (#1839) Signed-off-by: zehao-intel <zehao.huang@intel.com> commit 70a1d50 Author: Zixuan Cheng <110808245+violetch24@users.noreply.github.com> Date: Fri Jun 14 10:17:33 2024 +0800 fix 3x ipex static quant regression (#1864) Description fix 3x ipex static quant regression cannot fallback with op type name ('linear') dump wrong op stats (no 'Linear&relu' op type) --------- Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> commit 4e45f8f Author: zehao-intel <zehao.huang@intel.com> Date: Fri Jun 14 10:04:11 2024 +0800 Improve UT Coverage for TF 3x (#1852) Signed-off-by: zehao-intel <zehao.huang@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> commit 794b276 Author: xinhe <xin3.he@intel.com> Date: Thu Jun 13 18:02:04 2024 +0800 migrate export to 2x and 3x from deprecated (#1845) Signed-off-by: xin3he <xin3.he@intel.com> commit 0eced14 Author: yuwenzho <yuwen.zhou@intel.com> Date: Wed Jun 12 18:49:17 2024 -0700 Enhance INC WOQ model loading & support Huggingface WOQ model loading (#1826) Signed-off-by: yuwenzho <yuwen.zhou@intel.com> commit 6733dab Author: Wang, Mengni <mengni.wang@intel.com> Date: Wed Jun 12 17:08:31 2024 +0800 update mx script (#1838) Signed-off-by: Mengni Wang <mengni.wang@intel.com> commit a0dee94 Author: Wang, Chang <chang1.wang@intel.com> Date: Wed Jun 12 15:01:25 2024 +0800 Remove export_compressed_model in AWQConfig (#1831) commit 2c3556d Author: Huang, Tai <tai.huang@intel.com> Date: Wed Jun 12 14:46:14 2024 +0800 Add 3x architecture diagram (#1849) Signed-off-by: Huang, Tai <tai.huang@intel.com> commit 0e2cade Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed Jun 12 14:20:06 2024 +0800 Bump braces from 3.0.2 to 3.0.3 in /neural_insights/gui (#1862) Signed-off-by: dependabot[bot] <support@github.com> commit 5b5579b Author: Kaihui-intel <kaihui.tang@intel.com> Date: Wed Jun 12 14:12:00 2024 +0800 Fix Neural Solution security issue (#1856) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit e9cb48c Author: xinhe <xin3.he@intel.com> Date: Wed Jun 12 11:19:47 2024 +0800 improve UT coverage of PT Utils and Quantization (#1842) * update UTs --------- Signed-off-by: xin3he <xin3.he@intel.com> Signed-off-by: xinhe3 <xinhe3@habana.ai> commit 6b27383 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Wed Jun 12 11:11:50 2024 +0800 Fix config expansion with empty options (#1861) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 25c71aa Author: WenjiaoYue <wenjiao.yue@intel.com> Date: Tue Jun 11 17:54:31 2024 +0800 Delete the static resources of the JupyterLab extension after packaging (#1860) Signed-off-by: Yue, Wenjiao <wenjiao.yue@intel.com> commit 455f1e1 Author: Wang, Mengni <mengni.wang@intel.com> Date: Tue Jun 11 15:28:40 2024 +0800 Add UT and remove unused code for torch MX quant (#1854) * Add UT and remove unused code for torch MX quant --------- Signed-off-by: Mengni Wang <mengni.wang@intel.com> Signed-off-by: xinhe3 <xinhe3@habana.ai>

Merged from INC public master branch Squashed commit of the following: commit 27f3e2657b2667e8bca8fb9c02a50d55f404a7e6 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Oct 22 11:51:37 2024 +0800 Adapt autoround format (#2038) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 7775768245ec8beea3210910b166e95e2e730586 Author: Sun, Xuehao <xuehao.sun@intel.com> Date: Sun Oct 20 19:41:36 2024 +0800 remove autoround limit (#2036) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> commit 795aeb5bb6950a357da5e339ed97277bc73b4c5c Author: WeiweiZhang1 <weiwei1.zhang@intel.com> Date: Fri Oct 18 17:39:10 2024 +0800 Add vlm examples, bugfix (#2012) * add VLM examples Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * bugfix, add utils Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix docstring issues Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * bugfix Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * refine examples Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * fix scan issue Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * refine shell Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * refine scripts & requirements Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * typofix Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * refine docs Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * set attn_implementation for Phi3-vision Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * refine phi3 example Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix code coverage Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update config Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> * refine shells, docs and example. enable qwen2-vl quantization Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix ci Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * fix EOF error Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> * update qwen dir Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> * refine shell, add llama3.2 inference to doc Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * bugfix Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * bugfix Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * bugfix Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * refine eval shell Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * fix eval device issue Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> * refine eval dtype Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> --------- Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sun, Xuehao <xuehao.sun@intel.com> commit b5f3eb7ea604fc1bd235cce15dda87569b70390e Author: xinhe <xin3.he@intel.com> Date: Fri Oct 18 15:03:53 2024 +0800 add back missing image (#2035) Signed-off-by: xin3he <xin3.he@intel.com> commit 45b29d46a2e958b103c6f8a5539fead25809a89a Author: Huang, Tai <tai.huang@intel.com> Date: Thu Oct 17 15:23:26 2024 +0800 fix broken link to FP8 example (#2034) Signed-off-by: Huang, Tai <tai.huang@intel.com> commit 01bf4b2b3a0f12434b5f44f07a9c26abf96fb5f0 Author: Huang, Tai <tai.huang@intel.com> Date: Thu Oct 17 15:22:23 2024 +0800 update gaudi version mapping table for v3.1 (#2030) Signed-off-by: Huang, Tai <tai.huang@intel.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> commit 5fb21847e12acc51ed4f197eb86b066e6578934b Author: xinhe <xin3.he@intel.com> Date: Thu Oct 17 15:21:18 2024 +0800 Cherry pick Habana software 1.18.0 update (#2025) Signed-off-by: xinhe3 <xinhe3@habana.ai> Signed-off-by: Yi Liu <yiliu4@habana.ai> Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: yan tomsinsky <ytomsinsky@habana.ai> Co-authored-by: Uri Livne <ulivne@habana.ai> Co-authored-by: Dudi Lester <dlester@habana.ai> Co-authored-by: Danny <dsemiat@habana.ai> Co-authored-by: Tomer Gafni <tgafni@habana.ai> Co-authored-by: Eran Geva <egeva@habana.ai> Co-authored-by: Daniel Ohayon <danielohayon444@gmail.com> Co-authored-by: Roi Tiefenbrunn <rtiefenbrunn@habana.ai> Co-authored-by: Kamil Felskowski <kfelskowskix@habana.ai> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> commit d6149aac01c8142f6f5ffc18c03433c82f44150c Author: Yi Liu <yi4.liu@intel.com> Date: Wed Oct 16 14:02:29 2024 +0800 Update the PT2E CV example (#2032) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 08ec90866f9cbd770bed3d93c35aaaf0087d4fe9 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Wed Oct 16 09:20:33 2024 +0800 Remove itrex dependency for 2x example (#2024) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> commit d9377b826d24b2e1c206632bc40f39aab02b3d43 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Oct 15 15:28:37 2024 +0800 Support generation search for transformers examples (#2029) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 61f1e393b6374703b6516fe9406bafb0cc088009 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Fri Oct 11 17:07:14 2024 +0800 Support quant procedure on XPU (#2026) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit dfa6aabab3d280085fb166822b7d849a2dc9b36e Author: Sun, Xuehao <xuehao.sun@intel.com> Date: Fri Oct 11 16:36:42 2024 +0800 remove ITREX unit test CI (#2021) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> commit 2bb257e71353d87414ff7e410ca35bce5cc3dbc7 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Thu Oct 10 19:27:11 2024 +0800 Add woq examples (#1982) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> Co-authored-by: Sun, Xuehao <xuehao.sun@intel.com> commit 586eb88fc7b4bfe87bf8fed9f77951623e48bd88 Author: Huang, Tai <tai.huang@intel.com> Date: Wed Oct 9 09:22:39 2024 +0800 add transformers-like api link in readme (#2022) Signed-off-by: Huang, Tai <tai.huang@intel.com> commit 4e9c7641589c5f3eec20972f9a16022b7eb7e941 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Oct 8 13:13:45 2024 +0800 Remove itrex dependency for 3x example (#2016) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Co-authored-by: Sun, Xuehao <xuehao.sun@intel.com> commit a0066d4e55f3dc03a2e0b992286d8806509cf368 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Mon Sep 30 18:17:32 2024 +0800 Fix transformers rtn layer-wise quant (#2008) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> commit 802a5af3c3093941b665e6e9a92c706da1aeccdc Author: Huang, Tai <tai.huang@intel.com> Date: Mon Sep 30 17:02:52 2024 +0800 add autoround EMNLP24 to pub list (#2014) Signed-off-by: Huang, Tai <tai.huang@intel.com> commit 44795a1ae93f3676a595063cf0e6f680c41989b2 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Mon Sep 30 16:55:22 2024 +0800 Adapt transformers 4.45.1 (#2019) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Co-authored-by: changwangss <chang1.wang@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> commit d4662ad47a4af11a9ed8b45429aff007d8c1b605 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Mon Sep 30 15:52:17 2024 +0800 Add transformers-like api doc (#2018) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 72398b69334d90cdd7664ac12a025cd36695b55c Author: Wang, Chang <chang1.wang@intel.com> Date: Fri Sep 27 15:11:04 2024 +0800 fix xpu device set weight and bias (#2010) Signed-off-by: changwangss <chang1.wang@intel.com> Co-authored-by: Sun, Xuehao <xuehao.sun@intel.com> commit 9d27743705af66a66aafae7fb1d19e2ffad6b2a2 Author: Sun, Xuehao <xuehao.sun@intel.com> Date: Fri Sep 27 14:17:24 2024 +0800 Update model accuracy (#2006) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> commit 7bbc47373033a46fca92ce5ec312d4e051092eee Author: xinhe <xin3.he@intel.com> Date: Fri Sep 27 11:47:00 2024 +0800 add pad_to_buckets in evaluation for hpu performance (#2011) * add pad_to_buckets in evaluation for hpu performance --------- Signed-off-by: xin3he <xin3.he@intel.com> commit b6b7d7c3c415d67976e054ab5ad5be6b5d5b460d Author: Kaihui-intel <kaihui.tang@intel.com> Date: Thu Sep 26 17:21:54 2024 +0800 Update auto_round requirements for transformers example (#2013) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit ee600ba79c5197908ca119446b377be59a5a19e7 Author: Wang, Chang <chang1.wang@intel.com> Date: Fri Sep 20 13:54:06 2024 +0800 add repack_awq_to_optimum_format function (#1998) Signed-off-by: changwangss <chang1.wang@intel.com> commit 4ee6861d666a15c26bb796547d446879e17e6b11 Author: Sun, Xuehao <xuehao.sun@intel.com> Date: Thu Sep 19 22:27:05 2024 +0800 remove accelerate version in unit test (#2007) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> commit 24458114c0765e177b3f4dfbb73d7cfda6b196ab Author: WeiweiZhang1 <weiwei1.zhang@intel.com> Date: Sat Sep 14 18:13:30 2024 +0800 enable auto_round format export (#2002) Signed-off-by: Zhang, Weiwei1 <weiwei1.zhang@intel.com> commit 906333abd41c8be8a6f097da42c1931ea3bb37d5 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Sat Sep 14 16:17:46 2024 +0800 Replace FORCE_DEVICE with INC_TARGET_DEVICE [transformers] (#2005) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 443d00779acac739c3a185f384b78236eaac9643 Author: xinhe <xin3.he@intel.com> Date: Fri Sep 13 21:35:32 2024 +0800 add INC_FORCE_DEVICE introduction (#1988) * add INC_FORCE_DEVICE introduction Signed-off-by: xin3he <xin3.he@intel.com> * Update PyTorch.md * Update PyTorch.md * Update docs/source/3x/PyTorch.md Co-authored-by: Yi Liu <yi4.liu@intel.com> * rename to INC_TARGET_DEVICE Signed-off-by: xin3he <xin3.he@intel.com> --------- Signed-off-by: xin3he <xin3.he@intel.com> Co-authored-by: Yi Liu <yi4.liu@intel.com> commit 5de9a4f56c4cf3901b8ca75d56677255c4e8c833 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Fri Sep 13 20:48:22 2024 +0800 Support transformers-like api for woq quantization (#1987) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Wang, Chang <chang1.wang@intel.com> commit 9c39b429baa16591d4fe883e1a7279761f7f86a5 Author: chen, suyue <suyue.chen@intel.com> Date: Thu Sep 12 14:34:49 2024 +0800 update docker image prune rules (#2003) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 09d4f2d6fb1a6aa91874a0b87a967067800462cb Author: Huang, Tai <tai.huang@intel.com> Date: Mon Sep 9 09:24:35 2024 +0800 Add recent publications (#1995) * add recent publications Signed-off-by: Huang, Tai <tai.huang@intel.com> * update total count Signed-off-by: Huang, Tai <tai.huang@intel.com> --------- Signed-off-by: Huang, Tai <tai.huang@intel.com> commit 399cd44a35583bd96701bee58107c6969be0201e Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Sep 3 16:37:09 2024 +0800 Remove the save of gptq config (#1993) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 05272c48591567d0a1d36fe6cfe5c697d836887b Author: Yi Liu <yi4.liu@intel.com> Date: Tue Sep 3 10:21:51 2024 +0800 add per_channel_minmax (#1990) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 82d8c06c6b535d8db21a6c848a2e374b3b16288e Author: chen, suyue <suyue.chen@intel.com> Date: Fri Aug 30 21:21:00 2024 +0800 update 3x pt binary build (#1992) Signed-off-by: chensuyue <suyue.chen@intel.com> commit e9f06af240065fd48066d32ec4d856c0b7a62f14 Author: Huang, Tai <tai.huang@intel.com> Date: Fri Aug 30 17:49:48 2024 +0800 Update installation_guide.md (#1989) Correct typo in installation doc commit 093c9669692c8b9263cfbc16d7299da4170c8201 Author: Wang, Chang <chang1.wang@intel.com> Date: Fri Aug 30 17:45:54 2024 +0800 add quantize, save, load function for transformers-like api (#1986) Signed-off-by: changwangss <chang1.wang@intel.com> commit 4dd49a43dec86aea581db4f29c7ca36b0baf1f7c Author: xinhe <xin3.he@intel.com> Date: Thu Aug 29 17:23:18 2024 +0800 add hasattr check for torch fp8 dtype (#1985) Signed-off-by: xin3he <xin3.he@intel.com> commit f2c454f88c0ffbb4d30d66eedaa6fc56ad47f804 Author: chen, suyue <suyue.chen@intel.com> Date: Thu Aug 29 13:45:39 2024 +0800 update installation and ci test for 3x api (#1991) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 7ba9fdcb24a8ea1c1efc27844f39d0c128f83517 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Mon Aug 19 14:50:50 2024 +0800 support gptq `true_sequential` and `quant_lm_head` (#1977) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 68b1f8b734bff723dd4962da08ecdc0d22c5faab Author: Sun, Xuehao <xuehao.sun@intel.com> Date: Fri Aug 16 09:43:46 2024 +0800 Fix UT env and upgrade torch to 2.4.0 (#1978) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> commit f9dfd54272348483037cc70802cd85a085fec39c Author: Yi Liu <yi4.liu@intel.com> Date: Thu Aug 15 14:13:26 2024 +0800 Skip some tests for torch 2.4 (#1981) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 46d9192659f1c0dcf488e2e69f0f7dd7bd0b2f2e Author: xinhe <xin3.he@intel.com> Date: Thu Aug 15 09:57:22 2024 +0800 update readme for fp8 (#1979) Signed-off-by: xinhe3 <xinhe3@habana.ai> commit 842b7159fafa09300bc0e745c802910a2d60502e Author: chen, suyue <suyue.chen@intel.com> Date: Tue Aug 13 12:09:25 2024 +0800 bump main version into v3.1 (#1974) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 3845cdc4837e7f0ede12b9de0906b7d01899fc00 Author: Neo Zhang Jianyu <jianyu.zhang@intel.com> Date: Tue Aug 13 12:09:09 2024 +0800 fix online doc search issue (#1975) Co-authored-by: ZhangJianyu <zhang.jianyu@outlook.com> commit 7056720df96f17c706522bc6b0530df534d22ee7 Author: chen, suyue <suyue.chen@intel.com> Date: Sun Aug 11 20:58:34 2024 +0800 update main page (#1973) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 95197d1697e19323b124c2a32bdef7425d4d1c3e Author: xinhe <xin3.he@intel.com> Date: Sat Aug 10 23:28:43 2024 +0800 Cherry pick v1.17.0 (#1964) * [SW-184941] INC CI, CD and Promotion Change-Id: I60c420f9776e1bdab7bb9e02e5bcbdb6891bfe52 * [SW-183320]updated setup.py Change-Id: I592af89486cb1d9e0b5197521c428920197a9103 * [SW-177474] add HQT FP8 porting code Change-Id: I4676f13a5ed43c444f2ec68675cc41335e7234dd Signed-off-by: Zhou Yuwen <zyuwen@habana.ai> * [SW-189361] Fix white list extend Change-Id: Ic2021c248798fce37710d28014a6d59259c868a3 * [SW-191317] Raise exception according to hqt config object Change-Id: I06ba8fa912c811c88912987c11e5c12ef328348a * [SW-184714] Port HQT code into INC HQT lib content was copied as is under fp8_quant Tests were copied to 3.x torch location Change-Id: Iec6e1fa7ac4bf1df1c95b429524c40e32bc13ac9 * [SW-184714] Add internal folder to fp8 quant This is a folder used for experiments, not to be used by users Change-Id: I9e221ae582794e304e95392c0f37638f7bce69bc * [SW-177468] Removed unused code + cleanup Change-Id: I4d27c067e87c1a30eb1da9df16a16c46d092c638 * Fix errors in regression_detection Change-Id: Iee5318bd5593ba349812516eb5641958ece3c438 * [SW-187731] Save orig module as member of patched module This allows direct usage of the original module methods, which solves torch compile issue Change-Id: I464d8bd1bacdfc3cd1f128a67114e1e43f092632 * [SW-190899] Install packages according to configuration Change-Id: I570b490658f5d2c5399ba1db93f8f52f56449525 * [SW-184689] use finalize_calibration intrenaly for one step flow Change-Id: Ie0b8b426c951cf57ed7e6e678c86813fb2d05c89 * [SW-191945] align requirement_pt.txt in gerrit INC with Github INC Change-Id: If5c0dbf21bf989af37a8e29246e4f8760cd215ef Signed-off-by: xinhe3 <xinhe3@hababa.ai> * [SW-192358] Remove HQT reference in INC Change-Id: Ic25f9323486596fa2dc6d909cd568a37ab84dd5e * [SW-191415] update fp8 maxAbs observer using torch.copy_ Change-Id: I3923c832f9a8a2b14e392f3f4719d233a457702f * [SW-184943] Enhance INC WOQ model loading - Support loading huggingface WOQ model - Abstract WeightOnlyLinear base class. Add INCWeightOnlyLinear and HPUWeighOnlyLinear subclasses - Load woq linear weight module by module - Save hpu format tensor to reuse it once load it again Change-Id: I679a42759b49e1f45f52bbb0bdae8580a23d0bcf * [SW-190303] Implement HPUWeightOnlyLinear class in INC Change-Id: Ie05c8787e708e2c3559dce24ef0758d6c498ac41 * [SW-192809] fix json_file bug when instantiating FP8Config class Change-Id: I4a715d0a706efe20ccdb49033755cabbc729ccdc Signed-off-by: Zhou Yuwen <zyuwen@habana.ai> * [SW-192931] align setup.py with github INC and remove fp8_convert Change-Id: Ibbc157646cfcfad64b323ecfd96b9bbda5ba9e2f Signed-off-by: xinhe3 <xinhe3@hababa.ai> * [SW-192917] Update all HQT logic files with pre-commit check Change-Id: I119dc8578cb10932fd1a8a674a8bdbf61f978e42 Signed-off-by: xinhe3 <xinhe3@hababa.ai> * update docstring Signed-off-by: yuwenzho <yuwen.zhou@intel.com> * add fp8 example and document (#1639) Signed-off-by: xinhe3 <xinhe3@hababa.ai> * Update settings to be compatible with gerrit * enhance ut Signed-off-by: yuwenzho <yuwen.zhou@intel.com> * move fp8 sample to helloworld folder Signed-off-by: yuwenzho <yuwen.zhou@intel.com> * update torch version of habana docker Signed-off-by: xinhe3 <xinhe3@hababa.ai> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update readme demo Signed-off-by: xinhe3 <xinhe3@hababa.ai> * update WeightOnlyLinear to INCWeightOnlyLinear Signed-off-by: xinhe3 <xinhe3@hababa.ai> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add docstring for FP8Config Signed-off-by: xinhe3 <xinhe3@hababa.ai> * fix pylint Signed-off-by: xinhe3 <xinhe3@hababa.ai> * update fp8 test scripts Signed-off-by: chensuyue <suyue.chen@intel.com> * delete deps Signed-off-by: chensuyue <suyue.chen@intel.com> * update container into v1.17.0 Signed-off-by: chensuyue <suyue.chen@intel.com> * update docker version Signed-off-by: xinhe3 <xinhe3@hababa.ai> * update pt ut Signed-off-by: chensuyue <suyue.chen@intel.com> * add lib path Signed-off-by: chensuyue <suyue.chen@intel.com> * fix dir issue Signed-off-by: xinhe3 <xinhe3@hababa.ai> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update fp8 test scope Signed-off-by: chensuyue <suyue.chen@intel.com> * fix typo Signed-off-by: xinhe3 <xinhe3@hababa.ai> * update fp8 test scope Signed-off-by: chensuyue <suyue.chen@intel.com> * update pre-commit-ci Signed-off-by: chensuyue <suyue.chen@intel.com> * work around for hpu Signed-off-by: xinhe3 <xinhe3@hababa.ai> * fix UT Signed-off-by: xinhe3 <xinhe3@hababa.ai> * fix parameter Signed-off-by: chensuyue <suyue.chen@intel.com> * omit some test Signed-off-by: chensuyue <suyue.chen@intel.com> * update main page example to llm loading Signed-off-by: xinhe3 <xinhe3@hababa.ai> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix autotune Signed-off-by: xinhe3 <xinhe3@hababa.ai> --------- Signed-off-by: Zhou Yuwen <zyuwen@habana.ai> Signed-off-by: xinhe3 <xinhe3@hababa.ai> Signed-off-by: yuwenzho <yuwen.zhou@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: yan tomsinsky <ytomsinsky@habana.ai> Co-authored-by: Ron Ben Moshe <rbenmoshe@habana.ai> Co-authored-by: Uri Livne <ulivne@habana.ai> Co-authored-by: Danny Semiat <dsemiat@habana.ai> Co-authored-by: smarkovichgolan <smarkovich@habana.ai> Co-authored-by: Dudi Lester <dlester@habana.ai> commit de0fa21cd9d6291b521281b2b5fc8f6519cb84ae Author: Huang, Tai <tai.huang@intel.com> Date: Fri Aug 9 22:32:37 2024 +0800 Fix broken link in docs (#1969) Signed-off-by: Huang, Tai <tai.huang@intel.com> commit 385da7c7ed018a66fcba6e28658d1a5eea2e52e4 Author: Sun, Xuehao <xuehao.sun@intel.com> Date: Fri Aug 9 21:53:51 2024 +0800 Add 3.x readme (#1971) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> commit acd8f4f182eaccf03b221f765ec0ddb451be3415 Author: Huang, Tai <tai.huang@intel.com> Date: Fri Aug 9 15:24:14 2024 +0800 Add version mapping between INC and Gaudi SW Stack (#1967) Signed-off-by: Huang, Tai <tai.huang@intel.com> commit 74a4641390b4d8c11dc66ff8ef40df92c298b996 Author: Sun, Xuehao <xuehao.sun@intel.com> Date: Fri Aug 9 10:23:59 2024 +0800 remove unnecessary CI (#1966) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> commit b99abae5d937380cf9df80c9050fce18bddfb72d Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Aug 6 16:02:03 2024 +0800 Fix `opt_125m_woq_gptq_int4_dq_ggml` issue (#1965) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit b35ff8f0044bdf12da87647d0404b62ae5ff7d3d Author: Zixuan Cheng <110808245+violetch24@users.noreply.github.com> Date: Fri Aug 2 09:06:35 2024 +0800 example update for 3.x ipex sq (#1902) Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> commit 000946fce147a02ad6662538e337570c0a56329d Author: Zixuan Cheng <110808245+violetch24@users.noreply.github.com> Date: Thu Aug 1 10:19:32 2024 +0800 add SDXL model example to INC 3.x (#1887) * add SDXL model example to INC 3.x Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> * add evaluation script Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> * add test script Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> * minor fix Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> * Update run_quant.sh * add iter limit Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> * modify test script Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> * update json Signed-off-by: chensuyue <suyue.chen@intel.com> * add requirements Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> * Update run_benchmark.sh * Update sdxl_smooth_quant.py * minor fix Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> --------- Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> Co-authored-by: chensuyue <suyue.chen@intel.com> commit aa42e5edcd0b5196a21ee7bb68a7965125601fea Author: xinhe <xin3.he@intel.com> Date: Wed Jul 31 15:36:06 2024 +0800 replenish docstring (#1955) * replenish docstring Signed-off-by: xin3he <xin3.he@intel.com> * update Quantizer API docstring Signed-off-by: xin3he <xin3.he@intel.com> * Add docstring for auto accelerator (#1956) Signed-off-by: yiliu30 <yi4.liu@intel.com> * temporary remove torch/quantization and add it back after fp8 code is updated. * Update config.py --------- Signed-off-by: xin3he <xin3.he@intel.com> Signed-off-by: yiliu30 <yi4.liu@intel.com> Co-authored-by: Yi Liu <106061964+yiliu30@users.noreply.github.com> commit 81a076d7c59609be666ddddf64a574cacf1a5c36 Author: Neo Zhang Jianyu <jianyu.zhang@intel.com> Date: Wed Jul 31 13:51:33 2024 +0800 fix welcome.html link issue (#1962) Co-authored-by: ZhangJianyu <zhang.jianyu@outlook.com> commit 87f02c15a2f1047a8b4bcb5b7f443a4cecb4dfc7 Author: chen, suyue <suyue.chen@intel.com> Date: Wed Jul 31 10:09:47 2024 +0800 fix docs link (#1959) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 03813e2894871fce7a95fb4ee584aab6c5bb18f7 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed Jul 31 10:09:29 2024 +0800 Bump tensorflow version (#1961) Signed-off-by: dependabot[bot] <support@github.com> commit 3b5dbf681d8e9beb47eb0d1be4c5a58f4018d42a Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Jul 30 17:27:21 2024 +0800 Set low_gpu_mem_usage=False for AutoRound Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 41244d3bc65fd646d2d6d88ca2c6686f2ab65bc6 Author: chen, suyue <suyue.chen@intel.com> Date: Mon Jul 29 23:05:36 2024 +0800 new previous results could not find all raise issues in CI model test (#1958) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 190e6b2be6b31158a1101729bcf621bc93e85531 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Mon Jul 29 19:39:57 2024 +0800 Fix itrex qbits nf4/int8 training core dumped issue (#1954) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> commit 0e724a4d96ca0d6a170281688ca644b37fa340e0 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Mon Jul 29 16:22:13 2024 +0800 Add save/load for pt2e example (#1927) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 50eb6fb6f5924054b38d8ed99e78e0ebdab51f50 Author: chen, suyue <suyue.chen@intel.com> Date: Mon Jul 29 13:40:36 2024 +0800 update 3x torch installation (#1957) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 6e1b1da712d20d9291e5932974bc3167b00dd214 Author: Zixuan Cheng <110808245+violetch24@users.noreply.github.com> Date: Fri Jul 26 15:58:00 2024 +0800 add ipex xpu example to 3x API (#1948) Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> commit 19024b351372ca76934db33b0d230552c13bff39 Author: zehao-intel <zehao.huang@intel.com> Date: Fri Jul 26 14:52:01 2024 +0800 Enable yolov5 Example for TF 3x API (#1943) Signed-off-by: zehao-intel <zehao.huang@intel.com> commit d84a93f7db8eeb69099aa332a4c01a743c9f4090 Author: zehao-intel <zehao.huang@intel.com> Date: Thu Jul 25 14:45:19 2024 +0800 Complement UT of calibration function for TF 3x API (#1945) Signed-off-by: zehao-intel <zehao.huang@intel.com> commit fb8577931c11c3bdc55868e01576b73372d9912b Author: zehao-intel <zehao.huang@intel.com> Date: Thu Jul 25 14:04:25 2024 +0800 Update Examples for TF 3x API (#1901) Signed-off-by: zehao-intel <zehao.huang@intel.com> commit 6b30207d0a3b6d6d497ecf8f6bb5891765d798ba Author: zehao-intel <zehao.huang@intel.com> Date: Thu Jul 25 13:39:06 2024 +0800 Add Docstring for TF 3x API and Torch 3x Mixed Precision (#1944) Signed-off-by: zehao-intel <zehao.huang@intel.com> commit d254d508be9c6b14c474fd643ad448a4e261ca72 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Wed Jul 24 21:50:44 2024 +0800 Update doc for client-usage and LWQ (#1947) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit f253d35a152b8003cfc8738fd3c6db7930149264 Author: Neo Zhang Jianyu <jianyu.zhang@intel.com> Date: Wed Jul 24 17:48:05 2024 +0800 Update publish.yml (#1950) commit 6cda338a042073aba61ba411a6fc563fc8731889 Author: Neo Zhang Jianyu <jianyu.zhang@intel.com> Date: Wed Jul 24 17:31:19 2024 +0800 Update publish.yml (#1949) * Update publish.yml * Update publish.yml commit c80b68afdba7a55b19898b1b9ff3e21d18b57427 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Jul 23 21:26:53 2024 +0800 Update AutoRound commit version (#1941) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 9077b382259e2e56ff5796084a1f4275e4387537 Author: zehao-intel <zehao.huang@intel.com> Date: Tue Jul 23 17:04:37 2024 +0800 Refine Pytorch 3x Mixed Precision Example (#1946) Signed-off-by: zehao-intel <zehao.huang@intel.com> commit efcb2930be6b9d575b1fb8a6e86afdd6a09b5857 Author: Neo Zhang Jianyu <jianyu.zhang@intel.com> Date: Tue Jul 23 10:15:41 2024 +0800 Update for API 3.0 online doc (#1940) Co-authored-by: ZhangJianyu <zhang.jianyu@outlook.com> commit b787940ea2868e1fc8a56a81b94d62d4ea3d8454 Author: Wang, Mengni <mengni.wang@intel.com> Date: Tue Jul 23 10:12:34 2024 +0800 add docstring for mx quant (#1932) Signed-off-by: Mengni Wang <mengni.wang@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: xinhe <xin3.he@intel.com> commit 0c52e1243b78734e95fc348834303bc3c3cfe369 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Jul 23 09:59:17 2024 +0800 Add docstring for WOQ&LayerWise (#1938) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: xinhe <xin3.he@intel.com> commit 08914d6b0e365212fee6016d03dcdc087bd7e441 Author: Huang, Tai <tai.huang@intel.com> Date: Mon Jul 22 11:14:44 2024 +0800 add read permission token (#1942) Signed-off-by: Huang, Tai <tai.huang@intel.com> commit e106dea73471ddecdb1cfc702e90fcb1a5d41452 Author: zehao-intel <zehao.huang@intel.com> Date: Sun Jul 21 21:48:51 2024 +0800 Update Example for Pytorch 3x Mixed Precision (#1882) Signed-off-by: zehao-intel <zehao.huang@intel.com> commit 1ebf6987bd054b926d3cdd5630ae058c8d3a66c2 Author: Zixuan Cheng <110808245+violetch24@users.noreply.github.com> Date: Fri Jul 19 15:56:09 2024 +0800 add docstring for static quant and smooth quant (#1936) * add docstring for static quant and smooth quant Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> * format fix Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> * update scan path Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> * Update utility.py --------- Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> Co-authored-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> commit 296c5d4f1138e5bf33584fb75cea0f6ca5080122 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Fri Jul 19 15:08:05 2024 +0800 Add docstring for PT2E and HQQ (#1937) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 437c8e75706cff1767dcde115e428654766b3f18 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Thu Jul 18 10:00:41 2024 +0800 Fix unused pkgs import (#1931) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit ff3740146a829e845d79266acf233b202843d3fd Author: chen, suyue <suyue.chen@intel.com> Date: Wed Jul 17 23:11:15 2024 +0800 3.X API installation update (#1935) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 6c27c19c3ec7a318455bd12d6e66ad9bb757ab93 Author: zehao-intel <zehao.huang@intel.com> Date: Wed Jul 17 20:35:42 2024 +0800 Support calib_func on TF 3x API (#1934) Signed-off-by: zehao-intel <zehao.huang@intel.com> commit 53e6ee6b75d476bae0382c7d6fb9aa1348c2ab5e Author: Zixuan Cheng <110808245+violetch24@users.noreply.github.com> Date: Wed Jul 17 20:35:03 2024 +0800 Support xpu for ipex static quant (#1916) Signed-off-by: violetch24 <zixuan@aia-sdp-spr-117706.jf.intel.com> commit a1cc618df6efa823bb1834ff2f8be83531f91178 Author: chen, suyue <suyue.chen@intel.com> Date: Wed Jul 17 17:29:49 2024 +0800 remove peft version limit (#1933) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 30583882df76838ea3e4a719e25ddca7bb449b9b Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Wed Jul 17 15:31:38 2024 +0800 Add doc for client usage (#1914) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 29471df05a9e2c36c4ad8083c0b0b285011748d8 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Wed Jul 17 12:12:40 2024 +0800 Enhance load_empty_model import (#1930) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit fd96851f7f8339ec8bfabd602cf494ac6c31d17b Author: Kaihui-intel <kaihui.tang@intel.com> Date: Wed Jul 17 12:05:32 2024 +0800 Integrate AutoRound v0.3 to 2x (#1926) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit bfa27e422dc4760f6a9b1783eee7dae10fe5324f Author: Kaihui-intel <kaihui.tang@intel.com> Date: Wed Jul 17 09:33:13 2024 +0800 Integrate AutoRound v0.3 (#1925) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 5767aed4dbc9a400f65f74bdc9c09209f0a4c145 Author: xinhe <xin3.he@intel.com> Date: Wed Jul 17 09:16:37 2024 +0800 add docstring for torch.quantization and torch.utils (#1928) Signed-off-by: xin3he <xin3.he@intel.com> commit f909bca86cfe7881119b62c4e75ca1f330718764 Author: chen, suyue <suyue.chen@intel.com> Date: Tue Jul 16 21:12:54 2024 +0800 update itrex ut test (#1929) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 649e6b148755bda737009bc323b735b92231c579 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Jul 16 21:05:55 2024 +0800 Support LayerWise for RTN/GPTQ (#1883) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Co-authored-by: chensuyue <suyue.chen@intel.com> commit de43d851a24a5f4290fe148f7d3607cad6d8433f Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Jul 16 17:18:12 2024 +0800 Support absorb dict for awq (#1920) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit e9765955f991e1270e3b65635285f6b6cb8fc38c Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Jul 16 17:17:56 2024 +0800 Support woq Autotune (#1921) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit d56075c7e9f6e3e85385abbff9f1b0d07d157a04 Author: Huang, Tai <tai.huang@intel.com> Date: Tue Jul 16 15:21:06 2024 +0800 fix typo in architecture diagram (#1924) Signed-off-by: Huang, Tai <tai.huang@intel.com> commit 0a542397ac1ea8d6fe2edf04565d3cb673001b2c Author: chen, suyue <suyue.chen@intel.com> Date: Tue Jul 16 15:12:43 2024 +0800 update documentation for 3x API (#1923) Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: xin3he <xin3.he@intel.com> Signed-off-by: yiliu30 <yi4.liu@intel.com> commit be42d033b25c6dd3bcac0ead964699f25f939014 Author: xinhe <xin3.he@intel.com> Date: Tue Jul 16 09:48:48 2024 +0800 implement TorchBaseConfig (#1911) Signed-off-by: xin3he <xin3.he@intel.com> commit 7a4715c1d488441e383b7c999fd1b574a3f6ceda Author: Kaihui-intel <kaihui.tang@intel.com> Date: Mon Jul 15 14:59:03 2024 +0800 Support PT2E save and load (#1918) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 34f0a9f450b385aa3227f7f34e8d0f16460080a9 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Mon Jul 15 09:10:14 2024 +0800 Add `save`/`load` support for HQQ (#1913) Signed-off-by: yiliu30 <yi4.liu@intel.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> commit d3204604aad007f3db67c46dcb0575aa8f5cd584 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Fri Jul 12 14:48:12 2024 +0800 remove 1x docs (#1900) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 6c547f7c4cd71342e28a1b23f827a6aa7aa91bb8 Author: chen, suyue <suyue.chen@intel.com> Date: Fri Jul 12 14:42:04 2024 +0800 fix CI docker container clean up issue (#1917) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 17036587d84d2b42e0e9eb501d175e78d552c063 Author: chen, suyue <suyue.chen@intel.com> Date: Fri Jul 12 11:14:48 2024 +0800 Remove deprecated modules (#1872) Signed-off-by: chensuyue <suyue.chen@intel.com> commit f698c96c817c56292a66aee07b3e1396e074b966 Author: chen, suyue <suyue.chen@intel.com> Date: Thu Jul 11 18:00:28 2024 +0800 update Gaudi CI baseline artifacts name (#1912) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 4a45093c1418f34da2660a54052a2ff5c2b4edff Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Thu Jul 11 17:47:47 2024 +0800 Add export support for TEQ (#1910) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 16a7b11508c008d4d4180a0fe0e31c75b8e5d662 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Thu Jul 11 17:13:24 2024 +0800 Get default config based on the auto-detect CPU type (#1904) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 2fc72555c987dc7bce8476b389720e1a29159a43 Author: xinhe <xin3.he@intel.com> Date: Thu Jul 11 13:22:52 2024 +0800 implement `incbench` command for ease-of-use benchmark (#1884) implement incbench command as entrypoint for ease-of-use benchmark automatically check numa/socket info and dump it with table for ease-of-understand supports both Linux and Windows platform add benchmark documents dump benchmark summary add benchmark UTs incbench main.py: run 1 instance on NUMA:0. incbench --num_i 2 main.py: run 2 instances on NUMA:0. incbench --num_c 2 main.py: run multi-instances with 2 cores per instance on NUMA:0. incbench -C 24-47 main.py: run 1 instance on COREs:24-47. incbench -C 24-47 --num_c 4 main.py: run multi-instances with 4 COREs per instance on COREs:24-47. --------- Signed-off-by: xin3he <xin3.he@intel.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> commit de8577ef5874f85d39c3b08f63c98f22c3ce25c6 Author: chen, suyue <suyue.chen@intel.com> Date: Wed Jul 10 17:21:45 2024 +0800 bump version into 3.0 (#1908) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 01f16c4e816fec9d05d34f9d2bd7e425a59b803c Author: chen, suyue <suyue.chen@intel.com> Date: Wed Jul 10 17:19:57 2024 +0800 support habana fp8 UT test in CI (#1909) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 28578b96bf6217fa2b79699838e5a4af30843de4 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Wed Jul 10 13:19:27 2024 +0800 Add docstring for `common` module (#1905) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 5fde50f2c0476dbc08d59481b742515f5a210de1 Author: Wang, Chang <chang1.wang@intel.com> Date: Wed Jul 10 10:34:46 2024 +0800 update fp4_e2m1 mapping list (#1906) * update fp4_e2m1 mapping list * Update utility.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> commit 3fe2fd9aadda4991552d65fef09a75ba5127b5db Author: xinhe <xin3.he@intel.com> Date: Tue Jul 9 15:01:25 2024 +0800 fix bf16 symbolic_trace bug (#1892) Description: fix bf16 symbolic_trace bug, - cause abnormal recursive calling. - missing necessary attributes - By moving BF16 fallback ahead of quantization and removing bf16_symbolic_trace, we fix it. --------- Signed-off-by: xin3he <xin3.he@intel.com> Co-authored-by: Sun, Xuehao <xuehao.sun@intel.com> commit e080e06d38447d2ab869fe8265a04e464a732057 Author: Sun, Xuehao <xuehao.sun@intel.com> Date: Tue Jul 9 11:04:30 2024 +0800 remove neural insight CI (#1903) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> commit f28fcee6cc7bd6b3e1642157744f38686b1b9a91 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Fri Jul 5 15:47:37 2024 +0800 Remove 1x API (#1865) Signed-off-by: yiliu30 <yi4.liu@intel.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> commit 1386ac5ec7be40608dfac082d2275307b8e4d14e Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Thu Jul 4 12:18:03 2024 +0800 Port auto-detect absorb layers for TEQ (#1895) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 856118e36f7670136c8d83dfbc232010d13d72a6 Author: Wang, Chang <chang1.wang@intel.com> Date: Wed Jul 3 13:50:00 2024 +0800 remove import pdb (#1897) Signed-off-by: changwangss <chang1.wang@intel.com> commit f75ff4082bc7a22d9367d3e91a3ea2c7aaec2bd2 Author: xinhe <xin3.he@intel.com> Date: Wed Jul 3 13:07:48 2024 +0800 support auto_host2device on RTN and GPTQ(#1894) Signed-off-by: He, Xin3 <xin3.he@intel.com> commit b9e73f5cf34f824a9b84d74f725c6157dc6430a2 Author: chen, suyue <suyue.chen@intel.com> Date: Wed Jul 3 11:10:45 2024 +0800 tmp fix nas deps issue (#1896) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 63b29126b7c1958939af388d48e56fcceb85db6f Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Tue Jul 2 14:46:02 2024 +0800 Refine HQQ UTs (#1888) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 5592acc60562b7fccb308af0eaaba9cad53004a5 Author: zehao-intel <zehao.huang@intel.com> Date: Tue Jul 2 14:18:51 2024 +0800 Remove Gelu Fusion for TF Newapi (#1886) Signed-off-by: zehao-intel <zehao.huang@intel.com> commit 4372a762585189accc65196e081a0a7a85f5af9e Author: Kaihui-intel <kaihui.tang@intel.com> Date: Fri Jun 28 14:55:10 2024 +0800 Fix sql injection for Neural Solution gRPC (#1879) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 4ae2e87d2f98eb34c2e523a76ffa6ff77bf767e1 Author: xinhe <xin3.he@intel.com> Date: Thu Jun 27 09:56:52 2024 +0800 support quant_lm_head arg in all WOQ configs (#1881) Signed-off-by: xin3he <xin3.he@intel.com> commit cc763f5134f5f84b3020a8ea1bee409a60d15218 Author: Dina Suehiro Jones <dina.s.jones@intel.com> Date: Wed Jun 26 18:29:06 2024 -0700 Update the Gaudi container example in the README (#1885) commit 1f58f024d812b6c1f7f3430b62e61051599cd1b2 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Thu Jun 20 22:03:45 2024 +0800 Add `set_local` support for static quant with pt2e (#1870) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 0341295de95dce5d5c775fdba78de85e3d3a041d Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Wed Jun 19 09:40:11 2024 +0800 rm cov (#1878) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 503d9ef4136023f1952e397a2ab0f7f476040901 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Jun 18 17:12:12 2024 +0800 Add op statistics dump for woq (#1876) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit 5a0374e7db23cac209af78f1ace9b38d23bebbb0 Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Tue Jun 18 16:21:05 2024 +0800 Enhance autotune to return the best `q_model` directly (#1875) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 90fb43135397a035968b5334eba21931c18a83c0 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Tue Jun 18 16:06:04 2024 +0800 fix layer match (#1873) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Co-authored-by: Sun, Xuehao <xuehao.sun@intel.com> commit f4eb66073fc2c3f13d624c31056d94f2b6735076 Author: Sun, Xuehao <xuehao.sun@intel.com> Date: Mon Jun 17 16:12:06 2024 +0800 Limit numpy versions (#1874) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> commit 2928d856336d3cd2db9068950e054ce4f7c7bbe0 Author: chen, suyue <suyue.chen@intel.com> Date: Fri Jun 14 21:51:13 2024 +0800 update v2.6 release readme (#1871) Signed-off-by: chensuyue <suyue.chen@intel.com> commit 48c5e3a9c22b8f16446a6849d63fed0cdf4a0a7a Author: Kaihui-intel <kaihui.tang@intel.com> Date: Fri Jun 14 21:10:14 2024 +0800 Modify WOQ examples structure (#1866) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> commit 498af747839af0f54e8b1e946ac20fb52b0fbb89 Author: Sun, Xuehao <xuehao.sun@intel.com> Date: Fri Jun 14 21:09:36 2024 +0800 Update SQ/WOQ status (#1869) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> commit b401b02db2cc7d7f4f8412a815fa435e66e330a0 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Fri Jun 14 17:48:03 2024 +0800 Add PT2E cv&llm example (#1853) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit e470f6cdfbbad32fcf17be56903e649a05059780 Author: xinhe <xin3.he@intel.com> Date: Fri Jun 14 17:34:26 2024 +0800 [3x] add recommendation examples (#1844) Signed-off-by: xin3he <xin3.he@intel.com> commit a1415128a8d63af7e1d2798521f11b137eccec81 Author: zehao-intel <zehao.huang@intel.com> Date: Fri Jun 14 14:56:30 2024 +0800 Improve UT Branch Coverage for TF 3x (#1867) Signed-off-by: zehao-intel <zehao.huang@intel.com> commit b99a79d029e8010d234d3b4259994e598bec1a06 Author: Zixuan Cheng <110808245+violetch24@users.noreply.github.com> Date: Fri Jun 14 14:10:49 2024 +0800 modify 3.x ipex example structure (#1858) * modify 3.x ipex example structure Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> * add json path Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> * fix for sq Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> * minor fix Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> * Update run_clm_no_trainer.py * Update run_clm_no_trainer.py * Update run_clm_no_trainer.py * minor fix Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> * remove old files Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> * fix act_algo Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> --------- Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> Co-authored-by: xinhe <xin3.he@intel.com> commit 922b2471e617cc4c56376866e991302d0beb0640 Author: zehao-intel <zehao.huang@intel.com> Date: Fri Jun 14 12:33:39 2024 +0800 Add TF 3x Examples (#1839) Signed-off-by: zehao-intel <zehao.huang@intel.com> commit 70a1d501fdfee16a10e34385bca9f15eba4366b4 Author: Zixuan Cheng <110808245+violetch24@users.noreply.github.com> Date: Fri Jun 14 10:17:33 2024 +0800 fix 3x ipex static quant regression (#1864) Description fix 3x ipex static quant regression cannot fallback with op type name ('linear') dump wrong op stats (no 'Linear&relu' op type) --------- Signed-off-by: Cheng, Zixuan <zixuan.cheng@intel.com> commit 4e45f8f68bf126ca0c9dd655fce03b21a93ec151 Author: zehao-intel <zehao.huang@intel.com> Date: Fri Jun 14 10:04:11 2024 +0800 Improve UT Coverage for TF 3x (#1852) Signed-off-by: zehao-intel <zehao.huang@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> commit 794b2762c0bb2f076973e1fca5fdecd23efec774 Author: xinhe <xin3.he@intel.com> Date: Thu Jun 13 18:02:04 2024 +0800 migrate export to 2x and 3x from deprecated (#1845) Signed-off-by: xin3he <xin3.he@intel.com> commit 0eced1478c6796a5e2dcb254a65bbc96af4d1b8b Author: yuwenzho <yuwen.zhou@intel.com> Date: Wed Jun 12 18:49:17 2024 -0700 Enhance INC WOQ model loading & support Huggingface WOQ model loading (#1826) Signed-off-by: yuwenzho <yuwen.zhou@intel.com> commit 6733dabc4d48a6625e184e4a29a754949f415097 Author: Wang, Mengni <mengni.wang@intel.com> Date: Wed Jun 12 17:08:31 2024 +0800 update mx script (#1838) Signed-off-by: Mengni Wang <mengni.wang@intel.com> commit a0dee94dab0920ba30de049e871b19a72ddb8996 Author: Wang, Chang <chang1.wang@intel.com> Date: Wed Jun 12 15:01:25 2024 +0800 Remove export_compressed_model in AWQConfig (#1831) commit 2c3556d441de2f0963167db71ecdee7353bd76bb Author: Huang, Tai <tai.huang@intel.com> Date: Wed Jun 12 14:46:14 2024 +0800 Add 3x architecture diagram (#1849) Signed-off-by: Huang, Tai <tai.huang@intel.com> commit 0e2cade66f8c3951e6ce7de226421f6700d2ad85 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed Jun 12 14:20:06 2024 +0800 Bump braces from 3.0.2 to 3.0.3 in /neural_insights/gui (#1862) Signed-off-by: dependabot[bot] <support@github.com> commit 5b5579bf953cb24607dc18b3a01ffe1071c3b604 Author: Kaihui-intel <kaihui.tang@intel.com> Date: Wed Jun 12 14:12:00 2024 +0800 Fix Neural Solution security issue (#1856) Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> commit e9cb48c9462fdc671c523f93611b30b41b6cff98 Author: xinhe <xin3.he@intel.com> Date: Wed Jun 12 11:19:47 2024 +0800 improve UT coverage of PT Utils and Quantization (#1842) * update UTs --------- Signed-off-by: xin3he <xin3.he@intel.com> Signed-off-by: xinhe3 <xinhe3@habana.ai> commit 6b2738390dfdab543de1ccd9242fe541c78b6a2e Author: Yi Liu <106061964+yiliu30@users.noreply.github.com> Date: Wed Jun 12 11:11:50 2024 +0800 Fix config expansion with empty options (#1861) Signed-off-by: yiliu30 <yi4.liu@intel.com> commit 25c71aad5a55210d87d371257344f21762e3bb0e Author: WenjiaoYue <wenjiao.yue@intel.com> Date: Tue Jun 11 17:54:31 2024 +0800 Delete the static resources of the JupyterLab extension after packaging (#1860) Signed-off-by: Yue, Wenjiao <wenjiao.yue@intel.com> commit 455f1e1f0f0284e87b46d257b6d126ca76fe1748 Author: Wang, Mengni <mengni.wang@intel.com> Date: Tue Jun 11 15:28:40 2024 +0800 Add UT and remove unused code for torch MX quant (#1854) * Add UT and remove unused code for torch MX quant --------- Signed-off-by: Mengni Wang <mengni.wang@intel.com> Change-Id: I543550ffcc16143d3e612fac2f9ea3a31a1143e1

add save/load

e28fc3e

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

Kaihui-intel requested a review from yiliu30 July 16, 2024 07:04

chensuyue added this to the v3.0 milestone Jul 16, 2024

yiliu30 approved these changes Jul 16, 2024

View reviewed changes

chensuyue reviewed Jul 17, 2024

View reviewed changes

...i/pytorch/nlp/huggingface_models/language-modeling/quantization/weight_only/run_benchmark.sh Outdated Show resolved Hide resolved

Kaihui-intel added 4 commits July 17, 2024 09:37

update run_benchmark.sh

9a72393

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

update run_benchmark

c56a3b9

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

update func name

08700f6

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

fix typo

9f693c0

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

chensuyue reviewed Jul 18, 2024

View reviewed changes

...rch/nlp/huggingface_models/language-modeling/quantization/static_quant/pt2e/run_benchmark.sh Outdated Show resolved Hide resolved

chensuyue reviewed Jul 18, 2024

View reviewed changes

examples/3.x_api/pytorch/cv/static_quant/run_quant.sh Outdated Show resolved Hide resolved

Kaihui-intel and others added 6 commits July 19, 2024 10:54

update run_benchmark.sh

5706780

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

update cv benchmark & script

a7b7d88

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

update llm & set logger level

9aab159

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

update nlp performance

8d5c108

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

7753b40

for more information, see https://pre-commit.ci

Merge branch 'kaihui/pt2e_eg_save_load' of https://github.com/intel/n…

954af25

…eural-compressor into kaihui/pt2e_eg_save_load

Kaihui-intel requested a review from xin3he July 22, 2024 08:38

xin3he approved these changes Jul 22, 2024

View reviewed changes

examples/3.x_api/pytorch/cv/static_quant/main.py Outdated Show resolved Hide resolved

examples/3.x_api/pytorch/cv/static_quant/main.py Outdated Show resolved Hide resolved

Kaihui-intel added 7 commits July 23, 2024 10:08

add dynamic_shapes to cv

a2c8364

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

fix default iters

c4b351f

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

fix default iters

ed8637c

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

fix cv acc

63e0278

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

fix cv fp32 acc

c628331

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

enhance benchmark

61d9325

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

fix nlp benchmark

c760cf1

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

chensuyue merged commit 0e724a4 into master Jul 29, 2024
26 of 27 checks passed

chensuyue deleted the kaihui/pt2e_eg_save_load branch July 29, 2024 08:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add save/load for pt2e example #1927

Add save/load for pt2e example #1927

Kaihui-intel commented Jul 16, 2024

github-actions bot commented Jul 16, 2024

yiliu30 left a comment

Add save/load for pt2e example #1927

Add save/load for pt2e example #1927

Conversation

Kaihui-intel commented Jul 16, 2024

Type of Change

Description

Expected Behavior & Potential Risk

How has this PR been tested?

Dependency Change?

github-actions bot commented Jul 16, 2024

⚡ Required checks status: All passing 🟢

yiliu30 left a comment

Choose a reason for hiding this comment