[BENCHMARK][ADRENO] Adreno Benchmarks with texture #13675
Conversation
Benchmarks for various networks listed below with fp16 and fp32. resnet-18, resnet-34, resnet-50, vgg-16, vgg-19, densenet-121, inception_v3, mobilenetv1, squeezenet_v1.0, squeezenet_v1.1
srkreddy1238
force-pushed
the
texture_benchmarks
branch
from
df0facf to
3da412c
Compare
apps/benchmark/adreno/bench.py
Outdated
| # Test TVM | ||
| make cython3 | ||
|
|
||
| if [ "texture" == $1 ] ; then |
There was a problem hiding this comment.
Are there any reasons to run this script without texture parameter? If no, probably we can remove this argument.
There was a problem hiding this comment.
Have plans to support CLML benchmarks too.
| @@ -727,6 +727,13 @@ def add_subparser( | |||
| "./tests/scripts/task_python_adreno.sh " + os.environ.get("ANDROID_SERIAL", ""), | |||
| ], | |||
| ), | |||
| "benchmarks": ( | |||
There was a problem hiding this comment.
Are there any plans to run these benchmarks in the CI? @driazati probably you know, do we have any opportunities to run different performance tests e.g. once per week to check that no performance regressions were introduced to TVM mainline.
There was a problem hiding this comment.
Ideally this should be separate and as part of nightly instead of every push
There was a problem hiding this comment.
Yes, totally agree. Not necessary to run it for every push. Probably, a new command to @tvm-bot can be added to run performance test for a PR if it is needed.
I think it would be nice, and it is very important to have such tests in the CI.
There was a problem hiding this comment.
There's no concrete plans right now (we'd need to do some designing to figure out how to provision machines for benchmarks and where to store results) but I agree having benchmarks run on some cadence would be good for tracking our progress.
There was a problem hiding this comment.
Script is ready to be used with CI provided we have Snapdragon devices on CI. For now we are using it internally.
| sys.stdout.flush() | ||
|
|
||
|
|
||
| def tune_tasks( |
There was a problem hiding this comment.
Just as an idea. If we want to use such benchmarks in the CI we can use statistic from tophub to measure performance w/o tuning. If someone changes schedules for Adreno then he/she will upload updated statistic to the tophub. In this way, we can avoid performance regressions.
There was a problem hiding this comment.
Already done. tlc-pack/tophub#27
There was a problem hiding this comment.
Tuning option is left for tuning future Adreno variants if required.
| net, target=tvm.target.Target(target, host=target_host), params=params | ||
| ) | ||
| else: | ||
| print("WARNING: Benchmark running with out tuning cache file - ", tune_log) |
There was a problem hiding this comment.
| print("WARNING: Benchmark running with out tuning cache file - ", tune_log) | |
| print("WARNING: Benchmark running without tuning cache file - ", tune_log) |
There was a problem hiding this comment.
Technically we have tuning cache implicate now from tophub. Let me remove this line.
srkreddy1238
force-pushed
the
texture_benchmarks
branch
from
424d951 to
e8b9f23
Compare
* [BENCHMARK][ADRENO] Adreno Benchmarks with texture Benchmarks for various networks listed below with fp16 and fp32. resnet-18, resnet-34, resnet-50, vgg-16, vgg-19, densenet-121, inception_v3, mobilenetv1, squeezenet_v1.0, squeezenet_v1.1 * * lint error Co-authored-by: Siva Rama Krishna Reddy B <sivb@qti.qualcomm.com>
Benchmarks for various networks listed below with fp16 and fp32. resnet-18, resnet-34, resnet-50, vgg-16, vgg-19, densenet-121, inception_v3, mobilenetv1, squeezenet_v1.0, squeezenet_v1.1