Commit 3e18dd9
Add an awq_sym lm-eval test (#1702)
SUMMARY:
In response to
#1688, add an
awq_asym lm-eval test config.
TEST PLAN:
```
export CADENCE=weekly
export TEST_DATA_FILE=tests/lmeval/configs/w4a16_awq_asym.yaml
python -m pytest tests/lmeval/test_lmeval.py::TestLMEval::test_lm_eval -v
```
MISC:
As Llama is a gated model, I'm using a local model path instead of
meta-llama/Meta-Llama-3-8B-Instruct during local testing. I'm not sure
if this might affect the test flow / accuracy. It would be great if you
could help me verify it. That's also why I've marked it as WIP.
---------
Signed-off-by: aladerran <aladerran@gmail.com>
Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com>1 parent ad20cf1 commit 3e18dd9
File tree
2 files changed
+23
-0
lines changed- tests
- e2e/vLLM/recipes/WNA16
- lmeval/configs
2 files changed
+23
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
0 commit comments