update r2.8 LLM table (#3782)

* update r2.8 LLM table

* update html pages accordingly

* append qwen3-30b-a3b

Author: ZailiWang

README.md

@@ -55,6 +55,8 @@ and the phenomenal high-quality reasoning model DeepSeek-R1.
 |Qwen| Qwen/Qwen-7B-Chat | ✅ | ✅ | ✅ | ✅ |
 |Qwen| Qwen/Qwen2-7B | ✅ | ✅ | ✅ | ✅ |
 |Qwen| Qwen/Qwen2.5-7B-Instruct | ✅ | ✅ | ✅ | ✅ |
+|Qwen| Qwen/Qwen3-14B | ✅ | ✅ | ✅ |   |
+|Qwen| Qwen/Qwen3-30B-A3B | ✅ | ✅ | ✅ | ✅ |
 |LLaVA| liuhaotian/llava-v1.5-7b | ✅ | ✅ | ✅ | ✅ |
 |GIT| microsoft/git-base | ✅ | ✅ | ✅ | ✅ |
 |Yuan| IEITYuan/Yuan2-102B-hf | ✅ | ✅ | ✅ |   |
@@ -66,6 +68,7 @@ and the phenomenal high-quality reasoning model DeepSeek-R1.
 |Phi| microsoft/Phi-4-mini-instruct | ✅ | ✅ | ✅ |   |
 |Phi| microsoft/Phi-4-multimodal-instruct | ✅ | ✅ | ✅ |   |
 |Whisper| openai/whisper-large-v2 | ✅ | ✅ | ✅ | ✅ |
+|Whisper| openai/whisper-large-v3 | ✅ | ✅ | ✅ |   |
 |Maira| microsoft/maira-2 | ✅ | ✅ | ✅ | ✅ |
 |Jamba| ai21labs/Jamba-v0.1 | ✅ | ✅ | ✅ | ✅ |
 |DeepSeek| deepseek-ai/DeepSeek-V2.5-1210 | ✅ | ✅ | ✅ | ✅ |

docs/_static/htmls/tbl_deepspeed.html

@@ -188,6 +188,18 @@
       <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
       <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
     </tr>
+    <tr class="row-even">
+      <td><p>Qwen</p></td>
+      <td><p>Qwen/Qwen3-14B</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
+    </tr>
+    <tr class="row-odd">
+      <td><p>Qwen</p></td>
+      <td><p>Qwen/Qwen3-30B-A3B</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
+    </tr>
     <tr class="row-even">
       <td><p>GIT</p></td>
       <td><p>microsoft/git-base</p></td>
@@ -231,12 +243,18 @@
       <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
     </tr>
     <tr class="row-odd">
+      <td><p>Whisper</p></td>
+      <td><p>openai/whisper-large-v3</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
+    </tr>
+    <tr class="row-even">
       <td><p>DeepSeek</p></td>
       <td><p>deepseek-ai/DeepSeek-V2.5-1210</p></td>
       <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
       <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
     </tr>
-    <tr class="row-even">
+    <tr class="row-odd">
       <td><p>DeepSeek</p></td>
       <td><p>meituan/DeepSeek-R1-Channel-INT8</p></td>
       <td><p style="text-align: center; vertical-align: middle;"></p></td>

docs/_static/htmls/tbl_single.html

@@ -274,6 +274,22 @@
       <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
       <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
     </tr>
+    <tr class="row-odd">
+      <td><p>Qwen</p></td>
+      <td><p>Qwen/Qwen3-14B</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
+      <td><p style="text-align: center; vertical-align: middle;"></p></td>
+    </tr>
+    <tr class="row-even">
+      <td><p>Qwen</p></td>
+      <td><p>Qwen/Qwen3-30B-A3B</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
+    </tr>
     <tr class="row-odd">
       <td><p>LLaVA</p></td>
       <td><p>liuhaotian/llava-v1.5-7b</p></td>
@@ -363,30 +379,38 @@
       <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
     </tr>
     <tr class="row-even">
+      <td><p>Whisper</p></td>
+      <td><p>openai/whisper-large-v3</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
+      <td><p style="text-align: center; vertical-align: middle;"></p></td>
+    </tr>
+    <tr class="row-odd">
       <td><p>Maira</p></td>
       <td><p>microsoft/maira-2</p></td>
       <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
       <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
       <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
       <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
     </tr>
-    <tr class="row-odd">
+    <tr class="row-even">
       <td><p>Jamba</p></td>
       <td><p>ai21labs/Jamba-v0.1</p></td>
       <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
       <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
       <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
       <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
     </tr>
-    <tr class="row-even">
+    <tr class="row-odd">
       <td><p>DeepSeek</p></td>
       <td><p>deepseek-ai/DeepSeek-V2.5-1210</p></td>
       <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
       <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
       <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
       <td><p style="text-align: center; vertical-align: middle;">✅</p></td>
     </tr>
-    <tr class="row-odd">
+    <tr class="row-even">
       <td><p>DeepSeek</p></td>
       <td><p>meituan/DeepSeek-R1-Channel-INT8</p></td>
       <td><p style="text-align: center; vertical-align: middle;"></p></td>

examples/cpu/llm/inference/README.md

@@ -41,6 +41,8 @@ and the phenomenal high-quality reasoning model [DeepSeek-R1](#223-deepseek-r1-6
 |Qwen| Qwen/Qwen-7B-Chat | ✅ | ✅ | ✅ | ✅ |
 |Qwen| Qwen/Qwen2-7B | ✅ | ✅ | ✅ | ✅ |
 |Qwen| Qwen/Qwen2.5-7B-Instruct | ✅ | ✅ | ✅ | ✅ |
+|Qwen| Qwen/Qwen3-14B | ✅ | ✅ | ✅ |   |
+|Qwen| Qwen/Qwen3-30B-A3B | ✅ | ✅ | ✅ | ✅ |
 |LLaVA| liuhaotian/llava-v1.5-7b | ✅ | ✅ | ✅ | ✅ |
 |GIT| microsoft/git-base | ✅ | ✅ | ✅ | ✅ |
 |Yuan| IEITYuan/Yuan2-102B-hf | ✅ | ✅ | ✅ |   |
@@ -52,6 +54,7 @@ and the phenomenal high-quality reasoning model [DeepSeek-R1](#223-deepseek-r1-6
 |Phi| microsoft/Phi-4-mini-instruct | ✅ | ✅ | ✅ |   |
 |Phi| microsoft/Phi-4-multimodal-instruct | ✅ | ✅ | ✅ |   |
 |Whisper| openai/whisper-large-v2 | ✅ | ✅ | ✅ | ✅ |
+|Whisper| openai/whisper-large-v3 | ✅ | ✅ | ✅ |   |
 |Maira| microsoft/maira-2 | ✅ | ✅ | ✅ | ✅ |
 |Jamba| ai21labs/Jamba-v0.1 | ✅ | ✅ | ✅ | ✅ |
 |DeepSeek| deepseek-ai/DeepSeek-V2.5-1210 | ✅ | ✅ | ✅ | ✅ |
@@ -91,13 +94,16 @@ and the phenomenal high-quality reasoning model [DeepSeek-R1](#223-deepseek-r1-6
 |Qwen| Qwen/Qwen-7B-Chat | ✅ | ✅ |
 |Qwen| Qwen/Qwen2-7B | ✅ | ✅ |
 |Qwen| Qwen/Qwen2.5-7B-Instruct | ✅ | ✅ |
+|Qwen| Qwen/Qwen3-14B | ✅ | ✅ |
+|Qwen| Qwen/Qwen3-30B-A3B | ✅ | ✅ |
 |GIT| microsoft/git-base | ✅ | ✅ |
 |Phi| microsoft/phi-2 | ✅ | ✅ |
 |Phi| microsoft/Phi-3-mini-4k-instruct | ✅ | ✅ |
 |Phi| microsoft/Phi-3-mini-128k-instruct | ✅ | ✅ |
 |Phi| microsoft/Phi-3-medium-4k-instruct | ✅ | ✅ |
 |Phi| microsoft/Phi-3-medium-128k-instruct | ✅ | ✅ |
 |Whisper| openai/whisper-large-v2 | ✅ | ✅ |
+|Whisper| openai/whisper-large-v3 | ✅ | ✅ |
 |DeepSeek| deepseek-ai/DeepSeek-V2.5-1210 | ✅ | ✅ |
 |DeepSeek| meituan/DeepSeek-R1-Channel-INT8 |   | ✅ |
 
@@ -474,7 +480,7 @@ Please add the `quantization_config` field to the end of the file as below.
 +    "bits": 8,
 +    "group_size": -1
 +  }
- }
+}
 ```
 
 - Use the following command to run the test.
@@ -510,6 +516,22 @@ There are some model-specific requirements to be aware of, as follows:
 
 - For Llava models from remote hub, additional setup is required, i.e., `bash ./tools/prepare_llava.sh`.
 
+- For INT8 quantized Qwen/Qwen3-30B-A3B model, a `quantization_config` field needs to be added in `config.json`.
+
+  ```diff
+     "transformers_version": "4.46.3",
+     "use_cache": true,
+     "v_head_dim": 128,
+  -  "vocab_size": 129280
+  +  "vocab_size": 129280,
+  +  "quantization_config": {
+  +    "quant_method": "int8",
+  +    "bits": 8,
+  +    "group_size": -1
+  +  }
+  }
+  ```
+
 ## 2.3 Instructions for Running Multimodal LLMs
 
 Multimodal LLMs are large language models capable of processing multiple types of inputs,