File tree Expand file tree Collapse file tree 1 file changed +8
-7
lines changed Expand file tree Collapse file tree 1 file changed +8
-7
lines changed Original file line number Diff line number Diff line change @@ -253,15 +253,16 @@ def forward(self,
253253
254254 return attn_output
255255
256+ # .. dropdown::
257+
258+ ###############################################################################
259+ # Utilities
260+ # =========
261+ # In this section, we include a utility to generate semi-realistic data using
262+ # Zipf distribution for sentence lengths. This is used to generate the nested
263+ # query, key and value tensors. We also include a benchmark utility.
256264
257- ###############################################################################
258- # Utilities
259- # =========
260- # In this section, we include a utility to generate semi-realistic data using
261- # Zipf distribution for sentence lengths. This is used to generate the nested
262- # query, key and value tensors. We also include a benchmark utility.
263265
264- # .. dropdown::
265266 import numpy as np
266267
267268 def zipf_sentence_lengths (alpha : float , batch_size : int ) -> torch .Tensor :
You can’t perform that action at this time.
0 commit comments