Wrong number of hyperparameters in keras tuner?

[jhudsy]

I'm new to Keras Tuner, so apologies if this is simply a user error.

I'm trying to build a model with a time distributed layer, followed by several LSTM layers followed by dense layers, with various parameters defined using hp.Int, hp.Choice, etc. My model builder code is as follows.

import kerastuner as kt

def model_builder(hp):

    inputs = Input(shape=(40, 136)) #full tensor
    x = inputs

    #prepare hyperparameter tuning

    num_LSTM_layers = hp.Int('num_LSTM_layers',0,3)
    num_LSTM_units=[]
    for i in range(num_LSTM_layers):
        num_LSTM_units.append(hp.Int('lstm'+str(i+1)+'_units',
                                     min_value = 32,
                                     max_value = 64,
                                     step=8))
        
                                     
    num_dense_layers = hp.Int('num_dense_layers',1,3)
    num_dense_units = []
    dense_activation = []

    for i in range(num_dense_layers):
        num_dense_units.append(hp.Int('dense'+str(i+1)+'_units',
                                     min_value = 32,
                                     max_value = 128,
                                     step=16))
        dense_activation.append(hp.Choice("dense"+str(i+1)+"_activation",["relu", "leaky_relu"]))
    
    hp_learning_rate = hp.Choice('learning_rate', values=[1e-3, 1e-2])

    #make the NN
    x = TimeDistributed(Dense(hp.Int('td_dense_units',min_value=32,max_value=128,step=16),activation=hp.Choice("td_dense_activation",["relu","leaky_relu"])))(x)

    for i in range(num_LSTM_layers):
        x = LSTM(num_LSTM_units[i],return_sequences=True if i<num_LSTM_layers else False)(x)

    for i in range(num_dense_layers):
        x = Dense(num_dense_units[i],activation = dense_activation[i])(x)

    output = Dense(1,activation='relu',name="Elo")(x)
    
    model = keras.Model(inputs=inputs,outputs=[output])

    model.compile(optimizer=keras.optimizers.Adam(learning_rate=hp_learning_rate),
                    loss={'Elo':'mae'},
                    metrics={'Elo':'mae'})

    return model

When I run the model using the following

tuner = kt.Hyperband(model_builder,
                     objective='val_loss',
                     max_epochs=100,
                     factor=5)

stop_early = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=5)
save = tf.keras.callbacks.ModelCheckpoint('modelCP.keras', save_best_only=True,mode='auto',monitor='val_loss')

tuner.search(train_gen,validation_data=val_gen,epochs=100,callbacks=[stop_early,save])

I get output as follows:

Search: Running Trial #2

Value             |Best Value So Far |Hyperparameter
3                 |2                 |num_LSTM_layers
3                 |1                 |num_dense_layers
96                |64                |dense1_units
relu              |relu              |dense1_activation
0.001             |0.001             |learning_rate
48                |32                |td_dense_units
relu              |relu              |td_dense_activation
64                |32                |lstm1_units
56                |32                |lstm2_units
4                 |4                 |tuner/epochs
0                 |0                 |tuner/initial_epoch
2                 |2                 |tuner/bracket
0                 |0                 |tuner/round

As you can see, I have, for example 3 dense layers, but no hyperparameters for these are shown. Instead, the hyperparameters shown are for the values generated in trial 1 (i.e., in that trial I had 2 LSTM layers and 1 dense layer rather than the 3 I'm supposed to have in this trial).

[jhudsy]

Here's an example that (perhaps) better illustrates the issue. In my first trial there should be 3 LSTM layers and 2 dense layers, but parameters are only shown for 1 dense layer and 1 lstm layer: