Transfer learning on new shape gives ValueError: Depth of input (2) is not a multiple of input depth of filter

  keras, python, tf.keras, transfer-learning

I have the following pre-trained keras model:

input_1 (InputLayer)            [(None, 12, 4)]                                
__________________________________________________________________________________________________
input_2 (InputLayer)            [(None, 12, 4)]                                                 
__________________________________________________________________________________________________
conv1d (Conv1D)                 (None, 20, 2)        input_1[0][0]                    
__________________________________________________________________________________________________
conv1d_3 (Conv1D)               (None, 20, 2)        input_2[0][0]                    
__________________________________________________________________________________________________
lstm_1 (LSTM)                   (None, 12, 2)        conv1d_3[0][0]              
__________________________________________________________________________________________________
time_distributed (TimeDistribut (None, 12, 16)       lstm_1[0][0]                     
__________________________________________________________________________________________________
time_distributed_1 (TimeDistrib (None, 12, 1)        time_distributed[0][0]           
==================================================================================================

I need to do the transfer learning on the new small data set that has a bit different shape. In particular, instead of two inputs of the shape (None, 12, 4), the new model must have a shape (None, 12, 3).

Therefore, I want to replace the top layer with the new one. Then I want to train the new model on the new data, while keeping the weight coefficients of some layers frozen.

import tensorflow as tf  # 2.4.1
from tensorflow import keras

base_model = tf.keras.models.load_model("model")

def change_model(model, new_input_shape=(None, 12, 3)):
    # replace input shape of first layer
    model._layers[1].batch_input_shape = new_input_shape

    # rebuild model architecture by exporting and importing via json
    new_model = keras.models.model_from_json(model.to_json())
    new_model.summary()

    # copy weights from old model to the new one
    for layer in new_model.layers:
        try:
            layer.set_weights(model.get_layer(name=layer.name).get_weights())
        except:
            print("Could not transfer weights for layer {}".format(layer.name))

    return new_model

calibrated_model = keras.Sequential()
for layer in base_model.layers[2:]:
    calibrated_model.add(layer)
    
new_model = change_model(calibrated_model, new_input_shape=(None, 12, 3))

But I am getting a lot of problems with this approach and not sure how to proceed.

Source: Python Questions

LEAVE A COMMENT