ValueError: No gradients provided for any variable tensorflow keras

  keras, neural-network, python, tensorflow

I want to use neural network to select which data point is the best to be used for regression. For example, I have a sinewave with 100 data points in (-pi/2,pi/2), I want to choose the 3 most influential data points among these 100 so that the regression result is similar to the 100 data points sinewave. Assumed that at the beginning I only know one data point (X0,Y0), I send it into the first NN model, it output the index for the second datapoints (Xi1,Yi1). I send these two points (X0,Y0), (Xi1,Yi1) into the second NN model and get another index for the third datapoints (Xi2,Yi2). Then I will use these 3 datapoints to predict the whole sinewave from (-pi/2,pi/2), and backpropagate the mean squared error between the estimated and ground truth sinewave. Here is what my neural network looks like:

class PredModel(keras.Model):
def __init__(self):
    super().__init__()
    self.hidden0_1 = keras.layers.Dense(40, input_shape=(1,2))
    self.hidden0_2 = keras.layers.Dense(40)
    self.out0 = keras.layers.Dense(1)
    self.hidden1_1 = keras.layers.Dense(40, input_shape=(1,4))
    self.hidden1_2 = keras.layers.Dense(40)
    self.out1 = keras.layers.Dense(1)
    self.hidden2_1 = keras.layers.Dense(40, input_shape=(1,6))
    self.hidden2_2 = keras.layers.Dense(40)
    self.out2 = keras.layers.Dense(1)
  #  self.hidden1 = keras.layers.Dense(40, input_shape=(1,100,1))
  #  self.hidden2 = keras.layers.Dense(40)
  #  self.out = keras.layers.Dense(1)

    
def first_model(self, x):

    idx = keras.activations.relu(self.hidden0_1(x))
    idx = keras.activations.relu(self.hidden0_2(idx))
    idx = self.out0(idx)
    return idx

def second_model(self, x):



    idx = keras.activations.relu(self.hidden1_1(x))
    idx = keras.activations.relu(self.hidden1_2(idx))
    idx = self.out1(idx)
    return idx


def third_model(self, x):


    idx = keras.activations.relu(self.hidden2_1(x))
    idx = keras.activations.relu(self.hidden2_2(idx))
    idx = self.out2(idx)
    return idx

I think the main problem happens from here, there is no gradient provided. I need to find the data points according to the index after each model prediction. After finding all data points, I just used a pre-trained model MAML to estimate the sinewave, it works fine and I can get the MSE between its prediction and ground-truth, but the gradient can’t backpropagate.

idx=model.second_model(x_selected_whole)
idx=tf.cast(idx, tf.int64)


x_selected_whole=x_selected_whole.numpy().tolist()
x_selected_whole, x_selected_ITD, y_selected=choose_element(x,y,x_selected_whole,x_selected_ITD,y_selected,abs(idx[0][0]))
idx=model.third_model(x_selected_whole)
idx=tf.cast(idx, tf.int64)

x_selected_whole=x_selected_whole.numpy().tolist()
x_selected_whole, x_selected_ITD, y_selected=choose_element(x,y,x_selected_whole,x_selected_ITD,y_selected,abs(idx[0][0]))



logits= eval_sinewave_for_test(maml, x_selected_ITD, y_selected, x, y)
logits_tensor,y_tensor = np_to_tensor((np.array(logits), np.array(y)))
mse = loss_fn(y_tensor, logits)


return mse, logits,logits_tensor

Appreciate any help

Source: Python Questions

LEAVE A COMMENT