How to get input data shape when model is in predicting phase?

  keras, machine-learning, python, python-3.x, tensorflow

I tried to build Seq2Seq model by using TensorFlow 2.0.
It is not for NLP, but for predict timeseries data.

Since Seq2Seq model behave differently in training and predicting phase, I needed to write call function based on the phase.

In training phase, the decoder takes shifted timeseries data padded with zero.
(e.g. encoder input: [1,2,3,4], decoder input: [0,1,2,3])

And in predicting phase, the decoder takes single zero tensor at first, and it takes its output as input until it produces same length of input timeseries data (e.g. decoder input: 1) [0] — (decoder predict 1) –> [1] — … –> final output : [0,1,2,3])

Therefore, the call function in the predicting phase (training=False), it gets the input shape of encoder and makes a tf.zeros tensor based on the shape, and it goes to decoder’s input.

Here is some part of the class of model.

class Seq2Seq(Model):
    def __init__(self, **kwargs):
            self.encoder = layers.LSTM(5, return_state=True, input_shape=(None, 1))
            self.decoder = layers.LSTM(5, return_state=True, return_sequences=True,
                                       input_shape=(None, 1))
            self.dense = layers.Dense(1)

    def call(self, inputs, training=False, mask=None, **kwargs):
        decoded = None

        if training:
            encoder_input, decoder_input = inputs[0], inputs[1]
            encoded, state_h, state_c = self.encoder(encoder_input)
            decoded, _, _ = self.decoder(decoder_input, initial_state=[state_h, state_c])
            decoded = self.dense(decoded)
            encoder_input = inputs
            batch, time_step, features = tf.shape(encoder_input)
            encoded, state_h, state_c = self.encoder(encoder_input)
            decoder_input = tf.zeros(shape=[batch, 1, features])
            decoded = None
            for i in range(time_step):
                decoder_input, state_h, state_c = self.decoder(decoder_input, initial_state=[state_h, state_c])
                decoder_input = self.dense(decoder_input)
                if decoded is not None:
                    decoded = tf.concat([decoded, decoder_input], axis=1)
                    decoded = tf.identity(decoder_input)
        return decoded

In training phase, inputs has two input data. One is the encoder_input has shape [batch_size, time_step, features] and, another is the decoder_input has same shape as encoder_input.
But, in predicting phase, inputs is just same as the encoder_input of the training’s.

Training worked well, but in validation, it produces error., epochs=10, validation_data=(seq_test_data, seq_test_data))
# >>> train_dataset.element_spec
# (TensorSpec(shape=(2, None, 140, 1), dtype=tf.float32, name=None),
# TensorSpec(shape=(None, 140, 1), dtype=tf.float32, name=None))
# >>> seq_test_data.shape
# TensorShape([1000, 140, 1])
Epoch 1/10
5/5 [==============================] - ETA: 0s - MSE: 0.1487Traceback (most recent call last):
  File "/home/dbadmin/anaconda3/lib/python3.7/site-packages/IPython/core/", line 3417, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-91afa9c067da>", line 1, in <module>
    runfile('/home/dbadmin/woosung/DWT-AutoEncoder-Comparison/source/', wdir='/home/dbadmin/woosung/DWT-AutoEncoder-Comparison/source')
  File "/home/dbadmin/.pycharm_helpers/pydev/_pydev_bundle/", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "/home/dbadmin/.pycharm_helpers/pydev/_pydev_imps/", line 18, in execfile
    exec(compile(contents+"n", file, 'exec'), glob, loc)
  File "/home/dbadmin/woosung/DWT-AutoEncoder-Comparison/source/", line 112, in <module>
    history =, epochs=10, validation_data=(seq_test_data, seq_test_data))
  File "/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/", line 108, in _method_wrapper
    return method(self, *args, **kwargs)
  File "/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/", line 1133, in fit
  File "/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/", line 108, in _method_wrapper
    return method(self, *args, **kwargs)
  File "/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/", line 1379, in evaluate
    tmp_logs = test_function(iterator)
  File "/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/", line 780, in __call__
    result = self._call(*args, **kwds)
  File "/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/", line 823, in _call
    self._initialize(args, kwds, add_initializers_to=initializers)
  File "/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/", line 697, in _initialize
    *args, **kwds))
  File "/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/", line 2855, in _get_concrete_function_internal_garbage_collected
    graph_function, _, _ = self._maybe_define_function(args, kwargs)
  File "/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/", line 3213, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/", line 3075, in _create_graph_function
  File "/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/", line 986, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/", line 600, in wrapped_fn
    return weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/", line 973, in wrapper
    raise e.ag_error_metadata.to_exception(e)
tensorflow.python.framework.errors_impl.OperatorNotAllowedInGraphError: in user code:
    /home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/ test_function  *
        return step_function(self, iterator)
    /home/dbadmin/woosung/DWT-AutoEncoder-Comparison/source/ call  *
        batch, time_step, features = tf.shape(encoder_input)
    /home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/ __iter__
    /home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/ _disallow_iteration
        self._disallow_when_autograph_enabled("iterating over `tf.Tensor`")
    /home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/ _disallow_when_autograph_enabled
        " indicate you are trying to use an unsupported feature.".format(task))
    OperatorNotAllowedInGraphError: iterating over `tf.Tensor` is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature.

I guess the error comes in the reason of that it is not possible to get exact input shape until I put explicit data.

However, still not clear how to fix it.

Source: Python Questions