Using LSTM/RNN to predict a sequence of numbers

  keras, lstm, python, recurrent-neural-network, tensorflow

I am looking to apply RNN to a fairly simple problem, so as to grasp how it works. I followed this example which demonstrates how to use a LSTM layer to analyse input, and now I’d like to use it for output.

I decided to try to train an RNN to output doubles of an int given as input, up to a cap. So for example, using this data:

## MANUAL NP IMPORT AS VERSION CHANGE CAUSES ERRORS
import pkg_resources
pkg_resources.require("NumPy==1.19.5")
import numpy

from tensorflow.keras.models import Sequential
from tensorflow.keras import Input
from tensorflow.keras.layers import Dense, LSTM, Embedding, Dropout, SimpleRNN

def doubles(b,cap): 
    seq = [b]
    if b<=0 :
        raise ValueError('Base int must be greater than zero.')
    i = 1
    while seq[-1]<cap:
        seq.append(b*2**i)
        i +=1
    return seq

maxsize = -1
cap = 100
nums = [2,3,4,6,7,8,9,10,11,12]
mydoubles = []
for base in nums:
    myseq = doubles(base, cap)
    mydoubles.append(myseq)
    if len(myseq)>=maxsize:
        maxsize = len(myseq) +1

for s in mydoubles :
    while len(s)<maxsize:
        s.append(-1)
    print(s)


[2, 4, 8, 16, 32, 64, 128, -1]
[3, 6, 12, 24, 48, 96, 192, -1]
[4, 8, 16, 32, 64, 128, -1, -1]
[6, 12, 24, 48, 96, 192, -1, -1]
[7, 14, 28, 56, 112, -1, -1, -1]
[8, 16, 32, 64, 128, -1, -1, -1]
[9, 18, 36, 72, 144, -1, -1, -1]
[10, 20, 40, 80, 160, -1, -1, -1]
[11, 22, 44, 88, 176, -1, -1, -1]
[12, 24, 48, 96, 192, -1, -1, -1]

I would like to create a keras model that takes nums as inputs and outputs the corresponding sequence, using -1 as a ‘STOP’ indicator, seeing as I am looking to output only numbers.

I have tried creating a model like this:

mymodel = Sequential()

mymodel.add(Input(shape=(1,)))
mymodel.add(Dense(32))
mymodel.add(LSTM(64))

But it raises this error:

ValueError                                Traceback (most recent call last)
<ipython-input-30-24845ffeabd5> in <module>
      3 mymodel.add(Input(shape=(1,)))
      4 mymodel.add(Dense(32))
----> 5 mymodel.add(LSTM(64))
(...)
ValueError: Input 0 of layer lstm_2 is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: (None, 32)

What additional dimensions does it require? Am I using these layers incorrectly for wanting to output a "timeseries"?

Source: Python Questions

LEAVE A COMMENT