Category : transformer

am using Jupyter Lab to run. It has pre-installed tf2.3_py3.6 kernel installed in it. It has 2 GPUS in it. PyTorch Lightning Version (e.g., 1.3.0): ‘1.4.6’ PyTorch Version (e.g., 1.8): ‘1.6.0+cu101′ Python version: 3.6 OS (e.g., Linux): system=’Linux’ CUDA/cuDNN version: 11.2 How you installed PyTorch (conda, pip, source): pip I am saving the best model ..

Read more

I have a batch size of 1 and number of transformer layers is 1. I have images that are very big so I have created embeddings using ResNet18 as an intermediate representation for tiles of my images. Because my images don’t include different number of tiles, I also have used some sort of masking/zero filling ..

Read more

Consider the following training corpora: dataset1: composed of French instances dataset2: dataset1 + Arabic instances test_dataset (for both scenarios): composed of French instances (the same annotation guidelines were used for both languages). After analyzing the results of our preliminary experimental setup, we chose BERT as our baseline system. Considering the different languages involved, we experimented ..

Read more

I have been reading the official guide here (https://www.tensorflow.org/text/tutorials/transformer) to try and recreate the Vanilla Transformer in Tensorflow. I notice the dataset used is quite specific, and at the end of the guide, it says to try with a different dataset. But that is where I have been stuck for a long time! I am ..

Read more

I am working on a problem where I need to reconstruct an image in a Pix2Pix-like manner. The data has some attributes what would make a Transformer/Perceiver IO favourable in my opinion (e.g. the y axis contains information about the location, but is not necessarily neighboured to the row above, hence Convolutions assume a structure ..

Read more