Category : bert-language-model

Goal: Amend this Notebook to work with Albert and Distilbert models Error occurs in Section 1.2, only for these 2 new models. For filenames etc., I’ve created a variable used everywhere: MODEL_NAME = ‘albert-base-v2’ # ‘distilbert-base-uncased’, ‘bert-base-uncased’ Section 1.1, I’ve added in the additional functions: from transformers import (BertConfig, BertForSequenceClassification, BertTokenizer,) from transformers import (AlbertConfig, ..

Read more

I am currently trying to replicate the article https://towardsdatascience.com/text-classification-with-bert-in-pytorch-887965e5820f to get an introduction to PyTorch and BERT. I used some own sample corpus and corresponding tragets as practise, but the code throws the following: ————————————————————————— IndexError Traceback (most recent call last) <ipython-input-4-8577755f37de> in <module>() 201 LR = 1e-6 202 –> 203 trainer(model, df_train, df_val, LR, ..

Read more

This page has two scripts When should one use 1st method shown below vs 2nd? As nli-distilroberta-base-v2 trained specially for finding sentence embedding wont that always be better than the first method? training_stsbenchmark.py1 – from sentence_transformers import SentenceTransformer, LoggingHandler, losses, models, util #You can specify any huggingface/transformers pre-trained model here, for example, bert-base-uncased, roberta-base, xlm-roberta-base ..

Read more

I am getting the following error : attributeerror: ‘dataframe’ object has no attribute ‘data_type’" . I am trying to recreate the code from this link which is based on this article with my own dataset which is similar to the article tokenizer = BertTokenizer.from_pretrained(‘bert-base-uncased’, do_lower_case=True) encoded_data_train = tokenizer.batch_encode_plus( df[df.data_type==’train’].example.values, add_special_tokens=True, return_attention_mask=True, pad_to_max_length=True, max_length=256, return_tensors=’pt’ ) ..

Read more

I have a question concerning simpleTransformers, I have 2 datasets for my downstream task, there are 3 options: 1. append the 2 datasets and treat them as 1 large dataset: model_1 = ClassificationModel("bert","bert-base-cased",args=train_args) all_dataset=dataset1+dataset2 model_1.train_model(all_dataset) 2. train separately without concatinating but on the same model: model_1 = ClassificationModel("bert","bert-base-cased",args=train_args) model_1.train_model(dataset1) model_1.train_model(dataset2) 3. train separately without concatinating ..

Read more

def set_seed(seed: int): """ Helper function for reproducible behavior to set the seed in “random“, “numpy“, “torch“ and/or “tf“ (if installed). Args: seed (:obj:`int`): The seed to set. """ random.seed(seed) np.random.seed(seed) if is_torch_available(): torch.manual_seed(seed) torch.cuda.manual_seed_all(seed) # ^^ safe to call this function even if cuda is not available if is_tf_available(): import tensorflow as tf tf.random.set_seed(seed) ..

Read more

I have fine-tuned a model for sentiment analysis using BertForSequenceClassification. I’m trying to run a prediction on example sentiment (to find out whether it is negative or positive). Here is my code: model_path = "./path_to_directory_with_config.json" tokenizer = transformers.BertTokenizer.from_pretrained(‘TurkuNLP/bert-base-finnish-cased-v1’) txt = "This was a nice place" inputs = tokenizer(txt, return_tensors="pt") print(inputs) model = transformers.BertForSequenceClassification.from_pretrained(model_path, num_labels=1) trainer ..

Read more

I’m trying to summarize some text with "Text Summarization with BERT" by next steps: first, installation of: pip install transformers==2.2.0 pip install bert-extractive-summarizer secondly,import summarizer: from summarizer import Summarizer,TransformerSummarizer and thenI got a importError like that: cannot import name ‘AlbertModel’ from ‘transformers’ my reference: https://medium.com/analytics-vidhya/text-summarization-using-bert-gpt2-xlnet-5ee80608e961 Source: Python..

Read more

I’m trying to fine-tune BERT model for sentiment analysis (classifying text as positive/negative) with Huggingface Trainer API. My dataset has two columns, Text and Sentiment, it looks like this. Text Sentiment This was good place 1 This was bad place 0 Here is my code: from datasets import load_dataset from datasets import load_dataset_builder from datasets ..

Read more