How to install BigBirdTokenizer for NLP?

When I try to install BigBirdTokenizer I get the
Following error

AttributeError                            Traceback (most recent call last)
<ipython-input-12-c81eaf9abd32> in <module>()
----> 1 from summarizer import Summarizer
      2 f = open("macineLearning","r")
      3 full_text = f.read()
      4 model = Summarizer()
      5 result = model(full_text, min_length=60, max_length = 500 , ratio = 0.4)

4 frames
/usr/local/lib/python3.7/dist-packages/summarizer/__init__.py in <module>()
----> 1 from summarizer.model_processors import Summarizer, TransformerSummarizer

/usr/local/lib/python3.7/dist-packages/summarizer/model_processors.py in <module>()
      2 
      3 import numpy as np
----> 4 from transformers import *
      5 
      6 from summarizer.bert_parent import BertParent

/usr/local/lib/python3.7/dist-packages/transformers/__init__.py in __getattr__(self, name)

/usr/local/lib/python3.7/dist-packages/transformers/file_utils.py in __getattr__(self, name)

/usr/local/lib/python3.7/dist-packages/transformers/file_utils.py in __getattr__(self, name)

AttributeError: module transformers.models.big_bird has no attribute BigBirdTokenizer

I am doing this [error on first line of code below]

from summarizer import Summarizer
f = open("macineLearning","r")
full_text = f.read()
model = Summarizer()
result = model(full_text, min_length=60, max_length = 500 , ratio = 0.4)
summarized_text = ''.join(result)
print (summarized_text)

I’ve also installed:

!pip install gensim
!pip install git+https://github.com/boudinfl/pke.git
!python -m spacy download en
!pip install bert-extractive-summarizer --upgrade --force-reinstall
!pip install spacy==2.1.3 --upgrade --force-reinstall
!pip install -U nltk
!pip install -U pywsd
import nltk
nltk.download('stopwords')
nltk.download('popular')
!pip install sentencepiece
!pip install BigBirdTokenizer
!pip install sentence-transformers==0.2.5.1
!pip install transformers==2.6.0

But the BigBirdTokenizer always seems to fail..
I’m quite new to NLP but would like to understand why the issue.

Thank you

Source: Python Questions

One Reply to “How to install BigBirdTokenizer for NLP?”

LEAVE A COMMENT