Stemming a pandas dataframe

  pandas, python, stemming

I have tweet dataset (taken from NLTK) which is currently in a pandas dataframe, but I need to stem it. I have tried many different ways and get some different errors, such as

AttributeError: 'Series' object has no attribute 'lower'
and
KeyError: 'text'

I dont understand the KeyError as the column is definitely called ‘text’, however I understand that I need to change the dataframe to a string in order for the stemmer to work (I think).

Here is an example of the data

from nltk.stem.snowball import SnowballStemmer
stemmer = SnowballStemmer("english")

negative_tweets = twitter_samples.strings('negative_tweets.json')

negtweetsdf = DataFrame(negative_tweets,columns=['text'])

print(stemmer.stem(negtweetstr))

Source: Python Questions

LEAVE A COMMENT