Comments on: Text Classification for Sentiment Analysis – Eliminate Low Information Features

By: Alex Leykin

Alex Leykin — Wed, 12 Oct 2011 21:39:00 +0000

Yes, this seems like a lookahead bias here. Bestwords should be computed only on the train set (first 3/4 of the reviews)

By: Sriram

Sriram — Sun, 18 Sep 2011 05:19:00 +0000

Thanks a lot Jacob.Let me try it out.

By: Jacob Perkins

Jacob Perkins — Sat, 17 Sep 2011 16:53:00 +0000

You can average the probabilities from the NaiveBayes and Maxent classifiers, but DecisionTree doesn’t support prob_classify, so you’ll have to hardcode a check and use 100% or 0% in your averaging.

By: Sriram

Sriram — Sat, 17 Sep 2011 12:47:00 +0000

Hi Jacob,
Iam trying to implement prob_classify() method in the MaxVoteClassifier class which combines three classifiers(naive,decision,max ent) and does voting.I added the defenition of prob_classify to the MaxVoteClassifier class.But not sure how to pass feature_probdist to the MaxVoteClassifier class.Appreciating your help.

By: Sriram

Sriram — Mon, 22 Aug 2011 11:40:00 +0000

Thanks a lot Jacob.I already had separate classifiers.But didnt know how to use the Prob_classify() to calculate the positive and negative scores.It worked now.Thanks once again.

By: Jacob Perkins

Jacob Perkins — Sat, 20 Aug 2011 15:52:00 +0000

You’d need to train separate classifiers, one for each method, then instead of using the classify() function, call prob_classify() to get a ProbDist, which you can use to get the probability of each label.

By: Sriram

Sriram — Sat, 20 Aug 2011 14:40:00 +0000

Hi,
Thanks a lot for the all information.For a given test string for example “This is a great movie” if i need to measure the confidence level of classification of the above three methods(best words features,bigram etc) how do i do that?So that i can determine which output to take when i get different classification results from each method. Thanks once again for all the knowledge sharing.

By: Jacob Perkins

Jacob Perkins — Mon, 14 Mar 2011 16:04:00 +0000

Thanks for that paper reference, looks interesting.

Perhaps it would be more correct to only use the training set for info gain stats, but I see those stats as a kind of global filter that allows you to determine significant features beforehand (as you can often do in non-text classification). The exact numbers aren’t used directly, but the relative values are used to determine which words are more significant, and that significance should theoretically be similar in the training set, testing set, and both combined. And that assumption appears to be correct based on your tests.

By: Henrik Nordvik

Henrik Nordvik — Mon, 14 Mar 2011 15:29:00 +0000

When calculating information gain, you’re using the whole data set, including the test set. Aren’t you supposed to use only data from the training set when training?

I tested using only the training set to gather statistics, but it didn’t seem to affect the results much, just a tad lower accuracy.

Also, check out “Delta TFIDF: An Improved Feature Space for Sentiment Analysis” for a similar method.

By: Henrik Nordvik

Henrik Nordvik — Mon, 14 Mar 2011 15:29:00 +0000

When calculating information gain, you’re using the whole data set, including the test set. Aren’t you supposed to use only data from the training set when training?

I tested using only the training set to gather statistics, but it didn’t seem to affect the results much, just a tad lower accuracy.

Also, check out “Delta TFIDF: An Improved Feature Space for Sentiment Analysis” for a similar method.