Comments on: NLTK Classifier Based Chunker Accuracy

By: Learning to do natural language processing with NLTK | JetLlib Journal

Learning to do natural language processing with NLTK | JetLlib Journal — Sat, 03 Apr 2010 20:12:05 +0000

[...] NLTK Classifier Based Chunker Accuracy [...]

By: Jacob Perkins

Jacob Perkins — Tue, 30 Mar 2010 04:10:37 +0000

Sorry for the late reply, somehow the notification got caught in my spam filter. Anyway...

If you think the iterations are cutting off too quickly, you might want to try tweaking some of the cutoffs described here: http://nltk.googlecode.com/svn/trunk/doc/api/nl...

Wish I could help with the NE. All I can think of is to suggest checking out the names corpus: http://nltk.googlecode.com/svn/trunk/doc/api/nl...
Maybe you can train on that somehow.

By: Jacob Perkins

Jacob Perkins — Mon, 29 Mar 2010 21:10:37 +0000

By: James Smith

James Smith — Fri, 26 Mar 2010 16:00:57 +0000

I've got my computer finally setup so I'm training the stuff myself now.

I'm training a few different ones to evaluate but I suspect I may be doing something wrong as the iterations go 1, 2, Final. Any ideas?

Unlike yourself, I'm also interested in the named entity side of this. Unfortunately the NLTK default chunker is pretty bad at recognising the entity types so I'm using conll2002s chunked_sents as a training corpus. I'd also like to use the treebank NE material but suspect I would have to change the tags to the same names as those used in conll materials.

By: Jacob Perkins

Jacob Perkins — Thu, 25 Mar 2010 01:41:23 +0000

What corpus would you want it trained on, treebank, conll2000, or both? Any suggestions on where to put the file? (it's a couple megabytes)

By: James Smith

James Smith — Tue, 23 Mar 2010 19:23:15 +0000

I don't suppose you'd be willing to upload the trained chunker as a serialized object would you?