new icn messageflickr-free-ic3d pan white
Annotation Agreement | by Intelligent Enterprise
Back to photostream

Annotation Agreement

Is Sentiment Analysis an 80% Solution? Find out more about this image here:


The Univ. of Pittsburgh researchers found 82% agreement in the assignment, by two individuals, of phrase-level sentiment polarity to automatically identified subjective statements in a test document set. Polarity (a.k.a. valence) is the sentiment direction: positive, negative, both, or neutral.


The authors further report, "For 18% of the subjective expressions, at least one annotator used an uncertain tag when marking polarity. If we consider these cases to be borderline and exclude them from the study, percent agreement increases to 90%."

0 faves
Taken on March 29, 2010