Resting on Laurels

I am sending fawning, swooning appreciative texts to my engineers back home because I actually get what they are doing on a much deeper level and realize we’ve nailed some of the issues I was concerned about.

They text me back and ask what I am smoking.

My colleague, Bob Valeiko, confides… “It’s not like ‘absent that day’ missed it… some of the stuff I heard them talking about last night at the cocktail party was really weird!

How so?

He showed me a business card that had some dots on it connected by lines. “Do you have any idea what this is?”

I don’t.

“It’s a protein molecule.  This guy was talking to Leslie (Barrett) about it, and she jumped right in, knew what it was.  She said ‘Oh yeah, and you put it in a vector…’  

Welcome to Seth Grimes’ Sentiment Analysis Symposium in New York.

Or, for those of us who went through base camp together:  #SAS10.

Who was there?   Four categories:

1.  Research Scientists & Students

2. Press & Bloggers

3. Sentiment Analysis tool & service reps SERVING the SA industry (yes, folks, there is a subcategory.  I guess that means we’re official.)

4. Clients & Consultants who use Sentiment Analysis tools.

At one point I quipped to Seth Grimes, (via Twitter, of course):

…Loving the #SAS10 conference, the IQ in this room is in the stratosphere… But have you seen the movie “Best in Show”?

For those of us with non-stratospheric IQs, here are five takeaways from the conference & five things we’re doing to move the industry forward.

The five main points:

1.  What is Sentiment Analysis actually for?  It’s to look for outliers. An analyst should NOT use Sentiment Analysis on its own and expect to find all the answers.  She should look for spikes as indicators.  For example, Apple’s sentiment rose and fell based on the release of the iPad.  A ‘blip’ should be a signal to ‘look here’ and find out why the change happened.  Sentiment Analysis in and of itself is useless without a human to interpret and make the result actionable.

2. Machine based sentiment analysis will never nail correct sentiment 100% of the time…and that’s perfectly NORMAL. In any given situation, human beings don’t consistently agree as to whether something is positive/neutral or negative/neutral and occasionally positive/negative.  The generally accepted standard is 82%, meaning: between human beings, there will be 82% agreement on determining whether sentiment expressed toward something (positive, neutral, negative) is the same. Machines won’t do any better than the human standard.

3. We’re in the wild west: we’re all using different terms and standards. Just as in the days of early internet advertising, of which I am a proud veteran, we have problems with everything from being able to describe what a score is to claims of x% accuracy.  Just what does it all mean?

4. Different kinds of Sentiment Analysis exist: Counting adjectives is what I originally thought this industry entailed.  And that may be fine for certain applications.  But Natural Language Processing, & Statistical Analysis & Artificial Intelligence (NLP&SA&AI) give you much better shades of nuance. We understand which types of methods work with which kinds of data, and therefore which may be a better application for which industry.  And this is NOT an easy business, nor is it easy to build from scratch, hence the love letters to my confused but grateful engineers.  I wholeheartedly agree with the principle of being able to kick the tires and try before you buy. On that note, to try something today, I invite you to go to www.Sentimetrix.com and register for a free account (to the right of the graph); play with it on your own or request a demo and I’ll walk you through the less obvious features.  We’re trying a few new things and would love your feedback.

5. SA currently rocks a few core industries: financial, pharma, reputation management. Make no mistake, if you are in these industries, you simply HAVE to have this technology now or you will be toast. Each industry has its own particular requirements (i.e. speed and accuracy in financial, broad reaching ‘we don’t know what we’re looking for yet’ in pharma).  My prediction on the next ‘must have’ industry?  You guessed it: politics.

The five big things on the horizon:

1. Battle of the Bands (not!): Matthias Tyrberg of Saplo proposed side by side testing of the vendors to let us duke it out.  Or, as Nathan Gilliatt enthused,  “It’s Thunderdome.  Two go in, one comes out.”  Seth Grimes wisely talked us down, insisting there could not possibly be a fair competition because of tuning requirements.

However… What do you think about a a Sentiment Analysis petting zoo for vendors at next year’s symposium?

2. Extraordinary research by some of the finest minds in the space: I am very excited about the opportunity to bring back ideas to my technical team such as Shlomo Argamon’s automated authorship profiling (gender, age, native language, education level).  I’m giddy to think of how this will yield results in the elections space.  My new friend Leslie Barrett talked about polarities beyond positive and negative:  i.e. sweet vs sour, hot vs. cold.  We have a prospective client asking for just that, and because our engine is language agnostic, I’m eager to test her premises against a multinational backdrop(what is ‘hot’ in terms of taste compared to ‘mild’ may differ considerably from country to country).

3. Call for standards in the industry: It feels like we need something in this space.  Matthias Tyrberg and I raised our hands to start feeling our way forward and figure out what needs to be defined.  I am so thankful to have met Marshall Sponder at the conference and to be able to learn from his experience in the web analytics space. Do you want to participate?  Do you have any suggestions?  Respond to this post and we’ll get started.

4. Foreign languages: This is an urgent need.  And the methodologies differ greatly.  I predict this is the space where we will see the most development in the next year.

5. Consolidation: This conference was very open and friendly.  There was no 500lb gorilla in the room to be afraid of.  We’re all pioneers and there is room for everyone.  For now… but on the horizon we see Google.  Microsoft.  IBM (and yes, they were there.) True giants in the space are now turning their eyes and patents in our direction.   Microsoft is expected to launch a service in six months.  Seth seemed to know a bit about it and said we are in for a surprise.  I think as we all looked around at each other, the feeling was:  Today, competitors, tomorrow, collaborators or even, …ahem, …subsidiaries? Be nice to everybody.

I glance back at Seth.  He’s tweeting, but I haven’t seen a reply to my snarky comment.

I look around at the 90 or so people in the room and realize that next year’s symposium won’t be quite so intimate; the field is booming.  And Seth has been a huge part of our collective success.  He has been a tremendous advocate for us.  He’s generous with his time, contacts, and advice.  We’re lucky to have him and this conference in a class by itself.

My best in show comment got a retweet.  And of course, I want to do an air punch “YESsssss!” OMG, I got retweeted by Seth Grimes!!! ::psych::

Then I look back at my own comment.   I slap my forehead.   I feel like I have become one of the totally neurotic characters in “Best in Show”. (http://www.imdb.com/title/tt0218839/)

Woof.

Laurel Earhart is Vice President of Business Development at SentiMetrix, which offers Sentiment Analysis tools and services to industries that require highly granular, extremely high speed, multilingual capabilities.  The company was developed out of a project to serve the DOD following an incident after 9-11 and now offers its services to clients inside and outside the beltway. She can be reached at earhart@Sentimetrix.com.  Or follow her on Twitter - she tweets from SentiMetrix.