Energy Industry Use of Sentiment Analysis

Companies don’t rely solely on sentiment analysis to inform strategic decisions, but it is a powerful complement to traditional market intelligence. Like many things in the social data ecosystem, sentiment is a rapidly evolving tool. While challenges like accurately identifying and classifying  irony, sarcasm, and emoticons exist, companies are meeting that challenge with increasingly sophisticated, Twitter syntax-specific tools.

IHS, a leading analysis and information provider–and one we’re excited to announce as a new Plugged In partner–built a sentiment intelligence tool to facilitate its clients’ use of social data. This past summer, IHS released the U.S. Sentiment Index, a tool that assess realtime Tweets, providing a representation of the average mood of the United States.

We were curious to learn more about how sentiment analysis is being used across industries not typically known for their use of social data. IHS shared an example of how companies, in this case in the oil and gas industry, incorporated sentiment analysis to provide a deeper understanding of public opinion on hydraulic fracturing, commonly referred to as “fracking.” IHS looked at the sentiment of fracking-related Tweets globally, as well as in specific states like Colorado.  Analysis determined in which states the most Tweets about fracking originated and what keywords are most commonly associated with the topic. Both of these things contribute to the companies’ understanding of the drivers of public sentiment on the topic of fracking–valuable information.

To expose another layer of insight, IHS used network analysis to understand and measure the virality of messages. One of the takeaways from the research was that the content of a message is not as important as understanding which voices influence the dissemination of that message. Further, the number of followers an influencer has was not as important as whether one of those followers retweeted the message outside the influencer’s immediate circle of followers. These are just a few highlights from a more in-depth paper we wrote.

SGI Launches Global Twitter Heartbeat, Powered by Gnip

File this under cool news.

SGI’s Big Brain Computer has created a Global Twitter Heartbeat, allowing the supercomputer to analyze the Twitter stream for sentiment and geolocation to create a Twitter heartbeat telling us how the world is feeling based on emotions communicated via Twitter. Not only is this a cool undertaking by the folks at SGI, but we’re proud to announce that it is powered by Gnip’s decahose Twitter stream.

To make this happen, SGI partnered with Kalev H. Leetaru of the University of Illinois and Dr. Shaowen Wang of the CyberInfrastructure and Geospatial Information (CIGI) Laboratory at the University of Illinois at Urbana-Champaign.

This isn’t just some simple stream.  The SGI supercomputer analyzes every Tweet to assign location (not just GPS-tagged tweets, but processing the text of the Tweet itself) and tone values, then visualizing the conversation in a heat map that puts Tweet location, Tweet density and tone into a unified geospatial perspective. The entire process from ingestion to data analysis to producing the heat map runs at a speed that allows visualization of a map frame per second.

To see it live, check out SGI’s Facebook page.

You can also see videos of the Twitter Heartbeat for the Presidential Elections and Hurricane Sandy.

Sentiment Analysis & Social Data — Picking the Right Tool for the Job

Chris Moody of Gnip

We were recently at the Sentiment Analysis Symposium with many of the leaders in the sentiment space. Conference presenters spanned a spectrum: academics, engineers and executives involved in research and developing sentiment analysis tools. The event was full of great content, and we wanted to share some takeaways.

It was impressive to see the variety of use cases and solutions that were presented — both signs of a rapidly maturing market.  Because of that variety, it was also clear that a one-size-fits-all solution for sentiment analysis of social data is not on the immediate horizon.  An overarching message from the event was that the different use cases for sentiment analysis and the types of social content being analyzed demand tailored sentiment tools purpose-built for the job.

I was also struck by just how central social data was to the conversation and how most of the speakers talked about analyzing social data in some form.  Sentiment analysis has clearly become important to social data analysis, and vice versa, social data is really attractive to companies and researchers seeking ways to understand the moods and opinions of broad populations of consumers and citizens.

Here are some of the highlights of the business cases that were presented at the conference:

  • Mood Analysis Informs Market Decisions at ThomsonReuters — Scanning a broad swath of social content and news allows ThomsonReuters to build their MarketPysch Indices, including their “Gloom Index,” which they’ve demonstrated provides a leading indicator for market drops. ThomsonReuters’ Aleksander Sobczyk made an interesting point for our readers: the social content they focus on is long form only (i.e. blogs), and not short form content like Twitter.  Developing the depth of insight and certainty essential to their current solution requires a bigger text sample than Twitter provides.
  • Guiding Product Development at Dell — Through its Social Media Listening Command Center, Dell picks up on customer sentiment and uses it to drill in and learn more. For instance, they picked up negative sentiment about a new Alienware laptop right after a release: units were overheating. The negative sentiment tipped them off, and their product team took action to learn more from users. Through engaging their customers online, they were able to discover an idiosyncrasy in laptop use among power users. When plugged into big auxiliary monitors, users were keeping the laptop screen nearly closed, blocking the exhaust fans. Dell fixed the fan placement and eliminated the problem for future customers. Knowing very specifically what the negative sentiment was about was critical to catching a problem spot among loyal fans and power users.
  • Understanding Weibo Emotions (了解微博的情感) — Detecting sentiment in other languages with different linguistic structures and use habits can be difficult, which is why Soshio has tried to normalize around universal emotions to help Western brands make sense of what Chinese weibo (microblog) users are saying about their brands.  According to Ken Hu, Soshio’s founder, they’re actually having to translate written language “into sound” to address sentiment challenges like written puns that only make sense when pronounced audibly.

These are just a sample of the stories we heard from the companies at the event, and what we hear from our customers building sentiment tools.  Some serve customers who need to know about general sentiment (i.e. positive or negative), while are others are interested in specific moods or opinions. Some support tailoring to a specific cultural context or industry vertical, while others help evaluate conversation in a broad, general context.  A few are working hard to analyze sentiment in a global context — including native natural language processing for different languages, each with their own norms related to sarcasm, puns, irony, etc. Content format is a major focus as well. Evaluating 140 characters rich with hashtags and URLs on Twitter requires a much different approach than evaluating a WordPress blog post. These are all valid use cases for sentiment analysis, and all come with unique nuances and benefit from different techniques.

Sentiment analysis is a critical category of tool for broader social data analysis, but at the same time, it’s not a monolithic category. If you’re building a product to analyze social data and want to evaluate sentiment, the good news is that you can pick from a wide range of approaches or tools (or maybe even choose more than one?) for sentiment analysis that is suited to the business problem you’re trying to address, and the content you’re trying to analyze.

If you’re interested to learn more about the event and some other interesting developments and business cases in the sentiment analysis world, check out our recent interview with Seth Grimes, the lead organizer for the Sentiment Analysis Symposium.

Social Data and The Election

If you’re excitedly waiting the results of the election, and wanted to keep an eye what people are saying about Election 2012 on social media, we have a list of resources below:

  • I Voted Map – A realtime map by the good people at Foursquare allowing people to check in and say “I voted.” The map compiles checkins from voters.
  • The Twitter Political Index – Twitter’s official coverage measuring the sentiment between each Presidential candidate and trending topics related to the election.
  • Tumbling the Election – Coverage from Tumblr’s editorial team tracking top election related hashtags.
  • Facebook Stories on the Election – Watch Americans that said they voted on Facebook in real time.
  • Tumblr Election 2012 – Union Metrics visualization of trending election-related tags on Tumblr. Shows how many posts per second are about the election.
  • Infinigon Group – Tracking realtime political sentiment on social media.
  • 2012 Election Mood Meter – Netbase election mood meter tracks sentiment on social media for both candidates for President and Vice President candidates and breaks it down further by gender.
  • Electoral Map based on Tweets – The Guardian has created a map to show who would win the election based on Tweets.
  • Election Day on Twitter – Al Jazeera and Flowics show buzz volume on Twitter about each of the Presidential candidates as well as a live stream about Tweets on each candidate.
  • Rock The Vote Real-Time Politics – Splunk and MTV have created a visualization of hashtags on Twitter about each Presidential candidate.
  • The Crowdwire – Bluefin Labs has created what they’re calling a “Social Exit Poll” looking at how people are talking about how they voted on social media.
  • Yahoo! Election Control Room –  Attensity has teamed up with Yahoo! to show how America is feeling about the election and sharing select Tweets.
  • US Electoral Compass 2012 – Brandwatch allows you to select a state and date range to show what political issues each state is talking about.
  • NBC Politics – Using Crimson Hexagon to power their social media analysis.
  • Twitter Sentiment Analysis – USC is tracking sentiment around each candidate.

Who else are we missing? Also, as a bonus you can see our interview with Gabriel Banos of Zauber Labs on predicting the election with social data and Union Metrics comparison of the candidates using Tumblr data.

Social Data Around Voter Values

Dreamforce Hackathon Winner: Enterprise Mood Monitor

As we wrote in our last post, Gnip co-sponsored the 2011 Dreamforce Hackathon, where teams of developers from all over the world competed for the top three overall cash prizes as well as prizes in multiple categories.  Our very own Rob Johnson (@robjohnson), VP of Product and Strategy, helped judge the entries, selecting the Enterprise Mood Monitor as winner of the Gnip category.

The Enterprise Mood Monitor pulls in data from a variety of social media sources, including the Gnip API, to provide realtime and historical information about the emotional health of the employees. It shows both individual and overall company emotional climate over time and can send SMS messages to a manager in cases when the mood level goes below a threshold. In addition, HR departments can use this data to get insights into employee morale and satisfaction over time, eliminating the need to conduct the standard employee satisfaction surveys. This mood analysis data can also be correlated with business metrics such as Sales and Support KPIs to identify drivers of business performance.

Pretty cool stuff.

The three developers (Shamil Arsunukayev , Ivan Melnikov  and Gaziz Tazhenov) from Comity Designs behind this idea set out to create a cloud app for the social enterprise built on one of Salesforce’s platforms.  They spent two days brainstorming the possibilities before diving into two days of rigorous coding. The result was the Enterprise Mood Monitor, built on the platform using Apex, Visualforce, and the following technologies: Facebook API (Graph API),  Twitter API, Twitter Sentiment API, LinkedIn API, Gnip API, Twilio, Chatter, Google Visualization API. The team entered their Enterprise Mood Monitor into the Twilio and Gnip categories. We would like to congratulate the guys on their “double-dip” win as they took third place overall and won the Gnip category prize!

Have fun and creative way you’ve used data from Gnip? Drop us an email or give us a call at 888.777.7405 and you could be featured in our next blog.

Can Social Media Data Offset Market Volatility?

It’s been a volatile time for the markets the last few weeks. Today especially – the Dow closed down 635 points; S&P, -80; NASDAQ, -175. While there’s no shortage of opinions on how/why the market will/will not recover, one thing is for certain – having the right data to make decisions is more important than ever.

Part of the reason for this is that the markets are clamoring for trends – definitive information on stock trends and market sentiment. Which is why it’s exciting to see how our finance clients are using the Gnip realtime social media data feeds. In a time of increased volatility, our hedge fund (and other buy-side) clients are leveraging social media data as a new source of analysis and trend identification. With an ever-growing number of users, and Tweets, per day, Twitter is exploding, and market-leading funds are looking at the data we provide as a way to more accurately tap into the voice of the market. They’re looking at overall trend data from millions of Tweets to predict the sentiment of consumers as well as researching specific securities based on what’s being said about them online.

While the early-adopters of this data have been funds, this type of analysis is available to individuals as well. Check out some start-ups doing incredible things at the intersection of finance and social media:

  • Centigage provides analytics and intelligence designed to enable financial market participants to use social media in their investment decision-making process
  • SNTMNT offers an online tool that gives daily insights into online consumer sentiment surrounding 25 AEX funds and the index itself

For the first time in history, access to (literally) millions of voices is at our fingertips. As the market continues its volatility, those voices gain resonance as a source of pertinent information.

Customer Spotlight – MutualMind

Like many startups seeking to enter and capitalize on the rising social media marketplace, timing is everything. MutualMind was no exception: getting their enterprise social media management product to market in a timely manner was crucial to the success of their business. MutualMind provides an enterprise social media intelligence and management system that monitors, analyzes, and promotes brands on social networks and helps increase social media ROI. The platform enables customers to listen to discussion on the social web, gauge sentiment, track competitors, identify and engage with influencers, and use resulting insights to improve their overall brand strategy.

“Through their social media API, Gnip helped us push our product to market six months ahead of schedule, enabling us to capitalize on the social media intelligence space. This allowed MutualMind to focus on the core value it adds by providing advanced analytics, seamless engagement, and enterprise-grade social management capabilities.”

- Babar Bhatti
CEO, MutualMind

By selecting Gnip as their data delivery partner, MutualMind was able to get their product to market six months ahead of schedule. Today, MutualMind processes tens of millions of data activities per month using multiple sources from Gnip including premium Twitter data, YouTube, Flickr, and more.
Get the full detail, read the success story here.

Expanding Gnip's Facebook Graph API Support

One of our most requested features has long been Facebook support. While customers have had beta access for awhile now, today we’re officially announcing support for several new Facebook Graph API feeds. As with the other feeds available through Gnip, Facebook data is available in Activity Streams format (as well as original if you so desire), and you can choose your own delivery method (polling, webhook POSTing, or streaming). Gnip integrates with Facebook on your behalf, in a fully transparent manner, in order to feed you the Facebook data you’ve been longing for.

As with most services, Facebook’s APIs are also in constant flux. Integrating with Gnip shields you from the ever shifting sands of service integration. You don’t have to worry about authentication implementation changes or delivery method shifts.

Use-case Highlight

Discovery is hard. If you’re monitoring a brand or keyword for popularity (positive or negative sentiment), it’s challenging to keep track of fan pages that crop up without notice. With Gnip, you can receive real-time notification when one of your search terms is found within a fan page. Discover when a community is forming around a given topic, product, or brand before others do.

We currently support the following endpoints, and will be adding more based on customer demand.

  • Keyword Search – Search over all public objects in the Facebook social graph.
  • Lookup Fan Pages by Keyword – Look up IDs for Fan Pages with titles containing your search terms.
  • Fan Page Feed (with or without comments) – Receive wall posts from a list of Facebook Fan Pages you define.
  • Fan Page Posts (by page owner, without comments) – Receive wall posts from a list of Facebook Fan Pages you define. Only shows wall posts made by the page owner.
  • Fan Page Photos (without comments) – Get photos for a list of Facebook Fan Pages.
  • Fan Page Info – Get information including fan count, mission, and products for a list of Fan Pages.

Give Facebook via Gnip a try (, and let us know what you think