SGI’s Big Brain Computer has created a Global Twitter Heartbeat, allowing the supercomputer to analyze the Twitter stream for sentiment and geolocation to create a Twitter heartbeat telling us how the world is feeling based on emotions communicated via Twitter. Not only is this a cool undertaking by the folks at SGI, but we’re proud to announce that it is powered by Gnip’s decahose Twitter stream.
To make this happen, SGI partnered with Kalev H. Leetaru of the University of Illinois and Dr. Shaowen Wang of the CyberInfrastructure and Geospatial Information (CIGI) Laboratory at the University of Illinois at Urbana-Champaign.
This isn’t just some simple stream. The SGI supercomputer analyzes every Tweet to assign location (not just GPS-tagged tweets, but processing the text of the Tweet itself) and tone values, then visualizing the conversation in a heat map that puts Tweet location, Tweet density and tone into a unified geospatial perspective. The entire process from ingestion to data analysis to producing the heat map runs at a speed that allows visualization of a map frame per second.
We were recently at the Sentiment Analysis Symposium with many of the leaders in the sentiment space. Conference presenters spanned a spectrum: academics, engineers and executives involved in research and developing sentiment analysis tools. The event was full of great content, and we wanted to share some takeaways.
It was impressive to see the variety of use cases and solutions that were presented — both signs of a rapidly maturing market. Because of that variety, it was also clear that a one-size-fits-all solution for sentiment analysis of social data is not on the immediate horizon. An overarching message from the event was that the different use cases for sentiment analysis and the types of social content being analyzed demand tailored sentiment tools purpose-built for the job.
I was also struck by just how central social data was to the conversation and how most of the speakers talked about analyzing social data in some form. Sentiment analysis has clearly become important to social data analysis, and vice versa, social data is really attractive to companies and researchers seeking ways to understand the moods and opinions of broad populations of consumers and citizens.
Here are some of the highlights of the business cases that were presented at the conference:
Mood Analysis Informs Market Decisions at ThomsonReuters — Scanning a broad swath of social content and news allows ThomsonReuters to build their MarketPysch Indices, including their “Gloom Index,” which they’ve demonstrated provides a leading indicator for market drops. ThomsonReuters’ Aleksander Sobczyk made an interesting point for our readers: the social content they focus on is long form only (i.e. blogs), and not short form content like Twitter. Developing the depth of insight and certainty essential to their current solution requires a bigger text sample than Twitter provides.
Guiding Product Development at Dell — Through its Social Media Listening Command Center, Dell picks up on customer sentiment and uses it to drill in and learn more. For instance, they picked up negative sentiment about a new Alienware laptop right after a release: units were overheating. The negative sentiment tipped them off, and their product team took action to learn more from users. Through engaging their customers online, they were able to discover an idiosyncrasy in laptop use among power users. When plugged into big auxiliary monitors, users were keeping the laptop screen nearly closed, blocking the exhaust fans. Dell fixed the fan placement and eliminated the problem for future customers. Knowing very specifically what the negative sentiment was about was critical to catching a problem spot among loyal fans and power users.
Understanding Weibo Emotions (了解微博的情感) — Detecting sentiment in other languages with different linguistic structures and use habits can be difficult, which is why Soshio has tried to normalize around universal emotions to help Western brands make sense of what Chinese weibo (microblog) users are saying about their brands. According to Ken Hu, Soshio’s founder, they’re actually having to translate written language “into sound” to address sentiment challenges like written puns that only make sense when pronounced audibly.
These are just a sample of the stories we heard from the companies at the event, and what we hear from our customers building sentiment tools. Some serve customers who need to know about general sentiment (i.e. positive or negative), while are others are interested in specific moods or opinions. Some support tailoring to a specific cultural context or industry vertical, while others help evaluate conversation in a broad, general context. A few are working hard to analyze sentiment in a global context — including native natural language processing for different languages, each with their own norms related to sarcasm, puns, irony, etc. Content format is a major focus as well. Evaluating 140 characters rich with hashtags and URLs on Twitter requires a much different approach than evaluating a WordPress blog post. These are all valid use cases for sentiment analysis, and all come with unique nuances and benefit from different techniques.
Sentiment analysis is a critical category of tool for broader social data analysis, but at the same time, it’s not a monolithic category. If you’re building a product to analyze social data and want to evaluate sentiment, the good news is that you can pick from a wide range of approaches or tools (or maybe even choose more than one?) for sentiment analysis that is suited to the business problem you’re trying to address, and the content you’re trying to analyze.
If you’re interested to learn more about the event and some other interesting developments and business cases in the sentiment analysis world, check out our recent interview with Seth Grimes, the lead organizer for the Sentiment Analysis Symposium.
As we wrote in our last post, Gnip co-sponsored the 2011 Dreamforce Hackathon, where teams of developers from all over the world competed for the top three overall cash prizes as well as prizes in multiple categories. Our very own Rob Johnson (@robjohnson), VP of Product and Strategy, helped judge the entries, selecting the Enterprise Mood Monitor as winner of the Gnip category.
The Enterprise Mood Monitor pulls in data from a variety of social media sources, including the Gnip API, to provide realtime and historical information about the emotional health of the employees. It shows both individual and overall company emotional climate over time and can send SMS messages to a manager in cases when the mood level goes below a threshold. In addition, HR departments can use this data to get insights into employee morale and satisfaction over time, eliminating the need to conduct the standard employee satisfaction surveys. This mood analysis data can also be correlated with business metrics such as Sales and Support KPIs to identify drivers of business performance.
Pretty cool stuff.
The three developers (Shamil Arsunukayev , Ivan Melnikov and Gaziz Tazhenov) from Comity Designs behind this idea set out to create a cloud app for the social enterprise built on one of Salesforce’s platforms. They spent two days brainstorming the possibilities before diving into two days of rigorous coding. The result was the Enterprise Mood Monitor, built on the Force.com platform using Apex, Visualforce, and the following technologies: Facebook API (Graph API), Twitter API, Twitter Sentiment API, LinkedIn API, Gnip API, Twilio, Chatter, Google Visualization API. The team entered their Enterprise Mood Monitor into the Twilio and Gnip categories. We would like to congratulate the guys on their “double-dip” win as they took third place overall and won the Gnip category prize!
Have fun and creative way you’ve used data from Gnip? Drop us an email or give us a call at 888.777.7405 and you could be featured in our next blog.
It’s been a volatile time for the markets the last few weeks. Today especially – the Dow closed down 635 points; S&P, -80; NASDAQ, -175. While there’s no shortage of opinions on how/why the market will/will not recover, one thing is for certain – having the right data to make decisions is more important than ever.
Part of the reason for this is that the markets are clamoring for trends – definitive information on stock trends and market sentiment. Which is why it’s exciting to see how our finance clients are using the Gnip realtime social media data feeds. In a time of increased volatility, our hedge fund (and other buy-side) clients are leveraging social media data as a new source of analysis and trend identification. With an ever-growing number of users, and Tweets, per day, Twitter is exploding, and market-leading funds are looking at the data we provide as a way to more accurately tap into the voice of the market. They’re looking at overall trend data from millions of Tweets to predict the sentiment of consumers as well as researching specific securities based on what’s being said about them online.
While the early-adopters of this data have been funds, this type of analysis is available to individuals as well. Check out some start-ups doing incredible things at the intersection of finance and social media:
Centigage provides analytics and intelligence designed to enable financial market participants to use social media in their investment decision-making process
SNTMNT offers an online tool that gives daily insights into online consumer sentiment surrounding 25 AEX funds and the index itself
For the first time in history, access to (literally) millions of voices is at our fingertips. As the market continues its volatility, those voices gain resonance as a source of pertinent information.
Like many startups seeking to enter and capitalize on the rising social media marketplace, timing is everything. MutualMind was no exception: getting their enterprise social media management product to market in a timely manner was crucial to the success of their business. MutualMind provides an enterprise social media intelligence and management system that monitors, analyzes, and promotes brands on social networks and helps increase social media ROI. The platform enables customers to listen to discussion on the social web, gauge sentiment, track competitors, identify and engage with influencers, and use resulting insights to improve their overall brand strategy.
“Through their social media API, Gnip helped us push our product to market six months ahead of schedule, enabling us to capitalize on the social media intelligence space. This allowed MutualMind to focus on the core value it adds by providing advanced analytics, seamless engagement, and enterprise-grade social management capabilities.”
- Babar Bhatti
By selecting Gnip as their data delivery partner, MutualMind was able to get their product to market six months ahead of schedule. Today, MutualMind processes tens of millions of data activities per month using multiple sources from Gnip including premium Twitter data, YouTube, Flickr, and more.
One of our most requested features has long been Facebook support. While customers have had beta access for awhile now, today we’re officially announcing support for several new Facebook Graph API feeds. As with the other feeds available through Gnip, Facebook data is available in Activity Streams format (as well as original if you so desire), and you can choose your own delivery method (polling, webhook POSTing, or streaming). Gnip integrates with Facebook on your behalf, in a fully transparent manner, in order to feed you the Facebook data you’ve been longing for.
As with most services, Facebook’s APIs are also in constant flux. Integrating with Gnip shields you from the ever shifting sands of service integration. You don’t have to worry about authentication implementation changes or delivery method shifts.
Discovery is hard. If you’re monitoring a brand or keyword for popularity (positive or negative sentiment), it’s challenging to keep track of fan pages that crop up without notice. With Gnip, you can receive real-time notification when one of your search terms is found within a fan page. Discover when a community is forming around a given topic, product, or brand before others do.
We currently support the following endpoints, and will be adding more based on customer demand.
Keyword Search – Search over all public objects in the Facebook social graph.
Lookup Fan Pages by Keyword – Look up IDs for Fan Pages with titles containing your search terms.
Fan Page Feed (with or without comments) – Receive wall posts from a list of Facebook Fan Pages you define.
Fan Page Posts (by page owner, without comments) – Receive wall posts from a list of Facebook Fan Pages you define. Only shows wall posts made by the page owner.
Fan Page Photos (without comments) – Get photos for a list of Facebook Fan Pages.
Fan Page Info – Get information including fan count, mission, and products for a list of Fan Pages.