Putting the Data in Data Discovery – Qliktech & Gnip Partner Up

Gnip is excited to announce that Qliktech is the newest member of our Plugged In partner program. While we partner with many different types of companies – ranging from innovative social analytics products to well-known big data services and software providers – Qliktech is a unique and exciting addition to our program.
Qliktech is discovery software that combines key features of data analysis with intuitive decision-making features, including (to name a few):

  • The ability to consolidate data from multiple sources
  • An easy search function across all datasets and visualizations
  • State-of-the-art graphics for visualization and data discovery
  • Support for social decision-making through secure, real-time collaboration
  • Mobile device data capture and analysis

Our partnership means that joint Qliktech and Gnip clients can easily marry social data with internal datasets to create nuanced visualizations that surface performance indicators and real-time changes that can impact the decisions those clients are making.

To put the powerful capabilities of this new partnership to good use, Gnip will be co-sponsoring a partner hackathon on April 6th at Qonnections– the Qliktech Partner Summit.

Along with HP Vertica and Qliktech, we’ll enable partners to hack on behalf of Medair, Swiss based humanitarian organization that provides support for health, nutrition, water and sanitation, hygiene, and shelter initiatives to countries experiencing natural disasters or emergencies.

A series of recent academic papers have highlighted the usefulness that social media plays in obtaining real-time information following sudden natural disasters. This hackathon will follow in those steps, using Twitter data from during Typhoon Haiyan, which landed in the Philippines on Nov 8th, 2013. Using Gnip’s Profile Geo enhancement, we’ll provide data from the Philippines during that period, allowing other Qliktech partners to experiment with how Medair could leverage this data, within Qliktech, in future situations that require real-time analysis and response.

It will be a great time, but more importantly, will harness the power of the Gnip and Qliktech relationship to accomplish something everyone can be proud of. And that’s a pretty good start to a new partnership!

Data Story: Adam Sadilek on Tracking Food Poisoning With Social Data

Adam Sadilek has done some pretty ground breaking research around social data including tracking food poisoning with social data. When he was a Ph.D. student at the University of Rochester, he led a team that found a correlation between geotagged Tweets about foodborne illnesses that closely aligned with restaurants with poor scores from the health department. Adam is now a researcher at Google, and you can follow him on Twitter at @Sadilek

Tracking Food Poisoning via Twitter

 

1. Where did your interest in identifying health trends on Twitter come from?

First, it was studying how Twitter can predict flu outbreaks and then looking at identifying food poisoning outbreaks too.

We were interested in how much can we learn about our environment by sifting through the vast amounts of day-to-day chatter online. It turns out that machine learning can identify strong signals that can be used to make predictions about individuals as well as venues they visit. For example, in our GermTracker.org project, we predicted how likely is a Twitter user is to become sick based on how many symptomatic people he or she met recently. We leveraged geotags within the Tweets to estimate people’s encounters. In the nEmesis project, our model identified Twitter users who got sick after eating at a restaurant, which enabled us to rank food establishments by cleanliness.

2. Your machine learning can help assign scores to restaurants based on the chances of food poisoning that matches the Health Department based on Twitter data. Is there anyway to make Nemesis data public or as an add-on to services such as Yelp?

There certainly is — Henry Kaut’z group at the University of Rochester is working on an extending GermTracker to capture foodborne illness in real time as well.

3. What are the benefits and disadvantages of using social data over more traditional research on health patterns?

Online social media is very noisy, but significantly more timely. Many months pass between inspections of a typical restaurant. If they get a delivery of spoiled chicken a day after an A+ inspection, it will make their patrons sick anyway. Systems like nEmesis, on the other hand, can detect there is something going on very quickly. The flip side is that it’s hard to be certain on the basis of 140 characters. Therefore, we advocate for a hybrid approach, where inspectors use nEmesis to make better informed decisions. We can replace the current basically random inspections with a more adaptive workflow to detect dangerous venues faster.

4. What else do you think Twitter can tell us about public health?

We did a number of studies, focusing on multiple aspects of our health that can be informed by data mining online social media. Beyond flu and food poisoning, we looked at exposure to air pollution, mental health, commuting behavior, and other lifestyle habits. You can take a look at our publications at http://www.cs.rochester.edu/~sadilek/research/

If you’re interested in additional interviews with people using social data in research, check out our 25 Data Stories to hear about how researchers used social data to track cholera after Haiti’s earthquake. 

TrendyBuzz Helps an Aerospace Leader Use Social Data

Safran is a Paris-headquartered international high-technology leader in the aerospace, defense and security industries—building everything from commercial jet engines to the security systems that detect explosives in luggage. Given that they operate in more than 50 countries with thousands of employees, monitoring the social conversations in their core operating areas can be challenging – but extremely valuable.

We wanted to learn more about how Safran leverages realtime social data through our customer (and new Plugged In partner!), the TrendyBuzz Institut. TrendyBuzz is a French all-media monitoring and analytics platform that works with Gnip to provide access to both the full firehose of realtime and historical Twitter data through our PowerTrack solution. They help customers aggregate and identify relevant content, informing and improving the impact of future social media initiatives. In addition to helping customers monitor realtime data, TrendyBuzz can access historical Twitter data through Gnip for several scenarios, one of which they recently shared with us.

No company can predict every hashtag, mention, or user who will relate to their brand. As good as the TrendyBuzz platform is at surfacing items in realtime, it’s impossible for companies to know every content piece or term they should track, especially given the speed at which users on networks like Twitter create new viral items. Sometimes  a competitor makes an unanticipated announcement or there’s a breaking news story related to your industry, and you miss several hours or days of data. With access to historical data, companies don’t have to worry about missing important mentions or conversations that might impact their brand or industry.

We’re psyched the Plugged In program continues to expand in both number and geography with the addition of Paris-based,TrendyBuzz Institut!

Download the whitepaper on how TrendyBuzz helped Safran! 

TrendyBuzz Institute Dashboard

If Your Brand Falls in the Woods and No One Hears it, Does it Make a Sound?

For a percentage of iMessage users in the world, there was a frustrating period in recent weeks that has caused some angst, to say the least. There’s nothing worse than relying on a system that suddenly starts to fail you. Thankfully, for most, iMessage hasn’t quite hit the classification of enterprise level, but it’s definitely in play as a communication system for a lot of folks in the workplace. For a majority of users though, it just means they missed a note from their spouse about picking up some bread on the way home from work.

When dealing with software, there are differing levels of importance (and opinion) about the quality of your software and how it matters to its users. For folks on the space shuttle, it’s pretty important that software is functioning correctly and is tested without fail. For medical device folks, similar story.  Compare that to Candy Crush, and well, you get the point. What enterprises are finding more and more are the “mission critical” social aspects of their business. When thinking about the evolving industry of social data, it’s important to know that everything your population is saying about you is critical. It is absolutely mission critical to your business to see every Tweet, read every blog mention, see every comment about your business. You never know when the impact of that social activity will become viral. Consider the “United Breaks Guitars” video or the sponsored Tweet by @hvsvn for a complaint about British Airways. Those are both aggressively seeking resolution within social media, but there are countless others that aren’t as creative that can impact your business. The importance of my message really surrounds the old adage of, you don’t know what you don’t know. What happens if you think you’re paying attention to what everyone is saying about your brand, your company (your livelihood?) and you don’t see it?

Social data has completely become part of the critical IT infrastructure that needs to deliver in realtime, all the time. Some of our most successful partners are employing a wide variety of data sources into their product solutions so that they can see the whole picture, and monitor with confidence. At Gnip we refer to this as the social cocktail, well, mostly because we like cocktails, but also because it’s a blend of a varying set of ingredients that make a complete product. As my buddy Dave Heal points out in his latest blog post, they also want reliable, certifiable data that they can count on. You can’t rely on the timeliness of scraped data if you’re building any type of engagement product, so you need that reliability of a firehose in your infrastructure. If a complaint over Twitter goes viral and you don’t get notified in your system because of delays, you can imagine the impact. Sales leaders can’t go into meetings without the knowledge that something went viral about their product and marketing can’t respond to bad PR if they didn’t know something negative hit the wire.

It’s sometimes hard for big corporations to turn the ship on a dime, but it’s impressive to watch them change their product lines to digest streaming data from multiple publishers, adjust their rules on the fly, have the vision to ask for historical insight into social data so they can plan forward, and help drive industry change through the Big Boulder Initiative. When you get a phone call from your customer asking if you can help them build better signaling in their systems to alert them of the one activity in the 4 billion social activities Gnip delivers a day, it puts that missing text about picking up bread in perspective.

Social Data for Social Good in the Public Sector

Two weeks ago, I was invited to an event hosted by FEMA and White House Office of Science and Technology Policy (OSTP) with more than 80 top innovators across commercial and government sectors to discuss new ways to improve disaster response and recovery efforts. Gnip was taking part in this event because we and others believe that social data can play a critical role in natural disaster recovery and can provide some of the best feet on the ground reporting.

But we don’t think social data for the social good begins and ends with natural disasters. In fact, we think it’s only the tip of the iceberg. We’ve seen social data being used to track the spread of cholera in Haiti after their earthquake in 2010. Because official reporting methods in epidemiology often take two weeks to obtain official results while social data provides immediacy. Social data is even used to identify food poisoning outbreaks and where food borne illnesses are most likely to occur.

Uses are even expanding in ways that you wouldn’t necessarily expect including using social data to understand the effects and reach of governmental programs. Social data can provide insight into segments of the population that may have been traditionally hard to survey, such as veterans or younger generations that may not be inclined to reply to survey requests. Social data could even be used to identify the economic impact of government-based initiatives or events, through analyzing “check-ins” on platforms like Foursquare to understand how local economies may be positively affected by government based initiatives.

Recognizing that social data is still relatively new to the public sector, we put together a whitepaper to discuss what social data is, how it can be used, and the current implications of social data. While this whitepaper is primarily intended for those just beginning to look at social data, you can expect more content coming your way from Gnip about how to effectively leverage social data in the public sector.

 Download the whitepaper.

The 4 Ways People Use Social Media in Natural Disasters

Earlier this year I interviewed Brooke Fisher Liu with the University of Maryland about her research around how people used social media during natural disasters. She broke it down as this:

During natural disasters people tend to use social media for four interrelated reasons: checking in with family and friends, obtaining emotional support and healing, determining disaster magnitude, and providing first-hand disaster accounts.

I was reflecting upon this interview and how much I saw these four scenarios during the recent Boulder flood, which many in the community are still suffering from the aftermath. The Boulder Flood provided the perfect way to look at how people use social media in natural disasters.

1) Checking in with family and friends
People were using social media to let their friends and loved ones know they were safe, what their current status was, and offering (or soliciting) help. Across the community, there were people offering help to those who needed it whether they be strangers or family. For myself, I tried posting daily updates on Facebook, so I could keep people up to date and then focus on figuring out cleanup.

Social Media During Boulder Flood

2) Obtaining emotional support and healing

Twitter, Facebook and Instagram provided enormous amounts of emotional support along with concrete offers of help. The hashtag #BoulderStrong offered great encouragement to those suffering losses.

3) Determining disaster magnitude
Many of the people following the #BoulderFlood hashtag were following some of the official accounts including the @dailycamera (Boulder newspaper), @mitchellbyar (reporter at the Daily Camera), @boulderoem (Boulder Office of Emergency Management), @bouldercounty. As a community we were looking to hear about our homes, our schools, our neighbors and how they fared. We were looking to understand just how damaged our community was and how long it took to recover. One of the more interesting aspects I saw was people focused on determining road closures. While Boulder OEM was publishing their reports, many people were determining how to get in and out of Boulder. I can’t help but think how social data can represent more accurate information and real-time reporting than official sources.

4) Providing first-hand disaster accounts
While newspapers shared collections of horrifying images of the damages happening among Boulder floods, we were looking to our contacts on social media for first-hand accounts too. We were using our networks on Twitter to confirm what we were hearing online or even what we thought we were seeing.

Boulder Flood on Instagram

Our CTO Jud Valeski posted many shots of the flood on on his Instagram account that were picked up on the media. In fact, Michael Davidson at the Xconomy even wrote an article “Gnip Co-founder Jud Valeski on His Flood Shots Seen Around the World.”

The one aspect that really seemed to be missing from Brooke Fisher Liu’s research was the coordination that was taking across social media. People were offering to help strangers, organize cleanups, share tools, share bottled water and spare bedrooms, solicit donations, check on other people’s houses and a thousand other ways. Resource sharing was one of the major ways that social media played a role in the Boulder flood.

Continue reading

Tweeting in the Rain, Part 3

(This is part 3 of our series looking at how social data can create a signal about major rain events. Part 1 examines whether local rain events produce a Twitter signal. Part 2 looks at the technology needed to detect a Twitter signal.) 

What opportunities do social networks bring to early-warning systems?

Social media networks are inherently real-time and mobile, making them a perfect match for early-warning systems. A key part of any early-warning system is its notification mechanisms. Accordingly, we wanted to explore the potential of Twitter as a communication platform for these systems (See Part 1 for an introduction to this project).

We started by surveying operators of early-warning systems about their current use of social media. Facebook and Twitter were the most mentioned social networks. The level of social network integration was extremely varied, depending largely on how much public communications were a part of their mission. Agencies having a public communications mission viewed social media as a potentially powerful channel for public outreach. However, as of early 2013, most agencies surveyed had minimal and static social media presence.

Some departments have little or no direct responsibility for public communications and have a mission focused on real-time environmental data collection. Such groups typically have elaborate private communication networks for system maintenance and infrastructure management, but serve mainly to provide accurate and timely meteorological data to other agencies charged with data analysis and modeling, such as the National Weather Service (NWS). Such groups can be thought of being on the “front-line” of meteorological data collection, and have minimal operational focus on networks outside their direct control. Their focus is commonly on radio transmissions, and dependence on the public internet is seen as an unnecessary risk to their core mission.

Meanwhile, other agencies have an explicit mission of broadcasting public notifications during significant weather events. Many groups that operate flood-warning systems act as control centers during extreme events, coordinating information between a variety of sources such as the National Weather Service (NWS), local police and transportation departments, and local media. Hydroelectric power generators have Federally-mandated requirements for timely public communications. Some operators interact with large recreational communities and frequently communicate about river levels and other weather observations including predictions and warnings. These types of agencies expressed strong interest in using Twitter to broadcast public safety notifications.

What are some example broadcast use-cases?

From our discussions with early-warning system operators, some general themes emerged. Early-warning system operators work closely with other departments and agencies, and are interested in social networks for generating and sharing data and information. Another general theme was the recognition that these networks are uniquely suited for reaching a mobile audience.

Social media networks provide a channel for efficiently sharing information from a wide variety of sources. A common goal is to broadcast information such as:

  • Transportation Information about road closures and traffic hazards.

  • Real-time meteorological data, such as current water levels and rain time-series data.

Even when an significant weather event is not happening, there are other common use-cases for social networks:

  • Scheduled reservoir releases for recreation/boating communities.

  • Water conservation and safety education.

[Below] is a great example from the Clark County Regional Flood Control District of using Twitter to broadcast real-time conditions. The Tweet contains location metadata, a promoted hashtag to target an interested audience, and links to more information.

— Regional Flood (@RegionalFlood) September 8, 2013

So, we tweet about the severe weather and its aftermath, now what?

We also asked about significant rain events since 2008. (That year was our starting point since the first tweet was posted in 2006, and in 2008 Twitter was in its relative infancy. By 2009 there were approximately 15 million Tweets per day, while today there are approximately 400 million per day.) With this information we looked for a Twitter ‘signal’ around a single rain gauge. Part 2 presents the correlations we saw between hourly rain accumulations and hourly Twitter traffic during ten events.

These results suggest that there is an active public using Twitter to comment and share information about weather events as they happen. This provides the foundation to make Twitter a two-way communication platform during weather events. Accordingly, we also asked survey participants if there was interest in also monitoring communications coming in from the public. In general, there was interest in this along with a recognition that this piece of the puzzle was more difficult to implement. Efficiently listening to the public during extreme events requires significant effort in promoting Twitter accounts and hashtags. The [tweet to the left] is an example from the Las Vegas area, a region where it does not require a lot of rain to cause flash floods. The Clark County Regional Flood Control District detected this Tweet and retweeted within a few minutes.

 

Any agency or department that sets out to integrate social networks into their early-warning system will find a variety of challenges. Some of these challenges are more technical in nature, while others are more policy-related and protocol-driven.

Many weather-event monitoring systems and infrastructures are operated on an ad hoc, or as-needed, basis. When severe weather occurs, many county and city agencies deploy a temporary “emergency operations centers.” During significant events personnel are often already “maxed out” operating other data and infrastructure networks. There are also concerns over data privacy, that the public will misinterpret meteorological data, and that there is little ability to “curate” the public reactions to shared event information. Yet another challenge cited was that some agencies have policies that require special permissions to even access social networks.

There are also technical challenges when integrating social data. From automating the broadcasting of meteorological data to collecting data from social networks, there are many software and hardware details to implement. In order to identify Tweets of local interest, there are also many challenges in geo-referencing incoming data.  (Challenges made a lot easier by the new Profile Location enrichments.)

Indeed, effectively integrating social networks requires effort and dedicated resources. The most successful agencies are likely to have personnel dedicated to public outreach via social media. While the Twitter signal we detected seems to have grown naturally without much ‘coaching’ from agencies, promotion of agency accounts and hashtags is critical. The public needs to know what Twitter accounts are available for public safety communications, and hashtags enable the public to find the information they need. Effective campaigns will likely attract followers using newsletters, utility bills, Public Service Announcements, and advertising. The Clark County Regional Flood Control District even mails a newsletter to new residents highlighting local flash flood areas while promoting specific hashtags and accounts used in the region.

The Twitter response to the hydrological events we examined was substantial. Agencies need to decide how to best use social networks to augment their public outreach programs. Through education and promotion, it is likely that social media users could be encouraged to communicate important public safety observations in real time, particularly if there is an understanding that their activities are being monitored during such events. Although there are considerable challenges, there is significant potential for effective two-way communication between a mobile public and agencies charged with public safety.

Special thanks to Mike Zucosky, Manager of Field Services, OneRain, Inc., my co-presenter at the 2013 National Hydrologic Warning Council Conference.

Full Series: 

mBLAST Helps Brands Grasp the Trajectory of a Social Conversation

Officially announcing that mBlast is now a Plugged In to Gnip partner (which we’ve just done in this sentence) is especially exciting because of a particular kind of analysis they specialize in – called resonance tracking – that they’ll be able to do on this announcement!

As a leader in the social media analytics space that is helping companies understand the trajectory of social conversations, it makes sense to include mBLAST among Gnip’s growing group of partners. Companies who are Plugged In are driving innovation in the social media ecosystem through their products and emphasis on incorporating multiple social media sources into their analysis. One of the ways mBLAST is defining social media analytics is through their development of resonance tracking, a variation on influencer tracking with some distinctly different capabilities. Effective resonance tracking necessitates the monitoring of multiple different platforms, which piqued our interest on the topic.

Wouldn’t we all like to stop a rumor in it’s tracks — or at least have a good idea of who started it? Knowing how a story perpetuates across the social sphere and who is responsible for influencing those moves is a valuable tool for companies and brands. If you can pinpoint which of your fans, or critics for that matter, is talking about you the loudest, with the most influence, and in which channels, you can directly address the message.

mBLAST has measured resonance within the analytics applied to social, blog and media data tracked in their mPACT platform for literally millions of stories. All of this tracking and measurement has lead to some pretty interesting observations which mBLAST will be releasing in a whitepaper in September titled, “Social Intelligence: The Role of Resonance”. The more we learned about resonance tracking and the many ways it can be applied to everyday customer service, marketing, and public relations scenario, the more we couldn’t wait to spread the word on just how relevant a tool it is. So we put together a short summary of their resonance tracking whitepaper to give you a sneak peek — focusing on how resonance differs from more traditional influencer tracking and why it is a key part of social media analytics.

mBlast Resonance

So, how does resonance differ from traditional influencer tracking? A couple of ways, but probably the most significant is that it allows companies to identify which individuals are the most influential in a specific channel for a specific story and how that story is jumping from platform to platform. For example, a story about a CPG brand starts on a blog post and then hops from the blog post to Twitter and then to Facebook, etc. Perhaps the conversation is false or the product contents are reflected inaccurately. Whatever the case may be, the CPG company likely wants to engage with the people, and in the channels, where it will do the most to shape the conversation and protect the company’s reputation. Both can be done using resonance tracking.

We’re psyched to have mBLAST in Plugged In and look forward to working together. And we’ll be excited to track the resonance on this announcement!

If resonance tracking gets you excited or you want to know more, check out this webinar on the topic on September 12.

Data Stories: Stefan Papp of Blab on Predictive Social Intelligence

Data Stories is about telling cool stories about the amazing ways that social data is used. This week we’re interviewing Stefan Papp of Blab, a Gnip customer, about their predictive social analytics that are able to help customers understand the directions conversations online are headed in 24, 48, and 72 hour time frames. I absolutely loved this concept and wanted to understand more about how it worked.
Stefan Papp of Blab
1. Why did Blab decided to take a different approach instead of doing social media monitoring?

Typical social media monitoring tools use keyword-based searches – you put in your keyword and the tools return all historical results matching that keyword.  The catch is that you will only find the insights you are looking for. Our approach is to listen to the entire conversation as it organically evolves, allowing you to discover much more of what your users are discussing — including the unexpected insights you probably would not have thought to search for. Finding those non-obvious but vibrant discussions gives companies whole new opportunities to engage with their target audiences.

2. What makes social data a good source in predicting behavior?

When you monitor social data you have a unique opportunity to listen to the unfiltered personal voices of millions of users. This offers insight into not just what is trending in social but what people are really thinking and feeling; their beliefs and true opinions. Standard behavioral prediction methods use things like focus groups and polls. But these approaches have long been known to produce skewed data – both consciously and subconsciously, users tend to tailor their responses to what they think the “best” response should be. Social data has none of this user bias and as a result is an excellent source of raw unfiltered intelligence.

3. How do you think Blab fits into the trend of real-time marketing?

The challenge that real-time marketing presents is creating on-topic content and getting that content out the door before the conversation is old news. Blab not only monitors current conversations in real time but predicts which of those conversations are going to be hot 72 hours from now. This takes the reaction out of real-time marketing and for the first time gives control to the brand. Blabenables you to put relevant content in front of an engaged audience at the right time – before a conversation has grown so big that your voice can’t be heard. You can also strike a chord by engaging your audience in those non-obvious conversations happening right now that you would not have thought to join.

4. What’s some surprising findings companies have found when using Blab?

One of the major insights that companies have gained from using our Predictive Social Intelligence engine is how conversations evolve online. So many companies only think about Facebook and Twitter when they think social, but social is so much more than that. Blab enables you to watch a conversation evolve across the entire spectrum of social networks. You can follow a conversation that begins with a YouTube post, which then drives a larger conversation on Twitter, and ends up being predicted to explode on Tumblr 72 hours from now.  Without a holistic view like this companies can be led to believe that a conversation has ended when in fact it continues vibrantly on another untracked social platform.

Another interesting finding is that our clients get an unadulterated view of their standing in social discourse. One client, a global technology concern, was surprised and chagrined to find that while there was lively discussion around their competitors, there was no discussion at all about them in their area of expertise. As humbling as that was, it became a call to action and fueled enthusiasm for engaging more effectively. Blab helped that company discover a negative and use that knowledge to improve their position.

5. Clearly, Blab has huge implications when it comes to crisis communications. One of the things that has amazed me about social media is that you don’t need a huge following to start a fuss about a brand. How does Blab separate the wheat from the chaff when it comes to determining conversations that might spike?

We use two unique methods to identify truly relevant conversations and to make accurate predictions on when a conversation will spike.  First, we throw NLP out the window and use a proprietary contextual classification approach to find the conversations that are related to a given topic. Rather than filtering out words like “got” we let our engine tell us if a term or phrase should be included. And guess what? There is a thriving “got” conversation among people who are passionate about “Game of Thrones.” We embrace acronyms, slang, abbreviations and sarcasm in a language agnostic manner (from Klingon to emoticon). The result is that we give you a picture of the whole conversation, unfiltered yet relevant. The second unique method is our proprietary approach to determining which conversations will spike or cool down. As conversations ebb and flow on the social canvas they establish patterns of historical facts. We’ve discovered that regardless of the topic, these patterns tend to repeat themselves. So while there are a huge number of them, the universe of conversation patterns is not infinite. When we see a familiar pattern we can predict, and often with high confidence, how a conversation will progress up to 72 hours into the future.

Taken together, Blab gives brands the ability to find potentially troubling conversations as they emerge; to determine if action is important by predicting which conversation is likely to take off; to engage that conversation or take other remedial action; and to know if the engagement is having an impact by watching to see if the prediction of growth turns into a prediction of decline for the troubling conversation.

Continue reading

Social Data Mashups Following Natural Disasters

Exactly what role can social data play in natural disasters?

We collaborated with Gnip Plugged In partner, Alteryx, to do data mashups around FEMA relief related to Hurricane Sandy for a recent presentation at the Glue Conference. Alteryx makes it easy for users to solve data analysis problems and make decisions.

Gnip and Alteryx created a data mashup for Hurricane Sandy using six different data sources showing what kinds of tools we can create after natural disasters. Starting with mapping from TomTom, the data mashup also included data about businesses from Dun & Bradstreet, demographic data from Experian, registrations from FEMA, geotagged articles from Metacarta, and geotagged Tweets from Gnip.  We concentrated on FEMA efforts and reactions during spring 2013.

This kind of data mashup allows us to drill down into multiple aspects of evacuation zones. One of the easiest examples of this mashup is the ability to see what services and resources are available from businesses (from Dun & Bradstreet) while complimentary official efforts are organized.

FEMA Hurricane Sandy Maps

Or it can help prioritize which areas to assist first by mashing population densities with registrations from FEMA.

FEMA Hurricane Sandy Registrations

FEMA Hurricane Sandy Density Map

Using geotagged social data from Twitter is another way to identify areas that need help, as well as monitor recovering areas. Combining sentiment analysis with Tweets provides instant feedback on the frustrations or successes that constituents are feeling and seeing.

Hurricane Sandy Social Tweets

We think this type of data mashups with Alteryx is just the beginning of what is possible with social data. If you have questions or ideas for data mashups, leave it in the comments!