Historical PowerTrack Requests, Now Faster Than Ever

The Twitter Data Product Team is excited to share an update with you around recent enhancements to our Historical PowerTrack offering. In an effort to improve our customer experience for historical data requests, we’ve made substantial technology investments to reduce processing times as well as to support future adoption and usage patterns.

Historical data jobs have always been processed as fast as our infrastructure allowed, and now they are significantly faster than ever before. You may be asking yourself, “So just what does this mean for my business?” Well, here are some data points that should help to put these improvements in perspective:

  • A 1-year historical data job that previously took in the neighborhood of 144 hours to complete was recently processed in just 5 hours.
  • A 2-year historical data job that previously took nearly 288 hours to complete was recently processed in just 8 hours.

We no longer recommend breaking historical jobs down into smaller pieces to process the data faster, thereby affording our customers one more level of improved efficiency.  Your historical jobs will now actually process faster if they remain intact as a singular request and this should make the job management process easier for you and your team.

One final area of improvement benefiting your business is our more accurate estimates around job processing times. While these predictions will certainly remain just that, “estimates”, the deviation from the eventual processing times will be greatly reduced in most cases.

If you have any further questions around these improvements, feel free to reach out to our Product Team at info@gnip.com.

What Our Partners Are Saying About Plugged In To Gnip

Yesterday we launched Plugged In To Gnip, our partner program to recognize our partners providing comprehensive, reliable and sustainable social data. Our COO Chris Moody wrote about what Plugged In To Gnip means for us yesterday, but we wanted to share what our partners were saying about the program and what it means to them.

FirstRain is Proud to be Plugged In To GNIP!

And critical to the success of FirstTweets was, first, to have comprehensive, authorized and reliable access to the Twitter firehose—and for this FirstRain relies on Gnip. Gnip is the world’s largest and most trusted provider of social media data, and in addition to being a fantastic team of innovative social data ninjas, they’ve been terrific partners in our quest to transform the way corporate enterprises drive revenue through Web and Social customer intelligence and analytics beyond social monitoring.

Netbase and Gnip: Maintaining Streams of Quality Content

When I describe my job, I borrow from my childhood near Pittsburgh and explain that if NetBase were a steel mill, I’d be the guy making sure we have trains and barges bringing in high-quality iron ore and the other raw materials we need to make high-quality steel. Gnip, whose Plugged In program we are proud to have joined from its launch, operates one of the biggest, fastest “trains” delivering raw data to us. Out of the roughly 50 million social media postings we analyze and index each day, the majority come from Twitter and we rely on Gnip’s ability to operate a TGV-scale delivery platform. The TGV, if you don’t know that term, is France’s Tres Grande Vitesse, the fastest railway in the world. We also depend on Gnip for Disqus blog comments and much more. But they’re not just filling up buckets for us – Gnip adds value.

Although the TGV has a switching system, its scale is microscopic compared with the “switching” we need from our content suppliers. Although in many cases we tell our supplier “Give us everything you have,” we are able to rely on some of them, including Gnip, to filter the content to ensure that our customers receive 100 percent of the relevant tweets, posts and articles and as little spam and junk as possible. That makes them more a supplier of raw material; it makes them a partner. We also rely on them to put in place the technical and business processes to ensure that we are in compliance with licensing and other requirements created by the owners and distributors of the data they supply – and to help our customers do the same. Just keeping up with those processes and requirements is a challenge as the social media ecosystem evolves and matures.

Gnip Touts Big Data Partnerships

“Social data is a new and incredibly exciting dataset that our customers are beginning to leverage within IBM InfoSphere BigInsights and other IBM products,” said Bruce Weed, IBM big data program director, in a statement. “Via the Plugged In To Gnip business partner program, we make it incredibly easy for them to access that social data.”

Union Metrics is Proud to be Plugged In To Gnip

What else does this partnership mean? It means a number of things, but what’s most important to you – our valued customers – is that all our social analytics products are built on the highest quality, most comprehensive and reliable social data. We are committed to bringing you the data that you need to be successful with social media and our partnership with Gnip helps make that possible; full coverage, high-quality data is at the heart of all our analytics solutions.

Infochimps Plugged In To Gnip

Getting a handle on the immense volume of data produced by the social networks provided by Gnip often requires a sophisticated data infrastructure for the processing and control of feeds.  As a partner in providing solutions to customers needing to extract insight from this treasure trove of data, Infochimps can help by setting up customers with a best in class data platform for refining and working with Gnip’s feeds.

Joining Forces: BrandWatch Collaboration With Gnip 

As a Gnip partner, the social data we access through our Twitter Firehose integration is certified as the most comprehensive, reliable and sustainable source of social data. So in other words, as a Brandwatch client, this has always assured you that the insights you receive are based on the most complete access available for this data – period.

Clarabridge Partners with Gnip to Bring Real-Time Monitoring of Social Media Data to Customer Experience

We recognize that consumers are increasingly looking to social media as a critical means of communication, and businesses need to stay on top of consumer feedback in real time,” said Sid Banerjee, CEO, Clarabridge. “As a member of Gnip’s Plugged In program, our customers will be able to collect, analyze, operationalize and measure social media data in real time, as well as engage directly back with the customer. We are excited to partner with Gnip and enable Clarabridge customers to forge real and direct relationships with their consumers and elevate the customer experience they provide.

Greenplum Plugs In To Gnip

With data from social media platforms accounting for an ever-increasing amount of the Big Data deluge, collecting that data becomes an ever-moving target. As social networks proliferate, so do the respective services’ APIs and access policies. Social media aggregator Gnip aims to simplify the process, allowing businesses to focus on reaping social data insight, rather than tracking which services are hot, which are not, and which have changed their API policies.

Coinciding with its OpenChorus initiative, Greenplum announced a partnership with Gnip recently to make the company’s APIs accessible through the Greenplum platform. As Gnip details in its new Plugged In To Gnip campaign, users of Greenplum Chorus, UAP, Database, and HD can now easily access data from the Twitter, Tumblr, WordPress, StockTwits and Disqus firehoses.

uberVU is now Plugged In to Gnip

As an inaugural member of Gnip’s Plugged In program, this certifies that we are accessing social data, like Twitter, through the most comprehensive, reliable and sustainable source of social data. Now, our customers can look to this certification as an assurance that the insights they receive are based on the most complete access available for this data – period. Yes, ladies and gentlemen, we have the firehose.

UberVu Drinking from the Gnip Firehose

Get the Disqus Firehose With New Filtering Options

In February, we announced that the full Disqus firehose of public comments is available through Gnip. Our customers love the conversations in Disqus, but have asked for tools to filter the stream so they receive only the conversations they want. Today, we’re announcing our new Disqus PowerTrack offering. Similar to our Twitter PowerTrack product, Disqus PowerTrack offers powerful filtering so customers can filter the full Disqus firehose of public comments to extract the specific conversations they’re looking for. With over 500,000 comments created each day on Disqus, there are a huge range of conversations taking place and you don’t want to miss the ones about your brand or products.

With Disqus PowerTrack, you have a wide array of filtering options. You can filter for specific keywords. You can constrain that filter to specific websites. Or you can look for just the mentions that have links. So, if you’re looking for brand mentions of Apple, you can track conversations about the iPhone or brand mentions in general. You can also monitor for comments mentioning the iPhone that have links in them so you can understand what online stores are being promoted along with your products. See the full list of Disqus PowerTrack Operators in our documentation.

To see the power of the full Disqus firehose, check out this graph showing all mentions of Apple on Disqus. On a normal weekday, there are almost 10,000 comments about Apple. For big events, like WWDC, you see a spike to almost 40,000 comments per day. That’s a lot of conversations.

Get Disqus Firehose with New Filtering Options

We’re big proponents of the conversations that happen in comments, and we’re committed to making it easier for companies to understand and be able to participate. Our new Disqus PowerTrack makes it easier than ever to understand the types of conversations happening in comments.

If you have any questions about the new Disqus capabilities, please contact your sales rep or our sales team at info@gnip.com.

Rich Comment Data from Disqus Now Available Through Gnip

Imagine going to a dinner party and listening to the first thing each person said. You’d learn a few things, but you’d miss out on the meat of the conversation that happens in the give and take of the dialogue.

In the world of online public social conversation, blog posts are the monologue and comments provide the dialogue. Each is valuable on their own, but to see the complete picture, you need both. Conversations happen in comments, and it has been a huge struggle for brands to be able to keep up with comments to fill in their understanding of this key piece of the conversation.

I’m excited to announce that we’re making it easier to access these public conversations with the addition of the full Disqus firehose to our publisher portfolio. As the largest third-party commenting platform in the world with 70 million commenter profiles, the Disqus firehose provides coverage of more than 500,000 comments every day, spanning almost every topic imaginable and reaching over 700 million readers each month.

Comments last forever. They appear in search results and remain part of the discussion long after the day they were written.  With their staying power and depth of discussion, the commenting ecosystem provides an important — and different — social signal. Disqus further embodies this by allowing users to react to others’ comments with up or down “votes” creating significantly more engagement. The 2 million “votes” on Disqus each day provide insight into what comments are generating the most reaction.

Our Disqus API partnership provides authorized access for the first time ever to full firehoses of discussion content and interaction across the Disqus network. To the extent that any of this data has been available before, it’s been provided by technologies like content scraping/crawling that pulled pieces of the discussion, but did not guarantee full coverage in real time on a publisher-safe, consistent and reliable basis. Because this new service is being provided via a direct partnership with Disqus, with Gnip’s full firehose, you get low-latency streams that provide full coverage with the support of the publisher to ensure the availability of the data over the long term.

We’re thrilled to have data from Disqus available on our platform and can’t wait to see the amazing ways that our customers are able to apply it to their businesses. Email us at sales@gnip.com to learn more about Disqus and set up a trial so you can see the data for yourself.