Author Archive
In April 2010, the U.S. Library of Congress and Twitter announced that Twitter will donate its public Twitter conversation data to the Library of Congress archives, making it available for researchers to read and preserving it for generations to come. This was an historic moment: for the first time, a communication channel in the social media space had been identified as a relic of our culture, emblematic of the role technology has come to play in our interactions with one another.
Today we’re honored to announce that Gnip is playing a key role in the this project by delivering past and future Twitter data to the Library of Congress for historic preservation in the archives.
On a technical level, the magnitude of this project is enormous: with over a billion Tweets being generated weekly, the scale of data delivery involved in this project is historic. We’ve learned a lot from this project and it is clear that some of the outcomes from this project will benefit Gnip’s future product offerings. We look forward to sharing some of the technical details in the future at the appropriate time.
As a company, we couldn’t be more pleased to be a part of this momentous project. We’re eager to see what kind of research develops from the public archive made accessible by the Library of Congress. We’ve watched with awe as news of world-changing events like the Arab Spring and Osama Bin Laden’s death have flowed through our servers, and now we’re proud to be part of ensuring these historic events will be preserved for our posterity.
The archive of Twitter data which Gnip delivers to the Library of Congress will cater to the needs of researchers who wish to use limited amounts of public Twitter data for non-commercial purposes. Gnip will continue to serve those seeking realtime data, full-coverage data, and commercial use cases.
Thanks to Twitter and the Library of Congress for making this announcement possible. We couldn’t be more excited to be working with you both.
The response to the commercial Twitter streams we’ve made available has been outstanding. We’ve talked to hundreds of companies who are building growing businesses that analyze conversations on Twitter and other social media sites. As Twitter’s firehose continues to grow (now over 110 million Tweets per day), we’re hearing more and more requests for a way to filter the firehose down to the Tweets that matter.
Today, we’re announcing a new commercial Twitter product called Power Track. This is a keyword based filter of the full firehose that provides 100% coverage over a stream that you define. Power Track customers no longer have to deal with polling rate limits on the Search API and volume limits on the Streaming API.
In addition to keyword based filters, Power Track also supports boolean operators and many of the custom operators allowed on Twitter Search API. With Power Track, companies and developers can define the precise slice of the Twitter stream they need and be confident they’re getting every Tweet, without worrying about volume restrictions.
Currently we support operators for narrowing the stream to a set of users, matching against unwound URLs, filtering by location, and more. We’ll continue to add support for more ways for our customers to filter the content relevant to them in the future. Check the documentation to see the technical details of these operators and more.
Gnip is here to ensure the enterprise marketplace gets the depth, breadth, and reliability of social media data it requires. Please contact us at info@gnip.com to find out more.
Today we’re excited to announce the integration of the Google Buzz firehose into Gnip’s social media data offering. Google Buzz data has been available via Gnip for some time, but today Gnip became one of the first official providers of the Google Buzz firehose.
The Google Buzz firehose is a stream of all public Buzz posts (excluding Twitter tweets) from all Google Buzz users. If you’re interested in the Google Buzz firehose, here are some things to know:
- Google delivers it via Pubsubhubbub. If you don’t want to consume it via Pubsubhubbub, Gnip makes it available in any of our supported delivery methods: Polling (HTTP GET), Streaming HTTP (Comet), or Outbound HTTP Post (Webhooks).
- The format of the Firehose is XML Activity Streams. Gnip loves Activity Streams and we’re excited to see Google continue to push this standard forward.
- Google Buzz activities are Geo-enabled. If the end user attaches a geolocation on a Buzz post (either from a mobile Google Buzz client or through an import from another geo-enabled service), that location will be included in the Buzz activity.
We’re excited to bring the Google Buzz firehose to the Social Media Monitoring and Business Intelligence community through the power of the Gnip platform.
Here’s how to access the Google Buzz firehose. If you’re already a Gnip customer, just log in to your Gnip account and with 3 clicks you can have the Buzz firehose flowing into your system. If you’re not yet using Gnip and you’d like to try out the Buzz firehose to get a sense of volume, latency, and other key metrics, grab a free 3 day trial at http://try.gnip.com and check it out along with the 100 or so other feeds available through Gnip’s social media API.