Gnip Inc.

  • Product
    • Product Overview
    • Success Stories
    • Documentation
  • Industries
    • Social Media Monitoring
    • Finance
    • Business Intelligence
    • Government
    • Ad Tech
    • Other
  • Data Sources
  • Company
    • About
    • News
    • Careers at Gnip
    • Plugged In To Gnip
  • Blogs
    • Company
    • Engineering
  • Contact

888-777-7405

Company Blog

Tag Archives: metadata

Post navigation

← Older posts

Top 10 Reasons Gnip Data Collectors Outperform Direct API Connections

Posted on June 23, 2011 by Samantha Quist, Marketing

0 comments

Sometimes we’re asked why it makes sense to access social data from Gnip and not through direct access to the publicly accessible APIs. (We usually get this question from people who have never tried to access data from various social media APIs; those who have tried it understand how tedious and time-intensive data collection is and they can’t wait to hand their social data collection over to Gnip to manage for them.)

So, if you’ve never tried collecting data from multiple social media APIs at once… why would you use Gnip instead of connecting directly to the publicly accessible APIs? Here are 10 of the reasons…

#10 – Customer Support
When you use most public APIs, development teams are often busy, so they’re tough (if not impossible) for most developers to reach with questions. At Gnip, we actually want to talk to you. We offer enterprise-level support so clients can contact us at all odd hours and receive a thoughtful, thorough response. And we work closely with a variety of sources, so we can reach out to them directly if necessary.

#9 – Reliability
Public APIs are not contractually guaranteed; data availability and access levels may change at any time, with or without warning to users. Many businesses worry about building their businesses on data that doesn’t come with contractual agreements. When you subscribe to premium data such as the premium Twitter feeds available through Gnip, we provide you with a formal agreement. This locks in your access level, price, service, and terms of use for the duration of your agreement.

#8 – Rate limit recommendations
Instead of having to figure out rate limits for the various sources on your own, Gnip can recommend rate limits based on our own extensive experience with the various APIs.

#7 – Delivery in your protocol of choice: never poll for data again
A lot of developers think polling for data is tedious… and unfortunately, most APIs are polling-based. So if you go to the sources directly, you have to poll their servers for the data. By using Gnip, you can choose between polling for your data or to having your data streamed to you.

#6 – New feed setup in seconds
Without Gnip, it can take many hours (or days) of a developer’s time to set up a new API connection, parse the new feed, and start bringing data into your system. With Gnip, it can take as little as 30 seconds and no dev effort at all to start consuming the data.

#5 – Gnip is the only source for some data
Gnip can offer access to some data that’s not available from any other source (eg. Premium Twitter volume-based feeds like our Decahose and Halfhose).

#4 – Established premium data partnerships
Established partnerships with premium data publishers (Twitter, BackType, WordPress, etc.) make it quick and easy for Gnip customers to test and add premium data feeds.

#3 – Established relationships with all publishers
Because we manage data collection for customers all day every day, we’re among the earliest to know when API changes happen and the fastest to make any necessary changes to keep your data flowing.

#2 – APIs are generally hard to manage
Publishers change their APIs sometimes. Some APIs change frequently and without warning or documentation (cough, Facebook, cough) while others change less frequently. But no matter what, change is inevitable. Gnip manages your social media data delivery over time so you can keep your data flowing smoothly and reliably with minimal effort.

#1 – Enrichments
A variety of enrichments, or added metadata and features, come included with feeds delivered through Gnip data collectors. Some of the most popular enrichments include format normalization across sources (so you only have to write one parser for all your social media data), Klout Score inclusion (currently available for premium Twitter feeds), and language detection and filtering via a proprietary Gnip algorithm. We add enrichments all the time, so look for lots more to come.

We think Gnip is pretty cool (yes, we’re biased)… but even we know that Gnip isn’t for everyone. If you only need 1 feed from 1 source, the data you need is available through a publicly accessible API, you have an engineer who can monitor and optimize your data consumption regularly, and you’re certain that you will never need any other feeds forever and ever, then Gnip probably isn’t the right choice for you.

But if you’d like to ensure you’re receiving top-quality premium data access without requiring your engineering team to invest lots of time in data collection, we’d like to invite you to give Gnip a try. We’ve got lots of happy customers already and we just might prove valuable to you, too.

Categories: Product, Technology

Tagged: api, api rate limits, apis, backtype, data collection, data collectors, data delivery, data enrichments, data feeds, data filtering, data normalization, data publishers, data sources, data streaming, decahose, format normalization, gnip, halfhose, klout, klout scores, language detection, language filters, metadata, normalization, poll, polling, protocols, social data, social media, social media data, social media data delivery, twitter, twitter feeds, wordpress

Got Glue? Reflections from the 2011 Glue Conference

Posted on June 1, 2011 by bre

0 comments

Late last week, several members of the Gnip team attended the 2011 Glue Conference, including Gnip’s very own CEO Jud Valeski, who delivered a keynote presentation on High-Volume, Realtime Data Stream Handling. Check out Audrey Watters’ article, Gnip CEO on the Challenges of Handling the Real-Time, Big Data Firehose on ReadWriteCloud, it does a great job of summing up Jud’s presentation.

We were thrilled to once again sponsor such an innovative and informative event dedicated to the bits and pieces, APIs and metadata, standards, and connectors that help “glue” together the applications of the web. We would like to congratulate Eric Norlin (@defrag) and team for putting on a great conference.

It was exciting to have seen several of our customers and partners at the conference, including ReportGrid, a realtime analytics startup, who used Gnip data to power their demo at the conference. For those of you that we met at the conference, it was a pleasure! For those of you that we missed, give us a call or shoot us an email, we would love to hear from you. See you next year at the 2012 Glue Conference!

Categories: Events

Tagged: apis, data streams, firehose, glue conference, gnip, metadata, readwritecloud, realtime data, reportgrid, social media data analysis

New Gnip Enrichments: Klout Scores and Language Filtering

Posted on May 17, 2011 by Samantha Quist, Marketing

0 comments

We’re excited to announce two new enrichments today: we’ve partnered with Klout to deliver influence score data and we’ve enabled filtering by languages on our Twitter firehose-based premium data feeds. Combined with Gnip’s other enrichments (format normalization, URL expansion, etc.), we hope you’ll find it easier than ever to filter your Twitter feeds to precisely the data you want. (See all Gnip Enrichments)

KloutOur latest partner, Klout, is known as “the standard for influence.” Our friends there analyze Twitter and other social media data to determine how influential (or not) different Twitter users are and assign “Klout Scores” to them accordingly. (Last we checked, @gnip’s Klout Score was 41 on Klout’s scale of 1 to 100.) Klout is a Gnip customer as well, so we’re particularly pleased to work with them to bring Klout Score metadata to other Gnip customers and share the love.

Now when you access premium Twitter data through Gnip, you can opt to have each user’s Klout Score appended to their Tweets. Klout filtering capabilities are also available via Gnip — for example, when you use our Power Track feed, you can choose to receive Tweets only from users whose Klout Score exceeds a certain number. Although Klout data has been available upon request to existing Gnip customers for some time, today marks the official start of our partnership and Klout enrichment on Gnip feeds. Welcome to the family, Klout!

Our other new enrichment feature today, language filtering, has long been requested a wide variety of Gnip customers (our international clients in particular!). Starting today, language filtering too is available on Gnip’s premium Twitter feeds for 11 languages: English, Dutch, French, German, Italian, Japanese, Korean, Norwegian, Portuguese, Spanish, and Swedish (with more to follow).

To filter for English Tweets only, for instance, just append “lang:EN” to each relevant rule you’re querying. You can also enter “lang:EN” as a rule on its own if you’d like to receive all Tweets that our algorithm has identified as English language Tweets. Our language filtering option is based on our recently announced language metadata, built from the open sourced JTCL, using n-gram frequencies to categorize Tweets into given languages.

With these two new filtering capabilities you can construct a whole new class of streams using Power Track, such as:

  • All Tweets in German from users with Klout Score greater than @gnip (“lang:de klout_score:41”)
  • All geocoded Tweets written in Dutch (“lang:NL has:geo”)
  • Norwegian Tweets about Coca Cola from very influential users (“coca cola lang:no klout:50”)
  • and lots of others that we’re sure our customers will surprise us with!

Although Klout Scores and language filtering are only available on premium Twitter feeds so far, many of Gnip’s data enrichments come included with every Gnip Data Collector. Contact us to learn more or try Gnip’s enrichments for yourself.

 

Categories: News, Partners, Product

Tagged: data collectors, data enrichments, data feeds, firehose, format normalization, gnip, influence scores, klout, klout scores, language detection, language filters, metadata, power track, twitter, twitter feeds, url expansion

Language Metadata for Twitter

Posted on April 19, 2011 by Samantha Quist, Marketing

3 comments

Guten Tag and Buenos Dias! Gnip has been going global for a while. Customers are signing up from all over the world and we’re seeing increasing requests for language features on our feeds. 

Today we’re excited to announce language enrichment for the Decahose, Power Track, and other commercial Twitter feeds. In the past few years, many of our customers have asked us how they can identify which Tweets come from which language. Starting today, you can use Gnip’s enrichments to easily identify the language of your Tweets.

For instance, if you’re using Power Track to find all Tweets matching “Coca Cola,” now you can identify which of those are written in which language. We’re starting with support for eight languages: English, Spanish, German, French, Italian, Dutch, Portuguese, and Swedish, as available according to our confidence level for each language. You can expect more language support from Gnip in the coming weeks.

Starting from the open sourced JTCL, we’re using n-gram frequencies to categorize a Tweet into a given language. We’re thoroughly impressed with the accuracy levels thus far.

We’re excited about the use cases this enables across the industry. We know many of our friends are rapidly adopting Twitter (hello Japan!) and we’re glad to start providing better support for these global conversations.

Language enrichment is the first step toward a powerful language filtering capability for Twitter and Gnip’s 30+ other sources. If you’d like to try Twitter firehose filtering and language enrichment or to request support for your particular language, send us a note and say… ciao.

Categories: News, Product

Tagged: data enrichments, data feeds, decahose, firehose, gnip, language filters, metadata, power track, tweets, twitter, twitter feeds

Expanding the Twitter Firehose

Posted on January 12, 2011 by Gnip Admin

4 comments

We’ve spent the past few weeks at Gnip working on an infrastructure that allows us to expand shortened URLs as they come through the Twitter Firehose. Here’s an overview of our architecture:

architecture diagram

Note that I’ve removed redundant machines used for failover to simplify things a bit. Each rectangular node represents a Java process running on a separate EC2 instance.

Connector: A lightweight client that consumes the full Twitter Firehose via Streaming HTTP. On the way in each Tweet is wrapped in an internal (JSON) messaging format that allows us to insert arbitrary metadata for each Tweet. The messages are fanned out to n number of Deliverators and Enricherators.

Enricherator: Ingests the full Twitter Firehose from Connector via Streaming HTTP. Inspects each Tweet and plucks out any URLs that exist in the ‘entities’ payload. Compares these URLs to a list of known URL shorteners (t.co, bit.ly, etc..) and if there are any matches, ships those URLs over to Bosserator via Google Protocol Buffers (GPB) RPC to be expanded.

Also, as Enricherator receives each Tweet, the Tweet is inserted into a queue. The queue contains Tweets that are waiting to be decorated with an expanded URL. If a response comes back from Bosserator for a given Tweet, Enricherator adds the expanded URL metadata for that Tweet. After a predetermined number of seconds the Tweet removed from the queue and shipped off to Deliverator regardless of whether or not it has been decorated.

Bosserator: Holds onto and actively manages a queue of expand URL tasks from Enricherator. Exposes this task queue via GPB RPC to Fetchors. Listens for decorated responses from Fetchors, ships them over to Enricherator as they are received.

Fetchor: When Fetchor has available resources, it will ask Bosserator for n number of tasks via GPB RPC. For each task, Fetchor will issue a HTTP HEAD request against the shortened URL. Fetchor’s HTTP client will follow up to 10 redirects before returning a response. When a response is received, the expanded URL inserted into the relevant task and is shipped back over to Bosserator, who then immediately ships it to Enricherator.

Deliverator: Ingests the full firehose (in realtime) from Connector and an enriched firehose (delayed) from Enricherator via Streaming HTTP. Deliverator then determines what kind of stream that the client is asking for (User Mention Stream, Link Stream, etc..), what format they want it delivered in (original or JSON Activity Streams), and inserts any enrichments that have been enabled for that stream. Then the stream is made available to your Gnip Data Collector via Streaming HTTP.

If you would like to beta test the expand URLs feature, please contact us at info@gnip.com. Be sure to sign up for our newsletter at http://gnip.com/newsletter to hear about new features that we have coming in the pipeline. Feel free to leave comments and suggestions about how we can make URL expanding more useful to the problem you’re trying to solve.

Categories: Development, Product, Technology

Tagged: data collectors, firehose, gnip, http, http streaming, json, link stream, mentionhose, metadata, protocols, realtime data, tweets, twitter, twitter firehose, url expansion, url unwinding

Gnip Schema Update Adds More User Meta-data Support

Posted on September 1, 2009 by Eric Marcoullier, Co-Founder

0 comments

We continue to work on enriching the Gnip schema to provide second level meta-data on user generated activities .  Given we push 10s to 100s of of millions of activities around daily supporting more meta-data means a bit of work beyond just updating our schema.

Today we rolled out meta-data updates to the <actor> and <to> elements of the Gnip schema.   The updates today are new optional attributes that provide a place to map additional user information that is available on some social media services like Twitter and others.    Initially we will just add the new meta-data to activities where the information is available inline with activities and then in the near future we are adding more platform features to support the scenario where a second API call is required to add this meta-data to the activity.

Starting today the <actor> element has support for the numeric userID, friends, followers, and posts.   In addition, we are now mapping the fullname and username to individual attributes in order to better support services that allow end users to create custom screen names and change those names.   The <to> element was updated to provide a new attribute for numeric userID.

Overview of updates to <actor> and <to> elements of Gnip schema:

  • <actor> is the person who performed the activity on the service
    • <posts> is the number of updates made by the user
    • <followers> is the number of people following the user
    • <friends> is the number of people the user has friended
    • <fullname> is the descriptive name or screen name of the user
    • <username> is the username of the user on the service
    • <uid> the unique numeric ID for the user on the service
    • <metaURL> is the user profile link on the service
  • <to> is the person who the activity is in response
    • <uid> the unique numeric ID for the user on the service
    • <metaURL> is the user profile link on the servic


Categories: Development, News, Product

Tagged: activity data, api, data enrichments, gnip, metadata, social media, twtter

Gnip Announces Early Access Program for Facebook® Platform

Posted on May 18, 2009 by Eric Marcoullier, Co-Founder

0 comments

We are pleased to announce an early access program for a new Gnip data publisher to access and integrate data from the Facebook Platform Open Streams API.

Companies will realize immediate benefits from choosing to use the Gnip Platform for integrating data from Facebook.

  • Choose the specific Facebook users from among those that have authorized your applications and then Gnip will immediately begin collecting the relevant data, normalize it and deliver it in real-time to your applications.
  • Simplify the integration and data retention requirements for integrating with the Facebook Platform to your applications by using Gnip Notifications and Gnip Data Streams to work with and store either event meta-data or full-data based on the appropriate use case as defined by the Facebook Platform terms of use (i.e. the 24 hour rule, etc)

Developers and companies can sign up right now to be notified when the early access program is launched by sending an email to info@gnip.com with the subject: Facebook.  Any company signing up for the early access program will be eligible for three free months subscription service to the Gnip data publisher for the Facebook Platform once it is generally released.   At this time the early access program is planned to be launched in the summer.

And to provide a small taste of the upcoming integration here are two examples of what common Newsfeed actions on Facebook will look like when accessed via the planned Gnip data publisher.

1) Status update Example (fbids in this example were changed from actual one in my stream item)

<activities publisher=”facebook”>
<activity>
<at>2009-05-16T14:07:25.000Z</at>
<action>post</action>
<activityID>http://www.facebook.com/profile.php?aid=6&id=12345&ref=at</activityID>
<actor metaURL=”http://www.facebook.com/people/Shane-Pearson/12345″>Shane Pearson</actor>
<destinationURL=http://www.facebook.com/profile.php?id=12345&amp;story_fbid=12345</destinationURL>
<payload>
<body>It must be spring as my weekly trip to Lowes/Home Depot is back on the schedule</body>
</payload>
</activity>

2) Upload photo example (the below Gnip data schema maps to a Facebook activity stream example)

<activities publisher=”facebook”>
<activity>
<at>2009-04-06T21:23:00-07:00</at>
<action>upload</action>
<activityID>http://www.facebook.com/album.php?aid=6&id=499225643&ref=at</activityID>
<actor metaURL=”http://www.facebook.com/people/Snapshot-Smith/499225643″>Snapshot Smith</actor>
<destinationURLhttp://www.facebook.com/people/Snapshot-Smith/499225643</destinationURL>
<payload>
<title>Snapshot Smith uploaded a photo.</title>
<body><p><a href=”http://www.facebook.com/photo.php?pid=28&id=499225643&ref=at” caption=”A very  attractive   wall, indeed”/></a></p>
</body>
<mediaURL type=”thumbnail” > http://photos-e.ak.fbcdn.net/photos-ak-snc1/v2692/195/117/499225643/s499225643_28_6861716.jpg</mediaURL>
<mediaURL type=”content” > http://www.facebook.com/photo.php?pid=28&id=499225643&ref=at<</mediaURL>
</payload>
</activity>

 

Categories: News, Partners, Product

Tagged: activity data, activity streams, api, data normalization, data publishers, data streaming, data streams, enterprise applications, facebook, facebook api, gnip, metadata, normalization, realtime data

Beta 2 Technical Update

Posted on March 26, 2009 by Jud Valeski, Co-Founder and CEO

0 comments

api.gnip.com has been up for a couple of weeks now and I thought I’d take a moment to update folks on how things are going.

One thing I’ll note early is that our original production environment remains our on-call monitored environment. While we’re honing api.gnip.com it’s monitoring does not “page” us when there are issues. Plenty of internal alarms sound when things crater, but the “on-call” person doesn’t get paged.

The three new major components in this Beta:

  1. New Schema
  2. Polling Infrastructure
  3. Data Normalization

New Schema

We’ve vastly improved our schema. Now Gnip offers normalized meta-data across Publishers/services. We’re striving to minimize the guesswork you have to go through in order to display/process activities in your application. We digested our community feedback, and out popped the new schema. So far so good. A few minor issues have been brought to our attention, and we’ll address those in the next rev of the schema.

Polling Infrastructure

This one’s been fun. Along the road to “delivering the web’s data,” we realized we were going to need to acquire some of the data ourselves. Some Publishers aren’t interested in PUSHing to Gnip yet, however demand for their data remains strong. As a result, we’ve embarked on our own polling infrastructure to ensure our Consumer’s needs are met (visit gnip.uservoice.com and tell us which services/data sources/Publishers you’d like to see Gnip support next). We’ve built a traditional job scheduling/queuing model to poll variabled URL endpoints for data. Replete with various back-off algorithms and rate-limiting, the system grabs activities out on the network, normalizes the results, and publishes them back into Gnip’s PubSub API.

We’re plowing through bugs and the challenges that come with job scheduling (starvation, ordering, prioritization, fairness) at scale. We long for some of the linear, relatively simple, polling architectures some of our partners have built. Building polling infrastructure for a few hundred thousand endpoints is one thing; building polling infrastructure for a few hundred million endpoints is quite another.

Until we resolve these issues you may experience intermittent results from Polled Publishers in Gnip. Things can go from working smoothly, to sporadic gaps in data. Bottom line is we have starvation issues we’re working to address. Bare with us.

Data Normalization

We’re vastly expanding the breadth of our Publisher offering by leveraging our Polling Infrastructure. For Polled Publishers we’ve built a layer that translates from arbitrary feed/API formats, into Gnip XML. While our core is written in Java, we leverage Python’s Universal Feed Parser to ingest XML, and map fields to Gnip XML. When UFP can’t handle things (even it has its limitations), we punt out to text processing with our own simple mapping language (with regex support). Our investment in this layer has highlighted two things really well: one, sadly even with the RSS/ATOM/XML’ization of data, severe challenges remain with data handling on the web (rampant inconsistencies remain). two, Gnip’s value proposition continues to grow. We handle the headache of this kind of mapping/parsing once, and many of you get the reap the rewards.

AWS/Ec2

We remain exclusively in “the Cloud.” Some thoughts on our 12 month experience with it…

  • Cross instance latency remains high (say 5-10x higher than average non-Zen local interconnects). While we’ve been able to build a “real-time” system regardless, it’s certainly gotten in our way at times.
  • Inbound packet transmission “into the Cloud” is slow. It’s not uncommon for sustained, large, uploads (say moving builds onto Ec2 instances) to average 70kB/s. This is lame and needs to be fixed by Amazon. Happy to pay more for throughput, just give me the option.
  • Dedicated instances. One way to look at Ec2 performance is that the cheaper/smaller the instance, the more issues you’re going to have. Basically, Amazon heavily vslices and dices the cheaper boxes; the cheaper the box, the more loaded it can be with other apps. The larger instances (e.g. XL) equate to dedicated hardware just for you; your own CPUs, your own mem, your own NIC.
  • I get asked “how many times have you lost instances?” all the time. The answer is “rarely enough that it’s never on anyone’s mind and I can’t remember the last time it happened.” Maybe 3 instances out of 50 continuously running over the course of 12 months.
  • We continue to use the free version of RightScale to manage our deployments.

Please let us know what features you’d like to see going forward, or if you have questions. info@gnip.com

Categories: Development

Tagged: amazon, api, api rate limiting, api rate limits, atom, aws, data consumers, data feeds, data normalization, data publishers, data sources, ec2, gnip, java, metadata, normalization, poll, polling, push, python, realtime data, rightscale, rss, xml

Updated Gnip Schema Available to Beta Test Next Week

Posted on January 30, 2009 by Eric Marcoullier, Co-Founder

0 comments

After being very heads down on the Gnip platform this month we wanted to let everyone know that the test system will be available early next week with the new and improved schema.  Again, the new schema can be viewed in our forums.  Also, there were some data producer examples in my last post and in the our forums that should be helpful in understanding the broader meta-data we are now providing.

Also, we are still working on the length for the beta to allow people to prototype before it is moved to production, but expect this will at least be for the month of February.   We also are looking at how to continue to provide access to the current schema after making the update to the new one.   So, stay tuned for more information next week.

Categories: Development, Product

Tagged: data producers, gnip, metadata

Status Update on New Gnip Schema and Demo System

Posted on January 24, 2009 by Eric Marcoullier, Co-Founder

0 comments

As we come to the end of January it is time to provide some information on the upcoming new Gnip schema that we announced earlier this month.

First, a big THANK YOU to all the people who provided feedback, thoughts and examples of where you are using Gnip and why it was important to have the schema support your real-world use cases.   These were very important to our team as we made decisions on this major update to our schema.    The biggest decision we made was to keep all the existing meta-data in the Gnip Notifications that exist today, which was not what we had in the draft that was originally shared with the community.    We think this was absolutely the right thing to do in the end, but it did mean we ended up with a bit of re-work to get to the final version of the schema now being completed in our platform.   Our timing has slipped a couple weeks since the general announcement the first week of this month.  Therefore, we now plan to have the demo system up and running no later then the end of January to start allowing people to work with the new schema.   We will be pushing out more information on the updated schedule in the coming week.

The final version of the schema can be viewed in our forums.   In addition, here are some examples of data streaming from Gnip with the “existing” and “new” schema.    From the feedback we are receiving and from doing our own tests we believe the new schema is purely additive in terms of providing more rich meta-data to access, filter and integrate data.

1) Comparing “existing” Gnip Activity Notification schema to “new” schema from a random public tweet.

publisher=”twitter”, activity=”notice”

a) Existing Gnip schema

<activity source=”Tween”url=”http://twitter.com/statuses/show/1144685412.xml” action=”notice” actor=”kazusap” at=”2009-01-24T15:23:21.000Z”/>

b) New Gnip schema

<activity>
<at>2009-01-24T15:23:21.000Z</at>
<action>notice</action>
<activityID>tag:twitter.com,2007:statuses/1144685412</activityID>
<URL>http://twitter.com/statuses/show/1144685412.xml</URL>
<source>Tween</source>
<actor uid=”14202410″ metaURL=”http://twitter.com/kazusap”>kazusap</actor>
<destinationURL>http://twitter.com/kazusap/status/1144685412</destinationURL>
<to metaURL=”http://twitter.com/mossy_13″>mossy_13</to>
<regardingURL>http://twitter.com/mossy_13/status/1144684485</regardingURL>
</activity>

 

2) Comparing “existing” Gnip Activity Data Stream schema to “new” schema for a Delicious bookmark

publisher=”delicious”, activity=”bookmark”

a) Existing Gnip Activity Data Stream schema showing Delicious bookmark

<activity tags=”economists,economy,USA,recession,depression,politics,economics,barakobama,JohnMaynardKeynes” source=”http://delicious.com/url/35adba93e5e3fb83198b5dbabd2fbb9e#asterisk2a” regarding=”http://www.nytimes.com/2009/01/23/opinion/23krugman.html” url=”http://delicious.com/url/35adba93e5e3fb83198b5dbabd2fbb9e” action=”bookmark” actor=”asterisk2a” at=”2009-01-24T17:15:43.000Z”/>

b) New Gnip Activity Data Stream schema showing Delicious bookmark

<activity>
<at>2009-01-24T17:15:43.000Z</at>
<action>bookmark</action>
<activityID>http://delicious.com/url/35adba93e5e3fb83198b5dbabd2fbb9e#asterisk2a</activityID>
<actor metaURL=”http://delicious.com/asterisk2a”>asterisk2a</actor>
<destinationURL metaURL=”http://delicious.com/url/35adba93e5e3fb83198b5dbabd2fbb9e”>http://www.nytimes.com/2009/01/23/opinion
/23krugman.html</destinationURL>
<tag metaURL=”http://delicious.com/economists”>economists</tag>
<tag metaURL=”http://delicious.com/economy”>economy</tag>
<tag metaURL=”http://delicious.com/USA”>USA</tag>
<tag metaURL=”http://delicious.com/recession”>recession</tag>
<tag metaURL=”http://delicious.com/depression”>depression</tag>
<tag metaURL=”http://delicious.com/politics”>politics</tag>
<tag metaURL=”http://delicious.com/economics”>economics</tag>
<tag metaURL=”http://delicious.com/barackobama”>barackobama</tag>
<tag metaURL=”http://delicious.com/JohnMaynardKeynes”>JohnMaynardKeynes</tag>
<regardingURL>http://www.nytimes.com/2009/01/23/opinion/23krugman.html</regardingURL>
<payload>
<title>Op-Ed Columnist – Stuck in the Muddle – NYTimes.com</title>
<raw>H4sIAMBMe0kC/63UQW/TMBQH8DufwpQDB1jdJqtYIy9StXEZDNA6Dmja4dl+aa3EdmQ7RPn2eB1u
NjiAID5Zlt/vbztPYSqgLl+Qw2BBhQbLz+3Je0kubNNpo3wgJ2QbOlETZUjYI7nupGwwrn76dqs0
+rmwmtHH0gS1Hb+EgOUWwluSnZIrMCRbLNZk+a5YrorTnLxZxMFo2pgKd52SRPkv6DR8VKY+n1XQ
eJyV+xDaglKJjRLKdodU2rmG5iuQHNY5rjCv+Fm+XJ/xVVzhMqs4X+Mr8AGd8nUGjD7wx6wm+snt
+35uhpDuQx8OSxdLmuXUtsooa+K0dt1Og5nvg24YPZQnS4pCOIRgXcle3l1cbm43d2Pw/X3J6JMt
qSomaTTB//vtGD0aCe2rvvi5eOOPdIUo/fx5wPeMOu//MucXNqV52zmBJBrnsz9HjU8yK8f5a0+4
tbUGV3tGH8XxjWJ77KwbiLQalDmmPPdHjM5KFNZYHVs3aqn8/71hGuzrdjMN5FCg97Ezp+Ektm5K
r7WNCkpM+gkm4zg4ELXloGEa8MruzTUMBpz8gIPB387J6OFP+wNsAa+TcAUAAA==
</raw>
</payload>
</activity>

 

 

 

Categories: Development

Tagged: activity data, data filtering, data filters, data publishers, data sources, data streaming, data streams, gnip, metadata, push, twitter

Post navigation

← Older posts

Lijit Search

Follow Gnip

  • RSS
  • Facebook
  • Twitter
  • LinkedIn

Subscribe by Email

Want to Get Social Data from Gnip?

Tags

activity data activity streams api api rate limits apis Big Boulder boulder colorado data aggregators data collection data consumers data delivery data feeds data filtering data filters data normalization data producers data publishers data sources data streaming data streams delicious digg facebook firehose flickr gnip http keyword filters metadata normalization poll polling power track protocols push realtime data rest social data social media social media data tweets twitter twitter data xml xmpp

Copyright © 2013 Gnip, inc.

Gnip makes it easy to build social media tracking tools.

  • Home
  • Product
  • Industries
  • Sources
  • Blogs
  • Company
  • Contact
Gnip 1050 Walnut St, Suite 115 Boulder, CO 80302 info@gnip.com 888-777-7405
  • Gnip on Twitter