Integrating Gnip Notifications

Now that we’ve got some time under our belt, we’ve observed a pattern in Data Consumers who are using Gnip. Many Data Consumers already have event detection in place with polling infrastructure they’ve built. They poll various endpoints looking for changes, and when there’s a change, they consume all the associated data with that change, and digest everything into their application.

Gnip Notifications enhance this model, with very little effort on the Data Consumer’s part. As a Data Consumer, you can leave your existing infrastructure in place, un-touched, yet leverage Gnip’s latenency minimization. To do so, all you have to do is injest a Gnip Notification, then have that Notification bump a poll/crawl to the top of your existing stack. Customers have called this “hinting” or “accellerating.”

For example, today you have a queue/batch of jobs to poll for a user’s activities on serviceX. Let’s say you poll serviceX for that user every hour. Gnip can tell you precisely when that user made a change on serviceX, and what it does, you move that entry in your queue/batch to the top. Basically using Gnip as an acellerator to move various events in your system to a “high-priority queue.” That queue may indeed be a separate queue in your model, or a logical high-priority queue by just moving entries to the top of your stack.

Your users now see their actions (photo uploads, twitters, whatever) in near real-time, rather than potentially the eternity of 15 minutes (often longer).

Next I’ll blog about how to incorporate Gnip Full Data activities into your existing infrastructure.