Push & Pull

Over here at Gnip we’re knee deep in the joys of polling. Our mission to “deliver the web’s data” has us using several approaches to hook consumers up with publisher activities. While we iron out the kinks around our polled publishers, I’m reminded of how broken polling is for many types of data. From rate limiting, to minimum poll interval definition, polling inherently yields gaps between actions that take place along a timeline. In some cases those gaps are small, and potentially imperceptible, but in others they are large. Viewing one of our internal daily stats charts exemplified this push vs. pull dichotomy. Each color represents a separate publisher (top/black line == total). Guess which publishers are Push (event driven) and which are Pull (polling driven).

Gnip Publisher Daily Chart

The answer: consistent, connected, publishers/lines in the chart are Push (even driven), and more variable publishers/lines are Pull (poll) driven.

Our goal in life is to smooth those variable/jagged lines for the polled publishers, but along the path to data delivery nirvana, I thought I’d share a this visual.