Black-Market Data

This year’s Glue conference is right around the corner, and Gnip has a small spot that we’re going to use to discuss the emerging black market for data. Data providers, like Gnip, are under increasing pressure to provide their customers with “more data” than Publishers actually want them to have. Gnip has taken the moral high-ground position and complies with the terms of service that the Publishers we integrate with provide, yet we’re seeing more and more firms offer data that the Publishers explicitly disallow access to.

Our position is that we don’t want our customers building on a technical house of cards. While we can technically gather “more data,” doing so in a manner that violates the ToS of a given Publisher ultimately leads to an adversarial relationship between us and said Publisher. As a result, we’d be putting our own customers at risk when that scenario goes south (and it eventually would). The result is a widening gap between legitimate, and illegitimate data collection. This should signal high-demand Publishers to mature and build real solutions that market dynamics clearly require.

I don’t have my thoughts fully formed on this topic, but it should make for a good discussion at Glue! Hope to see you there!

  • http://blog.horseshowparent.com Steve

    What about the case were Publishers change their ToS, in some cases frequently? This can cause legitimate data to go ilegit, and vice versa. Also, there may be cases where Publishers have enhanced API’s for their use or “special partners” which isn’t part of the minimal public API’s. In that case the same data may be legit to some and ilegit to others.

    I applaud your stance, but perhaps morally higher~ish ground is a better stake than the moral high ground, if the ground is shifting ;-)

  • http://img365.eu Img

    What about the case were Publishers change their ToS, in some cases frequently? This can cause legitimate data to go ilegit, and vice versa. Also, there may be cases where Publishers have enhanced API’s for their use or “special partners” which isn’t part of the minimal public API’s. In that case the same data may be legit to some and ilegit to others.
    +1