Gnip moves billions of real-time public social activities to its customers everyday. Doing that efficiently, accurately, and reliably is an awesome challenge. Gnip also has a lot of product roadmap to build-out. We’re hiring across the board, but I wanted to provide some insight into the software side of things to give any software construction minded readers of this post a sense of the interesting technical challenges you’d get to work on if you were on our development team.

Big Streams: Long-lived, Stateful, Variable Throughput TCP Connections
While our web-app (Rails) looks/feels much like a typical web-app, looks can be deceiving. Our core system (Java) has uncommon TCP (HTTP for the most part) connection challenges that we’re constantly applying operational and business logic creativity to. We often describe the connectivity scenario as akin to a video streaming system (Netflix for example). The load balancing, tuning, connection handling logic (often in code (Java and/or C)), restart, buffering, flushing games we get to play keep one’s brain thoroughly engaged. Imagine getting to write code that runs in a pipeline moving at several hundred Mbps (sustained).

Big CPU: Filtering
We’ve built a real-time equivalent parallel to age-old SQL. Something powerful and efficient, yet simple and intuitive to use. The world has grown up with SQL on their minds, yet it generally doesn’t apply to the world of high-volume real-time nature of public social data streams. We spend a lot of time and energy crafting the language, as well as building the infrastructure underneath to ensure it can operate on a message (from small Tweets, to large blog posts) in ones of milliseconds. Very powerful. Very fun. We’d love your help in evolving this part of the system with us.

Big Blending: Enrichments
As data moves through Gnip’s infrastructure we do a variety of things to it. As a couple examples, we will enrich it with Klout scores and typed language classification scores. We’ll also unwind all those opaque URLs (and allow you to filter against the result) for our customers, so they don’t have to stand up horizontally scaled/parallelized infrastructure to do so on their end. We have a long list of enrichments we’re in the process of adding, and we’d like to do so with as minimal latency impact as technically possible. Speed matters. There’s amazing opportunity in the industry to blend other datasets into “the stream.” Help us do this.

Big Data: Historical
There are three organizations on Earth that have the complete public Twitter corpus on their servers. Gnip, Twitter & the Library of Congress. Gnip has dipped our toes into the “historical” offering with our 30-day reply product. Doing so has allowed us to meet many of the business critical backfill requirements of our customers. That said, there is still a huge opportunity in the historical space. We leverage a variety of parallelizable data access, query, and filter technologies at Gnip, but want more horsepower here. We’d love to have more people on the team with practical experience around map-reduce based data access models. Gnip personifies “Big Data” challenges and solutions. Show us what you’ve got (beyond textbook and academic understanding of the latest trends in this space)!

Big Customers: Impact
We’ve been delivering public social data for four years and serve the biggest, most demanding customers in the space. From 8 of the 9 largest social media monitoring companies to some of the largest hedge funds in the world, Gnip is expected to provide a bulletproof solution. Through our customers, Gnip serves more than 90% of the Fortune 500. We’re not just about “Big Ideas,” we’re about “Big Impact.” Want to work in a place where what you do matters? This is it.

Our Approach
We take an “every message matters” approach to our products. As a result, our development team is tightly coupled (one-and-the-same for the most part) with the production operation of the system. We don’t have a wall between the software being built and that software being run. Gnippers who write code also operate that code throughout the local, review, staging, and production environments. We believe this approach yields a better service for our customers; higher quality, better reliability, better consciousness. There is a strong sense of collective code ownership, as opposed to folks siloing into “their area.” This results in everyone’s creative and talented mind having an opportunity to impact the entire system, and subsequently yields better, more consistent, product.

jobs@gnip.com