Data Stories is Gnip’s project to tell the stories behind how people use data and why it matters. This week we interviewed Harper Reed, the Former CTO of Obama for America about the technology behind the scenes of elections, civic data and more. You can follow Harper on Twitter at @Harper.
1. In the next four years there will be massive changes in technology, what do you think will change in how campaigns use data in the 2016 election?
The big innovation this cycle was the analytics and how we found answers from the data we had. This follows the arc of the big data movement. When I first got involved in 2007/8 the conversations were all about collection and storage of data. Recently we have seen a shift from people not worrying about that because it is largely solved. Now, the big data space seems to have, thankfully, shifted to concentrate on gaining insight and getting answers from the data.
I think that this arc will continue. 2016 will be more about the answers that we will get from the data. Aggressively using modeling and data analytics to help make sure that there are no missteps.
2. Obama 2012 worked hard to remove the silos between tech and digital, what are some of the lessons you’ve learned between sharing data between departments?
This is obviously a work in progress for every organization. We made sure that there was a close physical proximity. That helps a lot.
Personally, the best lesson I learned was taught to me by John Maeda. He came in for a whirlwind visit and said “Manage by your Outbox. Not by your Inbox.” He then left. It was amazing and exactly the right amount of info. Later, he told me that this knowledge came from Larry Bacow.
Anyway, the idea is that you can work through political struggles, silos, etc by making sure that you are communicating out. Don’t expect or judge by the incoming communication.
This, for me, was the number one way to break down silos.
3. Political campaigns previously weren’t known for being full of tech savvy people. How do you reconcile the needs of campaign strategists and translate it to your team? Essentially, what did you learn about product development for campaigns?
Campaigns are organized like emergency response. Very top down. Lots of volunteers. Lots of downward delegation. Lots of managing and communicating up.
This type of organization does not mix well with standard software methodologies. It is hard to have a product manager when the stakeholders are unwilling to cede responsibility for the project to the PM.
Part of this is a lack of trust. Technology does not have a good history with campaigns (hopefully we helped instill more trust). Part of this is that in a top down type of environment, you need to negotiate more.
We found success by iterating on our process as quickly as we iterated on our software. We also would not have been able to do this without the great product team we had (Carol Davidsen, Mari Huertas, David Osborne, Jason Kunesh). They took the iteration seriously and made sure that the product development was successful no matter what.
4. What role did data scientists play in the reelection campaign?
We had an amazing set of data scientists – led by Rayid Ghani. They played the role of every scientist – thinking of crazy awesome things to test, testing experiments, making some of the coolest and more important of our discoveries and driving me crazy.
Working with them was awesome.
5. As one of your hacking projects, you took data from Chicago Transit Authority’s bus tracker and made it public. What other information would you like cities to make public?
I am going to pull the data hippie card here and say: ALL THE DATA!
I try and lead a very transparent and free life. There is not a lot of data that I think should not be public. Obviously personal data is a bit different (I would really like to have all my financial data, etc be available). But if it was available for everyone – it wouldn’t have such a stigma around it.
More realistically – the more civic data that is available, the better and more informed civic decisions we can make.
6. What’s next for you?
I have a small team of amazing people who I am working with. We are focusing on business tools. It should be fun.
Thanks to Harper for participating in the interview! Let us know in the comments if you have suggestions for other Data Stories.
Previous Data Stories:
Hilary Mason, Chief Data Scientist of bitly
Blake Shaw, Data Scientist of Foursquare on product development with data science
Simon Rogers, data journalist at The Guardian
Lada Adamic of Michigan on information networks
Mel Hogan of CU Boulder on digital archiving
Liv Buli of Next Big Sound, the world’s first music data journalist
Sherry Emery of UIC, studying social data and smoking cessation
Annicka Campbell of SapientNitro on the Digital Love Project
Gabriel Banos of ZauberLabs on predicting the election with social data