State of Social Media in the Financial Sector

In the two years that Gnip has been working with the financial industry, the state of social data and social media in the financial sector has changed dramatically. Here’s a look at where the industry stands and where we think it’ll go.

Social Media Moves Markets

Perhaps the highest profile event to involve social media in the financial sector this year was the Hash Crash.

On April 23, the AP Twitter account was hacked and tweeted that two explosions in the White House had injured Obama. The result? The Dow dropped more than 140 points within two minutes. It was an eye opener for many, on the power of social media to move markets. But what got lost in the coverage of this incident was that Twitter was part of the reason that the market rebounded so quickly. What Gnip has seen time and again is that rumors on Twitter are defeated just as quick as a rumor is started. Immediately after the initial Tweet, many others started to debunk the rumor giving live reports from the White House and setting the record straight.

The Hash Crash provided another, audible and clear reason for using social data if you are a participant in the financial markets. Our hedge fund and asset management customers have known this for sometime. If you weren’t following and analyzing social, you were most likely slower than others to understand what was happening in the market – in the dark.

SEC and Reporting on Social Media

Another big change to shape the industry was an official clarification in SEC policy on social media from the Securities and Exchange Commission allowing companies to announce key information on social media as long as investors knew that such channels would be used. As of today, more than 150 companies are using social media to report financial results or performance. Real-Estate Tech company Zillow, Nasdaq:Z ($Z) took this concept even further, opening their earnings Q&A up to questions from Twitter. Earnings calls have always been intended to provide color and transparency for all investors and potential investors of a publicly traded company, but the reality has been that they have been events attended and monitored almost exclusively by investment professionals. Opening up the announcement and especially the Q&A portion to Twitter isn’t as much a radical new move as it is a use of new technology to help re-align these events with their initial intent to give everyone access to information on the company to make investment decisions.

Social Data in the Markets

When Gnip first started looking at the ways the financial markets could use social data, we never would have guessed how fast the market would grow and how hungry people would be for data. In two years, we’ve seen large-scale growth of large hedge funds using Twitter social data as part of their trading strategies. Twitter provides a broad based stream that can answer questions about sentiment about companies, brands, ideas and rumors.  Investors are finding value both through intelligent aggregation and data mining. When a merger rumor is breaking, you can find speculative deal values on Twitter before official numbers have been released. In addition to Twitter, financial institutions have found value in similar content from Stocktwits as well. Stocktwits has a curated community of financial investors who buy into sharing their thoughts online. Stocktwits has been especially valuable for traders and hedge funds who don’t want to sift through the noise on Twitter.  If you search for Justin Bieber on StockTwits, you won’t find anything.

 

And earlier this year, Gnip signed a partnership with Estimize, a crowdsourced earnings estimates platform that provides open sourced financial estimates with incredible transparency, making it a valuable and unique set of social data. Estimize has a platform to capture and provide structure around the long explored concept of a whisper number. They’ve recently added Vinish Jha, a former Starmine Quant, to help add a layer of intelligent analytics on top of the open community, and to really work towards an open estimate that includes only the most accurate prognosticators.

The Adoption of Social Data in Trading Terminals

One of the oft passed around anecdotes at Gnip is how financial institutions talk about traders and analysts using their iPhones under the desk so they can keep an eye on Twitter. Due to regulation, most banks or brokerages don’t allow traders to post or use social media. To enable traders and analysts to access social media (but not to post) a number of banks and terminal providers have been adding social data to terminals  - thus enabling users to at least look up conversations and research online.  In the case of Bloomberg, for now they provide a curated feed, so it isn’t always the complete and full conversations.

New Uses – Risk Management

Over the next two years the acceptance of correlations between stock prices and social data will allow for deeper insights. The area I see making the most progress is in risk management. A good portion of making money in investing is figuring out how not to lose money.   With the S&P on a 5 year growth run, it’s no secret that there is a risk of a pullback, the big question is when?

Social data allows for risk modeling that removes one of the inherent biases of price/volume based modeling. Price and volumes of a security or asset only move when investors are ready to take action. Social media volumes and sentiment move around thought and discussion. Given the hope that thought and discussion still generally precedes action in the strategy of most investors, there exists a huge opportunity to pick up on early, previously undetectable correlations between companies and concepts.  A quick teaser example below shows normalized rolling 24 hour Twitter volumes for 2 related securities (LNKD and FB) and two unrelated securities (LNKD and IBM).  In the next year I expect more companies to start looking at these types of correlations for risk management, both between securities and concepts like “government shutdown”.

 

So Where Are We Headed?

Many of the initial uses cases have been reading social media for actionable trade ideas. The growing number of firms trying to offer social media based signals shows the success in this area. The next 1-2 years will be about expansion in two directions:  improvements in implementation/standardization and expansion of insights. Now that social data has made it through the sandbox phase for certain applications, the focus turns to integrating with existing processes and data sets. The most successful aggregators and indicators will partner with exchanges and traditional financial data vendors to help their data flow through to existing trading and research systems making the information more broadly accessible and cheaper to implement. On the raw data side, more tools will emerge to standardize linking data back to existing security/company identifiers and accepted industry and index classifications.

Using social data in the financial sector is fast becoming a must have, not a nice to have.

 

Tweets, Texts and Tickers

A look at social data in the financial markets with Tom Watson, Vice President of Global Market Data at NYSE Euronext; Brian Hyndman, Senior Vice President of Global Information Services at NASDAQ OMX Group, Inc., Rich Brown, Global Head of Elektron Analytics at Thomson Reuters and Heidi Johnson, Global Product Lead for Hub and Collaboration Services at Markit. 

Financial Social Data Panel

The use of social media data in the finance industry presents some inherent and unique challenges. This panel explored how social data could and should be used on Wall Street.

Leadership Vacuum
There is a great opportunity for leadership in this space, as no one entity is currently driving the charge on how to structure the use, the infrastructure or the verification of social data in financial markets. Firms need to focus on how to get this volume of data from the social channels and to the customers in order to trade and make investment decisions based on that data. Firms are receptive to using social data, but regulatory and compliance oversight make this tricky. Who is to say when information becomes public? Firms are looking for real-time data solutions but must examine this information within the context of historical models. Historical trends put real time data in perspective; both are critical.

Verification
How do firms vett social media accounts as the professional, official individuals and groups? A verification process or Klout-esque score or index is needed in order to confirm that the data from social channels or social sources are reliable, trusted, consistent: Is that the company or individual you think it is? Currently we passively consume this social data, but how can financial firms weed out false positives and anomalies? The nature of social data is that it moves so fast, people and companies react before verification can be made of a trusted source. Fake accounts can and have tanked stocks. Identity verification of expert, trusted sources is crucial. It’s not the first tweet, it’s the conglomerate of the tweets, blog posts, etc., but people often react to the first data they see instead of looking at trends and patterns, as well as the original source. Think snowball, not snowflake. Social amplifies data. Everyone has the ability to reach millions of people now on social media. Data needs to be corroborated with other sources of information.

“Hash Crash”
Earlier this year, the Associated Press Twitter account was hacked and sent out a falsified tweet about a bombing at the White House. The fake news event caused a real market event, dubbed the Hash Crash- a dip (and reversion) in the stock markets. But even from the start, a large proportion of overall Twitter conversation doubted the veracity of that tweet, and traditional news sources showed a different story. This disproved Tweet led to the V-shaped dip and recovery of the markets.

Social Data Examples in Financial Markets

  • Spikes in weather reports and crop prices
  • Violence in Iran and oil prices
  • Sentiment and Psychological index – fear, greed, optimism
  • Geospatial – Florida orange grove region and supply chain data

There is a great need for historical models, real time data analytics, data mining, and verification processes in the financial realm, and firms are receptive to finding these solutions.

Big Boulder is the world’s first social data conference. Follow along at #BigBoulder, on the blog under Big BoulderBig Boulder on Storify and on Gnip’s Facebook page.

Seth McGuire on Social Media and the Stock Market

Gnip’s Seth McGuire was on CNBC’s Squawk Box speaking to Andrew Ross Sorkin about social media and how investors can use data from social networks as part of their strategy. Gnip has been providing social data to the financial industry for more than a year, with clients including hedge funds, banks and signal/data providers.  Specifically, Seth spoke to how hedge funds and other traders are using social data as a variable in their algorithms, as well as a research product for deeper analysis of an equity.

Andrew and Seth also talked about how news frequently breaks on Twitter (the famous examples here is the death of Osama Bin Laden). This type of breaking news on StockTwits and Twitter provides a valuable signal that is frequently ahead of mainstream news. (As we’ve blogged before, natural disasters often are reported on Twitter before anywhere else.) Seth also talked about yesterday’s blog post from our data scientist, Scott Hendrickson, on JP Morgan’s $2 billion trading loss and how the news traveled through different social media publishers.

What Gnip has also seen is that while false stories might be shared on Twitter, Twitter is also quick to surpress the stories via crowdsourced response and questions as to the integrity of those false stories.

Squawk Box guest host Doug Dachille posed an interesting question on whether any of the financial regulators have reached out to use Gnip. While Gnip is serving government agencies in areas like disaster relief, right now it’s the actual compliance and data management departments at banks and funds who are more worried about social media. Most firms lock down the ability to post content on social networks, given SEC & FINRA restrictions, but when compliance officers walk the floor they see traders peeking at their iPhones or iPads to see breaking news and analysis on Twitter and StockTwits. From a compliance perspective, that’s dangerous…but they know the data is valuable so they’re seeking news ways (like Gnip) to bring that data in-house for controlled analysis.

Interested in learning more on social data and the stock market? Email info at gnip.com.

Launching Gnip MarketStream & Partnership with StockTwits

While the market has been on its roller coaster ride across the past month, Gnip has kept its collective head down and stayed busy on behalf of our Investment Management clients (hedge funds, HFTs, asset managers, etc.). That hard work has paid off and we have two exciting announcements to make today.

  • Launch of Gnip MarketStream: Our hedge fund clients have been quite vocal in their desire for a package incorporating the most relevant social data streams into a single low-latency, high-volume solution. We’re proud to answer their needs with the launch of Gnip MarketStream, a realtime data solution that packages the incredibly rich and broad “voice of the market” Twitter stream with the uniquely deep and targeted “voice of the trader” StockTwits stream.
  • Premium Partnership with StockTwits: An integral component of the Gnip MarketStream is StockTwits social media data. We’re thrilled to announce this partnership with StockTwits, the leading realtime financial platform for the investment community and creator of the $(TICKER) tag. The StockTwits stream is a curated, defined-demographic, realtime social data stream focused on investment decisions and analysis. Gnip now provides streaming access to the full StockTwits firehose of social data, and offers access to historical content as far back as 2009.

While the use of social media data by the investment community has included use of this data in news analysis and equity research, the primary adoption of this data across the last six months has been as a trading indicator. By combining the strengths of both the Twitter stream and the StockTwits stream, Gnip MarketStream provides investment professionals unparalleled access to relevant social data at time when social media has become an increasingly vital channel for news and market sentiment.

For more information about Gnip MarketStream or StockTwits data, contact trading@gnip.com.

Why Traders Use Social Media: Speed & Amplification

Gnip’s asset and investment management clients are consistently impressed by two aspects of our social data that differentiate this data from their other sources: Speed & Amplification.

Speed

Speed relates to the ability of social media content to be ‘instant’; an ability fueled by millions of global users who can break news and sentiment more immediately than traditional media sources always can.

A prime example is news of the death of Osama Bin Laden. Keith Urbahn, the former chief of staff for Don Rumsefeld, is widely credited with the breaking that story… through Twitter!

After Keith’s tweet, multiple retweets quickly followed. Within 19 tweets on this subject, a company called DataMinr had identified this as an important and breaking story. DataMinr, a “global sensor network for emerging events and consumer signals,” then issued a signal to their clients, alerting them to this important piece of information.

How does this play into the ‘speed’ characteristic? Because it would be over 20 minutes before that story appeared on traditional news sites. Access to a data stream that can beat traditional media sources by over 20 minutes requires no explanation as to its value for traders and investors.

Amplification

Amplification speaks to the ability of social media as a ‘crowd-sourced megaphone.’ The propensity of users to like, share, and retweet content from other users gives those consuming social media data an extremely easy mechanism to measure what content is most important to the world – and compare that content against other content in real time.

A prime example is the passing of Steve Jobs. We wrote about Steve Jobs’ passing a few weeks ago – that post is here – but there’s an important item to revisit:

The impact he had on us made his death that much more profound and the reaction on Twitter was immediate and immense. Word spread rapidly, peaking at 50,000 Tweets per minute within 30 minutes. At that point, Tweets about Jobs accounted for almost 25% of all Tweets being sent globally.

Access to Gnip’s social media data stream allowed our clients to measure, in the moment, the amplification of this story to measure the importance the world placed on this piece of news. While I doubt any of us needed to see those numbers to know Steve’s passing was an important piece of news, that’s a clear example of how ‘amplification’ works.

Our clients use amplification as a measure to weigh the importance of breaking news, upcoming events, market and product announcements, etc. against other stories. By capturing a realtime snapshot of what the market considers important – and what it doesn’t – they’re able to add an important factor to their existing algorithms.

None of this is to suggest that either social media data speed or amplification should be a sole factor in investing. But when the Gnip social media data stream provides clients with an additional factor to help understand or predict market fluctuations, the value is obvious.

We're off to Dreamforce!

There’s always a lot going on here at Gnip, but this week is especially packed with the team looking to make a big splash at Salesforce.com’s annual Dreamforce event. Salesforce is obviously a huge player in the software space and the theme of this year’s Dreamforce is “Welcome to the Social Enterprise” which fits really nicely with what we do.

At the conference, we’ll be speaking at two sessions and sponsoring the Hack-a-thon. In the first presentation, Drinking from the Firehose: How Social Data is Changing Business Practices, Jud (@jvaleski) and Chris (@chrismoodycom) will discuss the ways that social data is being used to drive innovation across a variety of industries from Financial Services and Emergency Response to Local Business and Consumer Electronics. They’ll also give a glimpse into the technical challenges involved in handling the ever-increasing volume of data that’s flowing out of Twitter every day. If you’re at Dreamforce, this session is on Tuesday (8/30) from 11am to noon in the DevZone Theater on the 2nd floor of Moscone West.

In the second presentation, Your Guide to Understanding the Twitter API, Rob (@robjohnson) will talk through the best ways to get access to the Twitter data that you’re looking for, examining the pros and cons of the various methods. You can check out Rob’s session on Tuesday (8/30) from 3:00 to 3:30 in the Lightning Forum in the DevZone on the 2nd floor of Moscone West.

And finally, we’re sponsoring the Hack-a-thon where teams of developers will create cloud apps for the social enterprise using Twitter feeds from Gnip and at least one of the Salesforce platforms (Force.com, Heroku, Database.com). The winning team stands to take home at least $10,000 in prize money. We’re really excited to see the creative solutions that the teams develop! All submissions are due no later than 6am on Thursday (9/1), so sign up now and get going!

Want to meet up in person at Dreamforce? Give any of us a shout @jvaleski, @chrismoodycom, @robjohnson, @funkefred.

Can Social Media Data Offset Market Volatility?

It’s been a volatile time for the markets the last few weeks. Today especially – the Dow closed down 635 points; S&P, -80; NASDAQ, -175. While there’s no shortage of opinions on how/why the market will/will not recover, one thing is for certain – having the right data to make decisions is more important than ever.

Part of the reason for this is that the markets are clamoring for trends – definitive information on stock trends and market sentiment. Which is why it’s exciting to see how our finance clients are using the Gnip realtime social media data feeds. In a time of increased volatility, our hedge fund (and other buy-side) clients are leveraging social media data as a new source of analysis and trend identification. With an ever-growing number of users, and Tweets, per day, Twitter is exploding, and market-leading funds are looking at the data we provide as a way to more accurately tap into the voice of the market. They’re looking at overall trend data from millions of Tweets to predict the sentiment of consumers as well as researching specific securities based on what’s being said about them online.

While the early-adopters of this data have been funds, this type of analysis is available to individuals as well. Check out some start-ups doing incredible things at the intersection of finance and social media:

  • Centigage provides analytics and intelligence designed to enable financial market participants to use social media in their investment decision-making process
  • SNTMNT offers an online tool that gives daily insights into online consumer sentiment surrounding 25 AEX funds and the index itself

For the first time in history, access to (literally) millions of voices is at our fingertips. As the market continues its volatility, those voices gain resonance as a source of pertinent information.

Financial Markets in the Age of Social Media

When you think about it, the stock market is a pretty inspiring thing.

Over the past several centuries, humans have actually created an infrastructure that lets people put their money where their mouth is; an infrastructure that provides a mechanism for daily valuation of companies, currencies and commodities. It’s just unbelievable how far we’ve come and reflecting on the innovation that’s led us here brings to light a common but powerful denominator: Information.

  • When traders began gathering under a buttonwood tree at the foot of Wall Street in the late 1800′s, it was because proximity allowed them to gossip about companies.
  • When Charles Dow began averaging “peaks and flows” of multiple stocks in 1883, his ‘index’ became a new type of data with which to make decisions.
  • In 1975, when the sheer volume of paper necessary for trades became unmanageable, the SEC changed rules to permit electronic trading, allowing for an entirely new infrastructure.
  • And in the 1980′s, when Michael Bloomberg and his partners began building machines (the now ubiquitous Bloomberg Terminals), they tapped into an ever-growing need for more data.

Those are just some examples of the history that is exciting for us @Gnip, because of the powerful signal the market is sending us about social media. Here are some of the more recent signals we’ve seen:

  • The Bank of England announcing they were using Google search results as a means of informing their “nowcasts” detailing the state of the economy.
  • Derwent Capital Markets launching the first social-media based hedge fund this year.
  • The dedication of an entire panel to Social Media Hedge Fund Strategies at the Battle of the Quants conference in London last week.
  • Weekly news articles that describe how traders are using social data as a trading indicator (here’s one as an example).
  • Incorporation of social data into the algorithms of established hedge funds.

In other words, the market is tapping into a new and unique source of information as a means of making trading decisions. And the reason social media data is so exciting is because it offers an unparalleled view into the emotions, opinions and choices of millions of users. A stream of data this size, with this depth and range, has never really existed before in a format this immediate and accessible. And that access is changing how our clients analyze the world and make trades.

We’ve been privileged to see these use cases as we continue to serve a growing number of financial clients. Most exciting to us, as we respond to the market’s outreach for our services, is understanding our pivotal place in this innovation. As the premier source of legal, reliable and realtime data feeds from more than 30 sources of social media- including our exclusive agreement with Twitter- we’re at the center of how firms are integrating this data as an input. And that’s incredible stuff.

Are you in the financial market looking for a social media data provider? Contact us today to learn more! You can reach us at 888.777.7405 or by email.

What Does Compound Interest Have to do with Evolving APIs?

Once Albert Einstein was asked what he found to be important discoveries. His answer did not mention physics, relativity theory, or fun stuff like Higgs bosons – but instead he said: “Compound interest is the greatest mathematical discovery of all time.”

I trust that most of you understand compound interest when it comes to investing or debt, but humor me and let’s walk through an example: Say you owe your credit card company $1000, and your interest rate is 16%. To make it simple, we assume the credit card company only requires you to pay 1% as your minimal payment every year, so the effective interest rate is 15%. After 30 years of compound interest you owe almost $60 000!

Compound Interest Graph

If there would be no compounding, you’d just owe a little bit over 5 grand!

What I find truly bizarre though is that when us software engineers throw around words like “technological debt” the eyes of our project managers or CEOs frequently just glaze over. Instead of doing the right thing – I’ll get back to that later – we are asked to come up with the quick hack that will make it work tomorrow and deal with the fallout later. Really? Sounds like we are using one credit card to pay off the other.

And we are even staying within terminology using “debt”! We could have said something like “Well, it would take us roughly 1 week longer to integrate our current J2EE backend with this 3rd party SOAP API instead of expanding our current custom XML parser, but then we would be done for good with maintaining that (POS) part of the app and can focus on our core IP.” But no, we keep it simple and refer to the custom XML parser as “technological debt”, but to no avail.

Now, the next time you have this conversation with your boss, show him the plot above and label the y-axis with “lines of code we have to maintain”, and the x-axis with “development iterations”, and perhaps a bell will go off.

Coming back to doing the right thing. Unfortunately determining what is the right thing is sometimes hard, but here are two strategies that in my experience decrease technological debt almost immediately:

  1. Refactor early and often
  2. Outsource as much as possible of what you don’t consider your core competency.

For instance, if you have to consume millions of tweets every day, but your core competency does not contain:

  • developing high performance code that is distributed in the cloud
  • writing parsers processing real time social activity data
  • maintaining OAuth client code and access tokens
  • keeping up with squishy rate limits and evolving social activity APIs

then it might be time for you to talk to us at Gnip!