Status Update on New Gnip Schema and Demo System

As we come to the end of January it is time to provide some information on the upcoming new Gnip schema that we announced earlier this month.

First, a big THANK YOU to all the people who provided feedback, thoughts and examples of where you are using Gnip and why it was important to have the schema support your real-world use cases.   These were very important to our team as we made decisions on this major update to our schema.    The biggest decision we made was to keep all the existing meta-data in the Gnip Notifications that exist today, which was not what we had in the draft that was originally shared with the community.    We think this was absolutely the right thing to do in the end, but it did mean we ended up with a bit of re-work to get to the final version of the schema now being completed in our platform.   Our timing has slipped a couple weeks since the general announcement the first week of this month.  Therefore, we now plan to have the demo system up and running no later then the end of January to start allowing people to work with the new schema.   We will be pushing out more information on the updated schedule in the coming week.

The final version of the schema can be viewed in our forums.   In addition, here are some examples of data streaming from Gnip with the “existing” and “new” schema.    From the feedback we are receiving and from doing our own tests we believe the new schema is purely additive in terms of providing more rich meta-data to access, filter and integrate data.

1) Comparing “existing” Gnip Activity Notification schema to “new” schema from a random public tweet.

publisher=”twitter”, activity=”notice”

a) Existing Gnip schema

<activity source=”Tween”url=”http://twitter.com/statuses/show/1144685412.xml” action=”notice” actor=”kazusap” at=”2009-01-24T15:23:21.000Z”/>

b) New Gnip schema

<activity>
<at>2009-01-24T15:23:21.000Z</at>
<action>notice</action>
<activityID>tag:twitter.com,2007:statuses/1144685412</activityID>
<URL>http://twitter.com/statuses/show/1144685412.xml</URL>
<source>Tween</source>
<actor uid=”14202410″ metaURL=”http://twitter.com/kazusap”>kazusap</actor>
<destinationURL>http://twitter.com/kazusap/status/1144685412</destinationURL>
<to metaURL=”http://twitter.com/mossy_13″>mossy_13</to>
<regardingURL>http://twitter.com/mossy_13/status/1144684485</regardingURL>
</activity>

2) Comparing “existing” Gnip Activity Data Stream schema to “new” schema for a Delicious bookmark

publisher=”delicious”, activity=”bookmark”

a) Existing Gnip Activity Data Stream schema showing Delicious bookmark

<activity tags=”economists,economy,USA,recession,depression,politics,economics,barakobama,JohnMaynardKeynes” source=”http://delicious.com/url/35adba93e5e3fb83198b5dbabd2fbb9e#asterisk2a” regarding=”http://www.nytimes.com/2009/01/23/opinion/23krugman.html” url=”http://delicious.com/url/35adba93e5e3fb83198b5dbabd2fbb9e” action=”bookmark” actor=”asterisk2a” at=”2009-01-24T17:15:43.000Z”/>

b) New Gnip Activity Data Stream schema showing Delicious bookmark

<activity>
<at>2009-01-24T17:15:43.000Z</at>
<action>bookmark</action>
<activityID>http://delicious.com/url/35adba93e5e3fb83198b5dbabd2fbb9e#asterisk2a</activityID>
<actor metaURL=”http://delicious.com/asterisk2a”>asterisk2a</actor>
<destinationURL metaURL=”http://delicious.com/url/35adba93e5e3fb83198b5dbabd2fbb9e”>http://www.nytimes.com/2009/01/23/opinion
/23krugman.html</destinationURL>
<tag metaURL=”http://delicious.com/economists”>economists</tag>
<tag metaURL=”http://delicious.com/economy”>economy</tag>
<tag metaURL=”http://delicious.com/USA”>USA</tag>
<tag metaURL=”http://delicious.com/recession”>recession</tag>
<tag metaURL=”http://delicious.com/depression”>depression</tag>
<tag metaURL=”http://delicious.com/politics”>politics</tag>
<tag metaURL=”http://delicious.com/economics”>economics</tag>
<tag metaURL=”http://delicious.com/barackobama”>barackobama</tag>
<tag metaURL=”http://delicious.com/JohnMaynardKeynes”>JohnMaynardKeynes</tag>
<regardingURL>http://www.nytimes.com/2009/01/23/opinion/23krugman.html</regardingURL>
<payload>
<title>Op-Ed Columnist – Stuck in the Muddle – NYTimes.com</title>
<raw>H4sIAMBMe0kC/63UQW/TMBQH8DufwpQDB1jdJqtYIy9StXEZDNA6Dmja4dl+aa3EdmQ7RPn2eB1u
NjiAID5Zlt/vbztPYSqgLl+Qw2BBhQbLz+3Je0kubNNpo3wgJ2QbOlETZUjYI7nupGwwrn76dqs0
+rmwmtHH0gS1Hb+EgOUWwluSnZIrMCRbLNZk+a5YrorTnLxZxMFo2pgKd52SRPkv6DR8VKY+n1XQ
eJyV+xDaglKJjRLKdodU2rmG5iuQHNY5rjCv+Fm+XJ/xVVzhMqs4X+Mr8AGd8nUGjD7wx6wm+snt
+35uhpDuQx8OSxdLmuXUtsooa+K0dt1Og5nvg24YPZQnS4pCOIRgXcle3l1cbm43d2Pw/X3J6JMt
qSomaTTB//vtGD0aCe2rvvi5eOOPdIUo/fx5wPeMOu//MucXNqV52zmBJBrnsz9HjU8yK8f5a0+4
tbUGV3tGH8XxjWJ77KwbiLQalDmmPPdHjM5KFNZYHVs3aqn8/71hGuzrdjMN5FCg97Ezp+Ektm5K
r7WNCkpM+gkm4zg4ELXloGEa8MruzTUMBpz8gIPB387J6OFP+wNsAa+TcAUAAA==
</raw>
</payload>
</activity>