After being very heads down on the Gnip platform this month we wanted to let everyone know that the test system will be available early next week with the new and improved schema. Again, the new schema can be viewed in our forums. Also, there were some data producer examples in my last post and in the our forums that should be helpful in understanding the broader meta-data we are now providing.
Also, we are still working on the length for the beta to allow people to prototype before it is moved to production, but expect this will at least be for the month of February. We also are looking at how to continue to provide access to the current schema after making the update to the new one. So, stay tuned for more information next week.
As we come to the end of January it is time to provide some information on the upcoming new Gnip schema that we announced earlier this month.
First, a big THANK YOU to all the people who provided feedback, thoughts and examples of where you are using Gnip and why it was important to have the schema support your real-world use cases. These were very important to our team as we made decisions on this major update to our schema. The biggest decision we made was to keep all the existing meta-data in the Gnip Notifications that exist today, which was not what we had in the draft that was originally shared with the community. We think this was absolutely the right thing to do in the end, but it did mean we ended up with a bit of re-work to get to the final version of the schema now being completed in our platform. Our timing has slipped a couple weeks since the general announcement the first week of this month. Therefore, we now plan to have the demo system up and running no later then the end of January to start allowing people to work with the new schema. We will be pushing out more information on the updated schedule in the coming week.
The final version of the schema can be viewed in our forums. In addition, here are some examples of data streaming from Gnip with the “existing” and “new” schema. From the feedback we are receiving and from doing our own tests we believe the new schema is purely additive in terms of providing more rich meta-data to access, filter and integrate data.
1) Comparing “existing” Gnip Activity Notification schema to “new” schema from a random public tweet.
publisher=”twitter”, activity=”notice”
a) Existing Gnip schema
<activity source=”Tween”url=”http://twitter.com/statuses/show/1144685412.xml” action=”notice” actor=”kazusap” at=”2009-01-24T15:23:21.000Z”/>
b) New Gnip schema
<activity>
<at>2009-01-24T15:23:21.000Z</at>
<action>notice</action>
<activityID>tag:twitter.com,2007:statuses/1144685412</activityID>
<URL>http://twitter.com/statuses/show/1144685412.xml</URL>
<source>Tween</source>
<actor uid=”14202410″ metaURL=”http://twitter.com/kazusap”>kazusap</actor>
<destinationURL>http://twitter.com/kazusap/status/1144685412</destinationURL>
<to metaURL=”http://twitter.com/mossy_13″>mossy_13</to>
<regardingURL>http://twitter.com/mossy_13/status/1144684485</regardingURL>
</activity>
2) Comparing “existing” Gnip Activity Data Stream schema to “new” schema for a Delicious bookmark
publisher=”delicious”, activity=”bookmark”
a) Existing Gnip Activity Data Stream schema showing Delicious bookmark
<activity tags=”economists,economy,USA,recession,depression,politics,economics,barakobama,JohnMaynardKeynes” source=”http://delicious.com/url/35adba93e5e3fb83198b5dbabd2fbb9e#asterisk2a” regarding=”http://www.nytimes.com/2009/01/23/opinion/23krugman.html” url=”http://delicious.com/url/35adba93e5e3fb83198b5dbabd2fbb9e” action=”bookmark” actor=”asterisk2a” at=”2009-01-24T17:15:43.000Z”/>
b) New Gnip Activity Data Stream schema showing Delicious bookmark
<activity>
<at>2009-01-24T17:15:43.000Z</at>
<action>bookmark</action>
<activityID>http://delicious.com/url/35adba93e5e3fb83198b5dbabd2fbb9e#asterisk2a</activityID>
<actor metaURL=”http://delicious.com/asterisk2a”>asterisk2a</actor>
<destinationURL metaURL=”http://delicious.com/url/35adba93e5e3fb83198b5dbabd2fbb9e”>http://www.nytimes.com/2009/01/23/opinion
/23krugman.html</destinationURL>
<tag metaURL=”http://delicious.com/economists”>economists</tag>
<tag metaURL=”http://delicious.com/economy”>economy</tag>
<tag metaURL=”http://delicious.com/USA”>USA</tag>
<tag metaURL=”http://delicious.com/recession”>recession</tag>
<tag metaURL=”http://delicious.com/depression”>depression</tag>
<tag metaURL=”http://delicious.com/politics”>politics</tag>
<tag metaURL=”http://delicious.com/economics”>economics</tag>
<tag metaURL=”http://delicious.com/barackobama”>barackobama</tag>
<tag metaURL=”http://delicious.com/JohnMaynardKeynes”>JohnMaynardKeynes</tag>
<regardingURL>http://www.nytimes.com/2009/01/23/opinion/23krugman.html</regardingURL>
<payload>
<title>Op-Ed Columnist – Stuck in the Muddle – NYTimes.com</title>
<raw>H4sIAMBMe0kC/63UQW/TMBQH8DufwpQDB1jdJqtYIy9StXEZDNA6Dmja4dl+aa3EdmQ7RPn2eB1u
NjiAID5Zlt/vbztPYSqgLl+Qw2BBhQbLz+3Je0kubNNpo3wgJ2QbOlETZUjYI7nupGwwrn76dqs0
+rmwmtHH0gS1Hb+EgOUWwluSnZIrMCRbLNZk+a5YrorTnLxZxMFo2pgKd52SRPkv6DR8VKY+n1XQ
eJyV+xDaglKJjRLKdodU2rmG5iuQHNY5rjCv+Fm+XJ/xVVzhMqs4X+Mr8AGd8nUGjD7wx6wm+snt
+35uhpDuQx8OSxdLmuXUtsooa+K0dt1Og5nvg24YPZQnS4pCOIRgXcle3l1cbm43d2Pw/X3J6JMt
qSomaTTB//vtGD0aCe2rvvi5eOOPdIUo/fx5wPeMOu//MucXNqV52zmBJBrnsz9HjU8yK8f5a0+4
tbUGV3tGH8XxjWJ77KwbiLQalDmmPPdHjM5KFNZYHVs3aqn8/71hGuzrdjMN5FCg97Ezp+Ektm5K
r7WNCkpM+gkm4zg4ELXloGEa8MruzTUMBpz8gIPB387J6OFP+wNsAa+TcAUAAA==
</raw>
</payload>
</activity>