The development of web applications for Higher Education lags that of the mainstream Internet, leaving the latest generation of students and researchers disenfranchised. Development of Internet applications has gone through paradigm shifts moving from monolithic relational stores to cloud based content storage. These shifts have changed the way in which applications are built, and the speed with which they are developed. This article explores the driving forces moving Sakai to join the new era of Social applications by adopting a content focused methodology. This exploration is performed by looking at the way in which the role of content has developed through various phases of the Internet, and how educational computing has leveraged those developments. The article then goes on to relate these developments to the way in which initiatives like OpenSocial are changing the nature of application development and hosting, discussing the impact of the new world of cloud aware and cloud based applications on the development of Sakai. From this analysis the author has made some predictions for the future of virtual teaching learning and research environment software and how these applications will become more a part of the educational users’ daily life. The article will be of particular interest to those readers considering the tensions between institutionally provisioned applications and global free to use web services.

Purpose: This article explores the driving forces moving Sakai to join the new era of Social applications by adopting a content focused methodology.

Approach: This exploration is performed by looking at the way in which the role of content has developed through various phases of the Internet, and how educational computing has leveraged those developments. The article then goes on to relate these developments to the way in which initiatives like OpenSocial are changing the nature of application development and hosting, discussing the impact of the new world of cloud aware and cloud based applications on the development of Sakai.

Findings: Shifting to a modern web development paradigm that includes heavy use of client-side programming methodologies such as AJAX, a content-centric architecture, and an implementation of social networking capabilities will increase student satisfaction while reducing development time and cost for systems like Sakai.

Value: The article will be of particular interest to those readers considering the tensions between institutionally provisioned applications and global free to use web services.

Keywords: Sakai, social software, distributed learning environments, Web 2.0, OpenSocial, agile development

In the brave new world of distributed learning management systems, connecting network API’s that enable rapid reactive UX design and implementation, everything is content, because everything we communicate in a distributed learning management system can be represented as content in files or to be more precise, a stream of bytes delivered over the network. It’s a strange concept having been lead to believe that in the implementation of systems we had to build data models that created a rich relational structure.

Fundamental to all human interaction, ever since the cave man, is the exchange of information and collaboration. The significant watersheds in the internet age have connected to this human desire. In CERN at the internet’s birth, rich information sharing was the fundamental driving force for layering the web and http on the emerging academic IP networks. As we pass the 18th anniversary of the first byte between web client and web server much has changed in the users experience, but in some respects the development path is returning to an experience those early users were hoping for, meaningful content with semantic markup.

So the mantra remains, content is king, not just is the sense that the creation of content is where the real value is, but in the process of creation and the interaction around creation and reuse. For me, in November 2007, close to a year on from the announcement of OpenSocial [OpenSocial] by Google, we passed another watershed in the internet age. Now the web is Social. In fact it always was social, but just as the semantic intention of the web declined, the collaboration and interaction of the social nature of the ‘we’ has not been expressed until now. With OpenSocial, we are starting to talk the same semantic language, and starting make it possible to share that language. There are very clear reasons why Google cares so much about OpenSocial and has become intent on layering this semantic knowledge of our relationships on the web. At present it has some information about an individuals relationship with information, through our use of its existing applications and is able to determine the likely value of advertising certain things to us, based on that information. Improving the quality and effectiveness of this information drives its business model and enables it to ‘turn the dials’ and achieve quarter on quarter results. There is a limit to how effective that knowledge is, and it is based on assumptions about why we might be interested in certain pages and search terms.

Google has undoubtedly learnt from GMail and Google Apps that there is more raw information to feed its core business model from user-created content than from traditional internet media, and it has will have certainly noticed that we all read emails from friends and delete spam from interlopers. Making the web Social and spreading the exposure of users to OpenSocial gives Google the opportunity, with other OpenSocial players to increase a users expression about their relationships. So this is the new web, richer with semantic markup associated with our collaborative interactions.

Higher education institutions are constantly being refreshed with new members. Research lead institutions like Cambridge nurture their incoming student population to develop first class researchers, loosing many to commerce and industry in the process. Institutions more focused on teaching and learning probably have a higher turnover of fresh blood, with the student community experiencing a complete refresh every few years. Nothing is static within this environment, and the experience that the constantly changing members of a University expect mirrors the influence of the wider internet. All of this is obvious, but so frequently institutions, capable of reacting to white heat of progress in research, fail to recognize or react to all that is around them. Perhaps its fear of inability to deliver stable systems, but I fear, more often it’s the systems themselves, the snail-like delivery of vaporware and broken promises that leave the institution with no option but to require students to use systems that barely engage. Students use the systems only out of necessity rather than choice, preferring to self create study groups on Facebook.

In research and everything we do in education surrounds communication of ideas between human beings. That communication is supported by relationships embellished with attributes. The value of these attributes are greater than is generally expressed in a Social network. The Social Friend in MySpace sometimes equates to ‘an old and valued friend’ but also equates to a ‘celebrity fan’. The Social Friend in an academic network has status, trust, funding, tenure and potential associated with it. Enhancing these social links is distributed content. The co-authored paper. The peer reviewed publication. The joint lecture series or the tutorial group. Content remains king, not to be hidden or abstracted inside silos the prevent re-use but to be celebrated and exposed. So the emphasis on the value of human relationships embodied by the social web is of direct relevance to academia.

When OpenSocial was announced, at a broadcast camp-side talk [Campfire] given by Google, the instigators talked enthusiastically about the tens of thousands of developers who are creating applications on the web. That army of developers, like a flock of roosting birds at sunset flow and redirect at the slightest encouragement. OpenSocial intends to direct these developers through encouragement to build the social web. OpenSocial aims to standardize the interactions of the applications those developers are creating, to make their applications work inside any OpenSocial container. The encouragement for the developer being the ability to reach a greater potential user community. One year on from the announcement the total user community of OpenSocial containers is greater than the population of most European countries, and yet, the standard has not yet reached version 1. The potential for the internet to create speed of growth is unsurpassed, and its ability to destroy endeavors that don’t move fast enough is ever present.

OpenSocial was developed as a standard to address the needs of all social networks. Without that ability, support and acceptance it would not become a standard. The early work on the standard analyzed the needs of social network applications and condensed that interaction into three services. A person service, delivering a user focused view into the social graph, where friends are connected and associated with the social application of gadget in question. Applications are unlikely to be of value to a user without the ability to persist user preferences and content. The second service is an application data service that stores, content for the user on behalf of the application. This parallels the concepts in Googles GData but in a gadget context. With all interactions, activities are performed, and so there is an Activity service that provides a mechanism for delivering a record of activities into the social network. The dissemination of this activity feed often appears as a mini feed on a friends home page. As the standard develops more services are being added to address specific areas of functionality, but it is the belief is that these services presented as Javascript API’s will make it easier and faster to develop social applications, those applications increasing the richness of the internet.

Social gadgets are developed in a way that plays to ease of development and speed of deployment. The majority use client side techniques to deliver a greater level of responsiveness and interactivity than is possible with server side technologies. The environment is purposefully concept light. Removing barriers to developer entry, and eliminating restrictions on creativity. Few web developers will tolerate frameworks that prevent them for delivering the user experience that they believe is desired. With this speed of development beliefs are soon tested and verified in production.

The primary model for development is client side, playing to the UI developers strengths, using pure HTML and Javascript to build the a UI that delivers a UX design connecting the user with OpenSocial Javascript API’s. The development cycle is distant from heavy server side frameworks, without the build systems and complicated deployment cycles. With larger applications this accelerates the development process. Its not that server side development strategies are inherently slow, but many require build and deployment cycles. Many heavy server side technologies are typified by the amount of developer time wasted on non-productive operations. A build and deploy that takes more than a minute will break a train of thought. All too frequently I encounter developers aimlessly reading email, surfing the web or doing ‘social research’ on Facebook whilst waiting for a build to complete, breaking concentration and focus. Its not the reading of email or even the ‘social research’ that I object to, its the fact the developers train of thought is broken by a frequently pointless and repetitive operation that only supports modifications to a few lines of code. In the past year I have observed an order of magnitude increase in developer productivity using concept light, prescription free client side approaches.

Not every developer is prepared to work client side, and some developers feel more productive developing Ruby, Perl, Python and PHP applications. The Javascript API’s of OpenSocial are backed by lightweight RESTfull wire APIs that are suitable for server-to-server consumption.

At Cambridge we have been paralleling this development. Shortly after the OpenSocial announcement had a moment of realization. We had been overrunning many of our development projects. With relief, we realized this was not due to the inability of our development teams, but the tools and framework that they were working with. Breaking the rules, we blame our tools or at least the environment we were working in. Sakai 2x, although it provides a complete environment for developing innovative applications for teaching learning and research, it does so at a cost. A full build of Sakai can take up to half an hour, fortunately the modularity and SOA nature of its architecture means modules can be individually built. Still, even with server side frameworks, an expert Java developer is required to translate the desires of a UX designer into reality. Sadly there are few Java developers who I would classify as fantastic UI developers. So the timescales of development delay implementation, loosing momentum and focus resulting in a greater level of rework and delay than is acceptable.

The realization that occurred after the OpenSocial announcement was that all of this was unnecessary. The torment we had been experiencing with stagnating productivity and delays could be eliminated by restructuring the development teams. We transfered responsibility for driving the process from the Java developer to the UX designer. New UI developers were hired who understood how to translate the needs of the UX designs into reality, and the UI developers told the Java developers what was needed to feed their client side applications. The majority of Java developers, being great at creating elegant and efficient data processing architectures were freed from the responsibility of delivering beautiful usable UIs, and started to produce wonderfully designed data feeds.

If this sounds too much like utopia; the reality is as by comparison it was. Our development cycles were cut by an order of magnitude, UX designers were able to conduct a productive relationship with users and best of all the project stakeholders saw delivery on time. In the period of 3 months we rebuilt the entire UX of CamTools, put it though QA and got it into production, while the main CamTools production team did a version upgrade.

We proved to ourselves that our realization was right, and or faith in the approach of OpenSocial was well placed. But nothing stands still and to continue to deliver improvements to our changing and demanding user community at Cambridge, we need to continue this pace.

The dramatic change in place did not go unnoticed amongst the Sakai community, many other institutions, facing pent up demand, could see the opportunity to draw on previously excluded resources in their own institutions.

The unexpected side effect of this work was to eliminate many of the technical interaction barriers imposed by the portal-like nature of the development environment. We adopted a widget [MYSAKAI] based approach that maintained the modularity and service separation of Sakai, but eliminated the need for heavy server side portal technologies, full page refreshes imposing significant server side load converted into incremental UI updates. This widget approach, close in nature to Google Gadgets but without the need for iframes provided a totally flat page experience. The first UX barrier was removed, normal web navigation semantics were restored to Sakai. I probably shouldn’t mention this, since every application on the web ignores normal navigation semantics and the normal operation of the back button at its peril. Sakai, due to the nature of its portal and its inception at a time before browser tabs had compromised the normal operation of the browsers back button to maintain portal state. With a widget based client side approach we have been able to restore this functionality. The impact on user acceptance has been dramatic, and we have started a journey to the CamTools becoming and peer in terms of acceptance with the likes of Facebook.

The second phase of that journey involves the entire community. Cambridge is not alone in recording user frustration and disengagement. Some schools in pilot phase have chosen commercial or competitive open source systems. This is not a tale of wide scale unrest, but simple an acknowledgment that the user experience in Sakai 2x is dated. That is all about to change.

We have two projects underway in the community that are rewriting Sakai from the ground up based on the development principals embedded in OpenSocial and the lessons we learnt rebuilding CamTools this year. Sakai 3 will be a significant change in user experience and functionality. This is not a promise of vaporware, the very early prototype of this work has been in production with the campus at Cambridge for six months, and the rewrite is well underway. The focus of this new environment plays true to the enthusiasm of the founders of the OpenSocial movement, targeting the UX designers and the UI developers by enabling them to deliver a great user interface with ease. As with OpenSocial this user interface communicates with the back end using RESTfull web services transporting JSON payloads. The processing in the client is simple, fast and efficient. We have already validated areas of the UI on a wide range of client side platforms including smart phones, small format web devices such as the iPhone and iTouch, as well as all the major web browsers. The network requirements are surprisingly low as the majority of interaction is in the form of extremely compact data packets as opposed to bulky full page refresh. This is the focus of the Sakai UX project.

Moving focus to the back end, with the Sakai Kernel 2 project, we have transitioned to a world were content truly is king. The core of the system is based on a RESTfull web service connecting to a Java Content Repository [JCR] conforming to the JSR-170 specification. The default provider will be Apache Jackrabbit. although other providers such as Oracle Beehive and Xythos are possible. The transactional nature of JCR enables us to eliminate the wasteful process of translating database records into JSON [JSON] feeds and back. The content is the feed, written by the client, stored as a byte stream in JCR, and delivered back to the client on demand. This saves cpu cycles and time serving each request and has additional side benefits. It also almost eliminates the need for a back end Java developer to painstakingly develop new feeds, as the UI developer can perform this task in the client, by simply creating the data structure required and persisting it on the server. The approach may sound over simplistic and limiting, but strangely it works extremely well, only rarely requiring a non-content-based feed to be created by the Java developer.

The approach has other benefits. We now know precisely when the content of the feed was last updated, so we can ensure that content is cached correctly. All of this leads the average amount of time that the server spends processing a request being lower leading to greater server responsiveness and greater scalability. The result for the all important UX is less latency between user actions although, with no page reload load these are already orders of magnitude less.

To mention content probably invokes the impression that we are talking about documents, and files. In this environment, that type of content is just one small category. Everything is content. The structure of a course expressed as a JSON data model, stored in a file; that is content. The description of a collaboration site, he structure of a discussion thread, the posts within it and their relationship to one another, are all content, stored in files in the JCR, expressed as JSON data models.

Why JSON? In the smallest, most power efficient mobile devices we have limited processing power. We have a duty not to waste that power. All browser technologies have native JSON parsers that are hyper efficient, and so, we use JSON. Even with powerful client machines choosing the most efficient data format frees processing time to deliver a great user experience with minimal latency.

Storing everything as content generates some difficulties. We need to be able to find items based on their metadata. Pure content stores have notoriously low performance when it comes to relational queries, and so, as content is updated we build a relational index of pointers in a database. This relational index allows us to build alternative hierarchies or views into the content store. With site descriptions as content, we can index the metadata and discover, for example, all engineering courses on campus in semiconductor technology, probably not many. But, from course section information, a view representing a lab session schedule becomes possible. This liberates the Sakai environment from content silos enabling all material to be stored anywhere within the content store. With metadata driven views into the content system we can easily transform the VLE from a course focused application into a person focused store for continuous professional development or portfolio. Each are different presentations of the same content sets.

Having liberated the data in Sakai from a relational store and provided unique addressable URI’s for each item of content, layering the standard http methods we have opened the possibility for cloud based storage [Chang, Dean et al, 2006]. The central Sakai server only needs to know about data that has been updated, and its location. Provided it can reliably retrieve that data from a URL derived form the items URI, the data could be anywhere. In the future we will be able to deploy content storage services connected by ultra light web services that use cloud based storage. That cloud based storage being either inside a Sakai cluster or further afield, potentially another Sakai cluster at a collaborating university or a cloud storage provider. Clearly distributing entire files would require an additional layer of cloud storage but with an intelligent streaming proxy it will be possible to reconstruct file fragments scattered into the cloud, addressed by a distributed hash table. Migration to cloud based storage, especially a full implementation is not something to be taken lightly, however we have already abstracted the underlying persistence management of content, through the JCR. The architecture of Apache Jackrabbit [Jackrabbit] gives use the perfect opportunity to achieve a full cloud based storage option by customizing its internal persistence manager through which all content bodies flow.

This approach is a vast simplification from previous models, and has lead to an acceleration in the development of Sakai. We are moving to a world where the next generation of Sakai will require significantly less central server capacity per user and deliver far greater interactivity and usability to the user. Sakai is not deployed at a scale where we have to think to hard about provisioning power and vast data-centers, but by using a client side user interface much of the processing is shifted from the server to the client. This raises a scalability bar and utilizes the processing power of every users machine in one massive parallel cluster rather than wasting idle cycles and heat in 1000’s of client CPU’s.

So far little has been done to address the expectations of the student and research body. We have made progress to bring Sakai level with other web 2 applications, but we have not at yet made it social. By the time this goes to press OpenSocial will have had its first birthday. The specification is racing towards the 0.9 version driven by a growing community of interested parties on the OpenSocial specification list. With this the number of social networking containers adopting OpenSocial is growing, and so is the user population. Many are using the Apache Shindig project as their core code base. Many vendors are not waiting for a release but are deploying into production from the code base under active development. iGoogle’s OpenSocial sandbox regularly deploys the code base from Shindig’s leading edge into production.

Over the past 6 months parts of the social server have been restructured to identify a Service Provider Interface (SPI). This Shindig’s interface between the implementation of the OpenSocial API’s and the Social Network Server. In preparation for embedding Shindig within Sakai I have written a Java Persistence API based implementation of the SPI. This will enable us to embed Shindig within the Sakai’s new Kernel, making Sakai an Academic Social Network Server. Unsurprisingly there are other areas of Shindig that align with Sakai, the default caching implementation uses the same infrastructure, we share the same version of Google Guice used to construct the server on startup.

OpenSocial in itself does not create a Social Network Server, and Shindig is limited to a Reference Implementation of the OpenSocial standard. Like Apache Jackrabbit it is firmly targeted at proving itself in heavy production, and so is not a ‘play’ reference implementation. There is much to be done around the sides. We need to complete the integration of Shindig and OpenSocial into Sakai. This will create a Social Networking platform for higher education that can interoperate with other OpenSocial adopters at the same level.

So much for the technical details; what will this mean in practice for an institution deploying Social Sakai?

A plant science course can use an OpenSocial gadget that retrieves protein structures from an institutional repository, to communicate key concepts in binding and folding. That Gadget no longer owned and developed by core IT support services, but rather created in HTML with Javascript and owned by the research group in the Chemistry Department that also teaches that lecture series. Previously the barriers to creating such an application would have been prohibitive. Barriers preventing the individuals from creating the application in the first place: with Sakai 2, good java skills would be required. Barriers in getting the application into production: in the past it would have required re-deployment of the core application. These barriers were not unique to Sakai, all traditional server based application suffer from the same barriers. Blackboard has building blocks, a server based extension mechanism, Moodle allows extension via custom php code. In contrast the OpenSocial chemistry gadget is hosted as a file on a web server in the Chemistry department. Most of the development of this application requires the creation of HTML and use of the OpenSocial Javascript client APIs, well within the reach of most departmental webmasters. Implemented as an OpenSocial Gadget, the reach is widespread. The same gadget will work within Linked In, MySpace and many of the other social networks that have adopted OpenSocial. Suddenly academic applications are reaching out into the environment in which students and researchers consider to be their own.

The development of many of the core applications within Sakai itself will use OpenSocial Gadget encapsulation, making a library of academic gadgets available to other social networks. Sakai 3 has the Shindig Gadget server embedded within it, so it is capable of allowing the user to select gadgets from other OpenSocial based social networks. It has become possible to see GMail within Sakai. Lists of Google Docs become available as do more business and professional focused gadgets from networks like LinkedIn. Rather than being an isolated silo within a rapidly changing world Sakai 3 has become an active player within the social web that is driving the next wave of web utilization. Perhaps we should call with web 3.0, a phrase everyone has been itching to connect with.

We have only started to scratch the surface of understanding the real nature of academic networks. We already know that the social model employed in OpenSocial is immature when applied to academia. The complexity and sophistication of relationships between those in higher education go beyond the friends relationship of so much interest to the mainstream social networks. The recommender system that informs the user of potential connections needs to draw on a wider range of information. Shared subject areas both with student and researches. Common conference interests, peer review relationships and joint publications. Even this publication represents a social graph of academic interests. Many of the authors know each other, some are not connected, and others directly collaborate. We might even classify ourselves as Friends. However we are unlikely to want to share photos of the wedding party we went to last Saturday, which we now barely remember due to that alcohol induced amnesia experienced by all social drinkers as we bury our head in hands on Sunday morning, hiding from the brightness of the day.

At Cambridge we are actively pursuing research in this area. Our reason for starting with Sakai was fueled by this research. The JISC Virtual Research Environment Project we ran several years ago deployed Sakai as a collaborative platform to the Teaching and Learning Resource Programme, and Higher Education Funding Council research project into teaching and learning over the UK. TLRP covers some 40 sub-projects with around 400 researchers. The VRE project used Sakai to investigate the impact of online support on collaboration. The social graph that this work uncovered delivered a greater understanding of the life-cycle of the TLRP, its members and their publications. This application of technology into teaching and learning and research has not stopped. Caret is building on this work with a funded project to investigate the impact of social networking in an academic network. We will be actively developing and deploying social applications in Sakai 3 over the next 2 years and monitoring their impact on the community. This work will not only inform the development of the applications in the form of OpenSocial gadgets, but will help us build out the components of an Academic Social Network Server.

Unlike other social networks that are only prepared to collaborate on superficial level, the schools that run Sakai share code and often intimate details of their production environment. As more schools deploy Sakai 3 we have an exciting opportunity to build and increasingly rich social network that builds on initiatives like the Access Federation within academia and standards like OpenID and OAuth from industry. Google’s vision for OpenSocial, the social web, is likely to become far more connected between schools running Sakai that would have ever been possible between commercial entities. Although we have policy restrictions, we may be able share with trusted parties. Perhaps this is many years out, but many academics, striving for tenure, live by their publishing record, peer review and the accolade of their colleagues, all of which form the core of their profile in higher education. The new development methodologies we have adopted, the speed and ease of development and the networked nature of OpenSocial leads me to believe that Sakai 3 there are exciting developments in Academic Social networking on the Horizon.

[OpenSocial] The OpenSocial Specification, http://code.google.com/apis/opensocial/

[Campfire] Video of the OpenSocial Campfire Announcement, http://www.youtube.com/watch?v=9KOEbAZJTTk

[MYSAKAI] Sakai Widgets, http://bugs.sakaiproject.org/confluence/display/MYSAK/MySakai+Introduction

[JCR] Java Content Repository, http://en.wikipedia.org/wiki/Content_repository_API_for_Java

[JSON] Javascript Object Notation, http://json.org/

[Chang, Dean et al, 2006] Bigtable: A Distributed Storage System for Structured Data Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deborah A. Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, and Robert E. Gruber. OSDI'06: Seventh Symposium on Operating System Design and Implementation, Seattle, WA, November, 2006. http://labs.google.com/papers/bigtable.html

[Jackrabbit] Apache Jackrabbit, JCR/JSR-170 Reference Implementation, http://incubator.apache.org/