A small (5-10) core team of people who work well together and who share the same goals and ideals for the project is key to the success of the project. In FHSST, the basic core team of Mark Horner, Sam Halliday, Rory Adams, Sarah Blyth and Spencer Wheaton, has stayed consistent over the six years of the project's lifetime. This helps to ensure consistency in the project and that the main aims and goals are clear when/if new team members join. Also, it is logistically simpler to communicate within a small team and ensure availability of people at group meetings.

To promote efficiency and allow people to take initiative in the core team it is important to have clearly defined roles. Within FHSST we mostly did not have clear roles and this sometimes led to certain team members taking on more tasks than was necessary which could have been avoided by better delegation of responsibilities. From our experiences, the roles we believe are required to make a successful core team are:

One person can take on more than one role and more than one person can fulfil a particular role depending on the situation. However, it's important to communicate who is responsible for what. Also, this requires a team with people who are multi-skilled. In our case, most of us could programme, all of us were qualified from our studies to be able to write content etc. We also found that the core team needs to be flexible in their responsibilities since the needs of content creation projects change over time from content gathering, to content editing, and finally to the printing and distribution phases. This requires open and regular communication.

The importance of open and regular communication between team members cannot be stressed strongly enough. During the main content creation phase of the project we held weekly meetings between the team members involved in that area which were particularly important since the team was geographically divided between continents. We held our conference calls using Skype to cut telephone costs. These meetings were always run with an agenda and included feedback sessions as well as team brainstorming sessions. Minutes were taken and used to inform the following meeting and help with project tracking. However, when regular meetings and communication stopped at one stage, due to various team members having other priorities, a valuable contributor to the project decided to leave. Better and more regular communication would most likely have allowed the team to avoid getting to that stage and would ultimately have required less time as they needed to be replaced which took a large amount of time.

New members to the core team need to be integrated properly to be most effective and productive. We found that it is very important to communicate the project history and goals upfront to new team members and to support them in new roles for some time before completely handing over responsibilities. It has to be remembered that for new team members coming on to the project need some time to assimilate the way things are done within the team and find their comfort zone. It is important to support new members and to have open and regular communication to iron out issues quickly.

It is important to understand the workflow associated with material. Opening the site up for anyone to contribute means that you will get random contributions which may or may not be in-line with the objectives of the project. You should allow anyone to participate and contribute but have a means for checking that those contributions are meaningful. You will also need to provide feedback on what was or was not appropriate about the material. The platform should allow you to automate as many management processes as possible.

FHSST identified the need to automate processes since coordinators were spending most of their time in back and forth email exchanges just to identify possible assignments for new volunteers. The automated processes removed this workload entirely and gave volunteers more independence in choosing their assignments.

It also allowed automation of management emails and ensuring that the correct people were automatically notified when a volunteer required feedback without the volunteer having to directly contact a particular person. This gave the team freedom to change reviewers without having to notify all volunteers and reduced the need for coordinators to send personal emails.

A book is not a loosely connected set of pages (think wiki) but a structured document with consistent structure and coherent flow and style. It is best to incorporate as much of the final structure into your online development as possible so that everyone knows what is expected and how it will all fit together. The structure should allow for the objectives (syllabus) to be displayed as part of the structure and allow for the relevant material to be written. Deciding how you want to approach this is important. We found that insufficient clarity and curriculum information about section requirements slowed down the content creation process because of the detailed communication required and the reliance on volunteer coordinators having time to respond.

Once you have decided on how you want to structure your books and manage the contributions, a platform needs to be chosen. We advocate using a content management system (CMS) because it includes authoring tools, provides workflow functionality, many community-related tools and acts as a repository for content.

A wiki solution is also a natural first thought and early in the project, FHSST ported two books to Wikibooks in an attempt to leverage off their community and infrastructure. However, we found that a wiki is not suitable for a number of reasons:

It is possible to add workflow and other functionality to a wiki-based platform but then, for all intents and purposes, it becomes a fully-fledged CMS!

During the very early stages of FHSST when we were using only the tools that were most familiar to the physics graduate students in the core team (CVS5, LaTeX6) we discovered that, despite our best efforts to train and support volunteers, the technical barrier to contributing proved a massive challenge and inhibited the project significantly. It is imperative to set the technical barrier-to-entry as low as possible. FHSST lost a number of potential volunteers through people not being able to configure their software appropriately or being intimidated to use LaTeX. It is also important to remember that you need to meet goals surrounding layout and typesetting of the final book which were well met by using LaTeX but also proved to be the major barrier to entry for the majority of volunteers.

If the final product, i.e. a book, looks inferior it will be judged as inferior. If the project is to write a book that includes equations and graphs then it is even more challenging to ensure a quality finished product. Two web-frameworks which require the volunteers to only have internet access to contribute which are worth considering for producing a printed book are:

We encourage the use of open-source software projects like these two and suggest, if possible, partnering with existing initiatives to strengthen projects that already exist.

To get the best 'return on investment' in terms of effort spent recruiting and number of volunteers recruited, a project needs to identify target groups for recruitment. FHSST had a lot of success with recruiting university students (mainly postgraduates) because this was the peer group of most of the core team and therefore accessible to recruit through word-of-mouth and also because students typically have more flexible schedules within which to include volunteer activities. Further, the students we recruited typically had similar skills to the core team and were familiar with the technology used to write content and so the barrier to entry for this group was low. In addition, these students had expertise in one of our book subject areas.

Scientists and university staff were also recruited to contribute, mainly, but certainly not always, through personal relationships with various university or laboratory departments.

Teachers were another large target group we focussed on recruiting and the project core team grew to include a teacher. In addition we recruited four further very experienced teachers to perform final editing of the content to ensure that the content applied properly to the curriculum and was useful from the point of view of people with experience in the classroom. We found that at first teachers were very sceptical about the project's prospects but as the body of content produced grew it added to our credibility.

We did attempt to recruit contributions from people in business and industry but were largely unsuccessful in this area mostly likely due to the time constraints on these people from their work. Our attempts here were modest but we believe that engaging management in large organisations in related areas could prove beneficial for future projects.

When a project has a large pool of volunteers, particularly when they are geographically dispersed, regular communication is an important tool to use to inform people of the latest project developments, plans and achievements. We also believe it is important to keep the communication of project strategies open to the larger volunteer base and we regularly shared our proposals and ideas with our pool of volunteers. We also asked for input on various issues from volunteers. Volunteer feedback was typically quite rare, however, we received feedback that they appreciated the inclusion.

There are various methods of communicating which can be used to fulfil various functions and which we found to be successful:

Holding regular hackathons was a way to facilitate a large volume of content to be created / edited in a very short space of time as well as a way of promoting a sense of team spirit and fun among volunteers. Our FHSST hackathons consisted of getting together a group of volunteers, sitting in the same physical location (either a coffee shop with wireless internet access or a university computer lab) and working on developing or editing content. We found this was a great way to create a face-to-face community within the project and also provided people with a set time per week/month to dedicate to the project where they felt motivated by the enthusiasm of the other volunteers to contribute.

To promote the aspect of fun, thank our volunteers for their contributions, and increase the length of time people would want to stay at hackathons, we also provided free pizza which was a big success – particularly for motivating university students! Also, having an interval in the middle of a hackathon for volunteers to socialise and meet each other added to the team atmosphere.

We also found that holding hackathons and reporting back on them on the website (including photos of pizza-eating volunteers) led to volunteers in other places holding their own hackathons. In this way we had a group of engineers from industry set up and hold their own hackathon to create FHSST content!

For a detailed how-to on holding hackathons please consult the ISKME Hackathon How-To.

A curriculum-aligned book structure which contains the syllabus requirements is necessary to guide volunteers regarding what they should write as well as ensure that the project stays focussed and on-track, or at least provides a well-defined measure for how off-track it is. FHSST used Drupal's book module with each chapter being a new node in the book hierarchy and also added a new field to each node which contained the curriculum syllabus and outcomes. These were only editable by the coordinator for each specific book.

The book coordinator must the be the person who configures, manages and modifies the overall structure. The FHSST rule-of-thumb for a coordinator is “someone who could write the entire book themselves if they had the time”. FHSST coordinators were required to set up the book structure and fill in the curriculum details.

Coordinators are vital in the workflow process to respond to queries, provide feedback to volunteers during the writing process, evaluate all submitted content and edit content .

Breaking the book structure into small, well-defined assignments allows users to browse and select a small section as an assignment. This significantly

In the case of FHSST, we found that collaborative development of individual assignments did not work and volunteers wanted exclusive editing rights and their assignment to be reviewed only when they felt it was ready. Initially we asked people to write chapters as assignments but found that this was intimidating to volunteers and often we did not receive anything from them at all. Breaking assignments down into smaller chunks resulted in better returns.

We found that the the administrative load of assigning initial work to volunteers could become quite significant, especially in periods where recruiting had been done which resulted in a significant volunteer increase for a specific book. By having a book framework with distinct units which could be listed as assignments, volunteers could browse available sections and select their own assignments. Once selected, the assignment would be removed from the assignment list so that only one person could work on it at a time. Volunteers were only allowed one assignment at first. Once this first assignment had been reviewed they could then have multiple assignments allocated to them. This was a means to assess the commitment and quality of the volunteer and ensure that volunteers who could not deliver were not able to “squat” on many assignments. Users could also relinquish assignments of their own accord, though this happened rarely.

We tried to build management dashboards (management overview page) on a book-by-book basis which showed which assignments had been dormant for long periods of time. We colour coded dormancy periods (green, yellow, red) which were configurable on a book-by-book basis (default settings were 21, 35, 42 days respectively). The tools then allowed bulk relinquishing of assignments through automated notification emails. This would put long-dormant assignments back into the pool to be chosen by other volunteers.

Ensuring that content development, with revision control, takes place on your website means that all contributions are effectively collected even if they are not completed assignments. In the early days of FHSST, before online authoring, volunteers would claim to have worked on, typically large, sections but not complete them. These partially complete sections were never sent in, what communication did take place indicated that the primary concern on the part of the volunteer was that the section was not complete. By working online all contributions can be captured. It also means that the entire project can be backed up from a central location.

If a project is very successful then the book coordinator should not be able to cope with reviewing all contributions. We implemented autmoated workflow to manage content quality:

We also used our workflow to help with the identification of special volunteers by reviewing a number of contributions from a volunteer. This allowed us to assess whether or not they understood the project, curriculum, language and style objectives sufficiently well to provide other volunteers with feedback. Volunteers who we felt met these criteria were promoted to an intermediate role (editor) between volunteer and coordinator. They were allowed to provide feedback to other volunteers. If they excelled at this they could eventually become a coordinator for the book.

By providing a metric for the review of submitted assignments an automated process of identifying special volunteers can be implemented. When we implemented our full CMS, we were already in the editing phase for the FHSST books so we were not able to implement this aspect fully but given our experience we felt the following to be a first set of metrics:

A user had to submit at least 5 assignments and have an average rating of >3.5 on a scale of 1 to 5 for the complexity ratings. It also allows for the training of users with a clear acknowledgement of achievement.

Hackathons, as defined earlier, are great events for content development which result in significant content generation in a relatively short period of time. There are some important content-related things to consider when holding hackathons which help to significantly improve their success. Hackathons should:

We also found that providing a progress report of the project at hackathons was well-received. Another volunteer favourite was when we provided pizza.

FHSST attempted to run competitions as an additional method to gather content and essays. Unfortunately, the competitions were relatively unsuccessful. In reviewing the competitions and considering current competitions in South Africa we attribute the failure to lack of effective advertising and a high barrier to entry.

It is prohibitively expensive to advertise on a large scale unless a project partners with a media-related partner or has significant public presence. Most current competitions have significant repeated exposure on TV and radio as well as a large prize. In contrast, FHSST was only able to afford to place a small number of print advertisements in a newspaper or a magazine for teachers as well as advertising on the Science in Africa online magazine. These adverts were possibly not targeted ideally and definitely not seen often enough by possible entrants. However, advertising was limited by our budget.

In most competitions, very little is required for entry; the barrier to entry is extremely low. For FHSST competitions, contestants were required to generate new content, a significant effort relative to other competitions, despite comparable prizes. We believe that this was directly responsible for our low entry rate.

In future, we think competitions could still work if significant exposure can be generated and the target audience is able to enter with minimal effort. For example, a significant advertising campaign for a competition with a cash prize, where teachers contribute an activity or exercise they have already constructed may be the best structure.

Another way to generate interest amongst contestants may be for the prize to become a status symbol. This could be possible if the content development project becomes widely known and respected and has significant enough impact to be viewed favourably amongst the target group.

A project developing material for a more general audience might try to get their competition listed on a news website like Digg or Slashdot which would result in significant exposure as well.

In preparation for the tests we constructed our baseline questionnaires to assess what teachers and learners did and didn't like about textbooks, what they thought could be improved, as well as what their favourite textbooks were and why. This was done to ensure that feedback on FHSST material was consistent and final results could be cross-checked. We were concerned that exposure to the FHSST material would in some way bias the results.

Content was delivered to 10 schools to use for 1 month. FHSST was not able to dictate how the material was to be used in the classroom and most teachers used the material in addition to their current primary resources. At the end of the month all teachers and learners completed questionnaires relating specifically to the FHSST material.

We collated the final results and cross-referenced them with the initial results. These were then presented to the participating teachers at a workshop we organised. The helped us to clarify any unclear answers and include the teachers in discussions around improvements. We received a lot of positive feedback from teachers, specifically about being included in the final workshop.

Ultimately the results from the trial feedback were fed into the final editing process which only started after the trials were completed.