Skip to content Skip to navigation

OpenStax-CNX

You are here: Home » Content » A Survey of Users of Connexions Music Modules

Navigation

Recently Viewed

This feature requires Javascript to be enabled.
 

A Survey of Users of Connexions Music Modules

Module by: Catherine Schmidt-Jones. E-mail the author

Summary: A short survey was attached to 103 Connexions modules in K-12 music theory, ethnomusicology, and acoustics. The survey was available for one year. 488 users responded, offering a glimpse of what they thought of the materials and why they were using them.

Introduction

Reader feedback has been very useful to me as an open education resource (OER) author, so for one year I formally asked for feedback. Near the end of each of 103 modules in the area of music theory, notation, acoustics, and ethnomusicology, I added a request to fill out a short survey. I was hoping for two things. As an OER developer with limited time and resources, I wanted a clearer idea which types of efforts would be most appreciated by the users. As a researcher, I hoped to share the results of the survey with others who are also interested in the perspectives of OER users. You can find an overview and analysis of the results in an article in the International Review of Research in Open and Distance Learning (IRRODL). (Results that have already been published in that article are not included below, with the exception of some figures that show proportions of responses.) In the module below you can find:

The survey was short (14 questions), voluntary, and anonymous. It is now closed, but anyone who would still like to comment on my materials, ask questions or make suggestions is welcome to contact me at caschmidtjones@gmail.com. I am sincerely grateful for the help of everyone who has contacted me or participated in the survey.

What do the numbers mean?

The report below includes many numbers, which may give the false impression that the results of the survey were quantitative (as in "four out of five people prefer..."). It was not possible to identify everyone who uses these materials and survey a statistically representative sample of them. Instead, I simply asked for volunteers among my readers. The response rates for voluntary online surveys are extremely low; for example, over a period during which the materials received more than a million visits, only 546 people began the survey and 488 completed it. Online surveys are also subject to biases that are difficult to measure. For example, those who decided to respond may have been more comfortable with the English language, more deeply engaged with the materials, and more positively disposed towards them, than all readers as a whole. This survey actually produced evidence of the positively-disposed bias (see below). I therefore consider this project to be a qualitative case study, a snapshot of a particular set of open-education materials, from the perspective of particularly engaged and enthusiastic readers, giving a picture of the range of users and viewpoints, along with a sense of whether a particular response to a question seemed to be common or unusual, and whether there are patterns that suggest connections between responses. The proportions of responses to a question should not be considered to be statistically representative of any type of open-education resource, or any particular population of open-education users.

Survey Results

Question 1: Which module are you reviewing?

I had hoped that the first 3 survey questions might give me some idea which modules visitors found more or less helpful, and why. In particular, I wondered whether "popular" (highly-visited) modules were also the most helpful modules. Asking which module the respondent had been viewing was necessary, because there are many links between the modules. The module that linked the respondent to the survey might not be the one that they recalled reading. In fact, many of the respondents gave very general answers to the first question, such as "music theory" or "notation." It is possible that they were reviewing their experience with multiple modules.

For most modules, the number of responses that could be definitely linked to that specific module was very low, and differences in satisfaction rates between modules were small. It was difficult, therefore, to draw useful conclusions about the relative usefulness of the modules from the perspective of the users. (Please see below for a discussion of the exception, a module that was clearly rated much less helpful than the others.) Popular modules - those that Google Analaytics reported receiving the most visits during the survey - consistently received more respondent ratings, but there did not appear to be any correlation between the average rating for a module and its popularity. This is not a surprising result; other research has also found little correlation between visit numbers and the value placed on a resource by those who do visit.

Questions 2-3: How helpful did you find the module and its various elements?

Figure 1: On a scale of 0-4, where 0="not" and 4="extremely", the mean of all responses was 3.00="very".
How helpful did you find the materials
Graph of responses

As discussed in the IRRODL article, users who were happy with the materials may have been more likely to respond to the survey, motivated by a sense of participation in the internet-based gift culture described by Raymond (1999). Bergquist and Ljungberg (2001) explain that, within these online gift cultures, accepting a free gift of information may create a feeling of moral obligation to pay back the giver in some way. The second survey question provided further evidence of this "gift culture" dynamic. On a scale of 0-4 (where a 0 rating found the materials "not at all helpful" and 4 found them "extremely helpful"), the mean score among those who finished the survey was 3.00 (a 3 rating found the materials "very helpful"). Among the 58 who began but quickly abandoned the survey, the mean was 2.62, which suggests that users who were less pleased with the "free gift" were less likely to reciprocate by finishing the survey. (Survey non-completers typically answered only two or three questions - not enough to suggest any pattern of responses - so they are not included in the results below.)

Although noticeably different, the satisfaction rate among survey non-completers is still closer to "very" than "moderately" helpful. Grouping modules by format (lesson plan, textbook, article) or subject matter (music theory, ethnomusicology, acoustics) also produced only small differences in satisfaction ratings (means between 2.68 and 3.21). Among modules that were rated by more than 5 respondents, mean ratings also fell easily within the 2.5-3.5 range (i.e. closest to "very helpful"), with one exception.

What aspects of the module made it useful? Were respondents reading the text or just studying the figures? Were they finding and using embedded media such as audio and video? Were they following hyperlinks or making print copies? Were they trying the suggested practice problems and exercises or ignoring them? Question 3 asked how helpful specific aspects of the module were. Any element might receive a low rating for a number of reasons: because the module did not have it, or the respondent did not see it, chose not to use it, or tried to use it and did not find it helpful. Respondents seem to value the online text most, with figures and practice problems also popular. The low rating for audiovisuals may be due to the fact that they are more sparsely scattered through the materials than other elements. Links, on the other hand, are present throughout the modules, and I was surprised by their comparatively low helpfulness rating. Other researchers have found that newcomers to a subject area may have difficulty using web-navigation aids like links to locate the information they need, and the survey produced some evidence that this is having an effect on some of the visitors to these modules. (See, for example, the mini-case study below.)

Figure 2: On a scale of 0-3, how helpful was this element? Reading the text online appeared to be the most helpful activity.
Which elements did you find helpful?
Bar Graph of responses

Which changes to the materials would be most helpful to you?

I would have preferred to leave suggestions for changes completely up to the imaginations of the respondents, but I feared that the typical response would be "I can't think of anything else I would want." I hoped that preceding the open-ended question with a multiple-choice question would help respondents think of specific changes they would like to see.

Question 4: Multiple-choice responses

Respondents were presented with a variety of suggestions for possible changes to the materials and were asked to choose one or two that they thought would be most helpful to them. Responses ranged from choosing none to choosing all of the options, but the overall results do seem to show some strong preferences among respondents.

Figure 3: More audiovisuals, practice problems, and bibliography/discography were the most popular choices. Least chosen were translation to another language and official approval of the materials by some accrediting body.
What changes would be most helpful?
Bar graph comparing preferences

More audiovisuals was clearly the most popular choice, which seems at odds with the results of question 3; but there are a few likely reasons for this. Audiovisuals are not as common in the materials as other elements; also, some respondents complained that they were not able to access them. Clearly, one good response to this survey would be to try to add more audiovisuals to my materials that are easily accessible regardless of the users' equipment.

Also popular with respondents (and much easier for me to create) were "more problems" and "more recommendations for books and recordings" (labeled "bibliography" in Figure 3).

The least popular choices were translation to a different language, official approval of the materials by some sort of accreditation process, and an audio file of the text read aloud. It is most likely that the majority of people who would prefer to see the materials in a different language are not looking at the English version at all or are uncomfortable taking a survey in English. The same may be true for those who would prefer to hear the text spoken rather than see it, or this group may already have the necessary equipment to hear the text read aloud. However, some research suggests that an audiovisual resource, created by combining an audio version of the text with animations or other visual presentations, might prove particularly to subject-area newcomers. For example, in an extensive survey of the users of MIT OpenCourseWare, Carson (2006) found that videos that provided an introductory overview to the subject were particularly appreciated.

I find the lack of interest in having the materials "properly accredited" to be particularly interesting. I have seen no research on this subject, but a very informal survey of my acquaintances suggests that many learners today have as much - or more - trust in "crowd-sourced" materials such as Wikipedia, in the recommendations of personally-known teachers or peers, or in their own ability to judge whether materials "seem trustworthy" than they do in official accreditation bodies.

Question 5: Free responses

In spite of my attempt to prime their imaginations with multiple-choice suggestions, more than half of the respondents who finished the survey (284 of 488) skipped this question, and 41 responded with some variation of "it's fine the way it is." Responses ranged from general satisfaction or dissatisfaction to very specific desired changes (for example, adding specific information) that would have helped with the respondent's search at the time of the survey. I could not understand what was being requested in eight of the responses.

Of those who made suggestions that I understood, responses were fairly evenly divided between requests concerning content (76 respondents) and requests concerning delivery (78). There was no significant agreement on the specific content that should be added. Responses concerning delivery included 9 complaints of elements (such as links) that were not working properly. The rest were fairly evenly split between requests for more basic or simpler information (20); formatting the information in a different way, such as in a table or as a one-page handout (22); and adding media, particularly images and audio examples (27). Since there is no clear agreement that a particular area of content needs expanding, it would appear that efforts to make the current content more complete, more accessible to novice learners, and more readily usable by teachers might make the most difference to users.

Making lessons more accessible to subject-area novices is particularly challenging in the open-education context. A face-to-face teacher, or even an online tutor, can take advantage of personal knowledge about the student's present understanding and capabilities. Without that knowledge, it is less clear how to assist learners in connecting new information to their present understandings. There is a great deal of research currently being done in this area, which from my viewpoint does not seem to have reached any widely-held consensus. Audiovisual materials might be particularly useful in connecting with implicit understandings that the learner may not realize they have (understandings about what an instrument looks like, for example, or the way music sounds), but I am also intrigued by the possibility of making materials that are more explicitly inquiry-based, leading learners through the process of identifying for themselves what they already know and connecting it to what they want to learn. I plan to explore both media and inquiry in my modules in the near future.

Question 6: How often have you looked at music modules in Connections?

Figure 4: Most respondents did not report previous visits.
How often have you visited these materials in the past year?
Bar graph how often visited

A clear majority of respondents reported that this was their first visit to Connexions (368 respondents) or that they did not know whether they had visited the site previously (33), strongly suggesting that the materials are being used in a piecemeal fashion, to support specific, immediate inquiries, rather than as entire courses. Google Analytics data on visits during this period seem to support this: 65% of logged visits arrived directly from a search for key words specific to the module.

Question 7-8: Why were you looking for this information?

Figure 5
Why were you looking at these materials?
PieChart

This involved two questions. First, a multiple-choice question asked the respondent to classify the reason for their visit. The choices were based on my expectations due to email from readers:

  • Teaching a class or lessons
  • Taking a class or lessons
  • Teaching myself about music
  • Playing or singing in a group
  • Directing a group
  • Just curious/other

The second question asked for further details, for example "what kind of group are you directing?" The answers to these two questions were, in my opinion, the most interesting result of the survey for those who offer or study OERs, so they are discussed in detail in the article in the International Review of Research in Open and Distance Learning.

Question 9: What music education and experience have you had?

Comparing the formal education and informal educative experiences of musicians around the world would require a very detailed and nuanced analysis which would have constituted a large research project in itself. In order to get some small idea of whether a respondent's previous knowledge of the subject had any influence on their experience with a module, I asked each respondent for an estimate of the amount of time they had spent as each of the following:

  • student in private music lessons
  • member of at least one school ensemble
  • member of non-school music ensembles
  • enrolled in college-level music course(s)
  • professional or semi-professional musician
  • music teacher
  • ensemble director

Again, the range of responses was very large, from 54 respondents who claimed no formal music experience in any category to 16 respondents who claimed more than 4 years' experience in each of the seven categories listed. As mentioned above, the amount of previous experience in music did not seem to affect the respondent's satisfaction with the materials.

Questions 10-11: Have you received the music education you wanted?

This involved two questions. Respondents were asked which of three choices best described them. Each choice led to a free-response question asking for further information. Percentages for the first question were:

  • I have always been able to take whatever music lessons or classes I wanted - 44%
  • I have not been able to take music lessons or classes that I wanted to take - 47%
  • I have never wanted to take music classes or lessons - 9%

A free-response question then asked for further information. Those who claimed access to the music education they desired were asked whether that education included lessons or classes that they did not want to take. 70% of this group said they had not been forced to take music courses that did not interest them. Clearly, a significant proportion of the respondents were well-satisfied with their formal music-education experiences. This is good news, but because one purpose of OERs is to make education more widely accessible and equitable (see for example, the Cape Town Declaration), it also raises the important question of whether these materials are only helping those who already have many other resources at hand.

Those who claimed that their preferred music education was not completely available shed some light on this question. They were asked what had restricted their options. The most common responses were:

  • Money/cost (at 38%, the most common response to this question)
  • The desired education was not available in their locale (22%)
  • Lack of time or scheduling options (16%)

Another group that should be of particular interest to OER providers were those who claimed that they did not want formal music education. This group was asked why they didn't want music classes or lessons. The most common responses were:

  • Prefer the freedom afforded by self-teaching
  • Not "musical enough" to pursue the subject formally
  • Too busy

Questions 12-14: Demographics

Figure 6
How old are you?
Graph of ages

Respondents were asked for their country (and state or province if applicable) of current residence, age, and gender. Reported ages ranged from 11 to 87 years. The high number of young people was expected; less expected was what appeared to be a rise in use after retirement, with respondents reporting turning to internet resources to pursue new interests or take up former hobbies that had been neglected during the working years. The male-female ratio of respondents was 55% - 45%. In most categories, the ratio was reasonably even, but teacher respondents were noticeably more likely to be female (61%-39%), and self-teachers who reported that music composition was their goal were much more likely to be male (88%-12%).

Responses arrived from 41 different countries, with the largest proportions coming from the U.S., the U.K., Canada, and Australia. (Other countries represented were India, the Philippines, Singapore, New Zealand, the Netherlands, Trinidad and Tobago, Sweden, Spain, China, Greece, Colombia, Germany, Mexico, Turkey, Pakistan, France, Jamaica, Romania, South Korea, Thailand, Moldova, Saudi Arabia, Malta, Italy, Israel, Mali, South Africa, Brazil, the Czech Republic, Japan, the Dominican Republic, Iceland, Portugal, Poland, and Bermuda.)

The Case of the Unsatisfactory Module

As noted above, satisfaction ratings between modules and among various groups of respondents did not vary much. This general consistency made the case of the unsatisfactory module all the more interesting. Both visit-log data and number of survey responses suggested that "Key Signature" was a popular module, but it received a mean rating of 2.25, closest to "moderately" helpful, and its low rating was caused by an unusually high number of "slightly helpful" and "not at all helpful" ratings. Why were visitors to "Key Signature" noticeably more dissatisfied than others? Clues in their comments suggested that unhappy users tended to be novices in music theory. One respondent requested "a beginner's section." Other comments included:

  • explanation of sharp and flat key signature very confusing! and what about natural? what's a natural? it doesn't explain in clear terms
  • Only reference material already presented....you referred to major and minor keys when determining key signatures when no prior discussion of those had occurred. I was lost. If there are 2 sharps, I wanted to learn that it is in the key of F, I think. Not major or minor or anything, just that and I didn't get it.
  • Please define "major", "minor", "circle of fifths". As written, this is incomprehensible!

However, it was not immediately obvious why this particular module should be more problematic than others. Survey results suggested that, in general, subject knowledge did not correlate with satisfaction, although dissatisfied high-subject-knowledge learners tended to express a need for further information, rather than confusion. Enrollment in a music course on the subject did not appear to help; among those with little music experience, "students" were actually more likely to be dissatisfied than "self-teachers." Also, lack of music experience did not seem to correlate with dissatisfaction with other modules, and some of the most positive comments specifically mentioned being a "beginner."

Perhaps the best example of this phenomenon came from the only instance I found that looked like the same person completing the survey twice. (Demographic information, as well as some unusual details about music-learning goals and previous experiences were identical.) Apparently, then, the same person who found the key signatures module "incomprehensible" shortly afterwards said of the "Clef" module, "No suggestions -- it's great as is!"

I use links extensively in my writing; a large proportion of the paragraphs in my text contain at least one link, and multiple links in a paragraph are common. This is my favorite feature of online writing, as I feel it allows me to avoid digressions that would clutter up each explanation with extra information that the reader may not need or want. Links allow all readers to choose any digression that interests them or to find crucial information that they need - essentially customizing their learning experience - while allowing me to write concise explanations. However, researchers have consistently found that learners who have little prior knowledge of the subject may learn better when offered materials organized in sequential order, while learners with more subject knowledge can make better use of the exploratory possibilities of hyper-linked materials (see, for example, Anderson, 2001; Wolfe, 2001).

In this case, visit-log data showed that the module had an unusually high number of referred visits, including over 400 visits each month from the "How to read sheet music" page at howtodothings.com. The high number of visits to a module from a single referring site was very unusual; no other module had hundreds of visitors arriving every month through a link from a single referring site. On January 29, 2011, I found howtodothings.com page to be the second highest result for a Google search for "how to read sheet music" and the fifth result (behind 2 other pages and 2 videos) for "how to read music," a very high ranking for such a basic query. Its high ranking may have reflected the page's placement at a popular site. The page itself was minimal, featuring advertisements, a short paragraph of text (with no figures) for each concept such as "treble clef" and "time signature," and no definitions of sharp, flat, and natural. The link to my module on key signature was the highest-place non-advertisement link on the page.

Someone who tried reading "Key Signatures" within five minutes of wondering how to read music would have good reason to feel lost and annoyed. In order to provide structured learning paths, I have placed that module within various collections in Connexions. For example, "Reading music: Common notation" includes that module as the fourth lesson, following a discussion of sharp, flat, and natural notes. The survey revealed two problems with this approach, one general and one specific to this module. In general, it is possible that visitors did not notice or understand the mention of "collections" in the sidebar. While investigating this case, I noticed that curriculum information (such as collections) in Connexions is placed very similarly to advertisements on commercial sites such as howtodothings.com. Learners who do not know basic music terms might search for "how to read music" rather than "staff" or "clef," arrive at Connexions in the middle of a course, and then fail to notice the offer of a structured curriculum.

The ensuing frustration may have been, using Dewey's term, miseducative in the sense of decreasing their long-term commitment learning the subject. Can an OER take steps to reduce such miseducative experiences? Other "beginner" modules in this survey also contained mention of, and links to, more advanced materials, but only "Key Signatures" discussed the more advanced material at length, a result of the centrality of this element of music notation to discussions of music theory. I believe the main problem with this module was a mismatch between my intended audience (those who are trying to understand music theory) and the actual audience (those who are trying to learn to read music). Although some users might be helped by making the structured curriculum (the collections) more obvious, a better solution in this case might be to move the "difficult" material to a module that is clearly not meant for beginners, as well as publishing an introductory "how to read music" module.

Note:

I have already begun to make changes to the modules based on the survey results. To see any module as it appeared during the survey, scroll to the bottom of the module to find "more about this module." Choose "version history" and open the version that is notated as "linked to survey."

Modules included

If you wish to view the modules as they appeared during the survey, look at the "version history" of the module, and choose the version notated as "linked to survey." The survey invitation was attached to the end of the following modules: Transposition, Transposing Instruments, Form, Major Keys and Scales, Minor Keys and Scales, Octaves and the Major-Minor Tonal System, The Circle of Fifths, Half Steps and Whole Steps, Interval, Triads, The Staff, Key Signature, Naming Triads, Clef, Pitch: Sharp, Flat, and Natural, Duration: Note Lengths, Time Signature, German Folk Songs, German Christmas Songs, Conducting Activities, Timbre, Frequency, Wavelength, and Pitch, Sound and Music Activities, Harmonic Series, What kind of music is that?, Message Drums, Korean Songs, Modes and Ragas, Counterpoint, Scales that aren't major or minor, Tuning Systems, Enharmonic Spelling, Harmonic Analysis, Syncopation, Textures of Music, Rhythm, Melody, Tempo, Dynamics and Accents, Harmony, Calypso and Found Percussion, Keys and Scales are Sets, Fractions, Multiples, Beats, and Measures, Intervals, Frequency, and Ratio, Powers, Roots, and Equal Temperament, The Shape of a Melody, A Melody Activity, Drone Harmony, Talking Drums, A Pentatonic Scale Activity, Parallel Harmony, Melodic Phrase, Theme and Motif, Articulation, Duration: Rest Length, Dots, Ties, and Borrowed Divisions, Percussion fast and cheap, Consonance and Dissonance, Beyond Triads, Consonance and Dissonance activities, Scott Joplin: King of Ragtime, Sound and Ears, Amplitude and Dynamics, Talking about Sound and Music, Transverse and Longitudinal Waves, Range, Cadence, Meter, Standing Waves and Musical Instruments, Indian Classical Music: Tuning and Ragas, Listening to Indian Classical music, Standing Waves and Wind Instruments, Mexican Fun Songs for Children, Ugandan Lullaby, Mexican Christmas Songs, Japanese songs, Pick-ups, Repeats and Musical Road Maps, Acoustics, Accent Activity, Resonance, Standing Wave Demonstrations, Meter activities, Form Activities, Harmonic Series: Timbre and Octaves, Harmonic Series: Harmonics, Intervals, and Instruments, Tempo Activity, Rhythm Activities, Timbre Activities, Music Texture Activities, Listening to Balinese Gamelan, Balinese Gamelan, Gamelan Dance Activity, Gamelan Form Activity, Gamelan Elaboration Activity, Gamelan Kotekan Activity, Didjeridu, Australian Aboriginal Storytelling, Didjeridu Activities, Mbira, Gourd Instruments, Vocables, Sega: The Music of Mauritius.

About this Project

The survey was conducted as part of the "early research" requirement as I work towards a doctorate in education, under the direction of Dr. Marilyn Johnston-Parsons, at the University of Illinois, Urbana-Champaign.

Please feel free to contact me with any questions, concerns, or observations you have about the survey or the project. More information about the survey can be provided or published if readers are interested. I can be reached at:

  • email: caschmidtjones@gmail.com
  • phone: 217-359-2038
  • address: 1214 W. Church St. Champaign, IL, 61821

If you have any concerns about the ethics of this project, you can contact Anne Robertson, Bureau of Educational Research, at 217-333-3023, or ber-irb@ed.uiuc.edu or the Institutional Review Board at 217-333-2670 or irb@illinois.edu

References

  • Anderson, M.D. (2001). Individual characteristics and web-based courses. In C. R. Wolfe (Ed.), Learning and Teaching on the World Wide Web. San Diego, CA: Academic Press.
  • Bergquist, M. and Ljungberg, J. (2001). The power of gifts: Organizing social relationships in open source communities. Information Systems Journal, 11, 305-20.
  • Carson, S. (2006). 2005 Program Evaluation Findings Report: MIT OpenCourseWare, retrieved June 28, 2011 from http://ocw.mit.edu/about/site-statistics/
  • Cope, B. and Kalantzis, M. (Eds.) (2009). Ubiquitous Learning. Urbana: University of Illinois Press.
  • Raymond, E. S. (1999/2000). Homesteading the Noosphere, version 3.0, Retrieved March 6, 2012 from The Cathedral and the Bazaar.
  • Wolfe, C. R. (2001). Learning and Teaching on the World Wide Web. San Diego, CA: Academic Press.

Content actions

Download module as:

Add module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks