Supporter To Reporter (S2R) Medals

S2R 1

Preface

The information contained in this project appendix was gathered from the original project proposal that was funded in 2012 and interviews with project leaders through the end of 2014.  This appendix and the overall design principles database from the DPD project does not reflect further evolution of the project or developments after the final interview that took place in 2014. As of the final interview, Suporter to Reporter (S2R) had developed a functioning badge system and issued badges. Based on this information, we classified the S2R badge system as an implemented (rather than a partial or suspended) badge system.

Summary

Supporter to Reporter (S2R) provides learning opportunities for young people to take on the roles of sports journalists, media producers, and mentors. S2R Medals will recognize and reward the skills and achievements gained by young reporters who learn and demonstrate a rich array of competencies acquired through their participation in the program.

Supporter To Reporter, an existing program developed by DigitalMe in the UK, participated in the Digital Media & Learning competition focused on Badges for Lifelong Learning. S2R won a grant from the MacArthur Foundation to implement Mozilla OBI-compliant digital badges in support of its educational objectives, helping young people to learn skills required in sports journalism and apply them in their communities.

“Badges allow us to capture these competencies in a way we have never been able to previously, allowing young people to create an online portfolio of their work to share with colleges and employers. S2R uses the safe Makewaves Open Learning community to run the program and connect schools across the UK” (HASTAC Q&A).

“We created a badge ecosystem based on the existing S2R learning program and will launch the badges into the Makewaves community of over 3500 schools and 60,000 learners worldwide. We expect the main take-up to be with youth ages 10 – 14 years old” (HASTAC Q&A). The development process for the S2R Badge system “is organized around (1) learning by doing, (2) collaboration, and (3) not making assumptions” (HASTAC Q&A). The purpose is to develop a program where students record interviews, produce videos and articles, and earn badges for reporting, producing, and mentoring others.

By implementing badges, Supporter to Reporter aims to recognize skills with transferable and long-term value, such as journalism and mentoring. A medal from the program should allow students to showcase their achievements as they apply to colleges and jobs:

“We expect S2R medals to feature on CVs and college and job applications, giving them worth and durability. Whilst S2R Medals use sports journalism as a learning framework, they demonstrate skills which have direct relevance to any workplace or educational establishment.” (DPD Project interview)

The project emphasizes that the skills they recognize are not tied strictly to journalism and that the medals embody some more general achievements as well: “Young people who take part in the program demonstrate increased confidence, aspiration and transferable skills that they can take into the world of work” (HASTAC Q&A).

Audience

“S2R Medals are designed primarily for young people ages 11-19. However, younger students can also complete badge activity if capable. To enable this audience we also addressed teachers, after school program facilitators and regional coordinators for sports and media provision for youth.” (q&a)

Supporter To Reporter is designed so that schools and supplemental education providers like after-school programs can implement the challenges, assess student achievement, and integrate with the Makewaves social platform to award badges and build learner e-portfolios.

Integration into curriculum

The badges design process was easier because S2R already had a pre-existing curriculum that was successful. What has provided to be the greatest challenge is helping students actually working with the Mozilla OBI system to actually push the badges to their backpacks.

Evolving Practices and Design Principles

What follows is a list of practices as they relate to the general and more specific design principles in each category of practice. The headings name a (a) General Principle, (b) Specific principle, (c) Specific practice. Below each heading, we indicate what stages this principle appeared in the project: as an (a) intended practice, (b) enacted practice, or (c) formalized continuing practice.

Design Principles for Recognizing Learning

A primary goal of the S2R team was the development of badges to illustrate competencies necessary for success in journalistic organizations and beyond. To this end, S2R developed their badge system in a manner that enabled the sharing and display of learning of journalistic skills, ability, and knowledge to journalistic organizations, colleges, and other institutions. The badge system also enabled the development of professional relationships with these organizations through the display of badges.

Use badges to map learning trajectory > Level badges > Badge levels in parallel tracks

Intended, enacted, formal.

Badges were intended from the beginning to be leveled in each of several categories based on different journalism roles. From the first proposal, the levels were intended though the categories evolved over time. Earners must pass through the lower levels in each category before attaining the higher ones. In the initial design, there were four levels: bronze, silver, gold, and then a coach/mentor level above gold (DML Stage 1 Proposal).

As enacted, levels and categories are still there but activities for the strands have been refined, particularly to include more of a face-to-face aspect. More skills are being added to each category through the “design days” that occurred with educators and students over the summer. Some changes were aimed to facilitate better practicality in school environment, and the coach strand now has more appropriate skills filled in. As far as the structure, the coach/mentor badges were split out into their own category. In order to get to the coach/mentor badges, students must have earned same level in one of the other two strands. (DPD Project Interview)

Supporter 2 Reporter designed and implemented a complex badge hierarchy that recognized both deep experience in strands of journalistic work but also breadth across learning the skills of journalists, producers, and mentors. It defines routes through journalistic competencies that require collaboration and practicing a variety of different tasks. The DPD Project has categorized this practice under the specific principle of “Level badges,” though the design as enacted and formalized includes elements of wayfinding. See the challenges section below for a detailed account of how S2R confronted the difficulties involved with building and supporting a complex leveled badge system. In order to ensure that the system would continue to thrive, S2R spent considerable effort designing tools and resources to support the teachers implementing the program.

Use badges to map learning trajectory > Provide routes or pathways > Badge missions

Not intended, enacted, formal.

Though it was not part of the initial design for S2R, Makewaves implemented a practice to improve the visibility of its various badge programs and get more participants. Named “Badge Missions,” the platform implemented a system for students to understand their progress as they moved through various badged tasks in different learning programs (DPD Follow-up Interview). This practice also constitutes a chance for students to discover learning opportunities, as described below.

Promote discovery > Discover learning opportunities > Discover badge missions and pathways

Not intended, enacted, formal.

Though it was not initially intended, after the badge system got rolling on the Makewaves platform, program administrators added additional features designed around discovering learning opportunities by following badge pathways. They called these “Badge Missions” (DPD Follow-up Interview)

Seek external backing of credential > Externally valued > Build relationships around recognizing S2R’s badges

Intended, not enacted. not formal

S2R intends its Journalist, Producer, and Coach medals to provide significant value to signal higher education providers and future employers about skills developed by badge earners.

S2R intended to ask organizations involved in sports journalism, including the BBC, to endorse the badge program and consider the badges in applications for internships and other opportunities. DigitalMe staff approached these organizations to broach the subject and found that collaborators are interested in badges but were unable to secure any specific commitments from national partners. S2R has adjusted their expectations and now feels that the value remains still at the local level. The next goal is to link with local universities and employers. There is already a collaborative process around the activities which are being badged and how meaningful the resulting badges will be. The project is putting a lot of effort into ensuring the badges will have external value, especially noting that the skills developed are not exclusive to journalism (DPD Initial Interview).

From the beginning, S2R aimed to create a badge system that would recognize skills and experience valued by both journalism organizations and generally across many careers. Their work toward gaining formal recognition of their badges is a continuing but not formally enacted effort for the project. See case study on the difficulty of recruiting endorsing partners for a detailed discussion focused on challenges to gaining formal recognition for digital badge pioneers like S2R.

Seek external backing of credential > Externally endorsed > Seek endorsing partners

Intended, not enacted. not formal

Supporter To Reporter hopes to work with external partners to gain official endorsement of their badges. “Endorsement of S2R medals by third parties would add significant value but is not essential. S2R has strong links with media organisations such as the BBC and with sports clubs such as Manchester United. We will build on these relationships to endorse our badges” (DML Stage 2 Proposal)

Supporter To Reporter has various partner organizations in the sports and media world. From conversations about endorsing S2R’s badge program, it appears third parties want to see evidence of what badges do/why they should care before they buy into the badges system (DPD Follow-up Interview).

Determine appropriate lifespan of credential > Never expires > Give students a permanent record of achievement

Intended, enacted, formal.

S2R intended to use badges without a set expiration: “S2R Badges aim to represent demonstration of skills with transferable and long-term value, such as journalism and mentoring. We expect S2R medals to feature on CVs and college and job applications, giving them worth and durability.” (DPD Project Interview)

As intended, S2R Badges are awarded without an expiration date. The project felt that though on their end, medals will be permanent, earners can choose to claim and display or hide at will depending on their beliefs about how relevant the badges are as their goals develop.

Supporter To Reporter thought through the audience for its students’ badges and aimed to recognize learning that would say something meaningful about the students for a long time, as they sought higher education opportunities and careers. The badges do not expire in order to enable students to display them as long as they are useful. Through the Makewaves platform, there is a long-term commitment to the validity of the badges.

Use badges as a means of external communication of learning >Badge relevance to careers

Intended, enacted, formal.

The project aimed to provide experience and credentials, which despite their sports reporting focus are transferrable to non-journalism careers: ”All the S2R badges are designed to reflect real-world transferable skills, including social media creation, reporting, interviewing, editing, time management, collaboration, responsibility, motivation, and mentoring others.” (HASTAC Q&A) Further, employers and higher education institutions can clearly view and understand these skills by exploring the badges’ metadata.

A lot of the work is online even though the actual interviewing is usually occurring offline. Supporter To Reporter built the program to be about developing life skills, not only journalist skills. To help students broaden their experience S2R is interested in linking with other projects to aid the discovery of learning opportunities (i.e. you completed badge “x”; you might be interested in these badges: “y” and “z”).

S2R exemplifies the principle “Use badges as a means of external communication of learning” through its efforts to speak publicly and to partners about the skills that participating students are developing. Through conversations with potential endorsing partners, they try to find what these parties need to know in order to recognize the value of the badge. The current continuing practice through the 2013-14 school year will include partnership with local sports organizations offering the S2R program: “the aim is to explore how some of those value pathways could work, and get some case studies of that working” (DPD Bloom Interview).

Have experts issue badges > Credentialed by both external accredited entity and community > Include peers in issuing badges

Intended, not enacted. not formal

As mentoring is a specific goal to support, S2R hopes to include peers as awarders of accomplishments, though not the OBI-compliant medals, in order to maintain prestige of the higher awards.

After earning a higher level medal, student reporters gain the privilege of approving accomplishments made by their peers. The “community achievements” given by peers do not translate into OBI badges. The project wishes to control accountability while also working to “encourage a culture of contribution”, thus the peer awarded achievements are not badges that are pushed beyond the community. “Through external and peer assessment, the risk of gaming is effectively limited for S2R badges. Moderators approve the work needed to gain [OBI] badges, and work needed for higher levels must be assessed by a staff member or senior student” (DPD Initial Interview). As planned, “community achievements” were developed and are not pushed to the OBI. Some of these are based on automated procedures, and the program is still investigating how peers may become involved with awarding, as well as exploring an expanded role for peer assessments involving the coaching strand of OBI-compliant medals.

S2R planned to implement peer awarded badges from the initial planning stages and remains committed to eventually allowing its highest-awarded students to play a role in the recognizing process. The organizers and participating teachers have to wait until a group of students progresses to the highest level in order to make it possible (DPD Follow-up Interview).

When assigning value to the badges they received, students seemed to discount the value of the peer-awarded badges: “Students also preferred to receive badges from ‘experts’ rather than peers. Experts ranked in order of preference would be external/industry, teachers, Makewaves, other students who had already achieved the badge or proved their expertise in some form” (HASTAC Q&A).

The DPD project categorized this as a sub-principle of “have experts issue badges,” recognizing that to the extent that their peers move toward “expert” level, they become more qualified to recognize learning themselves. It is clear from S2R’s reluctance to grant peers the ability to issue the most valuable badges in the system that they felt strongly about the appeal of expert-issued badges. S2R confirmed students’ preference for expert-issued badges with interviews, but noted that students “don’t mind getting a badge off a peer if they feel their peer has achieved a high enough level to be credible” (DPD Bloom Interview).

Recognize educator learning > Badges for participating teachers

Not intended, enacted, formal.

This was not an intended practice in S2R’s badge design, but they are now developing badges to recognize for teachers for their willingness to learn and use these new activities as well as engaging in assessing these activities for the sake of badges. Educators using the program in their classes can earn special educator badges that are different from the student badges.

S2R found that teachers were just as interested in earning badges as their young students. Their practice to recognize educator learning sprung up later in the process of building the badge system than many other principles, but the team now feels it is an important part of supporting their participating teachers’ professional development: “There’s as much value to be had for educators as for young people, in terms of lifelong learning.” S2R plans to issue badges to educators through local partner sports centers that administer the program (DPD Follow-up Interview).

Design Principles for Assessing Learning in Digital Badge Systems

In keeping with the development of sports journalism professionals, S2R developed their system to incorporate the achievement of badges based on standards and achievements necessary to professional journalism organizations. These practices primarily focused on the development of criteria and tools to enable multiple assessment outcomes from instructors, professionals, and peers.

Use leveled badge systems > Competency levels > Leveled assessments for leveled badges

Intended, enacted, formal.

Supporter To Reporter initially proposed a 4-tier hierarchy for each strain, integrating mentoring with the other skills as the top level: “Reporters progress through the programme, gain new skills and experiences, take on new responsibilities and push themselves. The journey has four levels:

  • Level 1: Gaining and demonstrating competency (content production)
  • Level 2: Using skills (interviewing, reporting and publishing)
  • Level 3: Understanding the workplace (reporting for real)
  • Level 4: Mentoring others” (DML Stage 1 Proposal)

Criteria and assessments for these levels were intended to become more involved participation and performances increase.

As initially enacted, the fourth level has been moved to its own “Coach” category. The badges still have levels of bronze, silver, and gold, but what constitutes each level of badge is being further defined and refined, though the specifics are often still up to the teachers involved in the direct assessment.

Use e-portfolios > Open to public and foster discussion around artifacts > Makewaves user portfolios

Intended, enacted, formal.

As with many projects, using e-portfolios has been central to S2R’s assessment methods. The project intended that when awarding badges, teachers would be able to pick out particular pieces of evidence to attach to a badge, or might link to the portfolio as a whole. The video, audio and text reportage that make up a student’s portfolio are open and public, although learners identities are obscured behind usernames to protect the privacy of the younger participants. The evidence attached to badges is then available to reference. Social features, Comments, and Likes are integrated into the Makewaves platform, and an elusive Golden Heart badge whose exact requirements are secret keeps students looking for more interaction with their stories. Seeing the number of students posting more articles expressly to gain the Golden Heart, Cliff Manning of Makewaves believes “Any badge system should have a mythic top level” (DPD Follow-up Interview).

Enhance validity with expert judgment > Use human experts and computer scoring systems > Judged by a combination of human and computer experts.

Intended, enacted, formal.

In order to provide a badge ecosystem that allows many low-level badges, yet provides prestigious high-level ones, S2R opted to employ automated systems as well as human arbiters. Some awards in the badge system are issued automatically based on submission of work and “community achievements” generated in the Makewaves system, but others are assessed manually. For human experts, in general, assessment was relegated to the professional educators and in some cases utilized peer assessment. This distributed model allowed for the assessment of the complex skills used in professional reporting (DPD Follow-up Interview).

Align assessment activities to standards > Internal standards > Align standards to professional community

Intended, enacted, formal.

As initially intended, the S2R team planned to utilize standards internal to professional educators within the Makewaves community. As this collaboration proceeded, the project developed and maintained its own standards in order to ensure that the silver and gold badges would continue to be scarce and prestigious and would represent true achievement of the competencies identified by their research with journalistic organizations

Use rubrics> Rubrics developed for assessment for specific artifacts> Rubrics used to assess journalistic artifacts

Intended, enacted, formal.

S2R initially intended to have artifacts assessed by both teachers and professionals. Teachers would be responsible for most of the assessment and would use developed rubrics to award badges for each category. For the highest level badges, professional reporters would assess the artifacts for the complexities inherent in journalistic practices. S2R encountered difficulty generalizing rubrics for professional assessment. As S2R’s badge system was fleshed out, an important component of maintaining consistent assessment across all the participating schools and sports centers was detailed rubrics, tailored to the type of artifacts students must generate to gain badges in each category. Earners’ teachers are primarily responsible for assessing artifacts based on a rubric.

 

Figure 1. Example of S2R Medal Rubric

Figure 1. Example of S2R Medal Rubric

Design Principles for Motivating Learning

S2R developed a badge ecosystem that incorporated a number of motivational factors supporting earning badges and developing skills sought by journalism organizations. A principle component for this development was the opportunity for active engagement with professional journalism organizations for achievements of particular badges. S2R also developed components into their badges that enabled continued engagement with the system through making the achievement of some of these opportunities and badges a rare event.

Recognizing Identities > Roles within a system > Participate in professional communities

Intended, enacted, formal.

The tracks of badges in the S2R program are named after the role that young people fill while earning the badges: journalist, producer, and coach. In addition, the tasks involved in filling these roles take students into their communities, covering sporting events in their schools, towns, and up to the London Olympics. This exemplifies two motivating principles identified by the DPD Project, both in motivating students by naming them as the role they want to fill, and by offering opportunities to do that journalistic work once they reach the appropriate level. The second practice is detailed under “provide privileges below.”

Provide Incentives> New activities > Access to reporting live events

Intended, enacted, formal

The top level badges are designed to unlock special opportunities, and as intended, part of the S2R team’s work involves arranging exclusive access to reporting from the sidelines at events, hopefully motivating young people to work steadily through the entire badge system.

Provide Incentives > Peer mentorships > Unlock peer support features.

Intended, enacted, formal

S2R’s badge levels also fell under providing privileges, as this became the specific mechanism through which peer mentorship and access to special opportunities were selected. As mentioned in the assessment principles, S2R aims to allow high-level students to participate in awarding badges to their peers.

Provide Incentives> Internships > Open internship opportunities to badge earners

Intended, not enacted. not formal

S2R intended to partner with organizations involved in sports journalism to provide internship opportunities to the best badge earners. In most cases, partnerships were more difficult to secure than initially imagined. (See case study for more detail about recruiting external partners for endorsement and integration into motivation practices.)

Engage with the community > Involvement in local community > Partnerships with local sports clubs to provide opportunities for journalism experience.

Not intended, enacted, formal.

S2R had initially intended more of an internship model as the privileges that could be opened up for badge earners, and they had targeted nationally known sports journalism organizations. When partnerships with these organizations were harder than anticipated to develop, S2R found several local sports teams that were open to having students visit for reporting opportunities, including a cricket club that brought some students to the famed Oval stadium in London. The local focus of opportunities has been fruitful for S2R and is now seen as a continuing formal practice within the system (DPD Follow-up Interview).

Stimulate competition > Scarcity of badges > Scarcity of badges through high requirements

Not intended, enacted, formal

As stated, S2R employed a hierarchical badge system that developed competencies in professional sports journalism through the achievement of bronze, silver, and gold medal badges. These badges were awarded across the supporter, producer, and coaching domains to enable a targeted development of professional skills. The requirements for gold-level badges are high enough that there will be relatively few students who earn them, preserving their prestige, and introducing an element of competition that may prove to motivate students who earn the medals they know will be rare. This practice was not specifically intended in the system’s early design documents, but it has turned out to be a characteristic that DigitalMe now counts as important to the program’s ability to maintain the value of high level badges.

Build outside value for badges> Real life application of knowledge> Badge enabled applications

Intended, enacted, formal

S2R initially intended to have badges afford opportunities for displaying journalistic ability through partnerships with several media corporations. As stated, these partnerships were difficult to develop and secure. However, S2R did, as described above, enable higher level of badges to connect learners to additional opportunities, both within and outside the badge system,  for  learning and development. Such opportunities afforded applications of skills and abilities towards relevant media programs and institutions. The application of knowledge and abilities in relevant contexts provides additional motivation for learners to continue through the badge system.

Utilize different types of assessment> Computer> Automatic recommendations for low level badges

Intended, enacted, formal

As stated in use the assessment section, S2R used a process in which some badges were issued automatically via a computer system. For low level badges, this system also provided recommendations on the next steps earners could perform in order to progress within the system. In this way, the use of computer assessments provided motivational components intended to engage earners in fully progressing through the badge system.

Design Principles for Studying Learning

Like many badge systems described by the DPD project, S2R did not initially intend to develop their system to study badges. However, the development of the system led to the acquisition of highly informative data that led the S2R team to take action in using this data indirectly for future refinements and developments of their program.

Study badge impact > Research OF badges > Research of learning

Not intended, enacted, formal

One of the features of the S2R badge system that most significantly affects the ability to study learning within it is the amount of data generated through the use of the Makewaves platform. Cliff Manning of Makewaves and Lucy Neale of DigitalMe both indicated that there is a strong need for research in order to provide evidence of the value S2R delivers to potential partners and employers.

The complexity of the S2R badge levels forced Makewaves to build a robust automatic badge issuing system with many potential triggers based on a range of social network activity and uploaded material. Cliff Manning says, “It’s forced us to stretch the technology a bit as well, so now we can do loads more than most badges require” (DPD Follow-up Interview).

S2R will be able to analyze the numbers of badges issued and quantify the activity leading up to students earning them. With the introduction of a new learning pathways feature, Badge Missions, even more data will be generated about students progress through the system.

As enacted, some limited analysis of certain metrics and trajectories have been performed in order to talk about activity within the system, partly in the extensive number of presentations about Open Badges delivered by DigitalMe staff. The specific practices to analyze this data on an ongoing formal basis have not yet been developed, but the project is committed to pursuing research into the activity within the badge system in some manner.

Recognizing Principles Assessing Principles Motivating Principles Studying Principles
Specific principles: Specific principles: Specific principles:
About

Loves open education, Open Badges, free culture, Progress of the Useful Arts and Sciences, people-powered politics, and local food production. Coordinator for the badges Design Principles Documentation Project at Indiana University.

0 comments on “Supporter To Reporter (S2R) Medals
1 Pings/Trackbacks for "Supporter To Reporter (S2R) Medals"
  1. […] Supporter To Reporter (S2R) Young people take on the roles of sports journalists, media producers, and mentors, demonstrating their work online, recognized by “medals” […]

Leave a Reply

Your email address will not be published. Required fields are marked *

*

Mozilla Open Badges
Digital Badges are web-enabled credentials of learning or accomplishment. -Erin Knight, director of the Badge Alliance
Badges contain detailed claims about learning, links to actual evidence of learning, and they're shareable over the web. -Dan Hickey, DPD Project Lead Investigator
To me, digital badges represent the bridge between formal learning & informal structures. -Alex Halavais, DML research competition winner
Open Badges can help people tell a verifiable story about their accomplishments. -Nate Otto, DPD Project coordinator
Regardless of where you start, it’s more than likely you’ll end up somewhere other than your intended destination. That’s okay. Systems are living things, and your badge system needs to be flexible. You must embrace a bit of chaos in its design. -Carla Casilli, Director of Design + Practice at the Badge Alliance