MOUSE Wins!

Preface

The information contained in this project appendix was gathered from the original project proposal that was funded in 2012 and interviews with project leaders through the end of 2014.  This appendix and the overall design principles database from the DPD project does not reflect further evolution of the project or developments after the final interview that took place in 2014. As of the final interview, MOUSE had developed a functioning badge system and issued badges. Based on this information, we have classified this badge system as an implemented (rather than a partial or suspended) badge system.

Summary

MOUSE logo (NEW)MOUSE offers opportunities for youth to develop skills and dispositions that can translate into the workplace and apply across professional settings. The program trains youth to join teams of help desk experts who provide technology support during the school day. This offers an environment that mirrors the workplace and enables students to gain practical experience. In school, youth receive the opportunity to lead in the field of technology, cultivating skills, and strengthening identities as valued contributors in technology-driven environments, in preparation for the careers and workplace that they would eventually enter. MOUSE explains that their “goal has always been to capture the milestones that emerge along the way as points of reflection (and as wayfinding devices) that empower the user to pursue pathways forward and demonstrate their expertise in learning and professional contexts where not enough of their experiences are being counted”  (HASTAC MOUSE Q&A). By introducing youth to communities of practice, MOUSE enables them to gain exposure and develop their skills and teamwork in an age-appropriate professional work setting. The program extends students’ experience after school by connecting them with a peer community with shared interests.

In 2011, MOUSE won a grant from the MacArthur Foundation through the Digital Media & Learning competition to implement a digital badge system to support lifelong learning. Through the grant, MOUSE integrated badge-based credentialing into its pre-existing network of students, educators, and institutions. This network forms a system of support for youth, focusing on developing their digital, computational, technical, and interpersonal skills. The badges were designed to recognize students for skills and community-building across this spectrum. The program is partnering with schools and organizations in New York, California, Chicago and 15 other regions.

The “MOUSE Wins!” badging system is partly designed to grow interaction between its established sub-networks to form a stronger online community at the national level. Youth can extend their experiences at the local sites to the broader community online by interacting with other individuals with shared interests. To this end, MOUSE has rolled out two kinds of badge types, called “Community Wins!” and “Certification Wins!”, recognizing learners for interaction and engagement with the community as well as for mastering a set of digital skills.

For Certification Wins!, the project essentially designed a system to recognize learners’ successful completion of learning modules, through which youth develop digital skills and literacies. The badge system charts the learning pathways of students as they develop the technical skills needed to help their local school communities. Students can earn micro-credentials for completing activities in a specific learning module, aggregating micro-credentials to ultimately earn the certification badge. By building an online credentialing system, MOUSE creates the ability for students to connect their achievements and pathways of learning. The badges also give focus to skills and accomplishments that do not receive much attention but are critical to the function of school communities and recognize students whose talents may otherwise escape notice.

Community Wins! aim to recognize students for fulfilling their duties on local MOUSE squads and build positive interactions between squads online. Students are recognized for closing tickets, posting blogs, and commenting on others’ work.

Learners are incorporated into the assessment process by empowering them with the ability to issue badges to other users, contributing further to the social process of learning and highlighting peer recognition of achievements. MOUSE described that “this growing ecosystem for learning, participation, and assessment targets domain-specific outcomes in the areas of technology, digital literacy, computation, and 21st Century skills” (DML Proposal). Along this thread, MOUSE designed a learning process that reflects the feedback loop established in industry contexts.

MOUSE Wins! helps carry student learning that begins in school beyond the school day. The online community provides a source of support wherein learners can engage one another, as they interact and earn badges for their learning and achievements. The program works to appeal to youth’s interest in media and technology, help them understand its practical applications, and keep them engaged in developing their skills and literacies within a thriving community. MOUSE aims to promote developmental outcomes and provide ongoing support to youth as they continue on their pathways of learning.

Timeline

MOUSE had started working on digital badges for internal use before the MacArthur/HASTAC Badges for Lifelong Learning competition, beginning the design process in 2009 and deploying their first set of badges in 2010. They became a beta partner for Mozilla’s Open Badge Infrastructure (OBI) in the spring of 2011 “to connect participants’ profile data to external social and professional network sites online with the goal of releasing public-facing badges for a pilot group of advanced leadership high school students from the MOUSE Corps program” (DML Proposal). In 2011, the program developed two features: the Wins!Tracker, a tool to aggregate micro-achievements into one place, and Microprojects, posted as monthly creative challenges in which users can earn achievements for successfully completing the task. Relative to Microprojects, MOUSE designed a peer to peer issuing system in February 2013 called Peer2Peer Awards, which enabled youth to issue Wins! to one another for demonstrating inspiration, creativity, and technical savvy on projects. In October 2013, MOUSE launched new features for their certification badges, which included a visual redesign of the graphics and increased flexibility in the sequence with which learners complete the learning modules.

Evolution of practices and relation to principles

What follows is a list of practices as they relate to the general and more specific design principles in each category of practice. The headings name a (a) General Principle, (b) Specific principle, (c) Specific practice. The paragraphs below each heading detail the project’s (a) intended practice, (b) enacted practice, and (c) how that practice relates to the specific and general principles.

Design Principles for Recognizing Learning in Digital Badge Systems

The MOUSE system aimed to use badges to recognize a range of hard and soft skills that would be used by its technically gifted students in their eventual workplaces. The achievements chosen for recognition reflect students’ experience in the school-based MOUSE tech support team program, but as students move through the badge system, they learn general skills like effective communication and teamwork that will be useful to them in a wide variety of careers.

Use badges to map learning trajectory > Provide routes or pathways> Credential as a means to track trajectory

Intended, enacted, formalized

MOUSE intended to employ badges as a way to map students’ pathways and learning trajectories. As described by MOUSE Senior Program Manager Meredith Summs, students progress from “users of technology to specialists in applied technologies and ultimately to makers and creators” (DPD Interview Feb 2013). One of the functions of badges is to connect students’ on-site learning of practical skills and applications to an online achievement system, which would chart the growth of youth and provide them with opportunities to build on the achievements they earn. Credentials are implemented as a formative tracking mechanism, allowing students to track their progress completing the modules and to make changes if their trajectory is not where they would like it.

In the initially enacted badge system, users start out with the certification curriculum and accumulate micro-achievements called Wins!, which later add up to macro-achievements, or badges. MOUSE is still in the early phases of mapping the trajectory. Equipped with the capacity to better track analytics, they are starting to see interesting data bubble up. For instance, the data shed light on how peer-issued Wins! affect the likelihood of youth to submit more projects. The badge system guides students toward the achievements deemed most important as they can see how micro-achievements add up to specific macro-achievements, which take the form of digital badges.

This practice is an example of how a badge system can use badges to map students’ learning trajectory. As intended, students move through the badge system along a trajectory from users to creators; the badges they have earned show them how far along they are on that path.

Have experts issue badges > Credentialed via community > Badges for community

Intended, enacted, formalized

The MOUSE Wins! system intended to include badges that recognize users’ participation and engagement. The initial Community Wins! were intended to reward the number of blog posts, cases (project tickets) case tracking software, comments posted involving troubleshooting issues or open dialogue and discussion on relevant topics. Figure 1: Intended MOUSE Community Wins!

Community Wins

Figure 1: Intended MOUSE Community Wins!

MOUSE started out with these four initial Wins! and then also introduced four new Community Wins! at the beginning of 2013, including the Creativity, Technical, Inspiration, and Motivator Wins! With the Motivator Win!, MOUSE offers recognition to learners for giving feedback to others and awarding Wins! to their peers.

As enacted, Community Wins! are dynamic and tied to users’ activity in the system, but they do not feed into a community badge. The feature of community badges is an important element of the system, as it supports culture of use, draws engagement to the system, and establishes the values of the community. While they do not lead to OBI issued Open Badges, there is the potential to issue these as badges in the future.

The initial Community Wins! depended on the computer system to recognize, based on MOUSE members’ interaction with the mousesquad.org website. However, as the system developed, MOUSE wanted to recognize achievements that could not be readily measured by the computer. The Creativity, Technical, and Inspiration Wins! are awarded through peer feedback instead, by a community of peer experts.

This reflects the design principle of “have experts issue badges,” specifically where the issuing party is the peer community itself. As students gain credentials through the system, they are moving along a trajectory toward expertise in the social and technical skills needed to perform technology support.

Have experts issue badges > Credentialed via community > Peer awarded Wins!

Intended, enacted, formalized

MOUSE intended to give peers the capacity to recognize one another’s learning through awarding Wins! This involves a crowd-sourced element of recognizing one another’s learning, where the community can issue badges for learning.

As enacted in early 2013, Peer2Peer Awards are issued under the umbrella of Community Wins! As of November 2013, MOUSE members are awarding these achievements, and there will be a much clearer picture of how students use it by the time they have had a full school year to grant each other peer awards. From early data, usage is on an upward trajectory.

The design of these Wins! as Peer2Peer awards is an example of community recognition, which we have classified under “have experts issue badges,” even though peers may not always be seen as experts. MOUSE’s intent for this practice was to mirror how peer recognition of team contributions occurs in the workplace. Recognition by others who are on the same level as the recipient is as important in technical workplaces as recognition by those higher up the corporate ladder.

Recognize diverse learning > Badges for both hard and soft skills

Intended, enacted, formalized

Community Wins! and Certification Wins! aimed to recognize a range of competencies in both hard and soft skills. MOUSE created certification badges that encompass micro-achievements that represent diverse skill sets. The focus of the program is a mix of computational and workplace literacies.

The project is awarding badges for “hard and soft skills.”The majority, or at least half, of badges are awarded through MicroProjects. The MicroProjects are monthly challenges in which youth can tackle by posting their work in an electronic or multimedia form. For example, in November 2013, MOUSE challenged students to create animated GIF images themed for ThanksGIFing, helping them develop a granular skill that is popular in the Internet culture. Peers can then award Wins! to recognize their work for aspects such as creativity, inspiration, and technical skill.

As enacted, the diverse focus of Community Wins! across hard and soft skills is critical to the program. MOUSE asserted that recognizing the importance of both hard and soft skills can prepare youth for a diverse range of communities and economies that they will eventually participate in, acting as a better reflection of the learning that is expected to happen (DPD Interview Nov 2013; DML Proposal).

Especially through the MicroProjects, but also generally through their whole program’s goal of providing Wins! across the wide range of skills needed in technical support, MOUSE’s badge system is an example of the principle “recognize diverse learning.”

Design Principles for Assessing Learning in Digital Badge Systems

Students are actively involved in the assessment process. MOUSE offers opportunities to learn and practice that resemble or reflect real-world contexts of work in technology, design, and engineering. Peers provided feedback that can complement performance measurements and assessments.

Enhance validity with expert judgement > Use human experts > Promote social learning

Intended, enacted, formalized

Through the selected assessment techniques, MOUSE intends for its badging system to promote social learning, providing local environments in which youth are supported by their school community and a nationwide community to interact with online. Young people are given chances to develop their skills and knowledge through interaction with the learning community of peers and adults. MOUSE intends to offer students the chance to assess one another. The badges are intended to enhance collaborative and individual learning (DPD Interview Feb 2013).

This has stayed consistent through the MOUSE Wins! enacted practices. MOUSE staff are proponents of social pedagogy, and they see users as “givers of cred” who in turn receive feedback and credit (DPD Interview Nov 2013). MOUSE promoted social learning in their enacted practice of involving educators and students in the assessment process, aiming to reflect the feedback loop and organic process of assessment in the workplace.

In the workplace, peers are experts at performing their roles, and all employees on a team have their own specialties. It is natural to seek feedback from peers and offer it in appropriate contexts. There are also automated processes that serve as measurements of worker performance, and the automated Wins! in the MOUSE system approximate the various metrics available to measure tech employees’ achievement and learning. As such, this practice is intended as an implementation of the “use a combination of human and computer experts specific principle of “enhance validity with expert judgement.”

Use performance assessments in relevant contexts >Use performance assessments

Intended, enacted, formalized

Mirroring the work roles that MOUSE students may eventually fill, the system intended to employ performance-based assessments. The integration of performance assessment can be used to measure and award learners for performance-based concrete and “soft” skills. The learning interaction and performance tasks involve face-to-face lessons combining hard and soft skills through project-based demonstrations over the course of their certification work.

The face-to-face tech support components of the system existed before MOUSE earned a grant through the DML competition. At that point, MicroProjects, which also constitute performance assessments, were already in an early stage of development. As enacted in early 2013, students upload the record of these accomplishments in varying media formats to meet the criteria of a monthly challenge.

The most important performance assessments in the MOUSE Wins! badge system are the times when students have to perform their function as technology help desk staff and solve a technology problem experienced by a member of their school community, often under a time crunch. Students can earn Community Wins! for cases closed as a quantitative measurement of these achievements.

MOUSE’s appreciation of the relevance of performance tasks to workplace environments and the program’s structure as a functioning technical support team make this system an excellent example of how to implement the principle “use performance assessments in relevant contexts.”

Enhance validity with expert judgement > Use a combination of human and computer experts > Badges are validated by a computer scoring system, experts, and peers

Intended, enacted, formalized

The badge system intended to build a validation system, the Wins!Tracker, in which a computer scoring system, experts, and peers can validate learners’ achievements based on the badge level. In their proposal, MOUSE explained that the “Wins!Tracker is a tool that helps program facilitators and mentors access an aggregate view of their whole Squad’s work for any given certification. The interface allows check-ins on student progress during certification work, 1:1 communication functionality to offer feedback, and a “award” control that allows the facilitator to approve certification when complete. All Wins! and Certification Badges awarded appear on students’ Profile pages within mousesquad.org” (DML Badge Proposal).

In the enacted practices, a coordinator views the Wins! and approves the badge to be issued, and the artifact or skill recognized by the badge is overseen by the computer, experts, and peers. This function illustrates what badging can afford to an organization like MOUSE. It enables them to “tap a bunch of different level points to get data” and helps them to scale their system.

This practice illustrates the principle of “enhance validity with expert judgment” by incorporating experts in the process of reviewing the performance of students and awarding them badges. In addition to a computer scoring system, experts or coordinators can play an integral role in appraising the work of students and thereby strengthen the validity of the badge issued.

Use leveled badge systems >Metabadges >Leveled badge system

Intended, enacted, formalized

MOUSE intended to combine the assessment of both hard and soft skills along the progression of levels set out in the learning modules. The level-like nature of micro-achievements can function as building blocks or assessment points in users’ learning progression, as students complete activities and develop the skills that count toward a specialist badge. Macro-achievements, then, function as metabadges, representing learners’ mastery of a broader set of subskills and foundational knowledge. The earned achievements reflect the degree that learners have accomplished learning goals, assessing the growth in learners’ skill and literacy development. As students engage in the activities, MOUSE intended to offer both Community and Certification Wins! as micro-achievements toward earning macro-achievements, or badges, which would be displayed on Mozilla’s OBI.

In their enacted practices, the MOUSE system continued to award micro- and macro-achievements. While the badges themselves are not divided into levels, the Wins! are leveled micro-achievements that can be earned toward the macro-achievements of badges. The granularity of the wins structure the pathway to achievement in the MOUSE squad network as well as higher education and industry. At the same time, MOUSE explained that the micro-achievements were challenging to digest and view as a point of engagement (DPD Interview Nov 2013). On the side of engagement, micro-achievements were less meaningful. During the summer of 2013, MOUSE redesigned and updated the Certification Wins! to make them more engaging.

In the project, as MOUSE builds the capacity for students to acquire Wins! that add up to badges, this practice reflects the principle of “use leveled badge systems.” The leveled system scaffolds students’ learning, providing structure to the experiences of students in gaining skills and extending their abilities.

Involve students at a granular level > Peer assessment and collaboration recognition

Not intended, enacted, not formalized

The initial release of the process did not include the functionality for learners to add collaborators to their projects. However, learners started to add collaborators on their own, and the project is working toward building in this feature.

In the functioning badge system, assessing and awarding recognition to collaborative teams poses technical and social challenges. In terms of MicroProjects, collaborating team members could either each submit the project for themselves or for the whole squad, which may include up to 10 students. Building a system to properly attribute how these projects are owned in code is complicated, but necessary to properly recognize participants. MOUSE is continuing to address this feature.

This practice is connected to the principle “involve students at a granular level,” enabling youth to actively participate in a synergistic learning process and engage in an exchange of knowledge and perspectives that further their development.

Promote “hard” and “soft” skill sets > Combination of collaborative learning and discrete skills > Develop assessments of collaborative and technical skills

Intended, enacted, formalized

MOUSE envisioned the development of assessments for both hard and soft skills. To this end, they have created ways to gauge the learning of students in the digital badge system. The badge system employs a combination of computer-based assessments and human reviewers to examine students’ learning. For instance, students can gain Wins! for the comments they post to other’s projects or the number of blog posts they publish. Peers can also evaluate one another on the basis of criteria such as inspiration or creativity in addition to technical or motivator qualities. The practice of “develop assessments of collaborative and technical skills” illustrate the principle “promote ‘hard’ and ‘soft’ skill sets” and the specific principle “combination of collaborative learning and discrete skills,” providing a barometer for the extent and quality of students’ learning.

Design Principles for Motivating Learning in Digital Badge Systems

Students play a central role in the badge system, and their active involvement in the program boosts motivation. The MOUSE badge system engages youth by recognizing skills that are not traditionally noticed in the classroom as well as rewarding them for their participation in the community. By enabling students to award Wins!, MOUSE demonstrates the social nature of learning and its potential to inspire growth.

Recognize identities > Roles within a system > Newly recognized skills in a given context

Intended, enacted, formalized

Badges are issued for both hard and soft skills, demonstrating the value of a range of skills recognized including computational and technological skills as well as interpersonal and communication skills. The project believes this is motivational because it enhances students options to complete the activities of interest to them in the sequence they prefer. The achievements act as reinforcement along the way to a badge.

After enacting their badge system, MOUSE’s Marc Lesser agrees that issuing badges for both hard and soft skills motivates learners. It illustrates different types of leadership.  Lesser explained, “Suddenly, for [youth] to be able to have cred for the things that they do well, at least our hunch, is a motivator for them to take themselves [seriously]. In part, it’s just an efficacy thing, if they’re taking themselves more seriously as part of the learning environment, they’ll be more likely to work harder at the things that they’re not naturally inclined to get cred for, or that their natural inclinations or skills are not being credited for” (DPD Interview Nov 2013). Recognizing learning with badges engages youth who are strong in technical or hard skills, helping them take themselves more seriously as part of the learning environment.

MOUSE employed the practice of rewarding students for a broad array of skillsets and establishing their ability. This practice thereby illustrates the principle of “recognize identities,” highlighting the roles that youth can acquire and empowering them to take greater ownership of their learning.

Give badges for small accomplishments to hook in learners >Reward for number of blog posts

Intended, enacted, formalized

Users are rewarded for the number of blog posts they write and the amount of participation in the community. They are recognized for their active interaction in the program and online. The program tallies up the number of posts made for a dynamic 5 Blog Posts Win!, 10 Blog Posts Win!, and so on.

This aspect has continued in their enacted practices. The program gets more value from learners who are motivated by Wins! for the number of blog posts than detriment to those who want to game this aspect of the system. They have spent little time in addressing learners who game the system. However, there is greater value in the organic interaction that emerges from user participation and the blog posts they create. This has formed very dynamic wins, in which they top out once they reach a level of “community ninjahood” and don’t need the number of posts incentive any longer to stay engaged. MOUSE assumes intrinsic motivation will become more important as students grow their accomplishments (DPD Follow-up Interview). MOUSE also created a badge awarded simply for registering, called a New User Win! Such an award has little value once students have moved on to more important accomplishments. MOUSE similarly hopes that students will be encouraged to continue blogging by the other benefits they get outside of minor badges based on their quantity of posts.

This practice demonstrates the principle “Give badges for small accomplishments to hook in learners.” MOUSE issues Wins! to users for actively creating blog posts, fostering sustained engagement and discussion that they feel will be self-sustaining once a student reaches a certain level of integration into the community.

Utilizing different types of assessment > Peer > Peer awarded badges

Intended, enacted, formalized

MOUSE intended to allow peers to award achievements to one another, aiming to engage users in the community and motivate them to participate and further develop their skills.  MOUSE described, “In addition to system-issued or coordinator- (adult) issued Wins! in development, MOUSE seeks further support to also build functionality for our Wins!Tracker and profile areas that enable youth- or peer-issued badges, with a goal of creating aspects of the system that place youth at the center of the assessment process” (DML Proposal).

MOUSE successfully implemented peer awarded Wins! for some achievements in early 2013. The program continues to view peer awarded badges as a motivator, strengthened by seeing what feedback looks like. MOUSE can watch the progression from users who are awarded a peer issued win and then go on to award them. This hints in the direction that their hunch is right that peer awards motivate more participation.

Reflecting the principle of “utilizing different types of assessment,” the practice includes youth at the center of the process, enabling them to acknowledge their peers’ achievements. Peer recognition serves as a motivating force, underscoring the social nature of learning.

Engage with the community > Involvement in digital community > Community engagement

Intended, enacted, formalized

MOUSE intended to reward youth for their engagement in the MOUSE squad community, contributing to their motivation to actively participate in the program. Community Wins! are distinct from peer-awarded badges. The program awards Community Wins! to members for closing cases, or project tickets, through their case tracking software and for posting comments to help members to troubleshoot problems or start a discussion on relevant topics.

In their enacted practices, MOUSE introduced the Peer2Peer Awards in 2013 and Community Wins! with Creativity, Technical, Inspiration, and Motivator Wins! The Peer2Peer Awards features the capacity for students to award badges to one another. They are still continuing this in their enacted practices so far.

MOUSE’s practice of integrating Community Wins! illustrates the principle of “engage with the community.” By awarding Wins! for users’ participation, the badge system promotes further interaction and learning within the community.

Display badges to the public > Learners do not choose which badges to share with others > Showcase badges on MOUSE Squad profile pages and Mozilla OBI

Intended, enacted, formalized

The badge development system enables learners to demonstrate their Wins! and achievements to one another on their MOUSE Squad profile pages. The project explained, “All Wins! and Certification Badges awarded appear on students’ Profile pages within mousesquad.org” (DML Stage 1 Proposal). To this end, all Wins! are shown on the page, and the social nature of sharing achievements affects students’ motivation to acquire badges. Additionally, badge earners can send their Wins! to the Mozilla Open Badge Infrastructure. MOUSE asserted that in “the Spring of 2011, MOUSE also became a beta partner for Mozilla’s Open Badge Infrastructure (OBI) to connect participants’ profile data to external social and professional network sites online with the goal of releasing public-facing badges for a pilot group of advanced leadership high school students from the MOUSE Corps program” (DML Stage 1 Proposal). The practice of showcasing learner skills and achievements, as represented by Wins!, exemplifies the principle of “display badges to the public” and the sub-principle of “learners do not choose which badges to share with others.” This principle has motivation effects on badge earners’ inclination to learn and contribute.

Utilizing different types of assessment > Computer > System-issued badges

Intended, enacted, formalized

MOUSE planned to employ a computer system to verify the learning of students, affecting the motivation of students in completing the activities. In the implementation, the project effectively carried this into practice, awarding badges to students through their computer system. This system of recognition and assessment can boost students’ engagement in the task on the MOUSE platform. The practice of “system-issued badges” illustrates the principle of “utilizing different types of assessment” and the specific principle of “Computer.” The computer-based assessment system plays a role in sustaining the interaction and completion of activities in the MOUSE badge system.

Design Principles for Studying Learning in Digital Badge Systems

From the implementation of the badge system, the project tracks and collects data on user behavior and the online experience to provide formative feedback. MOUSE leverages this data to inform decisions or adjustments that can enhance the learning process.

Improve badge impact > Research FOR badges> Use site analytics

Not intended, enacted, formalized

Prior to their DML proposal, MOUSE squad leveraged sitewide analytics for formative feedback loops to the organization for the purpose of the program, interface, system, and content development. They have researched tools beyond Google analytics to collect data at more granular levels, and they intended to employ the Kissmetrics web analytics tool. MOUSE explained, “With this tool we hope to get a better sense of which segments of our member population are most active, at different levels of granularity ranging from state to site, or in different demographic groups. We’ll also have the ability to do comparisons using these different metrics” (HASTAC MOUSE Q&A). The data can illuminate the kinds of interactions and user behaviors taking place across states.

While not initially intended in the proposal,  MOUSE fully deployed the KissMetrics analytics tool in June 2013. With this tool, they can track the number of logins and see, specifically for a user, when and how they are logging in to complete which activity. The function that they are employing this tool for is two-fold: in the long term, they can get smarter about how pathways connect and longer term outcomes, and in the immediate term, they can bring their feedback loop up to hyperspeed.  Marc Lesser explained, “For a given month, if I see tons of students in Chicago who are getting really into creative projects, I can actually blast those sites a note to say “you guys all have something in common here, and here’s how we can support that as a network”. I have data that helps support our sites and provide support to educators. Surveys at end of year can’t be put into place until next school year. Now data can be acted on quickly, in terms of our ability to deliver” (DPD Interview Nov 2013). The data from the tool can enable them to provide feedback more quickly and offer support to learners and educators with greater efficiency.

In this case, the feature of Peer2Peer awards was launched upon its completion in February, meaning that there was little time to introduce it to squads and gather data on its use before the end of the school year in the spring.However, little formative data is available about its effectiveness as of the end of 2013. MOUSE Education Director Marc Lesser notes a difficulty working with school-based users in that “K-12 cycles and web development cycles are not compatible” (DPD Interview Nov 2013).

This practice is related to the principle “research for badges,” as MOUSE gathers formative evidence to inform the implementation of the badge system. The data collected is used to make inferences and adjustments on ways to connect learners and enhance the user experience.

Challenges Faced by this Badge System

The following paragraphs offer ‘challenge vignettes,’ or the narratives of issues that the project encountered during the design and implementation of the badge system. The challenge vignettes illustrate the possible challenges that can arise as well as describe how this badge system chose to confront them. While these stories are grounded within the MOUSE context, the DPD Project investigated them more deeply because they seem likely to appear in other contexts as well.

Infrastructure Constraints on the Assessment of Learning

Within the assessment category of practices, infrastructure constraints posed challenges in the design of the badge system and, in specific, the portfolio system online. Similar issues surface when integrating badges as when carrying out an online portfolio system. The process is limited by the initial decisions made about the Drupal modules to build the platforms. MOUSE has taken into account numerous requirements, facing technical challenges and bandwidth limitations. For instance, a set of constraints are posed in schools with regard to the media that is supported by the MOUSE platform and the media (such as, Youtube) that is blocked by firewall settings. There is not a “piping standard” for schools. The program has also considered the image of the assessment process, and the perceptions of youth regarding their interest in the relevance of the badges. “When students reflect to us that things feel “schooly” and boring, it means that they’re bored that we haven’t come up with a more creative way to assess them” (DPD Interview Nov 2013). MOUSE has considered the standpoint of students in designing the user experience of the assessment process.

Challenge in Studying Learning: Changing the Badge System to Accommodate Research

During the process of designing the badge system,MOUSE realized that they needed to think seriously about the data and the factors to report on at an early stage.  However, they arrived at this realization only a year and half in the process, so they rearranged the design process of new features to include this consideration earlier. As they built new features, MOUSE made an effort to think about how data would be collected and used in the feature’s operation earlier in its design process. This informed their approach by the time they started development on the peer-to-peer issuing system. However, some challenges emerge with making decisions about upgrading and changing the system. For example, for awarded Wins! that are initially tied to specific content, if the content for the Wins! changes, then there are questions that arise with how to migrate the data over and whether or not the achievement remains valid. When changes are made to the initial badges, it prompts further thought about how this would translate for users, such as whether the system would migrate the previous data over or whether they would wipe the system and start from the beginning. In essence, they would focus on thinking about how the assessment data is related to the user and how to treat the user’s experience when making decisions about upgrading or evolving the content or system.

Challenge Across Strands: Developing a Culture of Use

Challenges exist across the strands of recognizing, assessing and motivating learning in building the online achievement system and connecting established subnetworks to grow at a national scale. Issues surround the ensuring of COPPA compliance. In addition, there is the element of fostering a culture of use. The program understood that it would take a period of time, especially in the K-12 setting, for a network of users to build a culture of use around usability. MOUSE set out in the first year to design badge for end users, which had initially been viewed as exclusively young people or learners in their programs. They later realized, however, that they also needed to consider the tools that educators of those learners would employ to curate and think about the various nodes in the learning environment of youth. This brought to the forefront the question of how to involve or get educators into the system and make it an efficient way for them to access information about their students in the assessment process. MOUSE developed the Wins!Tracker assessment tool to provide educators with a way to access them and understand what to do with the nodes. Part of this also includes the issue of badges to educators for their rockstar role in the culture of the network, involving aspects of constructing the recognition system.

 

 

Recognizing Principles Assessing Principles Motivating Principles Studying Principles
Specific principles: Specific principles: Specific principles:

Leave a Reply

Your email address will not be published. Required fields are marked *

*

Mozilla Open Badges
Digital Badges are web-enabled credentials of learning or accomplishment. -Erin Knight, director of the Badge Alliance
Badges contain detailed claims about learning, links to actual evidence of learning, and they're shareable over the web. -Dan Hickey, DPD Project Lead Investigator
To me, digital badges represent the bridge between formal learning & informal structures. -Alex Halavais, DML research competition winner
Open Badges can help people tell a verifiable story about their accomplishments. -Nate Otto, DPD Project coordinator
Regardless of where you start, it’s more than likely you’ll end up somewhere other than your intended destination. That’s okay. Systems are living things, and your badge system needs to be flexible. You must embrace a bit of chaos in its design. -Carla Casilli, Director of Design + Practice at the Badge Alliance