Design for America (DFA): A Badge Community for Innovation

Preface

The information contained in this project appendix was gathered from the original project proposal that was funded in 2012 and interviews with project leaders through the end of 2014.  This appendix and the overall design principles database from the DPD project does not reflect further evolution of the project or developments after the final interview that took place in 2014. As of the final interview, Design for America (DFA) had issued badges and developed a functioning badge system. Based on this information, we have classified the DFA badge system as an implemented (rather than a partial or suspended) badge system.

Summary

Design for America (DFA) is an interdisciplinary network of university students and community members. The project aims to create local and social impact by using the needs of the community members to guide design and interaction within the system. This Human Centered Design (HCD) approach allows the project to have a broad focus on design, while creating learning opportunities that meet specific needs of users within that broader domain. Students in the program have developed designs to conserve energy, reduce the potential of acquiring infections in hospitals, and promote reading literacy, among other topics. Founded at Northwestern University in 2009, Design for America has since expanded to other universities across the country. Currently, several studios of student teams are located nationwide, operating as extracurricular programs.

Within the badging system, mentors oversee students as they lead projects focused on social change. Student team members work with community partners to develop creative solutions to real-world problems and challenges, addressing issues in the local environment. DFA developed and organized the ecosystem badging according to the empirically-grounded model of Extracurricular Design Based Learning (Gerber, Olson & Komarek, 2011), which emphasizes student-directed learning in enhancing innovation education.  The project focuses on activities and projects that are user-centered.

DFA developed an open digital badge system that realized their vision of building an online learning community and acted as a guide through the design process, as employed by practitioners in the design field. In their DML proposal, DFA aimed to build a system that accommodated the specific needs of design education while encouraging participants to leverage the expertise and knowledge within the networked community. This dual focus on content and social skill development made the interaction within the system dynamic, in that learners engaged in meaningful disciplinary discussions that raised new questions and moved the community forward in their thinking.

In 2013, DFA designed and implemented the Digital Loft Badges System to promote further collaboration and greater sharing of ideas. On their site blog, DFA explained that the badge system would provide a platform for students nationwide to share their stories with one another. The project aimed to encourage “sea of stories” that illustrate what DFA members are creating (DFA Website). Through the online platform, university students can draw insights from the community’s collective understandings, sharing their knowledge and experiences in design and learning from one another’s projects, stories, and innovations.

DFA received an NSF Cyberlearning Grant to continue development of the badge system. There has been some changeover in staff, but research and development on the project is continuing (DPD Follow-up Interview).

Timeline

Design for America was founded in 2009. The organization launched a Summer Studio program that would later integrate the badge system. After winning the MacArthur grant, the organization set to work on developing the badging project to expand the program experience. The DPD project investigated the system beginning with the DML Stage One and Stage Two Proposal, tracing the practices of the project through collecting data and conducting interviews with the project in December 2012 and February 2014. Beginning July 29, 2012, DFA launched a six-week summer program at Northwestern University in the form of the Digital Loft Badge System, with about 20 learners and 4 professional mentors (DFA HASTAC Q&A). DFA received DML funding in September and was not paired with a technical team, so it took nine months to get a development team together. The project tested the system in a design class at Northwestern University and the findings informed the next iteration. Then DFA launched a closed beta session, rolling out the badge system to Northwestern University, Virginia Technical Institute, and the class of Northwestern instructor Matthew Easterday (DPD Follow-up Interview).

At the time of the follow-up DPD interview, the Loft had not yet been launched to the public. In its initial stages, the project does not mandate learners use the Loft, but allows teams to volunteer to do so. Participating teams receive mentorship and play a role in the design of the system; DFA conducts monthly interviews with learners to inform the process of the badge system. Eight teams from four DFA sites (Northwestern, Cornell, Virginia-Tech, and Barnard-Columbia) have used the loft, with the opportunity to receive mentorship and aid in the design of the Loft (DFA HASTAC Q&A).

Context of the infrastructure design and badge system development > Existing program process and responsive design/development of badge system

Initially, DFA planned to host the badge system on their website. When they released their DML Stage Two Proposal, the project saw the “system as being fully integrated with DFA’s current WordPress website” and “made up of three parts: Profiles, Project Pages, and Community Interaction Elements.” The project intended for users to be able to display badges on their own profiles, view a library or collection of projects, and interact with other community members on the website platform.

Over the course of the design, the project launched the badge system in the form of the Digital Loft Badge System, which is an online platform developed by DFA. Initially, DFA issued paper badges as a prototype of digital badges, and built a separate online platform that hosts the content of students project and the DFA process (LOFT). The project used a pre-existing curriculum, but made the categories of activities learners could complete more salient in order to make the badge earning process smoother (DPD Follow-up Interview). The Loft provides learners with the opportunity to reflect on three core components of the system (Understand, Create, and Implement). This directs more attention to the design process and permits greater latitude with which learners complete steps in the badge-earning process. Exact badges practices vary from studio to studio, but these core elements stay constant. The activities are not part of a formal degree or credentialing program so this freedom is not currently a concern.

In their paper, Easterday, Rees-Lewis, and Gerber (2013) describe the badges in the system as well as the kinds of interactions they aim to foster with formative feedback. In their description of the development of the Loft, the authors explain the challenges of orchestrating a learning experience with users of varying expertise and interests across different environments. In response to these challenges, the authors assert that the digital badge system may be a vehicle for creating a cohesive learning experience.

Figure 1. Digital Loft Badge Design badges (Easter-Day, Rees-Lewis, & Gerber, 2013)

Figure 1. Digital Loft Badge Design badges (Easter-Day, Rees-Lewis, & Gerber, 2013)

Evolving Practices and Design Principles

What follows is a list of practices as they relate to the general and more specific design principles in each category of practice. The headings name a (a) General Principle, (b) Specific principle, and (c) Specific practice. The paragraphs below each heading detail the project’s (a) intended practice, (b) enacted practice, and (c) how that practice relates to the specific and general principles.

Design Principles for Recognizing Learning

In its badge system development, DFA transitioned from simply recognizing a progression of learning to designing a badge pathway for a learning trajectory that is learner centered. DFA emphasized the role of peers and collaborators in recognizing one another’s performance and contributions to team projects in the design process. They proposed the possibility of learners creating their own badges to acknowledge one another’s achievements and quality of work. Badges highlight the different steps of students’ design process as they work toward their goals they had identified.

Use badges to map learning trajectory > Level badges > Recognition of trajectory

Intended, enacted, formal.

As outlined in the DML Stage Two Proposal, the project development team originally planned to mirror the university process from matriculation to graduation in the design of its badge system, creating different types of badges1. DFA conceived of the badge structure as a vehicle to guide students through the design process. With an existing program structure already in place, the project designed stages that badge earners would progress through as they employed the design process to promote civic innovation. They intended to issue badges based on “completion of certain milestones achieved during the design process” (DML Stage One Proposal). DFA’s plan was to create stepwise badges and to help learners move along the badge pathway. At higher levels, badges earned could unlock the opportunity to issue badges to one another. The project envisioned learners “leveling up” through the stages in the program as they complete activities involving research, creating and building on one’s ideas, and promoting their work. These steps were organized by the categories in the process flow2 described above.

As the project started to build the infrastructure, they moved away from the stepwise model and toward one with more flexibility, highlighting the learner’s needs. DFA has moved away from the graduated system; instead students choose from a group of activities and complete them in a particular stage of the badge pathway. For example, if a student has already conducted a fair amount of research, they do not need to carry out the research activity to the same degree as someone who had not done any research. They can then proceed to the next step in the design process (DPD Follow-up Interview).  As practices were enacted, it became apparent that the “piece-wise design process” was too directive for this project; it did not make sense to predetermine the steps for learners. The project decided to use a learner-directed process, allowing learners to create their own goals in the community (DPD Initial Interview).  Rather than prescribing a set structure for learners, the organization wanted to emphasize the role that learners play in defining their own goals and pathways. This shift from their initial plan to a greater focus on enabling learners to reach their own goals on their own terms was indicative of a major epistemological shift. The badges earned make learning transparent, encompassing the activities in each of the steps and making badge pathways that learners took in applying the design process and designing user-centered solutions salient.

Ultimately, DFA leveraged the leveled nature of their badging system to recognize learners’ unique trajectories. DFA recognized that while there is a kind of progression in a design process, learners may take many several steps in varying order as they design (Digital Loft Badge System Website). The badging process incorporates the DFA categories of Understand, Create, and Implement3. The DFA team concentrated on designing activities and a badge structure focused on these three categories, organizing learners’ experiences and trajectories. In addition, the badge system is exploring lower level badges for sub-activities that lead to an overall badge (DPD Follow-up Interview). Easterday, Rees-Lewis, and Gerber (2013) explain that “project badges…break the complex design process into a series of manageable mini-challenges” (p. 5). The project mapped the badges onto steps in the design process, awarding students for completing tasks as part of the various steps. These steps represent the activities that learners carry out activities and learning artifacts they create in reaching their goals. The project is cognizant that learners are not locked into the order as determined by DFA; learners may participate in the process through recursive cycles as opposed to following it in a linear fashion.

Have experts issue badges > Credentialed via community > Credential validation

Intended, enacted, formal.

DFA planned for the community to verify the validity of the learning recognized by badges. In their DML Stage Two Proposal, the project asserted their goal to build a community-based badge system in which “the forthcoming badge ecosystem [would] incorporate more badges that are created, assigned, and evaluated by individuals and their close collaborators” (DML Stage Two Proposal). The plan included peers awarding badges within the community to assess one another’s application of the design process. DFA intended to use “a hybrid system of rubric-based badges that represent a set curriculum that is determined by an organization and identity badges that are created and assigned by the community” (DML Stage Two Proposal).  The project planned to employ rubrics as a framework to guide assessment, and mentors and design professionals within the DFA community would be involved in the evaluation process of badges. They would work with students, give feedback, and then award badges as part of that feedback. In the program, some students are more experienced in the design process than others, so the project entertained the prospect of allowing them to attain mentor status or some sort of designation to see if other students should earn badges (DPD Initial Interview). Design professionals are local community experts with experience in the respective fields that studio teams are designing for, and it was planned for them to review learning artifacts based on rubrics and peer assessment.

The DFA community has design professionals who act as professional mentors within the community; these are different from site coordinators and site mentors.  In the prototype of the badge system, equal peers will not be awarding badges, but more expert peers may receive mentor status. Within the community, design professionals of varied expertise award badges. While rubrics were being used, many learners expressed that this assessment tool made the work feel too much like they were being graded in school. They also expressed that they did not want to grade each other. As part of its next steps, DFA plans to incorporate studio leads who issue badges (DPD Follow-up Interview).

DFA validates their credentials by having experts within the community issue them. Those experts range from design professionals to mentors. As experts review learning artifacts and project work, they follow guidelines based on a rubric in determining whether to issue a badge.

Promote discovery > Discover learning opportunities > Learner directed process

Intended, not enacted, not formal.

The project considered rebranding badges as goals with subactivities; this move was to make the process more learner driven. They planned to have no preset trajectory for learners and enable them to simply choose a project and select the activities that will assist them in creating it. The project intended a shift from from telling to guiding students, envisioning badges as a tool to help learners organize and discover the process of design and enabling them to set their own goals (DPD Initial Interview). With this they contemplated this with sub-badges for activities and meta-badges for project completion. Badges, in the sense, are issued to represent process, and not products, for they are awarded for the steps and milestones that learners attain. The plan at DFA is to have people suggest different activities and projects in the badge system. Currently, this is something to aspire to and is not yet being implemented (DPD Follow-up Interview).

The project did not rebrand badges as goals, because the community accepts the meaning conveyed by badges.  However, students can approach design from different angles, completing steps in the design process relevant to their project contexts. Rees-Lewis explained that, for instance, students who were working on a project, involving a design of a teddy bear for patients with diabetes, did not require much user research. The project jumped directly into the development stage. In another project, students designed a way to offer greater accessibility to education for high risks groups, and the team conducted a lot of user research to develop a wider breadth of experiences to understand the users they were designing for (DPD Follow-up Interview). Through the badge system, the idea is for learners to discover what steps to take to better inform their design and open up new avenues of learning and understanding.

The principle “promote discovery” and sub-principle “discover learning opportunities” demonstrates the practice of “learner directed process,” and the project demonstrates an awareness of the knowledge base that grows as students pursue their own trajectories.

Design Principles for Assessing Learning in Digital Badge Systems

Design for America encourages badge earners to interact with peers, their community, and mentors within the community as they create artifacts and share about their experiences. Peer feedback and collaboration is encouraged, but peer assessment as a formal practice is not being implemented. Community mentors review and validate badges, which are issued as earners progress through the levels of “Understand,” “Create,” and “Implement.” These levels have been revised since the initial proposal to better align the language with how students experience the learning process in capturing and measuring the outcomes of learning.

Promote “hard” and “soft” skill sets > Combination of collaborative learning and discrete skills > Badges promote social literacy and collaborative learning

Intended, enacted, formal.

The project planned to focus on developing assessments for a blend of hard skills and community-based, collaborative learning. The plan included a way for badge earners to log the progress of their project and share stories as they learn from one another in the DFA community (DML Stage One Proposal). With this capability, the project aimed for the badge system to promote and measure social literacy and collaborative learning as well as individual learning. DFA asserted the purpose of “producing quality feedback that helps our learners produce better design projects that have more chance of making positive social impact” (DFA HASTAC Q&A). DFA aimed for the assessment of different levels of collaborative learning, including the learning that happens as students work in groups as well as when they apply what they have learned and share their feedback, perspective, and experiences (DPD Initial Interview).

In the structure of the badge system, students select a goal that they work toward and then choose sub-activities (DPD Initial Interview). An example of a sub-activity would be an interview protocol, and after completing an interview, students would reflect on the process of what went into making the protocol and then publish it for feedback (DPD Initial Interview). Providing feedback on one another’s applications of the design process, peers further their learning and work collaboratively toward designing creative solutions to address an issue in the community.

The principle of “promote ‘hard’ and ‘soft’ skill sets” and sub-principle “combination of collaborative learning and discrete skills” is exemplified by the practice of “badges promote social literacy and collaborative learning.” As students work in studio teams, they gain skills of teamwork, communication, and problem-solving. In addition, they complete activities that include these skills, such as conducting an interview.

Enhance validity with expert judgment > Use human experts > Peer assessment and  collaboration

Intended, enacted, formal.

The project envisioned for peers and their collaborators to bolster the validity of the claims about learning represented by a badge in the assessment process. DFA proposed a community-based badge system in which individuals and their collaborators would create, assign, and evaluate badges (DML Stage Two Proposal). Based on a rubric, the project planned for peers and collaborators to review the learning claims of a badge.

DFA built in the capacity for this assessment in its badge system. Easterday, Rees-Lewis & Gerber delineated that “the Loft solicits feedback on students’ work from professional design mentors and peers who have previously completed the badge. The mentors and peers use the badge assessment rubrics to provide feedback to students” (Easterday, Rees-Lewis, & Gerber, 2013).  In this sense, mentors or peers who have earned the badge act as experts with experience in the specific aspect of the badge system and can then provide feedback. DFA designed a peer assessment system with prompts that are linked to lower-level badges, but it is not directly linked to the earning of the badges. Fellows will look at the feedback and use this to help them determine if the badge should be issued, but it is not the cause or identifying element that triggers the issuing of the badge (DPD Follow-up Interview).

The practice of “peer assessment and collaboration” provides an example of the principle “enhance validity with expert judgment.” DFA encouraged the culture of peer feedback and collaborative learning to inform one another’s thinking, as they work to earn a badge. Badges are not awarded specifically because of feedback given by peers, but the artifacts that earn badges may be influenced by that feedback. With construction of the badging system, community members validate the badges in an informal process, and peer feedback is given and used for refinement purposes on the project. The process of peer collaboration is purely formative. In this respect, peer feedback primarily serves to inform thinking about the design process, and students comment on each other’s progress rather than give each other badges in the system.

Use leveled badge systems > Hierarchical categories > Leveled badges

Intended, enacted, formal.

The initial project plan involved assessments progressing in a leveled structure. The badge system proposed a leveled structure of badges as participation and performances increase, assessing the learning of university students along a progression of activities and sub-tasks toward the mastery of a broader skill.

As part of the project’s badging practices, DFA renamed the stages to Understand, Create, and Implement and developed ways to assess learning of the design process. The badging effort decided on these names to reflect that the stages are more oriented toward the design process, rather than the progress of students through the program. In immersing students in the design process, DFA highlights the iterative and recursive nature in which students are free to move through the steps in a flexible order, and not necessarily a fixed, linear fashion in which they have been presented. This better aligns to how students think about the learning process and their own experiences.

The practice “leveled badges” shows the principle of “use leveled badge systems” and sub-principle “hierarchical categories.” While the project articulated the challenge of providing the right amount of structure, it recognized the value of granting students enough latitude to pursue their own course and apply steps in the design process of relevance to their projects and goals.

Use formative functions of assessment > Peer feedback > Building feedback loops

Intended, enacted, formal.

DFA intended to incorporate formative assessment in the badge system, strengthening the learning and design process with feedback loops. The project planned to gather the data and feedback that emerged from the badge system to inform the process and provide a model that can scale to other communities. In light of this, the data that emerged would include formative functions of assessment that contribute to the design of the badge system.

In the Digital Loft, the project sustains feedback loops on progress, informing the learning process. Easterday, Rees-Lewis, & Gerber (2013) described three forms of feedback loops: the crowd-critique loop, case development loop–which acts as case studies of successful projects–and a learner-driven instructional feedback loop that feeds back into the design of instruction. In specific, the crowd-critique loop involves providing feedback on learning artifacts for additional revision or adjustment. They show how these elements prompt reflection on the design process and directs attention to the students’ progress, process, experience and how they can adjust or iterate on their project toward addressing an issue. The project is geared more toward how students are thinking about the process and own experience. The critique section of the badging project is purely formative, but it does not trigger the badge issuing. Rather, it helps learners complete work that leads to the badges. DFA is looking for a way to evaluate peer feedback, but they do not have a system currently in place for it (DPD Follow-up Interview).

The practice “building feedback loops” captures the principle of “use formative functions of assessment.” DFA built feedback loops in its badge system that contribute to the improvement and adjustment of the projects.

Use rubrics > General rubrics > Integrating rubrics in the badge issuing process

Intended, not enacted, not formal.

Although not directly articulated in their DML proposal, DFA had created a hierarchical task analysis in which they set goals and designed activities. The rubrics are in place for fellows to use, but students do not react in the same way to the activities (DPD Follow-up Interview). To this point, the project planned to employ rubrics to establish a common basis for which to assess and award badges to students.

The principle “use rubrics” is illustrated by the practice of “integrating rubrics in the badge issuing process,” for the project planned to employ rubrics to guide the process of evaluating student work and guiding decision-making on whether to issue a badge. In the badge system, however, the project has not carried this principle forward. Rather, the badge system focuses on the steps in the design process.

Design Principles for Motivating Learning

This initiative seeks to utilize badges as a roadmap for students; ideally, the badge system will guide students through the steps necessary to create a successful design project (though these steps in and of themselves do not guarantee success). Originally, Design for America envisioned a badge system that would reward individual students for their work, granting copious privileges and mentorships. As the project got off the ground, however, it became apparent that this approach was not the best: students were wary of the “authoritarian” claims inherent in terms like “apply for a badge.” Additionally, each student project was so unique that mapping out universal privileges to award became problematic. Because of this, the project is moving toward using the term “goals” instead of “badges” and is downplaying the role of privileges in its badge system. The initiative considered the use of the term “goals” in lieu of “badges.” It is hoped that this will lead to greater alignment of the system to student projects (DPD Initial Interview). DFA asserted that there has been a reluctance to call the credentials “badges,” as it conveys a somewhat authoritative tone and suggests something that is not coming from the student’s end but is something that others issue to them (DPD Initial Interview). It hints that students are told what to do and which path to take. This top-down approach does not agree with the project’s aim to human-centered design (DPD Initial Interview). These design decisions and considerations have implications for the motivation of learners in terms of how the learning experience is framed around the design process.

Utilizing different types of assessment > Peer > Peer assessment

Intended, enacted, formal.

The project aimed to incorporate peers in the assessment of student learning and the possibility of student-created badges, promoting the motivation of learners. In particular, badges appear to motivate engagement by encouraging students to participate in peer assessment and the badge creation process. The project noted that students’ motivation can be boosted by the knowledge that their contributions can positively impact the designs and work of other community members. DFA described in their DML Proposal that “[p]articipation and sharing in online communities is bolstered when participants are made aware of the benefit that their contributions have to themselves and to others (Beenen et al., 2004; Karau & Williams, 1993)” (DML Stage One Proposal). In considering this aspect, the project thought about how they would build this function out. For instance, the proposal stated that “a badge system for storytelling or logging project milestones implemented on DFA’s knowledge management platform can encourage students to share more once it is apparent how beneficial their contributions are” (DML Stage One Proposal). Displaying the impact of one’s contributions on a broader cause can strengthen learner motivation.

With the badge system built, the initiative mostly moved away from concept of granting privileges via badges. This is because each team working with Design for America will be so unique that granting universal privileges across all projects would be difficult. Because of this, students are actively being involved in the badge creation process. The hope is that greater student involvement will lead to greater alignment of the badge system with student projects. Rees Lewis asserted that the setting of public goals works as a more prominent motivator than the practice of peer assessment. This process involves writing goals on a board and following it with a discussion. DFA’s plan is to migrate this process to the project page and display what badges that learners are working toward (DPD Follow-up Interview).

The practice of “peer assessment” illustrates the principle of “utilizing different types of assessment” and sub-principle “peer.” DFA designed an environment in which peers can offer one another feedback on the learning process and promote student engagement.

Recognize identities > Roles within a system > Role recognition privileges

Intended, enacted, formal.

The project planned to design identity badges to recognize the distinctive roles of students. DFA envisioned awarding Identity Badges “whenever badge recipients accomplish tasks or take on group identities outside of the project curriculum…providing an added level of information about students’ roles and contributions to their projects” (DML Stage 2 Proposal). They contrasted this with Role Badges, which “signify community status” (DML Stage 2 Proposal). The DPD project noted that badges appear to motivate engagement by recognizing roles played in a community and by granting awardees the role of mentor.

As the project implemented the badge system, the initiative has moved away from awarding students badges for roles played in a community. Over the course of the badge system development, the project found that “role” badges did not make sense in light of redefining badges as goals (DPD Initial Interview). The project has considered expert and novice privileges, but it is not a priority.  Among the social-oriented design community this format seems more pronounced (DPD Initial Interview). As the badge system is set up, the badges issued now can also work with identity badges. Before creating identity badges, however, DFA wants to find out what is happening in the community before they make the badges (DPD Follow-up Interview). Instead, special emphasis is being placed on the projects that students are doing and the goals accomplished. In the future, however, increased involvement of a developer on the project may lead to special user permissions or privileges.

The practice of “role recognition” sheds light on the principle of “recognize identities” and sub-principle “roles within a system.” DFA communicated the different identities that emerge from students’ interactions, and the project plans to observe user behavior prior to creating identity badges.

Setting goals > User-created badges > Users create badges they aim to earn

Intended, enacted, formal.

The project hoped that student involvement in system creation will lead to greater buy-in (DPD Initial Interview). Additionally, the project aimed to motivate students by granting content-creating privileges. DFA proposed the development of a badge ecosystem “to incorporate more badges that are created, assigned, and evaluated by individuals and their close collaborators, ” explaining that such a badge system is “more telling of recipients’ skills and achievements than badges that are created and administered in a top-down fashion” (DML Stage Two Proposal).  The project planned to provide students with greater freedom to pursue their own projects and goals, thereby also affecting the motivation of students. As described in their DML Stage Two Proposal, the badging project asserted that “[c]rowd-sourced badges have the potential to tell a rich story of learner achievement and identity” (DML Stage Two Proposal). As students create badges, they could feel a greater sense of ownership over their learning and the skills they strive to acquire. Moreover, DFA described that they saw “crowd-sourced badges as taking responsibility (and pressure) away from issuers to create badges that are relevant to their students, represent diverse student skills and achievements, and are constantly motivating students to achieve diverse learning outcomes” (DML Stage Two Proposal). This way, students can also create badges that are relevant and motivating, involving them in the process of considering which skills and criteria are key to assess.

DFA tried to rethink the aspect of having students coming up with own activities and own goals. During the planning process, DFA explored the possibility that badges could be self-created, enabling students to create the goal or badge in the system. As of the DPD initial interview, the project has been reworking that aspect (DPD Initial Interview).

Setting goals > Display of goal trajectory > Users can see their progress

Intended, not enacted, not formal.

The project intended to assign badges to users for completing certain subactivities or project steps in the stages. Users can track their progress by seeing what badges they have attained as part of a larger goal. Badges act as milestones that learners have reached in applying design thinking. The project originally started with preset goals and thought about the idea of goal recommendations depending on novice or expert group, selecting and creating goals if needed (DPD Initial Interview). DFA asserted, “Specific goals and well-defined challenges encourage legitimate participation and motivated performance when compared with a more free-formed approach to problem solving (Locke & Latham, 1990); for this reason, a badge system can guide and motivate students along the way to completing life-changing designs” (DML Stage One Proposal). In the badge system, this practice has not yet been fully implemented.

Setting goals > Provider-determined learning trajectory > A badge is awarded for earning badges in a similar domain

Intended, not enacted, not formal.

The project intended to issue badges that share characteristics of a meta-badge. They explained in their DML Stage Two Proposal that the “tenet completion badges represent a collection of Project Completion badges. A tenet badge piece is earned once the recipient has collected all of the project badges for that tenet” A tenet badge recognizes when learners have completed multiple badges in a similar domain.

The project has not yet carried out the practice of awarding badges in a similar domain through issuing a badge for the activities students complete in a specific step. The DPD project noted that this practice has motivating forces as students work toward goals. With respect to setting goals, the project provides structure for the steps and activities that users can take and complete, but users can choose from a menu or options of which steps to take (DPD Follow-up Interview).

Engage with the community > Involvement in digital community > Creating a badge community

Intended, enacted, formal.

The project aimed to promote a knowledge community in which learners could share insights and experiences in the design process. They stated, “We propose a badge system design that is largely community based” (DML Stage Two Proposal). Badges are awarded and created by the community, and the project encourages interaction among community members and offers opportunities for them to provide feedback and share ideas. They asserted, “As well being a signpost for skills and dispositions learned for effective HCD, we believe a badge system has the potential to motivate project teams to share their learning and foster reflection on projects by highlighting areas of excellent performance” (DML Stage One Proposal). With this vision, the project planned to strengthen engagement with the community in its digital badge system.

The practice “creating a badge community” brings into focus the principle “engage with the community” and sub-principle “involvement in a digital community.” DFA envisioned a community in which students and design professionals could share their experiences and perspectives, broadening the learning process.

Design Principles for Studying Learning

The project has studied the Digital Loft Badge System and published an article on their findings with regard to the various components and elements involved in structuring an effective learning experience. The project collects data and gathers feedback on the development and interactions of learners on the LOFT platform, aiming to simultaneously research, analyze, and develop creative solutions in addressing a need or issue within the community (Easterday, Rees-Lewis, & Gerber, 2013). In this way, the badge system would play a role in the gathering of data and collect evidence on student learning and participation that would shed light on students’ innovations and promote learning.

Improve badge impact > Research FOR Badges > Finding synergy between curricula and creative thinking

Intended, enacted, not formal.

DFA conducted research to inform the continual development of the badge system. The project has explored the questions of how to guide students through the design process while also allowing it to be flexible enough for them (DPD Initial Interview) to align with their creative thinking. The project focused on what they built and how to “best align the two, creating activities and goals that allow students to learn what they are supposed to do and be creative” (DPD Initial Interview). The project aimed for a platform in which community members share stories of the innovations produced by studio teams in addressing real-world challenges. DFA affirmed that “[c]ollectively, this research will increase DFA’s measurement and evaluation data to more deeply understand  innovation self-efficacy and informed design practice” (DFA Website).

The project employed a design-based research approach in developing solutions that have practical applications. DFA aimed for research to strengthen practice as well as to communicate to the broader community how such a model can scale to similar contexts (HASTAC Q&A). The project team included learning science researchers from Northwestern, who conducted studies and published results on the Digital Loft Badge System. Elizabeth Gerber, co-founder of DFA, developed “a validated measurement of innovation self-efficacy to be used to measure the impact of such interventions in academia and industry” (DFA Website).   The project is trying to place one foot in practice by having conversations with designers and seeing what they are doing, too. While this scenario has not come up yet, if an undergraduate community wants to start working on a project that designers do not deem worthy, it is uncertain what outcome would follow (DPD Follow-up Interview). It crosses the intersection of academics and industry across fields and disciplines, and the organization collaborates with community partners to approach local issues.

The project built in feedback loops that contribute further to the badge system development. For example, the case development loop and learner-driven loop provides feedback that can inform the decisions made in the program and design of the user experience. Regarding the case development loop, the project curates a library of successful studio projects to serve as models that illustrate the design principles, supplementing the learning experience of students. The project can leverage case studies as an instructional strategy. With respect to the learner-driven loop, the data that emerges from this can directly impact the instructional programs and topics chosen for further exploration. These feedback loops apply to the broader design of the badging project. In the badge system, the project has not yet continued to formalize this practice.

The principle “improve badge impact” and sub-principle “research for badges” is exemplified by the practice “finding synergy between practice and curricula.” The project built feedback loops that contribute to the development of the badge system, and it is gathering data in the process of testing the system to apply to and further refine the design.

Challenges this Project Faced

In the process of building the badge system, DFA encountered technical constraints and the development of an assessment system for the learning and studio team projects. In addition, the project faced challenges in striking the right balance between providing sufficient structure to guide students through the design process and enabling them freedom to pursue their own goals by taking the steps appropriate for their projects.

Technical challenges and assessment considerations

The project faced infrastructure constraints and technical challenges, as the developer did not supply what the project wanted. As the developers did not give the project what they needed, the project’s efforts were concentrated toward fundraising and HCI (DPD Follow-up Interview). After the project organized a technical team, they set to work on implementing the badge system. DFA carried out initial rounds of testing in a design course at Northwestern University and other design studios at Georgia Institute of Technology. As of the DPD follow-up interview, the project issues paper badges and not digital badges. In the process, they have developed a rubric system and cultivated a learning environment that fosters peer feedback. However, givers and receivers of critique have something similar to rubrics to test out, but they did not respond well to giving someone an evaluation. The users did not want to grade each other, as this felt too much like school. Based on the interactions that emerged from the badge system, the project integrates rubrics primarily for fellows in providing guidelines on assessing a project and deciding whether to issue a badge. As the project continues to test additional iterations of the badge system, it collects data from the system to inform continual adjustment and improvement.

Challenges in motivation and studying/recognition

DFA asserted the challenge in finding the right balance between providing students with enough structure and granting them latitude to pursue topics and projects of interest to them. Easterday, Rees-Lewis, & Gerber (2013) described that to leverage students’ “intrinsic motivation to address real world problems… [the project] must accept a certain level idiosyncrasy of projects.” By permitting students to set their own goals, they may feel more inclined to work toward them as opposed to following a plan provided by an authority. The team further raised the issue that once “different groups work on different, more complex problems, at different speeds, working with clients in the community, and so on, it becomes almost impossible for a single teacher to orchestrate learning in a productive way” (Easterday, Rees-Lewis, & Geber, 2013). This poses a challenge of orchestration in carrying out a learning experience that takes into account different individuals, interests, backgrounds, and projects. Additionally, as the project finds an optimal balance undergrads and design professionals, the projects wants the experience to ultimately be valuable to students. Different users want different things, and students respond better to studio leaders dictating things as opposed to DFA national. This adds another layer of communication to the badge system (DPD Follow-up Interview).  The project is working toward balancing these elements as they gather user feedback and focuses the design on the usability of the badge system.

Notes

1 Initially, the project created three core stages through which badge earners would progress: (1) Matriculation, (2) Progress, and (3) Graduation (DML Stage Two Proposal). While these stages are not necessarily badges, they serve as groups or categories for a set of badges. In line with this progression, the project planned to begin the badge structure with matriculation. This would help incorporate activities such as research in the progress stage, organize goals in building and prototyping and promote the project in the graduation stage (DPD InitialInterview).
2DFA proposed four badge types that would support the program experience: (1) Project Completion, (2) Tenet Completion, (3) Identity, and (4) Role. Project Completion Badges are awarded when students successfully complete their goals, while Tenet Completion Badges are awarded to learners after completing three goals related to one of the DFA tenets. The Identity Badges are issued to learners for assuming or taking up identities, such as tech guru and so on,  in groups, and the Role Badges are a sign of one’s status within the community.
3 Each of the three stages contain three more specific steps. Understand involves Identify, Immerse, and Reframe. Create includes Ideate, Build, and Test, and the Implement stage includes Plan, Prove, and Sustain. The three stages represent components of the human-centered design principles, which includes the process of conducting research, prototyping, and rolling out the product or design solution to the consumer. In the research phase, students collect data on their users to learn more about them and understand their circumstances or point of view.

References

Design for America. (n.d.). Design for America DML Stage 1 Proposal. Retrieved from

http://dml4.dmlcompetition.net/dml4.dmlcompetition.net/Competition/4/badges-projects.php%3Fid=2784.html

Design for America. (n.d.). Design for America Community Badges DML Stage 2 Proposal.

Retrieved from http://dml4.dmlcompetition.net/dml4.dmlcompetition.net/Competition/

4/badges-projects.php%3Fid=3235.html

Easter-Day, Rees-Lewis, & Gerber. (2013). Formative Feedback in Digital Lofts: Learning

Environments for Real World Innovation. Retrieved from

http://ceur-ws.org/Vol-1009/0809.pdf

Project Q&A with: Design for America. (2013, July 1). HASTAC. Retrieved from

http://www.hastac.org/wiki/project-qa-design-america-badge-community-innovation

Rees-Lewis, Daniel. (2014, February 5). DPD Follow-up Interview.

Shyrokova, Anya. (2012, December 7). DPD Initial Interview.

 

Recognizing Principles Assessing Principles Motivating Principles Studying Principles
Specific principles: Specific principles: Specific principles:

Leave a Reply

Your email address will not be published. Required fields are marked *

*

Mozilla Open Badges
Digital Badges are web-enabled credentials of learning or accomplishment. -Erin Knight, director of the Badge Alliance
Badges contain detailed claims about learning, links to actual evidence of learning, and they're shareable over the web. -Dan Hickey, DPD Project Lead Investigator
To me, digital badges represent the bridge between formal learning & informal structures. -Alex Halavais, DML research competition winner
Open Badges can help people tell a verifiable story about their accomplishments. -Nate Otto, DPD Project coordinator
Regardless of where you start, it’s more than likely you’ll end up somewhere other than your intended destination. That’s okay. Systems are living things, and your badge system needs to be flexible. You must embrace a bit of chaos in its design. -Carla Casilli, Director of Design + Practice at the Badge Alliance