PBS News Hour Student Reporting Labs

Preface

The information contained in this project appendix was gathered from the original project proposal that was funded in 2012 and interviews with project leaders through the end of 2014.  This appendix and the overall design principles database from the DPD project does not reflect further evolution of the project or developments after the final interview that took place in 2014. As of the final interview, PBS News Hour had issued badges and developed a functioning badge system. Based on this information, we have classified this badge system as an implemented (rather than partial or suspended) badge system.

Summary

Student Reporting Labs is a educational program offered by PBS News Hour in which participating high school teachers offer their students the opportunity to write and record a story about their community with the chance of getting it broadcast on local or national television. This program was among two other components (Roadtrip Nation and StoryCorpsU) of a proposal the Corporation for Public Broadcasting (CPB) put forward in the 2012 Digital Media and Learning Badges for Lifelong Learning grant competition. All three programs “serve at-risk high school students from largely underserved populations and struggling schools,” and each “seeks to leverage the resources of public media to help students find relevance and confidence through storytelling, relevance that may help propel them to high school graduation and a successful post-secondary education or career pathway” (HASTAC Q&A).

The program summarizes its motivation as a belief that “journalism and storytelling are ‘killer apps’ that use the best practices of project-based learning to build engaged and digitally-literate young citizens,” meaning that the skills students using project based learning and the technology of the badge system are valuable enough to prove the worth of those designs (Student Reporting Labs).

Timeline

News Hour’s education program predated the DML Competition, but digital badges were a new addition that came with the program and were developed in partnership with Pragmatic Solutions. Through the 2013-14 school year, Student Reporting Labs implemented the system, debugged technical problems, and successfully issued badges. The program sought and received  continuing funding from CPB.

Evolving Practices and Design Principles

What follows is a list of practices as they relate to the general and more specific design principles in each category of practice. The headings name a (a) General Principle, (b) Specific principle, (c) Specific practice. Below each heading, we indicate what stages this principle appeared in the project: as an (a) intended practice, (b) enacted practice, or (c) formalized continuing practice.

Design Principles for Recognizing Learning

The Student Reporting Labs badge system offers 12 component badges leading up to a program completion badge and as the program evolves, they are adding additional high level badges like a “career ready” and “SRL All Star” badges for the highest performers. High schools, after school clubs, and other programs are eligible to run a Student Reporting Lab, matched by PBS and News Hour staff with mentors from a local PBS station.

Use badges to map learning trajectories > Level badges > Program completion metabadge

Intended, enacted, formal.

News Hour defines the curriculum, but not levels of achievement or evaluation methods. The program does offer a metabadge, which constitutes a 2nd tier that is only earned when students complete all 12 components of the program. This badge, the Student Reporting Labs badge, is the only one in the system that carries the PBS logo. Stacy Kruse from technology partner Pragmatic Systems reports, “Teachers requested leveled badges, because they wanted more control to be able to distinguish the students,” but that feature is not implemented yet, because it needs more investigation. The program is collecting more input and a “360 degree evaluation” of the possibilities (DPD Follow-up Interview). “Badges are all multi-step, and they all represent team collaboration”, so in some sense the intention to level badges separates those badges from their underlying criteria components, but it doesn’t constitute a use of leveled badges outside of the intention. Some higher forms of recognition, like students’ stories appearing on TV are not directly connected to badges but are still an important component of the learning program.

Align badges to standards > Use national or international standards > Map badges to competency frameworks

Intended, not enacted. not formal

The complete Student Reporting Labs badge describes the competency frameworks it maps to, but it doesn’t recognize competency in particular skills themselves. A later addition to the program is three “career ready badges,” which as of early 2014 are in development. These include a “project leadership” badge, based on an effort to recognize identified 21st Century Skills. The program intends that these badges will recognize star students, who would be ready for an internship program, for example.

Have experts issue badges  > Credentialed via external accredited entity > News Hour experts approve final badges

Not intended, enacted, formal.

While the program intended to leave badge assessment and issuing in control of local participating teachers, as the program got started, News Hour staff found themselves in the role of performing final verification for badge issuing. “There is always an artifact requirement and a certification requirement, [where certification is] at the News Hour level.” Earners “have to check a box that says they have read the requirements and certify that they have done them”, and then a subject matter expert at News Hour is able to performs a review (DPD Follow-up Interview). In addition, News Hour staff will evaluate and issue the career track badges themselves, based on nominations from teachers, student work reviews, and classroom visits (DPD 2nd Follow-up Interview). For the time being, this is a continuing practice in the system.

Use badges as a means of external communication of learning > Badges indicate proficiency to partners.

Intended, enacted, formal.

Student Reporting Labs staff knew that local PBS affiliates had hired their students in the past and would continue to do so as the badge system practices were enacted. With the badge program, and especially the upcoming career readiness badges, there is now an even stronger intent to share the employability of star students with this audience. They see this as part of the process of building relationships with local stations.

Recognize educator learning > Badges for participating teachers.

Not intended, enacted, formal.

News Hour created a badge for teachers after a those teachers requested them. PBS awards this badge as participating teachers complete the boot camp training from the national News Hour education staff, and those staff also sign off on 16 hours of professional development, though they are not a fully accredited provider of PD. The actual awarding of teacher badges is done manually, and News Hour separately recognizes teachers who fully implement the program in their classrooms. There is a separate “technology innovator” badge conferred on mentor level veteran teachers, though the criteria for this badge had not been solidified as of the beginning of 2014.

Promote discovery > Discover learners Promote learners for futuer opportunities

Not intended, enacted, formal.

Student Reporting Labs staff view badges as a means by which their students could be discovered for opportunities. Leah Clapman, the managing editor for education at News Hour, says, “Badges will help this conversation with stations. It’s one thing to say they’re a great videographer, but another to say they’ve earned the badge” and to show evidence of the student’s achievement (DPD 2nd Follow-up Interview).

Design Principles for Assessing Learning in Digital Badge Systems

There are assessment points at nearly every step of students’ work in the Student Reporting Labs program, all of which were in place before badges were implemented. Both News Hour staff and mentors at local stations provide feedback on submitted artifacts. Badges did change where students needed to submit artifacts, which was a technical challenge, as parts of the system encountered technical difficulties (DPD Follow-up Interview).

Use formative functions of assessment > Expert feedback > Mentor and expert assistance.

Intended, enacted, formal.

Close contact between students and staff from both local PBS stations and the national News Hour team was always intended to help students as they developed their stories. News Hour staff and mentors from local stations provide feedback to students as they help students develop their stories. These are conducted via Google Hangout video chats to give feedback. Leah Clapman emphasized that this is a very informal process and also an expected “key requirement” of the space (DPD 2nd Follow-up Interview).

Use leveled badge systems > Metabadges > Program completion badge

Intended, enacted, formal.

As initially planned, in the enacted and continuing system design, there is a “News Hour” metabadge for completing all 12 processes in the curriculum. These processes are not arranged in a linear fashion. The 2nd-tier News Hour metabadge is the only badge with the PBS logo on it. There is no additional assessment that occurs to earn the metabadge, but all component badges must be completed.

Enhance validity with expert judgment > Use human experts > News Hour staff perform final badge approval.

Intended, enacted, formal.

The technical system was designed and implemented to allow students to assert when they had completed the criteria for a badge, upon which, their work would be evaluated by their classroom teachers, and if the teachers approve, News Hour staff would approve or reject the badge. Teachers have the option to send the work back to the student with no penalty to give them a chance to amend their work and resubmit.

Use rubrics > General rubrics > Rubrics for career ready nominations

Not intended, enacted, formal.

Student Reporting Labs didn’t originally see rubrics as necessary, but in developing higher value badges for “career ready” students, the program began developing some rubrics to guide classroom teachers in who they should nominate for those awards. SRL made sure to “hit all the important things” for video production skills.

Use e-portfolios > Open to public > Collecting and maintaining video content

Intended, not enacted.

Implementing the badge program meant changing how Student Reporting Lab handled student work. There were significant technical hurdles to building a working upload and portfolio system for the type of content created by students in News Hour’s program. Students’ finished videos have always been posted to the SRL website, but they had not been organized into portfolios. As SRL begins considering the nomination process for “career ready” students and the SRL All Stars travel opportunity, they plan to collect peer testimonials as well as written reflections about the decision making behind the process of making and editing a video. There is some discussion around how a portfolio could bind all of these elements together (DPD Follow-up and 2nd Follow-up Interviews).

Align assessment activities to standards: create measurable learning objectives > National/state standards > Matching between curriculum and standards.

Intended, not enacted.

There was some loose matching between curriculum elements and components of standards, but program staff felt it was a fairly loose alignment. Outcomes are not specifically tested against standards.

Design Principles for Motivating Learning

SRL found that one-on-one interaction between students and News Hour staff during classroom visits is really valuable. There are a lot of points where they can offer opportunities that are rewarding, including interviewing students or even retweeting announcements of a student’s video appearing on the News Hour website. In addition, SRL is beginning to use an All Star badge and associated prizes, including trips and other special opportunities. Leah Clapman says while these other elements of the program may be more powerful motivators, the badges “do make it clear what they need to do or keep doing to get these rewards and opportunities” (DPD 2nd Follow-up Interview). Unique among programs investigated by the DPD Project, Student Reporting Labs enacted all four of the specific principles under the general principle “Provide privileges,” which is further analyzed in a case study by Katerina Schenke.

Provide incentives > Prizes > All Star trips and associated rewards

Not intended, enacted, formal.

SRL All Stars, a way to honor their top students, was not originally part of the badging system. Students get to come to Washington DC and gain the title of “student ambassador.” Students get to come to a workshop, and in some cases may lead components of workshops. One All Star got to join a video chat with News Hour host Gwen Ifill (DPD 2nd Follow-up Interview). Stacy Kruse from technology partner Pragmatic Solutions had recommended a really great reward for top students as a “super-motivator”, which has taken the shape of the All Star program. Leah Clapman sums up the project’s feelings about this kind of reward when she says that, “There is a lot of validation that what they are doing is special” (DPD 2nd Follow-up Interview). The motivational rewards seem to be at least as much about introducing the most promising students to exciting possibilities along career pathways in journalism as a means to encourage these students to continue in the future rather than strictly as a carrot in front of them as they participate in the SRL program in high school.

Provide incentives > New activities > Video publication and promotion

Intended, enacted, formal.

Students who produce stories through Student Reporting Labs have the chance to have their work appear on a public-facing News Hour website or even as a segment on television. These achievements represent high-level rewards for students who produce the best work. In addition, the chance to travel to Washington DC events is a possibility for select “All Star” students and could be considered a new activity as well as a prize.

Provide incentives > Peer mentorship > Badge for peer mentorship

Intended, enacted, formal.

News Hour created a badge for those students who were recognized as being allowed to mentor others in the system, as initially intended (DPD Follow-up Interview).

Provide incentives > Internships > Access to partner station internship programs

Intended, enacted, formal.

PBS affiliates and News Hour staff keep their eyes open for high-achieving students to feed into local affiliates’ internship opportunities. As intended, contact with students through News Hour employees and participating mentors from local stations, as well as the “career ready” badge nomination process provides a gateway for students to access existing internship programs. Stations seeking interns passively and actively look for the most promising students, and the evidence attached to SRL badges could be an important collection point for

Recognize identities > Roles within a system > Recognize video journalism component roles

Intended, enacted, formal.

Some of the component badges of the SRL program recognize fun and attractive parts of filmmaking, like the “Camera Operator” badge. SRL staff feel those kinds of things were motivating. Staff also hoped to adjust the system to narrow the available goals around filmmaking, perhaps aiming toward shooting, editing, and leadership.

Recognize identities > Targets a specific group  > Telling stories about student communities

Intended, enacted, formal.

Student Reporting Labs tries to partner with Title I or schools with low resource levels, though schools that are “well-resourced” also apply to join. The program aims to enable those kids to tell stories about their communities, where News Hour feels unable to tell stories about those communities on their own without their participation. Leah Clapman feels that students do recognize that News Hour values the stories about their communities. Sharing that value is part of the package of motivational practices in the system.

Display badges to the public > Learners can choose to share their badges with others > Basic social media sharing for motivation

Intended, enacted, not formalized.

110 Intended, Enacted. At first, SRL thought that it would be motivating enough for students to be able to share their badges out to Facebook, but staff realized there needed to be a bigger reward for the badges to have more meaning. Alterations to motivation practices are aimed to make the badge system easier for teachers to operate and also to increase the level of honoring students for their work with means beyond the badges and their shareablility (DPD Follow-up Interview).

Build outside value for badges > Evidence for outside opportunities > Internship opportunities

Intended, enacted, formal.

As detailed in a case study on establishing the value of badges to potential earners, students are often skeptical of program administrators’ claims that badges will have external value. Across all three Corporation for Public Broadcasting badge projects, the programs intended to provide badges that would have “real life value.” Across all three, students “pushed back and demanded to see what the outside value would be. What jobs? What internships, where?” Stacy Kruse, from technology partner Pragmatic Solutions, said “You have one chance to make an empty promise,” and that the at-risk students targeted by these programs don’t automatically assume that their tomorrows will be better. “They will be waiting for you to bring them the light at the end of the tunnel.” She said they received that kind of feedback across many of the schools implementing the program and were not prepared to hear it (DPD Follow-up Interview).

However, in Student Reporting Labs, there was an intentional connection to internship opportunities through PBS partner stations. Each Student Reporting Lab partners with a local PBS station, and the badge system facilitates internship access, as noted above. Teachers requested this practice to help build outside value for the badges and experience their students gained through the program (DPD Follow-up Interview).

Set goals > Display of goal trajectory & User-determined learning trajectory > Program completion pathways

Intended, enacted, formal.

All the CPB Programs were self paced and not intended to be absolutely sequential. This allowed some student choice in selecting their own trajectory.

For News Hour, this took the form of a checklist of necessary steps to completion, where some sequential ordering was necessary. Program completeness was marked by a filled-out checklist. Leah Clapman says, “The process itself has lots of obvious opportunities to set goals, from the process of coming up with a news story, what is newsworthy, working with a mentor. The mentor helps you get from that story idea to a finished report, and there are goals all along the way (DPD 2nd Follow-up Interview). The whole process of news reporting is about setting goals and then hitting them.” SRL tried to make the specific goals into badges, but there was difficulty separating everything out. Clapman indicates that it was hard to pinpoint the exact problem with the initial implementations, but that it was clear there were too many badges and too much difficulty. This led to the system lending more weight to the finished product of the news report and the overall SRL badge. The fact that students are able to tell their stories is the most important accomplishment, and it is the output of an involved process.

Stimulate competition > Scarcity of badges > Some limited availability badges for high performing students.

Not intended, enacted, formal.

SRL added the SRL All Star and Career Ready badges after the initial planning phase, and these badges create some scarcity-based competition, but program staff feel it is a healthy competition. “Teachers know that not all the kids will be nominated for an All Star badge,” says Leah Clapman. They know that there will be about one special kid who stands out in each class and teachers are unable to nominate too many (DPD 2nd Follow-up Interview). 

Design Principles for Studying Learning

Unlike many programs funded by the DML competition, Student Reporting Labs implemented a limited formal survey to study the system, partly with the goal of determining why some participating Student Reporting Labs chose not to implement the badging component of the curriculum.

Improve badge impact > Research FOR badges > Implementation survey of teachers

Intended, enacted, formal.

Student Reporting Labs designed and implemented a study designed by Jim Marshall at San Diego State University to determine why some participating teachers were not using the badges.

 

Recognizing Principles Assessing Principles Motivating Principles Studying Principles
Specific principles: Specific principles: Specific principles:
About

Loves open education, Open Badges, free culture, Progress of the Useful Arts and Sciences, people-powered politics, and local food production. Coordinator for the badges Design Principles Documentation Project at Indiana University.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

Mozilla Open Badges
Digital Badges are web-enabled credentials of learning or accomplishment. -Erin Knight, director of the Badge Alliance
Badges contain detailed claims about learning, links to actual evidence of learning, and they're shareable over the web. -Dan Hickey, DPD Project Lead Investigator
To me, digital badges represent the bridge between formal learning & informal structures. -Alex Halavais, DML research competition winner
Open Badges can help people tell a verifiable story about their accomplishments. -Nate Otto, DPD Project coordinator
Regardless of where you start, it’s more than likely you’ll end up somewhere other than your intended destination. That’s okay. Systems are living things, and your badge system needs to be flexible. You must embrace a bit of chaos in its design. -Carla Casilli, Director of Design + Practice at the Badge Alliance