Best Practices in Course-Embedded Assessment

April 27, 2008

We’re just finishing up three day’s at NC State’s Assessment Symposium. 500 educators from around the USA have come together to talk about student learning, “closing the loop,” and accreditation.

Many of the sessions are focused not just on data-gathering, but on teaching and learning. A number of attendees have talked about the change they’ve seen since even last year: a focus on bringing assessment into the process of teaching (!). That is, avoiding the mad dash to develop data just for accreditation that often results in two databases of student learning outcomes. One presenter said that on her campus administrators referred to the “shadow database,” which reminded me of a business owner keeping two sets of books – one for the IRS and one for the real world.

We gave a 60 minute presentation on best practices in course-embedded assessment. We must have had at least 50 people in attendance…not to learn about Waypoint as much as to gain insight into how schools execute.

I spoke in three general areas:

  1. Getting faculty help with the challenges of formal assessment
  2. “Closing the loop” – using data to inform changes in curricula
  3. Using a sampling approach to gather data quickly and efficiently for benchmarking purposes

Getting faculty help with the challenges of formal assessment:

We increasingly talk to senior administrators about the need to look at authentic assessment and course-embedded assessment as more than a challenge in software training. This work is not about clicking the right buttons in Blackboard or Waypoint. Read the rest of this entry »


Designing Authentic Tasks…

March 21, 2008

We set out with this blog to create a resource for educators interested in assessment, and particularly authentic assessment. That is, designing tasks that are complex, that require critical thinking and synthesis of information. Tasks that are more real world and, more importantly, enjoyable for students and teachers.

A lot of the readers who come to this blog are searching the web for resources on rubrics – often for specific disciplines like nursing and engineering.

So I’d like to take the opportunity to describe a relatively simple philosophical shift when designing work for students. I’ll start with an example many of us can relate to: memorizing and reciting a poem. Read the rest of this entry »

Teaching Efficiency…

March 16, 2008

As teachers, we tend to focus on pedagogy as the great hope for improvement.

We spend our professional development and meeting time talking improved syllabi, new theories of learning, better designed assignments, or improved feedback.


(Click on image for a larger view)

And these are clearly, and massively, important.

But long ago (1980s), engineers, lawyers, accountants and other professionals looked to technology to save time. Spreadsheet softeware, CAD (Computer Aided Design) software, and other tools are now integral to these professions.

Educators are famously reluctant to look to technology for help. Sure, every school has its 10 to 15% (my estimate) of technology early adopters. But there hasn’t been a compelling reason for educators to embrace technology. In 1982 you either learned CAD as an engineer, or looked for another job (okay, or got promoted to management).

Thankfully educators are not usually under such autocratic rule, but perhaps it is no accident that the two industries with the least productivity over the last 20 years, and the largest year-on-year cost increases, are medicine and education. Both are people intensive. If you want to teach more students and cure more patients, you have to hire more people. Technology companies and accounting companies do not have their productivity tied to people-power in the same way, and so many medium-sized companies that had full-time accounting departments 20 years ago can now look to a single person or two, or simply outsource the work.

Read the rest of this entry »

PACT – Performance Assessment for California Teachers – and Waypoint Course-Embedded Solutions

March 1, 2008

We have been excited to receive a number of inquiries from California Colleges of Education in the last few weeks. We quickly learned that PACT – a consortium of CA schools – had recently released a library of rubrics designed to assess teacher candidates. Here’s their blurb:

PACT (Performance Assessment for California Teachers) is a consortium of teacher preparation programs at a number of California Universities. These institutions have joined together to develop a teacher performance assessment. Successful completion of the teaching performance assessment will be required to earn a California Preliminary Multiple Subject or Single Subject Teaching Credential.

The teaching performance assessments consists of Embedded Signature Assignments (ESAs) and the Teaching Event. Together, the Embedded Assessments and the Teaching Event measure all thirteen Teacher Performance Expectations (TPEs)

With a growing number of Colleges of Education using Waypoint nationwide (look for a listserv created by profs at William Paterson University to start soon for Colleges of Ed using Waypoint), we are in a great position to help the consortium.

A number of the inquiries have come from schools running an assessment/portfolio solution (the kind where students have to pay the cost) to satisfy other accrediting bodies, but these tools are separate from the institution’s Course Management System (Blackboard, Moodle etc.) and way too complicated for faculty to use on a regular basis to assess students.

So Waypoint’s course-embedded solution – embedded rubrics AND potentially embedded in the Course Management System – can have schools up and running in a matter of days.

We’re building in the entire library of PACT rubrics so they are available for our clients and look forward to seeing how they work on the iPhones, iPod touch, and Windows Mobile devices for assessing teachers in the field.

Undergraduates “fighting for feedback”

February 6, 2008

The University of Pennsylvania‘s student newspaper, The Daily Pennsylvanian, recently included a student-written piece on the lack of feedback in higher education. The article is called “Fighting for Feedback,” and it is a show stopper.

If we want to talk about accountability and outcomes assessment, what greater measure is there than students receiving feedback from their professors? David Kanter’s article begins,

The first time I got a paper back from a professor here at Penn, I was a little confused.

Other than a few perfunctory, illegible comments found scribbled in the margins, insightful, constructive criticism was nowhere in sight. I thought (incorrectly, I suppose) that I would receive extensive feedback on each assignment. I soon learned that unmarked papers and vague comments were the norm.

This is a message we’ve heard over and over again. It isn’t a Penn thing. It isn’t an Ivy League thing, or a private school thing. It’s just the old paradigm of pushing information at students rather than helping them discover knowledge for themselves. Some of us are more talented pushers than others, and can make that mode of education work through charisma and talent. But the majority of experiences we’ve all had in our educational careers are closer to what David describes than a true dialogue over issues and ideas. Read the rest of this entry »

Plug and Chug and Crank…

November 23, 2007

From middle schools through world-class MBA programs, the more we talk and listen, the more the plug and chug and crank approach to learning clearly doesn’t work.

That line is a famous quote from a professor I had years ago. He still teaches Physics, and is supremely dedicated to his job and his students. A great entertainer, I remember enjoying his lectures for his good-natured teasing, funny stories, and huge energy. But he was teaching engineers basic physics and the math that goes with it. Read the rest of this entry »

Student-centered rubric design

August 25, 2007

Here’s an example of a holistic writing rubric response for a student who has been judged as “approaching” (approaching what? Competence? Excellence? Bethlehem?)…

“Writer presents a wandering, vague, or unfocused controlling purpose or thesis. The paper moves awkwardly from a weak introduction to a conclusion that does not adequately represent the body of the paper. Basic paragraphing exists, but often fails to support or even recognize a central idea, and the use of evidence and examples is inadequate. Sentence and paragraph transitions are often unclear, awkward, indirect, and/or illogical. Tone and diction are often inconsistent and/or inappropriate for the subject and its implied audience. Mechanics (grammar, punctuation, spelling and documentation, if needed) are not well executed and may, at times, obscure meaning.”

Now, it could very well be that the author of this massive paragraph intends the rubric to be used for data gathering, not response to the student. Let’s assume that is the case (I won’t include a source for the above excerpt to avoid getting personal with a hard-working composition teacher somewhere).

At the other end of the spectrum, here’s a criteria from a rubric designed for a marketing class:

“Grammar, clarity of presentation, adequacy of content, attention to details, use of concepts
discussed in class:  …………….”

This criteria was worth 20 points (out of a 100)…so I’m assuming the prof wrote in “16” or similar on the dotted line, and that this rubric was designed expressly for communicating with a student.

First up, it’s great to see educators working hard at including writing in their courses and thinking about what is important in an assignment. The marketing example was included in the class syllabus, and therefore known to students in advance (something many writing teachers don’t do). Kudos on all those points.

Read the rest of this entry »