Should Standards and Assessments be Piloted?

A call for piloting standards and assessments has been raised by educators around the world who are faced with the problems inherent in moving to outcomes-based learning.  This from David B. Cohen in the US is representative of the sorts of things I’ve heard:

Tweet Let's pilot new standards and assessments

That verb to pilot has a couple of common meanings.  Its most common meaning is to lead or guide, typically in difficult conditions.  That definition doesn’t fit the context of the Tweet. I suspect many of  Cohen’s Twitter followers would say the standards and assessments are the difficult conditions.

The second meaning is probably closer to what Cohen has in mind, but even it is not a good match for the context. To pilot can mean to set a course and see that the vessel arrives at its destination.  That meaning does not suggest that there’s flaw in the vessel, only that it requires a skilled operator.  I doubt that the folks who are opposed to new standards and new assessments would be caught dead suggesting that better quality teachers would have no problem using them.

It seems to me that although Cohen uses pilot as a verb, he wants the word to be understood in its adjectival meaning of testing or experimental, as in the phrase “a pilot program to train monkeys to run cash registers.”

Even assuming Cohen wants a  limited pilot program to test standards and assessments, I still see a problem.

Standards just are

By themselves, the number of people who meet a particular standard doesn’t tell anything about whether a standard is good or bad.

If the carnival ride has a requirement, “people must be 48-inches high to ride the Cyclone,”  having a random sample of 1,000 people line up against the standard won’t tell whether they standard is good or bad.  The standard might be set too high for the ride to be profitable for the operator or too low to allow people to ride in relative safety, but those determinations cannot be made just on the basis of the percentage of people who meet the standard.

Educational standards are supposedly the sort everyone can meet, while standards for joining the Rockettes or the Navy Seals are intended to be those which only a few can meet.  Both types of standards can be inappropriate for a multitude of reasons.

One of the criticisms of the Common Core State Standards is that some of the grade-level standards are not appropriate to students’ developmental level at that grade. If true (and I think it is), that’s a serious problem.

It is not, however, a problem that’s like to be solved through small-scale experiments. The folks responsible for the overall standards will have to be convinced by seeing lots of data over a few years—with some assistance from experts in child and adolescent development—that the objective needs to be moved to a different grade level.

Assessments are testable

Unlike standards, assessments can and should be tested.  Assessments, however,  are evaluated in terms of how well they measure achievement of the standards.

To a considerable extent, assessments can be tested by small groups of the intended users to get rid of the least valid, least reliable assessments. Of course, if the standards were inappropriate to begin with, the assessments are going to be out of whack, too.

I have some sympathy for teachers who feel they are being forced to work with new standards and assessments without adequate preparation. I’m also willing to grant that first couple years of new standards and new assessments are going to be a pretty tough slog.

However, I believe teachers can work with (and around) new standards and assessments if they put their minds to it.

A workable approach

A District of Columbia ELA teacher who spoke at a webinar I attended recently told about how she implemented the Common Core in her classroom. She chose a few grade-specific standards that she thoroughly agreed with and worked all year teaching in-depth to achieve that standard.

When the standardized test showed her students didn’t do well on those standards, she said that test was not a valid assessment of students’ understanding of those topics.  She knew her students knew that material well.

She did the same things the next year that she’d done the first year.

The second year her students did very well: Between the first and second years, the standardized test was changed so the test items aligned better with the standards.

I suspect that  teachers will find that if they work consistently through the year toward a few of the standards they feel comfortable working with their students to achieve,  do their own assessments to show students’ learning, and not change their teaching to align with a poor test, they’ll be successful, too.

Set annual objectives in concrete

Teachers and schools need not only goals, but annual objectives. Annual objectives restrict the amount that must be taught while increasing the ways and number of times it can be taught.

It’s much easier to get all students to meet an objective if they’ve had multiple times to learn the content and skills it requires than if everyone is expected to be ready and able to master that content in a unit taught between Oct. 10 and Oct 19.

Objectives are valuable only if they are unchangeable as the laws of the Medes and the Persians. In his book Good to Great: Why Some Companies Make the Lead … and Others Don’t, Jim Collins quotes Amgen co-founder George Rathmann on the subject of annual objectives.

When you set your objectives for the year, you record them in concrete. You can change your plans through the year, but you never change what you measure yourself against. You are rigorous at the end of the year, adhering exactly to what you said was going to happen. You don’t get a chance to editorialize. You don’t get a change to adjust and finagle, and decide that you didn’t intend to do that anyway, and readjust your objectives to make yourself look better. You never just focus on what you’ve accomplished for the year; you focus on what you’ve accomplished relative to exactly what you said you were going to accomplish—no matter how tough the measure. (122)

Rathmann’s observations are as applicable to education as they are to business.

Collins, Jim. Good to Great: Why Some Companies Makes the Leap … and Others Don’t. New York: Harper Business, 2001. Print.

Teachers should set teaching content

Teachers wail that the Common Core State Standards don’t tell them what to teach.

And they are right.

The Common Core State Standards are educational objectives: they specify what students have to be able to do with knowledge, not what specific facts they need.

The business of deciding what information students need to know to get to the objective is a job for professionals.

That means it’s a job for teachers.

Teachers need to figure out the minimum information they must teach to enable students to do higher level learning. The authors of A Taxonomy for Learning, Teaching, and Assessing (the revised Bloom’s taxonomy) stress that students can learn complex processes without first memorizing the terminology experts use to discuss those processes.

It’s perfectly fine for students to talk about this thing and that thing until they learn how this thing and that thing work together to make the rocket go up.

If Josh and Caitlin want to go into STEM fields and build rockets, they’ll have to learn the terms that people in STEM fields use for this thing and that thing. But they do not need to memorize those terms in elementary school to enable them to learn what makes a rocket soar.

Don’t leave the work of deciding what information students need to know to bureaucrats.

It’s a job for teaching professionals.

Are you professional enough to do it?

[Link to material no longer available removed 04-03-2014.]

From lesson plans to educational objectives

Educational objectives are giving teachers fits. Faced with a set of annual objectives, such as the Common Core State Standards, teachers don’t know what to do. They are comfortable with lesson plans. They can handle unit studies. But the idea of working an entire year toward an objective boggles their minds.

I have five suggestions that may help.

1. Get a copy of the educational standards.

Don’t expect to learn how to implement educational standards from a 90-minute webinar. Don’t wait for your district to give a workshop. If you found this blog, you can find the standards you need.

2. Read the standards carefully.

Don’t read what bloggers think; read the documents yourself. You’ll have to teach students to find and use primary documents to comply with Common Core. You might as well start now.

3. Identify lesson plans that lead toward that objective.

Identify lesson plans in your repertoire that will work within the educational standards you must use. I’m sure you teach some lessons that are geared toward aspects of major educational standards. The basic skill students use every daily across the curriculum are mentioned in educational standards. From reading and writing to technology use,  educational standards include them all.

4. Identify other ways to teach toward those objectives.

You know how you always gripe that you don’t have time to get everyone up to speed by the end of the lesson or end of the unit? You’re right. That’s why educational standards let you  teach multiple ways, multiple times to get students up to speed on the most important skills, the cognitively demanding ones.

Instead of preparing a lesson plan or a two-week unit on persuasive writing, think about teaching persuasive over 36 weeks. Better yet, think about teaching persuasive writing as a tool that students will use to learn some other course content rather than as an end in itself.

5. Decide how to meet educational objectives and your educational goals.

If the educational objective is that students read and write nonfiction competently but your goal is to have students learn to value literature, figure out some way to combine the two. Remember, people don’t need to write plays to enjoy a performance of Shakespeare. Even a movie critic has to learn to write nonfiction.

Thinking in terms of educational objectives means getting beyond lesson plans and unit plans. To meet educational objectives, you need to finish the job.

7 things you never knew about Bloom’s taxonomy

Benjamin S. Bloom, commonly referred to as the author of  “Bloom’s Taxonomy,” called it one of the most cited and least read books on education.

The full title of the 1956 book is A Taxonomy of Educational Objectives: The Classification of Educational Goals, Handbook 1, The Cognitive Domain.  Bloom was not the author. That honor goes to a “Committee of College and University Examiners.”

Bloom was the taxonomy’s editor. He was assisted by four other men, including David R. Krathwohl, who became one of the two lead editors of the 2001 revision of the book.

Here are 7 other things you probably don’t know about Bloom’s taxonomy.

1. The authors’ taxonomy consists of six classes of educational outcomes, which are organized in hierarchical fashion:

1.0 Knowledge
2.0 Comprehension
3.0 Application
4.0 Analysis
5.0 Synthesis
6.0 Evaluation

The higher the number, the more difficult the educational task. That’s why someone may be able to recall facts, concepts, and procedures but be unable to apply them in a new situation.

2. The famous Bloom’s taxonomy pyramid does not appear in the book.

The only triangular shapes in the book are in the math problems in the illustrative materials in part 2 of the book.

3. Stress on lower level learning came from teachers.

The authors wanted to craft definitions of terms used in the objectives so that a researcher in Maine and a kindergarten teacher in Monterey would mean the same thing when, for example, they used the term comprehend.

The team that prepared the taxonomy began by collecting samples of objectives from their own institutions and the literature.  When they did that analysis,  they found many more objectives about the lowest level of learning—knowledge—than about higher ones. The authors of the taxonomy write,  “Because of the simplicity of teaching and evaluating knowledge, it is frequently emphasized as an educational objective out of all proportion to its usefulness or its relevance for the development of the individual” (p. 34).

4. Consciousness and cognition go together.

The higher the educational objective in the taxonomy, the more likely learners are to be conscious of the cognitive processes they are using. In other words, the tougher the educational task, the more likely students are to pay attention to what they are doing.  (The authors hypothesize that this fact may be the reason that highly complex tasks may become automatic responses.)

5. Learning difficulty is learner dependent.

The authors acknowledge that the taxonomic level of any objective depends on the learner. A problem that is difficult for students to solve the first time they encounter that particular type of problem because they have to work, say, at the analysis level, may be easy the next time because the second time they need only recall what they did before.

6. Application is the focus of most education.

The authors of Bloom’s taxonomy say most of what is taught in schools is intended for transfer to real life. “The effectiveness of a large part of the school program is therefore dependent upon how well the students carry over into situations applications which the students never faced in the learning process”  (p. 122) [italics added]. They continue:

The general consensus seems to be that training will transfer to new areas most readily if the person is taught in such a way that he learns good methods of attacking problems, if he learns concepts and generalizations (rather than how to use certain facts in specific instances), if he  learns proper attitudes toward work, and if he develops proper attitudes of self-confidence and control. It is obvious that the objectives in the application category, as they embody the meaning of transfer of training, are extremely important aspects of the curriculum. Further, the evaluation of the extent to which the application outcomes are being achieved becomes one of the most important aspects of the entire evaluation process.

7. Teachers must choose essential knowledge.

Since there is a vast universe of discrete facts teachers could require students to learn, teachers must select those bits of information are essential for students to master at a given point in their academic careers.  (E.g. Does the student need to know this now, or is it enough that he knows the information exists and can be found when he does need to know it?)  The authors of Bloom’s taxonomy say teachers must:

  • Distinguish between student-level knowledge (or terminology, for example) and expert-level knowledge.
  • Determine the degree of precision to require of the student.
  • Decide how to organize the information to facilitate learning.
  • Distinguish between immediate and future needs for information  (pp. 36-37).

The authors make clear that the amount of instructional effort expended on knowledge undoubtedly exceeds its importance in learning for application to non-instructional settings.

[Broken links removed 04-03-2014]