(This article was originally published at Learn and Teach Statistics and Operations Research, and syndicated at StatsBlogs.)
The most useful thing I learned in my teacher training at Auckland College of Education in 1985 was to write learning objectives. Not many years, and two babies later, I began lecturing at the University of Canterbury in Management Science/Operations Research. I was the only academic staff member to have formal teacher-training. My first task, when put in charge of MSCI210, Statistical Methods for Management, was to write learning objectives. This was revolutionary, but the idea infiltrated through other courses over the years.
A learning objective states specifically what a student should be able to do.
Here are some examples of good learning objectives:
Students will be able to:
- Identify different levels of data in new scenarios.
- Explain in context a confidence interval.
- Determine which probability distribution out of binomial, poisson or normal is most appropriate to model in an unfamiliar situation.
- Compare two time series models of the same data and evaluate which is more appropriate in a given context.
Learning objectives need to be specific and measurable.
Here are some things that people might think are learning objectives, but are not:
- Students will understand the central limit theorem. (The term “understand” is not measurable)
- Students will learn about probability trees (“learn” is not measurable, and does not specify the level. Do students need to be able to interpret or create probability trees?)
There are vast numbers of resources on learning objectives online.
Here is one I liked, with Bloom’s taxonomy of levels of learning. These are higher and lower levels of learning objectives, ranging from being able to state principles, through to synthesis and evaluation.
http://teachonline.asu.edu/2012/07/writing-measurable-learning-objectives/
And here are some useful verbs to use when writing learning objectives;
http://www.schreyerinstitute.psu.edu/pdf/SampleVerbs_for_LearningObjectives.pdf
It is not difficult to find material on developing learning objectives.
Not just learning objectives
A course is more than the set of its learning objectives. The learning objectives specify the skills, but there are also attitudes and knowledge to be considered. The starting point for course design is the attitudes. What do we want the students to feel about the topics? What changes do we wish them to contemplate in their thinking? Then the skills and knowledge are specified, often starting at a quite general level, then working down to specifics.
For example, we might wish to teach about confidence intervals. We need to determine whether students need to be able to calculate them, interpret them, estimate or derive them. We need to decide which confidence intervals we are interested in – for means alone, or proportions and slopes as well? Sometimes I find there are concepts I wish to include in the learning objectives, but they don’t really work as objectives. These I put as “important concepts and principles”.
I have put an example of learning objectives and concepts and principles at the end of this post.
Learning objectives tell students what is important
Without learning objectives it is difficult for students to know what they are supposed to be learning. In a lecture, a teacher can talk extensively about a case, but unless she states explicitly, it can be difficult for the students to know where to direct their attention. Do they need to know the details of that specific case or what principles are they supposed to glean from the example? Or was it just a “war-story” to entertain the troops? Students can waste a great deal of time studying things that are not necessary, to the detriment of their learning as a whole. The uncertainty also causes unnecessary anxiety.
Learning objectives enable good assessment development
Each year as we wrote our assessments we would go through the learning objectives and make sure they were assessed. This way the assessment was fair and applied to the course. If we found it difficult to write a question to assess a learning objective we would think again about the learning objective, and what it is we really want the students to be able to do. It made it easier to write fair, comprehensive assessments.
Learning objectives encourage reflection and good course design and development
As instructors write and review the learning objectives in a course, they can identify the level of learning that is specified in each. At an entry-level course, it is acceptable to have a number of lower level learning objectives. However, there needs to be some serious thinking done if a post-graduate course is not mainly made up of higher level learning objectives. I have seen tests in stage 2 and 3 papers that tested mainly recall and common-sense. It was evident that the instructor had not thought clearly about the level of learning that was expected.
Sometimes we find we are assessing things we have not specifically taught the students. The use of learning objectives, linked with assessment design, helps us to identify the background knowledge that we assume students have. One colleague was frustrated that the students did not seem able to apply the statistical results to a managerial context. However, nowhere had she specified that students would be required to do so, and nowhere had she actually taught students how to do this. She also assumed a level of understanding of business, that was probably not appropriate in undergraduate students.
Like it or not, assessment drives learning
I spoke recently to a maths advisor who informed me that teachers should be teaching to the curriculum not to the assessments. I felt he was idealistic, and told him so. My experience is that university students will learn what is assessed, and nothing else. I don’t know at what age this begins, but I suspect National Testing, the bane of good education, has lowered the age considerably. How wonderful it would be if our students learned for the sheer joy of learning! Where there are assessments looming, I fear this is unlikely.
When we write exams we are also writing learning materials for future students. One of the most common ways to prepare for an assessment is to do exercises from previous assessments. So when we feel that students were not really coming to grips with a concept, we include questions in the assessment, that can then be used by future students for review.
Information promotes equity and reduces unnecessary stress
The use of learning objectives can help reduce the “gaming” aspects that can proliferate in the absence of clear information. This is apparent at present in the world of Year 13 Statistics in New Zealand. The information regarding the external standards for 2013 is still sketchy (1 July 2013). The exams are written by external examiners and will take place in November of this year. However there is still only vague and sometimes incorrect information as to exactly what may or may not be included in the exams. Because of this, teachers are trying to detect, from what is or isn’t in the formula sheet and the (not totally correct) exemplars what might be in the finals, and what to include in the school practice exams. I suspect that some teachers or areas have more information than others. The way to make this fairer is to specify what is included in the material that may be included, as learning objectives. Let us hope that some clarity comes soon, for the sake of the teachers and the students.
Your learning
So what were the learning objectives for this post?
As a result of reading this post, readers will
- Reflect on their methods of course development and assessment with respect to using learning objectives.
- find further resources on the internet regarding learning objectives.
- Make comments on the good and bad aspects of this post! (oops – I didn’t teach that one)
Coda – Example of some learning objectives
Here is a set of learning objectives for the final section of a service course in quantitative methods for management. It is based on Excel and traditional (normal-based) methods of statistical analysis. They are far from perfect, including several ideas in many of them.
Evidence Section Learning Objectives
E1. Explain the process underlying hypothesis tests.
E2. Interpret a p-value in context for a given set of hypotheses.
E3. Formulate a null and alternative hypothesis in words for problems involving means, proportions, differences of two means and differences of two proportions.
E4. Use Excel to perform a hypothesis test on one or two means and interpret the results.
E5. Use Excel to perform a hypothesis test on one or two proportions and interpret the results.
E6. Use Excel and PivotTables to perform a Chi-sq test on table data, and interpret the results.
E7. Explain the concept of Type I and Type II errors and identify which (or neither) has occurred in a given situation.
E8. Use Excel to plot bi-variate data, find the correlation; interpret the output.
E9. Use Excel to fit a linear regression line; interpret the output.
E10. Evaluate the validity of statements about the nature of statistical thinking, including the concepts of causation, sample size, models, experimentation, statistical significance, effect size and subjectivity.
E11. Determine which test is most appropriate in a given situation, from: test for a mean or a proportion, difference between proportions, difference of two means: independent samples or paired data, chi-sq test for independence, regression.
Important concepts or principles
E12. Inferential statistics uses information collected in a sample to make predictions or judgements about the population from which the data is drawn.
E13. An effect is statistically significant when there is evidence from the sample to reject the null hypothesis.
E14. The p-value for a hypothesis test of a claim about a population parameter is the probability of getting, by chance, a sample as least as extreme as the observed one if the null hypothesis is true.
Please comment on the article here: Learn and Teach Statistics and Operations Research
