1999 Project Team Report
The English Study
Design in Detail
A Guide to Planning
Cedric Hall: Report on the 1998
year 12 trial
Terry Locke: "Assessment
Standards: What's in a Name?"
The 1999 Reference
Update and Forum
Update and Forum
This site has been set up in connection with the English Study
Design project for the purposes of making available information
related to the Achievement 2001 project, especially as
it relates to the subject of English and encouraging comment
on aspects of the development work in progress. Viewers wishing
to comment on any aspect of the information posted can email
Terry Locke (firstname.lastname@example.org) and your comments will be
posted in an appropriate place in a box.
To obtain Ministry of Education information on the NCEA click
Be prepared to patient, you may have a long wait for the
page to come up. This site will provide you with the official
version of the NCEA and includes policy information, FAQ, communications
and information about the professional development programme.
You will also find Achievement standards and assessment resources
Most of the files will require Adobe Acrobat Version 4 to
read and download them.
Click on NZQA
for their NCEA site. It's early days in its development, but
the idea is to include implementation updates, sample external
assessments, FAQs and timelines.
The project management team is headed by:
Elizabeth Eppel (Ministry of Education)
Liz Seymour (NZQA)
Joint Advisory Group (JAG)
Qualifications Development Group
Tim McMahon (Project Manager)
Paul Ackerley (facilitator)
Catherine Carter (facilitator)
Rawiri Gibson (facilitator)
Geoff Gibbs (facilitator)
Linda Glogau (facilitator)
Barbara Purvis (facilitator)
Lynette Chappell (support staff)
Bronwyn Stewart (support staff)
Joshua Williams (support staff)
Vicky Woodgate (support staff)
Te Karearea is a cross-agency group called together
to provide Maori-specific input into the work of the QDG.
Some work to date:
- The Ministry has processed more than 800 nominations for
subject expert panels (and the English panel has been appointed
- A cross-disciplinary group met in March, 1999, to debate
proposals for the general structure of the achievement standards.
A preferred common template was developed (see below) and
it is intended that this group will test its preferred achievement
standard template by drafting standards in a range of "subject"
areas. It will write assessment activities and marking schedules
to check the workability of the chosen model.
- The English experts panel has already met and has begun
work on an outline matrix of achievement standards for levels
1 to 3/4 and internal and external achievement standards for
- The QDG has given some attention to the question as to which
subjects should have achievement standards developed for them.
No decision has been made on the (approximately 50) subjects
that might potentially qualify.
Back to page menu
This section of the page looks at the terms of reference the
Ministry established (via the QDG and its cross-disciplinary
group) for the various experts panels. The information is based
on overheads that were presented to the English panel in its
April, 1999, meeting.
- Useability: The standards will be useless if they
don't convey any information about the people who have achieved
them, and if the various stakeholders can't interpret them
for their purposes
- Manageability: Assessors must be able to determine
and record students' success with minimum negative impact
on learning programmes.
- Credibility: All stakeholders must be able to be
convinced that the standards are reliable and valid no matter
who has met them or by whom or how they were assessed.
- Flexibility: Structure of standards must not limit
student's access to the qualification that they wish to
2. What we've learned...
from the development and implementation of unit standards
is that there is a need to
- limit the number of achievement standards to be
developed, and the degree of specificity of each, given
the impact on manageability and workload.
- ensure the equivalence of levels and comparability
of credit values distributed across subject/learning areas.
- recognise meritorious and excellent performance.
from previous investigations possible designs of achievement
standards suggest the need to address:
- the implications for reliable and consistent assessment
resulting from the degree of specificity chosen for the
- how much (if any) flexibility there can be in the design
of standards, within and between "subjects", without
creating any (real or perceived) "hard path" /
"easy path" distinction
- how much sampling is necessary for sufficient evidence
to be gathered to reliably conclude that a student has met
3. QDG proposes...
- The achieving of a standard should require the same rigour
of evidence whether the standard is to be externally or
internally assessed, so standards will need a commonality
of format that conveys this intention.
- both external and internal standards will indicate the
minimum amount of evidence that will suffice as confirmation
of the student's having achieved the standard.
Standards will be relatively broad
- achievement standards should have no more than 3 levels
of description (NB trade-off: More detail in standard
vs. more help with interpretation).
- to develop a generic definition of what will be required
for a student to be awarded merit or excellence in such
a way that it can be reliably interpreted by expert panels
in relation to each subject/standard.
It's vital to get early agreement on the components of
this generic structure.
Subject panels have been told to attempt to come up
with a 50/50 split between internal and external forms
of assessment. In a letter to me (25 May 1999), Tim McMahon
writes: 'The decision that standards be either externally
or internally assessed, and that at least half
of the credits from achievement standards be externally
assessable, has already been taken by government. the
requirement for external assessment is forced on us by
the need for public credibility for the qualification.
With such a constraint, it would appear that the English
Study Design's plan to internally mark Close Reading and
Writing at Year 12 and to have the grades moderated by
a Reference Test is out of court as far as the government
is concerned. (See an ESD-based
achievement standard.) Despite this, I'd want to continue
argument for an external/internal mix for such achievement
standards as a writing folio. On the face of it, it is
an easy matter to suggest that a writing folio be externally
marked (though it would be costly). However, there is
a serious pedagogical issue at stake here.
The ESD borrowed some of its thinking from the 1989
version of the English Study Design in Victoria. For a
number of reasons, the VCE in Victoria has undergone a
process of transformation and restructuring since 1990.
One of the issues which pertains particularly to writing
in English is authentication. One of the problems with
writing folios is that there is no way of effectively
ensuring a consistency of teacher input into the final
product. Even if one attempts to control input from outside
the school by, for example, keeping work in process under
lock and key in the classroom until it is sent off for
marking, it does not deal with the problem of teacher
The solution which the Victorians have recently adopted
to the problem of authenticity has been to adopt a strategy
of short pieces of writing in classroom conditions without
teacher input. The folio has been dispensed with. Frankly,
I believe that this solution is a classic instance of
throwing the baby out with the bath water. All the research
into writing that I know about stresses the value of teacher
input into the drafting process (a sort of master/apprentice
model). That's why I think that the writing folio is an
excellent idea because of its pedagogical implications.
So, how to deal with the problem of "authenticity"
I would argue that the Reference Test which the ESD
Project Team are testing this year as part of the Year
12 English Study Design project continuation is precisely
the kind of moderation instrument that solves this problem.
(And yes, we have indicated to the Ministry that they
are welcome to purchase the outputs of this research at
a rate that will be too good to turn down!) The year 12
Reference test moderates both Response to Text and Writing
at Year 12. That is, a piece of writing each year 12 student
does at the end of Term 3 as part of an externally set
and marked Reference Test contributes towards a range
of grades which is used to moderate the grades achieved
by the school's year 12 students in their internally assessed
writing folios. With such a set-up, the amount of teacher
input into student writing at school level is irrelevant,
since in the Reference Test, the student is "on their
I'd argue that such a moderation device for a crucial
component of any English course is:
- Cheaper than having writing folios marked externally
- Solves the problem of authentication (which remains
for externally assessed writing folios)
- And preserves the best model we know for teaching
Terry Locke (June 3, 1999)
Expert panels will analyse curriculum documents, existing
prescriptions, current unit standards and course statements
- determine the key outcomes to be recognised in qualifications
- locate and eliminate overlaps that address the same curriculum
- decide which outcomes are better suited to written external
assessment and which to internal assessment
- recast the key outcomes identified in existing prescriptions
and unit standards into the agreed achievement standards
format [In recasting the
standards it will be important to recognise where teachers are currently
"at". The changes will need to be implemented as evolutionary
changes from current practice: not cataclysmic upheaval]
- recommend criteria for the award of merit or excellence
and build these into the standards description.
'The relationship with ENGLISH in the New Zealand
Curriculum (ENZC) is problematical. The following excerpts
are from an exchange of letters between myself and Tim
"Tim McMahon wrote (25 May, 1999): 'In your
notes on the draft material you have commented on the
pedagocial flaws in unit standards and your dislike
of aspects of English in the New Zealand Curriculum.
Suffice to say that dissatisfaction with the unit standards
development was a major policy driver for this development.
To the extent possible, within the notion that this
will be a standards-based system by Cabinet decree,
we will ensure those flaws are not reintroduced. Cabinet
specifically requested that the qualifications be brought
closer to the New Zealand Curriculum to minimise confusion
for students and teachers. I do not have the licence
to alter English in the New Zealand Curriculum in
this development. Nor do I believe, in any case, that
qualifications assessment requirements should dictate
learning outcome requirements of public education policy.
There will be an opportunity for the community to re-visit
aspects of its English curriculum is reviewed. In the
meantime we have an opportunity to refine the qualifications
system that is based on the existing curriculum.'
"I replied (31 May, 1999): While everyone
acknowledges that ENZC is flawed and urgently needs
reviewing, I accept that the panel does not have a "licence
to alter English in the New Zealand Curriculum
in this development." We're agreed here. I'm simply
arguing that it is possible to come up with a matrix
that is true to the intent of ENZC that does not mimic
its errors. The panel, for example, has already decided
to view the "Processing" strands as underpinning
teaching rather than generating their own specific achievement
standards. That seems sensible to me (and was a decision
the developers of the English Study Design also made.)
Deciding not to use the terms "transactional"
and "poetic" while preserving the curriculum's
emphasis on a range of texts would be a similar type
of step. Finally, recognising that the curriculum has
it wrong in not recognising that for English the broad
outcomes are pretty much the same for all ages and stages
and shaping one's matrix accordingly (see An
ESD-based achievement standard matrix ) is hardly
an act of infidelity. It is simply acknowledging that
for assessment purposes and, specifically, for discriminating
between broad learning outcomes for years 11 to 13,
the curriculum document is unsatisfactory."
Terry Locke (3 June, 1999)
5. The policy drivers
- Sector dissatisfaction with aspects of unit standards
- micro definition of outcomes
(i) assessment and recording workloads
(ii) applicability to conceptual learning
- lack of recognition for partial achievement or superior
- Workload and manageability for teachers/assessors
- Public disquiet about perceived lack of credibility
- NQF white paper decisions
6. Crucial understandings
- A qualification is a "ticket" for an individual,
it is not a course of study
- Assessment for qualifications is undertaken, on behalf
- not to certify attendance
- not merely to recognise what an individual
knows and can do but
- to certify that the person knows and/or can
achieve or perform at or above the standards required
by the qualification
- Its purpose is distinctly different from "assessment
for better learning"
7. A National Certificate
- Needs public confidence in assessor comparability and
reliability for credibility
- External assessment and moderation of internal assessment
- To ensure this credibility NCEA will be awarded
only if some credit is gained from external assessment
and some by internal assessment
- This means every NCEA will contain at least some
credit from achievement standards
8. Reality Check
- No increase in student fees
- Revenue constraint
- Cost constraint
- Possible limitation on the range of subjects for which
external assessment and high quality moderation can be
- Reality of administering external assessment:
- entry taking
"It would appear that part of the helter-skelter
pace being set by this process is being driven by fiscal
constraints. Having squandered huge sums of money on unit
standards development and trials, there appears to be
little money left in the pot. Hence:
- Examination options are being favoured at every
turn simply on the basis that (at least superficially)
they offer the cheapest moderation option. The July
questionnaire has the following extraordinary statement
that 'The only reason for considering internal assessment
should be validity; i.e. what is to be assessed cannot
actually be observed or measured in an external assessment.'
- There appears to be no interest in research, trialling
or even purchasing existing research outputs.
- The huge issue of the impact of the proposed assessment
regimes on actual classroom practice does not even seem
to be being asked.
- There are suggestions that subjects will need to
have a certain number of students enrolled in them before
they will be deemed 'conventional subjects' and have
achievement standards developed for them."
Terry Locke (August 3, 1999)
Back to page menu
THE ENGLISH EXPERTS
||Hagley Community College
||Auckland College of Education
||Whangarei Girls High
||St Andrews College
||Upper Hutt College
||Rotorua Boys High School
||Mt Roskill Grammar
"It's hard not to see this panel as having a
strong unit standards orientation, with only two out of
twelve members representing the strong body of opinion
which resisted the imposition of unit standards. There
are only two South Island schools represented, and these
two are atypical institutions which were strong in their
advocacy of unit standards. No English Study Design trial
school is represented, even though at least one of its
English HODs applied for the panel. Yet the English Study
Design is, we would argue, the only standards-based assessment
option currently in use that is not rooted in a
unit-standards-type, competence-based orientation. Moreover,
it is being constantly monitored and independently evaluated."
Terry Locke 1/6/99
Back to page menu
There are a number of wider issues.
1. Other Providers
The shape and structure of the broader National Certificates
of Educational Achievement. The Ministry's "Progress
Report #1" (undated) states that "In essence, the
externally and internally assessed achievement standards in
conventional school subjects will provide one route [our
italics] for students to gain NCEA credits. Credit sill
probably also be able to be gained by students achieving in
ITO-developed and other unit standards as well as examinations
or standards registered by other organisations (such as the
The relevant OHT presented to the English experts panel is
GAINING A NATIONAL CERTIFICATE
NCEA is "owned" and regulated by Government.
Individual standards (subjects) need not be developed or
operated by Government
In a letter to Tim McMahon (20 May, 1999), I asked:
'I would be interested in information which could tell
me how the English Study Design can be recognised as awarding
standards which will allow students in our [trial] schools to gain credit on the NCEA. (What
are the steps? Whom do we approach?)
"Tim McMahon wrote (25 May, 1999): 'With regard
to alternative qualifications there are two possible
answers. Under the proposals in the National Qualifications
policy developments (see the Green paper) any qualifications
owner will be able to seek to have their qualification
registered on the National Qualifications Framework.
So it is with the International Baccalaureate. Further,
however, we are exploring the idea that if a student
pursues some other qualification that is registered
on the NQF that is obviously curriculum related, it
should provide credit towards a student's National Certificate
of Education Achievement.'
"'In the latter case however, you will appreciate
that there would need to be regulation to prevent a
student's gaining credit for the same learning outcomes
through more than one route. While we are working out
the way such a regulation should be enforced in practice,
if there are standards registered for the NCEA that
are broadly accepted by the community and the sector,
the Ministry would not want any other standard, examination,
or qualification that addressed the same outcomes to
"I replied (31 May, 1999): 'I'm confused. Sorry.
What I think you're saying is that, say, a qualification
such as International Baccalaureate (in English, or
Theatre Arts) could be registered on the framework.
However, if this qualification appeared to be made up
of sets of achievement standards that resemble the ones
that you are developing via your subject expert panels,
then there will be regulation to prevent students being
awarded these achievement standards via these alternative
"Despite my confusion, I would like to know
who we should approach in the Ministry (and how) in
order to have the English Study Design registered as
a qualification on the NQF. I mean, we can hardly be
accused of imitating achievement standards in English
that have yet to be developed could we?'"
Terry Locke (3 June, 1999)
The QDG and its related policy groups is considering two
- Level 4 standards to be developed to complement each level
3 achievement standard.
- Inviting other institutions or agencies with examinations
experience to develop appropriate standards and to examine
candidates against them.
Back to page menu
Back to English Study Design index