English Study Design


Home

Project Team

1999 Project Team Report

The English Study Design in Detail

 A Guide to Planning and Assessment

Cedric Hall: Report on the 1998 year 12 trial

 Terry Locke: "Assessment Standards: What's in a Name?"

 The 1999 Reference Test Trial

 Achievement 2001: Update and Forum

Contact Us

 

Achievement 2001:
Update and Forum

This site has been set up in connection with the English Study Design project for the purposes of making available information related to the Achievement 2001 project, especially as it relates to the subject of English and encouraging comment on aspects of the development work in progress. Viewers wishing to comment on any aspect of the information posted can email Terry Locke (t.locke@waikato.ac.nz) and your comments will be posted in an appropriate place in a box.

 Background information

 Broad principles

Where did NCEA come from?

 The English experts panel

 Achievement standard matrices

An ESD-based achievement standard matrix 

 The NCEA and achievement standards

 An ESD-based achievement standard

The July 1999 Questionnaire

 Wider issues: Other providers, scholarships

The Ministry Principal/PPTA Forums

  Critical Comment on the NCEA

BACKGROUND INFORMATION

To obtain Ministry of Education information on the NCEA click on Ministry. Be prepared to patient, you may have a long wait for the page to come up. This site will provide you with the official version of the NCEA and includes policy information, FAQ, communications and information about the professional development programme. You will also find Achievement standards and assessment resources for English. Most of the files will require Adobe Acrobat Version 4 to read and download them.

Click on NZQA for their NCEA site. It's early days in its development, but the idea is to include implementation updates, sample external assessments, FAQs and timelines.

The project management team is headed by:
Elizabeth Eppel (Ministry of Education)
Liz Seymour (NZQA)

Joint Advisory Group (JAG)
Graeme Aitken
Margaret Bendall
Marilyn Davies
Martin Eadie
Graeme Mccann
Timoti Maru
John Taylor

Qualifications Development Group
Tim McMahon (Project Manager)
Paul Ackerley (facilitator)
Catherine Carter (facilitator)
Rawiri Gibson (facilitator)
Geoff Gibbs (facilitator)
Linda Glogau (facilitator)
Barbara Purvis (facilitator)
Lynette Chappell (support staff)
Bronwyn Stewart (support staff)
Joshua Williams (support staff)
Vicky Woodgate (support staff)

Te Karearea is a cross-agency group called together to provide Maori-specific input into the work of the QDG.

Some work to date:

  • The Ministry has processed more than 800 nominations for subject expert panels (and the English panel has been appointed (see below).
  • A cross-disciplinary group met in March, 1999, to debate proposals for the general structure of the achievement standards. A preferred common template was developed (see below) and it is intended that this group will test its preferred achievement standard template by drafting standards in a range of "subject" areas. It will write assessment activities and marking schedules to check the workability of the chosen model.
  • The English experts panel has already met and has begun work on an outline matrix of achievement standards for levels 1 to 3/4 and internal and external achievement standards for level 1.
  • The QDG has given some attention to the question as to which subjects should have achievement standards developed for them. No decision has been made on the (approximately 50) subjects that might potentially qualify.

Back to page menu

 

BROAD PRINCIPLES

This section of the page looks at the terms of reference the Ministry established (via the QDG and its cross-disciplinary group) for the various experts panels. The information is based on overheads that were presented to the English panel in its April, 1999, meeting.

1. Principles

    • Useability: The standards will be useless if they don't convey any information about the people who have achieved them, and if the various stakeholders can't interpret them for their purposes
    • Manageability: Assessors must be able to determine and record students' success with minimum negative impact on learning programmes.
    • Credibility: All stakeholders must be able to be convinced that the standards are reliable and valid no matter who has met them or by whom or how they were assessed.
    • Flexibility: Structure of standards must not limit student's access to the qualification that they wish to gain.

2. What we've learned...

from the development and implementation of unit standards is that there is a need to

    • limit the number of achievement standards to be developed, and the degree of specificity of each, given the impact on manageability and workload.
    • ensure the equivalence of levels and comparability of credit values distributed across subject/learning areas.
    • recognise meritorious and excellent performance.

from previous investigations possible designs of achievement standards suggest the need to address:

    • the implications for reliable and consistent assessment resulting from the degree of specificity chosen for the standards
    • how much (if any) flexibility there can be in the design of standards, within and between "subjects", without creating any (real or perceived) "hard path" / "easy path" distinction
    • how much sampling is necessary for sufficient evidence to be gathered to reliably conclude that a student has met a standard

3. QDG proposes...

    • The achieving of a standard should require the same rigour of evidence whether the standard is to be externally or internally assessed, so standards will need a commonality of format that conveys this intention.
    • both external and internal standards will indicate the minimum amount of evidence that will suffice as confirmation of the student's having achieved the standard.

Standards will be relatively broad

    • achievement standards should have no more than 3 levels of description (NB trade-off: More detail in standard vs. more help with interpretation).
    • to develop a generic definition of what will be required for a student to be awarded merit or excellence in such a way that it can be reliably interpreted by expert panels in relation to each subject/standard.

It's vital to get early agreement on the components of this generic structure.

Comment

Subject panels have been told to attempt to come up with a 50/50 split between internal and external forms of assessment. In a letter to me (25 May 1999), Tim McMahon writes: 'The decision that standards be either externally or internally assessed, and that at least half of the credits from achievement standards be externally assessable, has already been taken by government. the requirement for external assessment is forced on us by the need for public credibility for the qualification.

With such a constraint, it would appear that the English Study Design's plan to internally mark Close Reading and Writing at Year 12 and to have the grades moderated by a Reference Test is out of court as far as the government is concerned. (See an ESD-based achievement standard.) Despite this, I'd want to continue argument for an external/internal mix for such achievement standards as a writing folio. On the face of it, it is an easy matter to suggest that a writing folio be externally marked (though it would be costly). However, there is a serious pedagogical issue at stake here.

The ESD borrowed some of its thinking from the 1989 version of the English Study Design in Victoria. For a number of reasons, the VCE in Victoria has undergone a process of transformation and restructuring since 1990. One of the issues which pertains particularly to writing in English is authentication. One of the problems with writing folios is that there is no way of effectively ensuring a consistency of teacher input into the final product. Even if one attempts to control input from outside the school by, for example, keeping work in process under lock and key in the classroom until it is sent off for marking, it does not deal with the problem of teacher input.

The solution which the Victorians have recently adopted to the problem of authenticity has been to adopt a strategy of short pieces of writing in classroom conditions without teacher input. The folio has been dispensed with. Frankly, I believe that this solution is a classic instance of throwing the baby out with the bath water. All the research into writing that I know about stresses the value of teacher input into the drafting process (a sort of master/apprentice model). That's why I think that the writing folio is an excellent idea because of its pedagogical implications. So, how to deal with the problem of "authenticity" and equity?

I would argue that the Reference Test which the ESD Project Team are testing this year as part of the Year 12 English Study Design project continuation is precisely the kind of moderation instrument that solves this problem. (And yes, we have indicated to the Ministry that they are welcome to purchase the outputs of this research at a rate that will be too good to turn down!) The year 12 Reference test moderates both Response to Text and Writing at Year 12. That is, a piece of writing each year 12 student does at the end of Term 3 as part of an externally set and marked Reference Test contributes towards a range of grades which is used to moderate the grades achieved by the school's year 12 students in their internally assessed writing folios. With such a set-up, the amount of teacher input into student writing at school level is irrelevant, since in the Reference Test, the student is "on their own".

I'd argue that such a moderation device for a crucial component of any English course is:

  • Cheaper than having writing folios marked externally
  • Solves the problem of authentication (which remains for externally assessed writing folios)
  • And preserves the best model we know for teaching writing.

Terry Locke (June 3, 1999)

4. Methodology

Expert panels will analyse curriculum documents, existing prescriptions, current unit standards and course statements to:

    • determine the key outcomes to be recognised in qualifications
    • locate and eliminate overlaps that address the same curriculum objectives
    • decide which outcomes are better suited to written external assessment and which to internal assessment
    • recast the key outcomes identified in existing prescriptions and unit standards into the agreed achievement standards format [In recasting the standards it will be important to recognise where teachers are currently "at". The changes will need to be implemented as evolutionary changes from current practice: not cataclysmic upheaval]
    • recommend criteria for the award of merit or excellence and build these into the standards description.

Comment

'The relationship with ENGLISH in the New Zealand Curriculum (ENZC) is problematical. The following excerpts are from an exchange of letters between myself and Tim McMahon.

"Tim McMahon wrote (25 May, 1999): 'In your notes on the draft material you have commented on the pedagocial flaws in unit standards and your dislike of aspects of English in the New Zealand Curriculum. Suffice to say that dissatisfaction with the unit standards development was a major policy driver for this development. To the extent possible, within the notion that this will be a standards-based system by Cabinet decree, we will ensure those flaws are not reintroduced. Cabinet specifically requested that the qualifications be brought closer to the New Zealand Curriculum to minimise confusion for students and teachers. I do not have the licence to alter English in the New Zealand Curriculum in this development. Nor do I believe, in any case, that qualifications assessment requirements should dictate learning outcome requirements of public education policy. There will be an opportunity for the community to re-visit aspects of its English curriculum is reviewed. In the meantime we have an opportunity to refine the qualifications system that is based on the existing curriculum.'

"I replied (31 May, 1999): While everyone acknowledges that ENZC is flawed and urgently needs reviewing, I accept that the panel does not have a "licence to alter English in the New Zealand Curriculum in this development." We're agreed here. I'm simply arguing that it is possible to come up with a matrix that is true to the intent of ENZC that does not mimic its errors. The panel, for example, has already decided to view the "Processing" strands as underpinning teaching rather than generating their own specific achievement standards. That seems sensible to me (and was a decision the developers of the English Study Design also made.) Deciding not to use the terms "transactional" and "poetic" while preserving the curriculum's emphasis on a range of texts would be a similar type of step. Finally, recognising that the curriculum has it wrong in not recognising that for English the broad outcomes are pretty much the same for all ages and stages and shaping one's matrix accordingly (see An ESD-based achievement standard matrix ) is hardly an act of infidelity. It is simply acknowledging that for assessment purposes and, specifically, for discriminating between broad learning outcomes for years 11 to 13, the curriculum document is unsatisfactory."

Terry Locke (3 June, 1999)

5. The policy drivers

    • Sector dissatisfaction with aspects of unit standards
      • micro definition of outcomes
        (i) assessment and recording workloads
        (ii) applicability to conceptual learning
      • lack of recognition for partial achievement or superior performance
    • Workload and manageability for teachers/assessors
    • Public disquiet about perceived lack of credibility
    • NQF white paper decisions

6. Crucial understandings

    • A qualification is a "ticket" for an individual, it is not a course of study
    • Assessment for qualifications is undertaken, on behalf of society
      • not to certify attendance
      • not merely to recognise what an individual knows and can do but
      • to certify that the person knows and/or can achieve or perform at or above the standards required by the qualification
    • Its purpose is distinctly different from "assessment for better learning"

7. A National Certificate

    • Needs public confidence in assessor comparability and reliability for credibility
  • External assessment and moderation of internal assessment
      • To ensure this credibility NCEA will be awarded only if some credit is gained from external assessment and some by internal assessment
    • This means every NCEA will contain at least some credit from achievement standards

8. Reality Check

    • No increase in student fees
  • Revenue constraint
  • Cost constraint
  • Possible limitation on the range of subjects for which external assessment and high quality moderation can be mounted
      • Reality of administering external assessment:
        • entry taking
        • scheduling
        • reporting

Comment

"It would appear that part of the helter-skelter pace being set by this process is being driven by fiscal constraints. Having squandered huge sums of money on unit standards development and trials, there appears to be little money left in the pot. Hence:

  • Examination options are being favoured at every turn simply on the basis that (at least superficially) they offer the cheapest moderation option. The July questionnaire has the following extraordinary statement that 'The only reason for considering internal assessment should be validity; i.e. what is to be assessed cannot actually be observed or measured in an external assessment.'
  • There appears to be no interest in research, trialling or even purchasing existing research outputs.
  • The huge issue of the impact of the proposed assessment regimes on actual classroom practice does not even seem to be being asked.
  • There are suggestions that subjects will need to have a certain number of students enrolled in them before they will be deemed 'conventional subjects' and have achievement standards developed for them."

Terry Locke (August 3, 1999)

Back to page menu

 

THE ENGLISH EXPERTS PANEL

Member

Institution

Margaret Adeane Wellington College
Phil Coogan Unitec
Mike Fowler Hagley Community College
Barry Gough Avondale College
Ngaire Hoben Auckland University
Judine Ladbrook Auckland College of Education
Mike McMenamin Whangarei Girls High
Vicki McLennan St Andrews College
Nicola Meek Upper Hutt College
Merle Ramsay Rotorua Boys High School
John Wilkinson Mt Roskill Grammar
Chris Williams Aorere College

Comment

"It's hard not to see this panel as having a strong unit standards orientation, with only two out of twelve members representing the strong body of opinion which resisted the imposition of unit standards. There are only two South Island schools represented, and these two are atypical institutions which were strong in their advocacy of unit standards. No English Study Design trial school is represented, even though at least one of its English HODs applied for the panel. Yet the English Study Design is, we would argue, the only standards-based assessment option currently in use that is not rooted in a unit-standards-type, competence-based orientation. Moreover, it is being constantly monitored and independently evaluated." Terry Locke 1/6/99

Back to page menu

 

WIDER ISSUES

There are a number of wider issues.

1. Other Providers

The shape and structure of the broader National Certificates of Educational Achievement. The Ministry's "Progress Report #1" (undated) states that "In essence, the externally and internally assessed achievement standards in conventional school subjects will provide one route [our italics] for students to gain NCEA credits. Credit sill probably also be able to be gained by students achieving in ITO-developed and other unit standards as well as examinations or standards registered by other organisations (such as the International Baccalaureate.)

The relevant OHT presented to the English experts panel is reproduced below:

GAINING A NATIONAL CERTIFICATE

NCEA is "owned" and regulated by Government.

Individual standards (subjects) need not be developed or operated by Government

Comment

In a letter to Tim McMahon (20 May, 1999), I asked: 'I would be interested in information which could tell me how the English Study Design can be recognised as awarding standards which will allow students in our [trial] schools to gain credit on the NCEA. (What are the steps? Whom do we approach?)

"Tim McMahon wrote (25 May, 1999): 'With regard to alternative qualifications there are two possible answers. Under the proposals in the National Qualifications policy developments (see the Green paper) any qualifications owner will be able to seek to have their qualification registered on the National Qualifications Framework. So it is with the International Baccalaureate. Further, however, we are exploring the idea that if a student pursues some other qualification that is registered on the NQF that is obviously curriculum related, it should provide credit towards a student's National Certificate of Education Achievement.'

"'In the latter case however, you will appreciate that there would need to be regulation to prevent a student's gaining credit for the same learning outcomes through more than one route. While we are working out the way such a regulation should be enforced in practice, if there are standards registered for the NCEA that are broadly accepted by the community and the sector, the Ministry would not want any other standard, examination, or qualification that addressed the same outcomes to be registered.'

"I replied (31 May, 1999): 'I'm confused. Sorry. What I think you're saying is that, say, a qualification such as International Baccalaureate (in English, or Theatre Arts) could be registered on the framework. However, if this qualification appeared to be made up of sets of achievement standards that resemble the ones that you are developing via your subject expert panels, then there will be regulation to prevent students being awarded these achievement standards via these alternative providers.

"Despite my confusion, I would like to know who we should approach in the Ministry (and how) in order to have the English Study Design registered as a qualification on the NQF. I mean, we can hardly be accused of imitating achievement standards in English that have yet to be developed could we?'"

Terry Locke (3 June, 1999)

 

2. Scholarships

The QDG and its related policy groups is considering two options:

  • Level 4 standards to be developed to complement each level 3 achievement standard.
  • Inviting other institutions or agencies with examinations experience to develop appropriate standards and to examine candidates against them.

Back to page menu

Back to English Study Design index page

 

 

 

 

Sport & Leisure Studies | Graduate Studies | Staff home | Student home | Research | Events calendar

 



(c)opyright 2000 University of Waikato | about