This toolkit supports the NSW Evaluation Community of Practice and seeks to help government agencies implement the Evaluation Guidelines.
Steps in managing an evaluation project
- The 7 steps of an evaluation project
- 1 Develop program logic and review needs
- 2 Develop the evaluation brief
- 3 Commission the evaluation project
- 4 Manage development of the evaluation design
- 5 Manage development of the evaluation workplan
- 6 Manage implementation of the work plan, including production of report(s)
- 7 Disseminate report and support use of the evaluation
How to use this guide
- Who is this guide designed for?
- What does the guide provide?
- Links to BetterEvalution website
- Your feedback will improve the guide
Evaluation in the NSW Government
1. Develop program logic and review needs
An important early step in planning a program evaluation is to describe how the program is intended to work, using program logic. In some cases it is important to review the needs the program aims to address, through a needs assessment.
2. Develop the evaluation brief
The evaluation brief is a document that is used to gain agreement on an evaluation and develop a Request for Tender (RFT) to commission. an external evaluation or to develop agreements for an internal evaluation.
- Purpose of evaluation – formative or summative
- Type of evaluation needed – process, outcome and/or economic
- Scope and focus of the evaluation
- Key stakeholders
- Key evaluation questions
- What is already known about the program?
- Reporting and communication
- Decide on balance of internal and/or external evaluation
- Develop an evaluation strategy (for large programs)
- The investment in the evaluation
- Governance mechanisms and stakeholders engagement strategy
3. Commission the evaluation project
This step covers preparing a Request for Tender (RFT) that requires detailed, rigorous, feasible and ethical submissions from tenderers. It builds on the evaluation brief and the evaluation design including whether to do the project internally or to seek an external provider for all or part of the evaluation project.
- Preparing the RFT
- Ensuring quality and rigorous designs in proposals
- Procurement: seeking competitive responses
- Choosing the right consultant
4. Manage development of the evaluation design
The design of a program evaluation sets out the combination of research methods that will be used to provide evidence for the key evaluation questions being addressed in the evaluation brief. The design defines the data that is needed for the evaluation, and when and how it will be collected. The evaluation design needs to ensure that the evaluation will be as rigorous and systematic as possible, while meeting needs for utility, feasibility and ethics.
- Who should develop the evaluation design?
- What type of evaluation should be used?
- Designs for process evaluation
- Designs for outcomes evaluation
- Designs for economic evaluation
- Synthesizing evidence into an evaluative judgment
- Research design issues
- Ethical and cultural issues
5. Manage development of the evaluation workplan
Planning for the implementation of an evaluation project should incorporate good practice in project management. Developing a clear workplan, with realistic timeframes, clear deliverables and milestones is crucial for ensuring the evaluation stays on track and achieves the intended results. The processes are similar for internal and external evaluation projects.
- Who develops the evaluation workplan
- Clear milestones and deliverables
- Stakeholder communication strategy
- Provisions for revising the plan
- Clear lines of management communication
6. Manage implementation of the workplan, including production of report(s)
The actual conduct of the program evaluation involves implementing the workplan to collect and analyse the data, and to prepare reports. The development of the workplan was discussed at Step 5.
7. Disseminate report and support use of the evaluation
Findings from a program evaluation may be received by different audiences and used for different purposes. The results of an evaluation should be communicated to relevant audiences and used to inform future practices