The ABCs of Evaluation - Boulmetis & Dutwin

I've rediscovered another older book (2005) on the bookshelf. I was browsing round the home library searching for inspiration in developing an evaluation for a leadership development program. I remembered what I liked about this book the first time round.

The authors have produced a book which is set out logically. It covers the choices you have and steps to take to carry out an evaluation. I found the section describing different evaluation models and which one to choose was just what I was looking for:
1. Discrepancy Model: use when the program is so interwoven within the organisational context it is difficult to discover freestanding changes
2. The Goal Free model focuses on the participants needs and is independent of the organisation's needs.
3. The transactional model is similar to the goal free model with the exception that the evaluator plays a more active role in the evaluation process.
4. The decision makingg model focuses less on the success of the program and more on the longer term sustainability
5. The most popular model is the goal-based where the outcomes are assessed against the organisation's objectives (or program objectives)
The authors do also list other evaluation models and I would be interested in using systems analysis with my healthcare programs. The version using art criticism where you bring in an expert in the field to review progress is an interesting one I had never thought of. The adversary model where participants present their views and a "jury" hears their evidence might be useful for the rapid assessments we sometimes need to make.

The evaluator's role is covered as a continuum from the start of the program through to the end as well as the level of active participation in the evaluation process.

I was interested in what an evaluation is supposed to measure. Having got used to most healthcare evaluations answering "has the program delivered its objectives" it was good to see that "was the program efficient in terms of cost and staff time - was the expense worthwhile?". What we often mean when we say sustainability, the authors use impact. I like this as it covers not just the sustainability of the results but also the magnitude of the results as contributing to the organisation's long term objectives.

The book also includes the types of data, how to present it and the process of writing up your report.

I'd recommend this book if you're starting out in evaluation or need a refresher. If you're a seasoned program evaluator I suspect you will find other books go to more depth.

http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0787979023.html

(Book Cover image from Wiley)

Rapid Transformation; A 90 Day Plan for Fast & Effective Change - Tabrizi

The change, improvement and OD fashion at the moment seems to be about speeding up the process. Part of me is a bit suspect about this because I believe change is a very personal and behaviour driven thing. However, another part of me believes that getting some momentum and steam behind an initiative is incredibly important.

Rapid Transformation was an easy read and I found many of the exhortations obvious, reasonable and at times felt like nothing new. What is new, however, is the exhortation to get your skates on and do stuff in 30 days that you might previously have taken 12 - 36 months to do. Another underpinning theme of the book is that incremental change is not enough. To survive and thrive organisations need to transform their businesses - and do so continuously and quickly. (Leaves me breathless just thinking about it...)

According to this book - and also in my experience - planning is everything.

Pre-planning stage: this is about ensuring you've fully diagnosed what the problem is that you're trying to solve. There seems to be no clear timeline on this and maybe this is where the devil lies. Maybe all change takes a long time to figure this out. SO if this preplanning stage takes months and years then the "rapid" is lost.

1st 30 days: Do the assessment; gather data and information and look to turn it into knowledge. This is a key stage and one which may at times be left out. Trick here is you get a max of 30 days to do this activity. Now that is new.

2nd 30 days: Get your future vision sorted, make sure all the goals are linked together and get this vision out to the organisation. You get 30 days to do this. In my experience the method of creating the vision and the method of communication is important. Somehow the author leaves me feeling this is an easy task to be done in 30 days.

3rd 30 days: Develop your plan. This is where you sort out the schedule for change etc.

So where is the implementation?

Ah, implementation begins on day 91.

I can see how for organisational transformation which is quite complex, that the implementation will take a long time (well, more than 90 days). However, with the preplanning stage and the implementation stage taken out of the way, this book focuses on just the getting started phase. While it does provide some useful ideas and methods for getting the right wax on the snowboard so you get a good combination of grip and slide, I do feel it is only part of the transformation story.

You can get more information about the book, the method and the author here:
http://www.rapidtransformation.com/index.html