QUALIFICATIONS

I have only spent three days reading and thinking about the content of your draft. It is highly detailed and the issues are most complex. You stimulate many, many thoughts and my comments would undoubtedly be of much greater potential value if I could spend longer on the draft and developing my comments but unfortunately I cannot do this. Being away from my office, I am also unable to refer to materials, such as the Gland Report and my Stockholm report as well as my files, to refresh my memory. You should also understand that I was only a distant viewer of the Commission while it did its work. My closer involvement was from the late fall of 1996 through to the first meeting of the Commission in 1998 and then again in early 2001 through the final meeting of the Forum.

Besides this involvement in the establishment of the WCD and the final meeting of the Forum, I bring several perspectives to your report that might be potentially useful. I have been involved in the design, implementation and facilitation of multistakeholder and various forms of stakeholder involvement processes for more than 20 years and for even longer have worked on water resources issues in Canada and internationally. My experience is as a practitioner, teacher and researcher. I teach courses on negotiation, facilitation and mediation in sustainability planning and governance. In recent years I have offered a course entitled Innovations in Multistakeholder Processes, which is primarily taken by the graduate students who want to undertake research with me in these areas. Many of them undertake studies designed to assess experience with MSPs. Much of my own writing is focused on the assessment of MSPs. Two particularly relevant examples can be found through links on my web page. One is an assessment of my experience in establishing and chairing a unique MSP, the Fraser Basin Management Board during 1992-94 <<http://www.interchg.ubc.ca/dorcey/chcwra/fccwra.html>�. The other is an assessment of the experience in Canada with the design, implementation and assessment of citizen involvement processes in sustainability governance (including particularly MSPs) over the last 50 years <<http://www.interchg.ubc.ca/dorcey/trends/draft2.html�> .

VALUE OF YOUR REPORT

When I was reviewing the literature in 1997 to advise on the design of the WCD, I wish I had had accounts like yours for earlier commissions such as Brundtland's. At the time there was relatively little information around addressing the breadth of assessment questions that you have considered. It is exceptionally valuable to have this detailed account of the WCD and for you to have done it while events are still relatively fresh in people's memories and you could also observe some of the events unfolding. We are desperately short of assessments of MSPs and badly need to emphasize what we can learn from experience.

I am obviously biased but I do believe that the processes leading up to the establishment of the WCD and the process itself are pathbreaking examples of MSPs and that, all things considered, they have been extraordinarily productive. It is therefore particularly valuable to have your account and assessment of the experience.

Just as the WCD process was an heroic venture in itself, so was your assessment of it. You too faced a task that was daunting in its scope, complexity and controversy. Likewise, you had limited time and, I suspect, very limited resources relative to what was needed. Both have to be looked at in this context and highly commended for all that they have achieved given the constraints upon them.

As is often the case in work such as yours, it tends to generate more questions than it answers. In many places your discussion stimulated me to scribble questions in the margins reflecting my desire to know so much more about the specifics and details. But to expect this of you would be unrealistic. Rather this should be seen as one of the contributions of your report, a potential that future researchers will pursue.

ANTICIPATING THE REACTIONS (MAXIMIZING THE VALUE, MINIMIZING THE DAMAGE)

As you recognize, your report will generate all kinds of reactions and responses. Just as has been the case throughout the WCD process, different stakeholders will see your work in different ways and because of its importance will comment on it and exploit its content to advance their own agendas. It is remarkable how closely the questions about your report parallel those that you are addressing about the WCD!

My concern is that the value of your work and the conclusions that relate to advancing and expanding use of MSPs not be lost under the weight of potential negative comments. I will therefore concentrate the rest of my comments and suggestions on ways in which you might anticipate and strategize to mitigate negative reactions. I focus particularly on general issues and leave it to others, who are much better informed than I, to give you feedback on the "accuracy" of your account and interpretations of events. I will indicate where I think your assessment is vulnerable and then suggest how this might be addressed recognizing that you probably have very limited time and resources for making revisions.

CONSTRAINTS

As indicated above there were major constraints on what you were able to do in your assessment. I would encourage you to preempt criticisms and put your work in the most favourable light possible by speaking explicitly and fully about them and their implications at the beginning of the report. Many of the potenial criticisms I'll suggest below can be mitigated at least in part by your discussion of constraints and their consequences.

FRAMEWORK OF ANALYSIS

There are some significant ways in which the framework of analysis for the study might be strengthened so as to provide a clearer rationale and focus for the specific questions that are being addressed in the study. Chapter 2 does an excellent job of putting the WCD in the context of the history of global commissions and governance. However, while it includes a general consideration of their evolving use of MSPs, it does not provide a comparable consideration of MSPs as such. For this the report primarily relies on the short section in Chapter 1 (pp. 3-5) and one general reference, Hemmati et al. (footnote 4).

There had already been a great deal of experience with MSPs before 1996 when the idea of the Gland meeting first began to be discussed. The majority of this experience had been in North America with diverse processes from the national to the local level. When George Green first approached me about the Gland meeting, it was because of my experience with these processes in Canada and because of the perceived potential to extend them into the global arena. The accumulated experience with MSPs and their facilitation was a major consideration among those who were designing the Gland meeting and this carried over into the design and implementation of the WCD. Achim Steiner who played such a pivotal role in design and implementation of both was highly experienced and skilled in facilitating such processes.

Thus it is hugely important in establishing the context for assessing the WCD as an innovative form of governance to highlight the status of the art and science of MSPs that influenced its design and implementation. Some of this experience was explicitly referred to in my introduction to the Gland and Stockholm reports (if I recall correctly; remember I don't have a copy with me to check). Beyond this you should have some information on this from the interviews with myself, Achim and others. But more fundamentally you can refer to the published literature at the time (my recent paper that is available from my website summarizes some of the major points and provides references to seminal work that preceeds Hemmati et al.)

I believe it is important for you to consider at least three questions. First, what did the literature in 1997 indicate were the key principles that should guide the design and implementation of MSPs (i.e. what were the normative ideals and how did these translate into criteria)? You can then relate these to the ones used in the Gland and WCD processes. Most importantly, you can relate them to your own choice of assessment criteria. Notably, while there are many ways of expressing it, your criteria don't include any consideration of what some call "cost-effectiveness" and others "efficiency". Criteria of independence, transparency and inclusiveness, along with other key considerations such as the breadth of consensus, each have to be related to the other dimensions of costs and benefits (broadly defined) (e.g. more time, money and other resources generally enables more of each of the othe criteria.)

Second, apart from what people were proposing as principles for MSPs, what was the state of practice in 1997? This field was (and still is) no different from many others in that the state of practice often varies considerably from espoused principles. There are all kinds of reasons that range across the extremes of being willing but unable through to being unwilling and deceiptful. Particularly important for your study is a sense of what was feasible in practice as opposed to the ideals of principle (e.g. how well did earlier MSPs succeed in being inclusive and representational even with the best will in the world?)

Third, what was perceived in 1997 to be the importance of convening and facilitation to the relative success of MSPs? As you will see in my recent paper (and as mentioned in the Gland and Stockholm reports), I have argued that each was viewed as crucially important to the productivity of MSPs. By "convening" I mean who was involved and how in bringing the MSP into being and establishing its mandate (e.g. the Gland process was unique as a means of convening and establishing the mandate for a global commmission). By "facilitation" I don't just mean the individual involved in doing this but the varied individuals and potentially institutions/organizations involved in providing this function. I believe these are two components of an analytical framework for assessing the WCD that are critically important in assessing the overall performance and productivity and which don't receive enough attention in your present draft.

METHODS

There are a number of methodological weaknesses in the draft, some of which can be addressed by adding information, others of which you probably can only mitigate in part by recognizing them.

Good practice in qualitative research such as your assessment study involves being more explicit about (i) the investigators' backgrounds, (ii) the rationales for the methods used and (iii) the experience with implementing them. This transparency gives greater understanding, credibility and legitimacy to the assessment.

<paraindent><param>right,left</param>What is the background and experience of the four authors? What are their biases? How does this relate to the goals and objectives of the organizations conducting the assessment? What special precautions were taken to control for biases?

What and why were the particular methods chosen for collecting data, analysing it and writing it up? What precautions were taken to ensure consistency of approach among the four investigators? What strategies were employed to enhance the validity/credibility of your conclusions by triangulation (e.g. testing for corroborating evidence through interviews with multiple individuals, and comparing interview results with written documents)? How were interviews conducted and recorded? What was the form of the raw data and how did you collate, organize and analyze it (e.g. were interviews tape recorded, then typed up and their content analyzed using software such as Nudist)? How did you determine your conclusions (i.e. how did you assess your evidence and different perspectives among the investigators on this, in determining where the weight of the evidence lay)? How were recommendations developed? How did you select and respond to reviewers comments?

What actually happened in practice (i.e. how did things change from what you intended in the design of the assessment)? How well did the strategies and methods work? What additional insight can you give the reader by describing your experiences and impressions in carrying out the study?

I would recommend putting this information up early so that it conditions the reading of the report. Likewise I would put the key considerations with respect to methods early in the Executive Summary.

MORE SPECIFIC OVERALL CONCERNS

I turn now to some more specific concerns, some of which would be addressed by the above suggestions with respect to the analytical framework and methods. Some of these concerns were not always present because of different approaches being taken in different parts of the text (i.e. there is an uneveness of treatment among parts of the draft).

Confusion of voices

In numerous places it is not clear where the voices of the investigators take over from those of the stakeholders. It is good practice to keep these clearly differentiated. In many of the boxes you need to clarify what material relates to whom and provide an unambiguous context for quoted material (e.g. Box 4.1). This is particularly important for evaluative comments.

Middle ground voices

In discussing the WCD process you frequently mention the extreme interests (as illustrated in the opening paragraph contrasting Medha Patkar and Goran Lindahl) but give relatively little attention to the middle ground interests and how they vary. These participants had key roles and impacts among the Commissioners and members of the Forum from my limited observations. Do you feel you have the evidence to conclude they were insignificant?

Dominence of voices

The greater activism and effectiveness of the NGOs in the WCD process seems to have been replicated in your assessment. I hear their voices and views much more than the other stakeholders. This leads to an apparent lack of balance in your assessment, even bias. You need to do all you can to dispell this. Ask whether you have adequately reflected the breadth of perspectives on a given issue. Consider whether your assessment is being swayed by the differential access to and the tactics employed by the interviewees. This is the reason why it is important to clearly separate your voice from that of the stakeholders. you discuss this briefly on p. 46, I would suggest, as above, that this problem and your handling of it be addressed earlier and for the whole study. Did the same active reviewers of WCD materials, play a similar active role in influencing your assessment (p.76)? Was industry engaged in a "damage control" mode (p.109)?

Weight of evidence

Ideally you should be presenting the reader with the range of views and evidence and then arguing your perspective on the conclusions that you believe are supported by the weight of evidence. The first paragraph on p.79 illustrates an instance where this is well done. On other occasions your conclusions fall short of this standard. Conclusions are sometimes drawn without strong supporting evidence (e.g. What evidence do you have that significant bias did result from the imbalances in gender? p.51 and footnote 37 seem to be your strongest evidence but hardily strong enough to support the strength of your conclusions. Likewise, what is the evidence to support the implied criticisms resulting from make-up of the secretariat, p.50? And biasing impacts of funding sources?). You need to ask whether such conclusions are supported by the weight of the evidence and in particular the ultimate consequences.

Compared with what?

The assessment would be greatly strengthened if there was greater clarity about the standard or basis of comparison. Comments by stakeholders and yourselves are often not explicit about this. Is the WCD process being compared with some idealized, in-principle view from the literature? Or is it in comparison to some previous best practice experience? If so, is the WCD situation comparable? Is the assessment comment considering the multiple objectives/criteria or just focusing on one or a subset and not recognizing the implicit weightings? Particular interesting is the with vs. without question. How does it compare overall with others, such as Brundtland? Clarification of the analytical framwork as suggested above, would provide the basis for making the basis of comparison much clearer.

Lessons

In concluding you focus most appropriately on a list of lessons. However, they are not as valuable as they could be because of the weaknesses in your analytical framework. To a very large extent your conclusions as stated reflect commonly recognized and experienced shortcomings and consequent remedies of MSPs. All of these were known at the time of designing and implementing the Gland and WCD processes and attempts were made to anticipate and avoid them. I would therefore suggest that you focus on two more specific questions in drawing your conclusions. First, in what ways did the Gland and WCD processes (a) avoid and (b) experience the same kinds of problems as earlier MSPs? Second, what conclusions can be drawn about strengths and weaknesses of the Gland and WCD processes in global contexts (i.e. in comparison to the previous experience with MSPs that was predominently at national to local levels).

Recommendations

Your recommendations in Chapter 9 could be greatly strengthened by drawing on the insights in the literature on MSPs to go beyond the relatively limited evidence that you have from what happened around the single process experience with the WCD. I believe there are three areas in which you could most usefully do this:

(i) The role of the convening or sponsoring organizations in the MSP ( in this case the WB and IUCN) in order to ensure buy-in and commitment to implementation of agreements. I recognize that there were strong feelings at the time of Gland that made it difficult to think of including the WB in the Commission directly but this challenge is not uncommon. Often it is the government or state that is setting up the MSP and similar concerns exist at the outset. However, it has been recognized time and time again that it is essential to include all stakeholders in the process who can impact the final outcome of any agreement. As a result creative ways have been found to include the key stakeholders (e.g. government agencies that ultimately have to make the implementing decisions) as participants in the process. As a facilitator of the establishment discussions, in retrospect, I believe I should have argued the rationale for this and proposed ways for doing it much more vigorously than I did. If this had been done I think the final report might well have been crafted in such a way that the WB was more bought in to the conclusions and recommendations.

(ii) The role of the Forum in providing opportunities for the Commission to build a wider consensus among the diverse stakeholder groups. I am under no illusions as to how difficult and time consuming this would have been but I believe more could have been done in exploiting the potential of this mechanism for testing ideas and shaping consensus. Again the literature suggests creative ways in which this might be done. For this to work, the Commission would have to be committed to acting resolutely no matter what came out of the Forum. I believe there could have been ways to build towards greater buy-in to the Commissioners ultimate conclusions and recommendations but it would have had to have been included from the beginning. If this had been done, it would have been more more feasible to involve the Forum in various ways in shaping the final conclusions and recommendations. Again, as the facilitator and advisor, I should have given much greater attention to this in the preparatory stages. If the Forum had been more fully engaged I believe that the concerns of some of the stakeholders who have been hostile to the final report could have been at least partially met.

(iii) The role of facilitation in making MSPs work. Facilitators and facilitating organizations play an absolutely crucial role in enabling MSPs to work in practice. In the two above points, I have indicated what I might have done better as a facilitator. I think the literature could help provide insight into what the Chair, Commissioners and Staff might have done differently ('better") in their critical facilitation roles. Many of the issues you address in Chapter 9 could be addressed at least in part by "better" facilitation (e.g. in particular pp.139-140).

A FEW SPECIFICS

In concluding I'll very briefly mention a few specific comments and suggestions.

I would suggest that you consider the Gland and subsequent establishment process as MSPs in their own right and separate from what followed under the WCD. It would be useful to have your separate conclusions and recommendations on each of them per se.

I would like to see some commentary on the success throughout in meeting the time deadlines. There was immense concern at Gland that the slippage that commonly occurs in such processes not happen on the dams issue. This was a key consideration in the establishment process and all things considered the Commission was established and launched pretty much on schedule. Likewise the Commission delivered its report very close to the deadline set. Meeting these deadlines obviously imposed trade-offs but I think they were well considered given the importance that was attached to timeliness by all stakeholders.

I would recommend including a brief summary of the OED study that was a key basis for the Gland meeting. The emphasis on the critiques of it suggest a bias in your approach. The time sequence of events is also confusing as reported because it doesn't become clear until later that the critiques were released on the eve of the Gland meetings as part of the NGOs strategy.

I believe it is important that you outline the process that was used to solicit nominations and input and assess the candidates for Commissioners prior to the selection procedures. A lot of effort went into this. It was part of the unique characteristics of the establishment process for this commission. Your focus on the inevitable contention that arose around the final make-up of the commission misses out important aspects of the process. How would you recommend changing the process in future?

I was most interested in your accounts of some of the debates or points of contention that arose during the WCD process. Issues such as eminence vs representation, consensus within vs without, and the Forum as advisory vs decision making, had all been given extended discussion before the establishment of the Commission and I thought relatively clear positions had been taken with regard to them in the preparatory phases. I am not clear from your accounts why this contention happened. To what extent did this result from the effectiveness of stakeholders in challenging previous decisions? How did this arise and get resolved within the Commissioners? How well was this handled? Do you have insights into the "style" of the Chairmanship? It is important to understand this better so that it can be avoided in future.

Unless I missed it, you don't comment on the fact that the number of Commissioners increased significantly from what had originally been suggested at Gland. This commonly happens when people confront the difficulty of constituting a multistakeholder body that is sufficiently representative of the diversity of interests. The major trade-off to be weighed is the increased difficulty and costs that result from working with larger numbers. What are your conclusions about the merits of the trade-offs in this instance?

FINAL COMMENTS

To the extent that your opportunity for making revisions is constrained, I would recommend (a) giving major attention to ensuring that your major points are not buried among the multitude you have considered and (b) that you focus on mitigating the major threats that might undermine your most valuable and important report. I would particularly emphasize the revision of the Executive Summary as it will inevitably attract greatest attention. Finally, you might recommend what further assessment and work on your conclusions and recommendations merits greatest attention and how it might be done (e.g. what further work might be done in developing your recommendations by drawing on the literature and wider experience with MSPs).

I hope these quick comments are useful to you and my apologies for not being able to spend more time on it. I will be back at the University on August 8th and you could contact me there (or my home 604 737 2167) after that if it would be helpful to discuss this further over the phone.

I very much look forward to seeing your final report.