CARRF The Canadian Association for Responsible Research Funding

For its first two decades CARRF operated through a bulletin board located at York University, Canada. Now CARRF is best accessed by way of the Twitter site kindly organized by Johannes P. Wheeldon Click Here

 


The Internal Brain Drain

Reform of the Canadian Research Councils

Bill C-13. The Establishment of the CIHR

Poulin and Gordon (2000) on Reforming CIHR

Forsdyke (2001) Opening Remarks to Standing Committee on Industry, Science and Technology

REPORT of Standing Committee. June 2002

Hunter's Report (2004) on the NSERC President


The Internal Brain Drain

To The Minister of Health, the Hon. Allan Rock,
House of Commons, Ottawa, ON., K1A 0A6.


copies to:

The Rgt. Hon. Jean Chretien, Prime Minister
The Hon. Paul Martin, Minister of Finance
The Hon. John Manley, Minister of Industry
The Hon. Ronald Duhamel, Secretary of State for Science
Presidents of Granting Councils
Presidents of Universities.

No increase in medical research budget without improvements in distribution system.

November 23rd 1998

Dear Minister Rock,

      The purpose of this letter to draw your attention to the opinion of many members of the Canadian research community that it is scientifically pointless and economically unjustified to increase the budget for medical research without a simultaneous radical correction of the way research grants are distributed among researchers.

     We, the undersigned members and supporters of the Canadian Association for Responsible Research Funding (CARRF) understand that a letter writing campaign is underway wherein members of the scientific community are asking you to increase the level of funding for medical research through the Medical Research Council (MRC) and other agencies.

     Much as we would appreciate increased levels of funding, we believe that the acute problems now faced by medical researchers are not due entirely to shortage of dollars but also to a very flawed distribution system. The flaws begin with the process of peer review which is treated as an exact science, without much allowance for the fact that it is not. The effect of the present peer review process is to concentrate research funds into the hands of the few, thereby depriving too many others of an opportunity to deliver their creativity and discovery for the benefit of Canada and the world.

     Researchers who are deprived of the opportunity to do research represent a serious "internal brain drain" within Canada that is much more serious than any brain drain leaving the country. Not only are such disenfranchised researchers lost to the research enterprise, they also continue to be paid by their universities for work the present system prevents them from doing -- a two-fold waste of taxpayers' dollars.

     The old adage that "two heads are better than one" still applies. Only in this case we are talking about thousands of good heads across Canada who cannot deliver the goods for which they are paid because research funds are literally monopolized by the few who see themselves as the truly excellent researchers according to their own skewed yardsticks. Thus, simply asking for more government dollars to further enrich these privileged few will not solve the problem but rather create a new problem of over-funding and waste.

     We believe that the peer review system is generally only good enough to differentiate between roughly the top two thirds and the bottom third or 1/4 of research projects. It is not sharp enough to divide the top 20% of researchers from all the rest, which is how it operates now. On the basis of the present, flawed, distribution system, you would have to increase funding by up to 500% in order to involve 70-80% of all qualified applicants. If such increments are out of the question, then the only reasonable alternative is to radically revise the distribution system in order to make it more inclusive of Canada's talent pool. One way or another the bulk of qualified researchers must be brought back into productivity.

     Without some pressure to change, the present entrenched system will not change itself. Its present beneficiaries will see to that. And without changes to the current distribution system, any new money will, in our judgement, be used inefficiently and be channeled preferentially to those who are already well funded.

     There are realistic and reasonable solutions to our distribution system such as introduction of basic no-frills research grants, with a system of gradual ranking instead of the present "all or none" allocations with sharp cut-offs that produce zero funding for grant applications below the line that are every bit as good as those above the line. This system was developed by Donald Forsdyke of Queens University and the details can be found at: http://www.queensu.ca/academia/forsdyke/peerrev.htm

      We urge your Government to refrain from allocating any new money to the granting councils in the next federal budget. Instead, the Councils should be asked to implement a new distribution system based on a broad consensus among researchers, including the Canadian Association for Responsible Research Funding (CARRF). No extra funding should be released until the conditions for proper distribution of such funds are adequately established.

     We believe that such a complete overhaul of the research funding system will do more to restore integrity and effectiveness of Canadian scientific research, than further infusions, at any level, of the Canadian taxpayers' money.

     MRC officials will no doubt brand our view as marginal.  Many researchers are understandably reluctant to speak out. But we assure you that our views are widely felt within the scientific community, most especially by the 50-70% of researchers who have been effectively excluded from the practice of research for which they trained many years and for they still draw salaries.

     To this letter we attach an article by A.A.Berezin,  G.Hunter, J.J.Pear, and C.Rangacharyulu, "Canadian Research Councils need Fundamental Reform", which was published in the Fall 1998 issue of Ontario Confederation of University Faculty Association (OCUFA) Forum. It provides more details regarding the defects of the existing funding system and proposes plausible alternatives. The article in its entirety can also be accessed on the OCUFA web page (Forum, Fall 1998 issue): http://www.ocufa.on.ca

       Contrary to what Dr. Henry Friesen (and other research administrators) are saying publicly, there are far better and more efficient ways to operate the research funding system. It IS possible to do more and better research with the same overall research budget and to involve a much larger proportion of Canada's researchers who are now disenfranchised. Bringing such researchers back into productivity will do more to reverse Canada's "internal brain drain" than any increase in budget without reform of the system of allocation. The best solution would be both reform and an increase in the research budget.

     However, it will be a cardinal political mistake to provide extra funding without requesting councils to change their grant allocation principles and practices.


Yours truly,

Mohammed A. Ali, Universite de Montreal
Alexander A. Berezin, McMaster University
Donald R. Forsdyke, Queens University
Richard Gordon, University of Manitoba
Geoffrey Hunter, York University
Christophe Jankowski, Universite de Moncton
Waldemar W. Koczkodaj, Laurentian University
Margarida O. Krause, University of New Brunswick
Serge Larochelle, Universite de Montreal
Daniel Osmond, University of Toronto
Joseph J. Pear, University of Manitoba
Klaus E. Rieckhoff, Simon Fraser University

Keith Slater, University of Guelph
Maria L. Torres, (On leave from) University of Regina
Anthony Whitehead, McGill University

OCUFA FORUM, Fall 1998 issue, pp. 8-13
(OCUFA = Ontario Confederation of University Faculty Association).

CANADIAN RESEARCH COUNCILS NEED FUNDAMENTAL REFORM

by Alexander A. Berezin, Geoffrey Hunter, Joseph J. Pear, and Chary Rangacharyulu

The effectiveness of the federal support of academic research at Canadian universities by the granting councils is of concern to all Canadians and especially to the university community. Here we outline the key issues regarding effectiveness, and express our view that the research councils need to undergo fundamental reform. Our focus is mainly on NSERC, since we are more familiar with it than with the other granting councils. Nevertheless, we believe that our inferences are (to a greater or lesser extent) applicable to MRC and SSHRC as well. NSERC is the largest of the three research councils, with an annual budget close to $500 million. Its clientele is mainly 10,000 to 12,000 professors of science and engineering who are eligible for NSERC research grants.

THE PROBLEM

 Some recent articles in the public press convey the impression that the problem is simply government underfunding of NSERC, MRC and SSHRC. This view is widely held; it is actively promulgated by the councils themselves. Recent increases given to the budgets of the three councils (in the February 1998 Canadian budget), although generally praised, have been criticized as "insufficient" doing little more than restoring funding that was taken away just a few years ago.

We believe that the "underfunding" bogie is a "red herring" that detracts from the real problems with Canadian university research; it diverts attention from the inherent flaws in the operations of the granting councils themselves. We believe that an overhaul of the research funding system will do more to restore integrity and effectiveness to Canadian scientific research, than further infusions of the Canadian taxpayers' money, whether this be at the level of 20%, 50%, or even 100% (as has been proposed recently in the United States).

    Without changes to the policies of the councils the new money will, in our judgement, be used inefficiently, being channelled preferentially to already well-funded (often overfunded) researchers. In fact this has actually happened in the case of NSERC; at its meeting in June the NSERC Council decided to simply do more of the same: a 10% increase across the board (regardless of actual need) to professors who already hold research grants.

     We filed a proposal (proposed jointly by 17 professors) to NSERC in May that 2% of its total budget (15% of the $60-70 million of new money) be allocated to give modest grants averaging $15,000 to some of the 709 applicants for research grants who were awarded "nil" grants in the 1997-98 grant-awarding process. This proposal was entitled "Restoring Integrity to the Research Base", a title that was amply justified by the details and rationale for the proposal. This proposal was deflected from the NSERC Council to NSERC's Committee on Research Grants and as a consequence it was rejected with an incoherent pretext that failed to counter our rationale for the proposal. This deflection and rejection of a very well thought-out proposal exemplifies NSERC's recalcitrant resistance to change. The 709 "nil" awardees remain de-facto unemployed as research scientists (notwithstanding substantial salaries to do research), and with the continuation of the climate of "competition" (for grant money) the coercion on funded professors to produce lots of easily publishable (but substantially inconsequential) research papers will continue; i.e. the support and promotion of routine research at the expense of innovations that have far greater potential to benefit Canada and Canadian society.

    One might well wonder how such a meritorious proposal requiring only 2% of NSERC's budget (less than 1/6th of the new money) could founder? The explanation is straightforward; the people whom NSERC appoints to its many committees (including the Committee on Research Grants) are typically professors who hold high (well above average) research grants. They are instructed (by the NSERC bureaucracy) to be "selective", and since this 'selectivity" has favoured them with big grants, (the key to advancement in their professorial careers), self- interest motivates them to resist any change to the system - regardless of the plight of their peers. Thus academic researchers are pitted against each other in a struggle for survival as funded researchers, rather than being encouraged to collaborate as is desirable for the advancement of science.

   The May 1998 proposal to NSERC is only one of many representations that have been made to NSERC, some of them dating back to 1984 - only 6 years after NSERC was formed. These representations have been brushed aside mindlessly by successive NSERC presidents and other executive officers. NSERC shields itself from public discussion of its policies and practices, because continuance of the present system ensures that an overbloated bureaucracy (with a staff of about 100 clerical jobs, costing about $17 million annually) maintains its existence, whereas some of the alternatives that have been proposed would result in substantial staff reductions.

THE HIGH PERCENTAGE OF "NIL" AWARDS

 The core of NSERC's philosophy (and practice) is the principle of "selectivity", i.e. the policy of deliberately not funding a significant percentage of the applicants. This policy was inaugurated by NSERC's first President, Gordon McNabb, apparently to convince the government that NSERC was "running a tight ship"; i.e. not giving out money indiscriminately.

   Ironically this policy has back-fired; in effect (if not in intent) it has been counter-productive of the research excellence that it was intended to support and promote; by keeping about one-third of our science and engineering professors unemployed (as researchers), the competition for funding has promoted (among the funded) "safe" (routine), "productive" (of easily publishable papers) research at the expense of intrinsically more valuable innovations, that had the potential (among other benefits) to enhance the economy of Canada as a technically advanced nation.

   Twenty years (since NSERC was formed in 1978) is long enough to prove whether "selectivity" works or not; the global proof that it has failed is the huge decline in the purchasing power of the Canadian dollar vis-a-vis other currencies during that 20-year period. Not only has "selectivity" been "effective" in retarding the Canadian economy, it has also been instrumental in deceiving the Canadian government into believing that NSERC's funding policy is responsibly discriminatory, and it has also been successful in keeping criticism from the research community in check; you don't "bite the hand that feeds you", especially when you fear the threat of being cut off from life-giving funding.

   It is important not to confuse NSERC's policy of selectivity with the weeding out of incompetent proposals; identification of such proposals is a valid use of peer-review - necessary to circumvent all and sundry from dipping into the NSERC pot. However, it is clear to anyone even remotely familiar with the Canadian university system, that the high level of competition for faculty appointments hardly ever results in an "incompetent" being appointed; such instances are extremely rare and for all practical purposes can be safely ignored in designing funding policies. To presume (as NSERC's current funding policy does) that one-third of all Canadian professors are incompetent (i.e. deserve no research funding) is a ludicrous insult to the entire University community.

   Some NSERC documents attempt to provide lame excuses for this high rate of "nil" awards by saying that, in fact, not all those who are given "nil" awards are "incompetent", but NSERC is "forced" to deny them any funding simply because it does not have enough money to fund all deserving applicants. This sop to Cerberus is no comfort to the researcher whose graduate students are unfunded, especially when he/she sees his/her highly funded colleagues "blowing" excess funding on vacation ("conference") trips to Europe and Japan.

   The pretext of inadequate overall funding does not hold water. Of course, NSERC does not have enough money (and likely never will have) to satisfy the overall demand for funding accumulated in the budgets of all of its grant applications. Researchers can always do more (up to a point) by hiring more graduate students, technicians and post-doctoral fellows to do their work for them; one well-known researcher has had in excess of 15 assistants working in his laboratory, which is clearly unmanageable and must result in inefficiency in the form of doubling and tripling up on specific research projects.

   The well-known rule "first dollars are the most efficient" makes the case for giving (say) $15,000 operating grants to six qualified experimentalists rather than a single grant of $90,000 to only one of them. This and similar arguments were recently put forward by several scholars (i.e. see Terence Kealey, "The Economic Laws of Scientific Research", Macmillan, 1996).

   Thus it does not make sense that competent researchers should be given "nil" grants. All researchers who pass a basic integrity check (as detailed in our May proposal to NSERC) should receive a minimal (base) grant to allow them to remain active as researchers. Funding above that minimal level should be based upon actual need for funds as assessed by tangible criteria such as the current number of supervised graduate students and the nature of the research, whereas at present the dollar value of a professor's research grant is "what he/she got last time" (four years ago) plus or minus up to 50% depending upon how the biases of the Grant Selection Committee (GSC) feel about the applicant's "prowess" as a researcher.

   It should be appreciated that for an active academic researcher, there is an enormous difference between having even a small grant and having no grant at all. In some areas, especially theoretical and computational, even a grant as low as $3,000 to $5,000 per year can cover the major portion of research expenses such as computer hardware and software, attendance at professional meetings, accommodation of short term visitors to enhance research collaboration, etc. More expensive (and productive) activities include the vital role of supervising graduate students - at a cost of a minimum of about $10,000 per annum per student.

   Furthermore, funding by NSERC (regardless of the dollar amount) often provides critical leverage for obtaining non-NSERC funding from provincial and other sources; "matching grant" programs are now commonplace - in a society conditioned by reticence to being the first to say: "this is worthwhile, I will support it unconditionally".

    This is not to suggest that all grants should be at the $3,000 to $5,000 level, but to presume (as a recent NSERC document does) that $10,000 is a "minimally useful grant" is a arbitrary assumption. In reality no amount of research funding is useless.

   NSERC's total budget of $500 million divided by the number of researching professors (around 12,000) yields a figure of $40,000 per researcher. This is much more than some researchers need, and hence there is ample funding to keep all researchers active while providing higher levels of funding to those who really need it. At present, i.e. the awarding of a graduate scholarship takes no account of whether the supervising professor already has adequate funds for the support of the student (from his/her NSERC research grant), and hence double-funding of the same fiscal need often occurs.

   Another blatant example of this double-funding is that holding a research grant (the most common kind) is a prerequisite to obtaining a strategic grant (we are unaware of any exceptions); the researcher with both only has as many hours in the day to do his research (with double-funding) as the "nil" awardee twiddling his thumbs in idle frustration.

   If one asks why NSERC mounts all these programs, the answer is that they serve to impress the politicians of the federal government with all of the things ("highly qualified personnel" in training, "strategic" research, "centers of excellence", etc.) that NSERC supports; thus political expediency rather than fiscal rationality is the framework of NSERC policies and practices.

TOP, BOTTOM, AND MIDDLE OF THE ROAD RESEARCHERS

 What about the privileged "top" researchers vis-a-vis the rest of the research community? It is of course a fact of the history of science that some researchers attain exceptionally high levels of achievement and occasionally receive Nobel prizes and other signs of public recognition. Canadian Nobel prize winners number about 12 (of whom only four or five remain in Canada); i.e. one in a thousand (0.1%) of the 12,000 researchers. So most scientists do not fit into this category of publicly recognized "excellence"; most researchers are simply competent and industrious. Any attempt to discriminate among the 99.9% majority of run-of-the-mill researchers will yield inconsistencies that reflect the personal biases of who you ask, and yet it is this arbitrary and inconsistent form of assessment that NSERC uses to determine who shall have large grants, smaller grants, or no grant at all.

   Furthermore, even if one could (objectively) distinguish some researchers as being much better than others, it does not follow that they need significantly higher funding levels. Funding should be based upon fiscal need - not upon an egotistical system that equates worth as a researcher with the size of his/her research grant.

    In science there is rarely a strong correlation between the impact of the research and money spent to do it. On the contrary, too much funding can actually be detrimental to the research program because it locks in its recipient into the grantsmanship game of seeking ever more postdocs, who need more money, who increase the productivity of papers, which increases the value of the grant at the next competition, etc. etc., all done in a mindless cycle based upon the dictates of money management rather than upon what is scientifically most interesting and promising.

   Science is a highly dynamic and non-linear process. The pattern of what will have a lasting impact and what is routine is rarely recognized at the initial stages of the research. This fact (attested to by many historians of science) makes a strong case for a more uniform funding system rather than the currently in-vogue principle of "selectivity".

   Furthermore, new research discoveries can only be firmly incorporated into the corpus of scientific knowledge if they become part of the on-going research activity of many members of the scientific community. Even if it were true that there exists a small elite group of scientists who consistently make the important discoveries, and that these individuals can be and are reliably detected by the GSCs, the findings of these elite scientists cannot be preserved and perpetuated if other scientists are denied even the minimal funds necessary to carry out what might be regarded as relatively routine follow-ups of these important discoveries.

ARBITRARINESS OF THE GRANT AWARDING PROCESS

 The funding councils are proud of their peer review process; it is invariably praised as objective, fair and capable of reliably picking out the "best" researchers worthy of grants well above the average.

   In reality though, peer review has a high margin of error and its choice of the "best" is often as arbitrary and subjective as a choice among finalists in a beauty contest. All that peer review can do with any reasonable reliability is to identify clearly incompetent or trivial proposals. (And even here the history of science is replete with cases of research originally thought to be wrong or trivial that turned out with hindsight to be pivotal, and, conversely, with cases of research thought to be of great importance that turned out to be of no lasting significance).

   What is even more disturbing, is that even within its presumed terms of reference and limited validity, peer review at NSERC is applied arbitrarily and inconsistently. While it is true that copies of each grant proposal are sent to the applicant's peers - i.e., recognized researchers in the applicant's field - the GSCs are free to give whatever weighting they wish to these reviews, and it is quite common for panels to completely override the reviewers' opinions or to take account of only those reviewers who criticize the applicant based upon school of thought and personal biases against the applicant. Thus it is typical practice at NSERC that the applicant who receives mixed peer reports (say, two positive and one negative) is given a "nil" award, despite the fact that his or her work clearly has merits, as attested to by the positive reviews.

   NSERC's GSCs operate on the negative Achilles heel principle of killing any applicant with any, even minor, discernible weakness, rather than the positive principle of giving the applicant the benefit of the doubt (since he/she is the most expert of all in his/her own particular field) when the external referee reviews are mutually inconsistent. Furthermore, innovative work (because it is new and not yet understood by many people) is more likely to get bad reviews than routine research whose basis is widely accepted; thus there is a strong bias in the "selection" process against innovative research.

CONFLICT OF INTEREST

 It has been pointed out by several critics that the fact that most GSC members are grant recipients themselves constitutes a fundamental conflict of interest. Recently NSERC introduced a rule that GSC members whose grants go up for renewal should temporarily resign from their GSCs. Such a measure (which, in fact, is an implicit recognition by NSERC that there is a conflict of interest), does not, however, solve the problem in an effective way. Members of GSCs are well aware of the next time that their own grants will be up for renewal, and hence their generous treatment of their fellow members is the best guarantee that they will get equally good treatment next time; the game of "I fund you, you fund me" prevails despite NSERC's procedures designed to prevent such favouritism.

   Moreover, given our above argument that experts are unable to reliably assess which proposals (disregarding clearly incompetent ones) are likely to yield the most fruitful findings or the most important scientific advances, a sharp downsizing of the number of "experts" in the system seems warranted.

   It should also be noted that it is very unclear as to how members of the GSCs are chosen; as far as we can tell (it is secretive) the process is confined to a closed circle of highly funded grantees and their friends in the NSERC bureaucracy. We are unaware of any process by which new members are democratically nominated and elected by the academic community; not even their candidacy is known (to allow public discussion and feedback) before they are actually appointed to Grant Selection and other committees.

PUBLIC ACCOUNTABILITY AND DISCRETIONARY POWERS

 Many critics point out that the secretive nature of the funding system obviously permits relatively easy misappropriation of the ideas of grant applicants by those (i.e., peer reviewers and GSC members) who have privileged access to their research proposals. Cases of this kind of academic dishonesty (occasionally reported in magazines such as "Nature" and "Science") indicate that exhortations against such practices do not provide a sufficient safeguard against theft of ideas or data; you cannot stop a reviewer or a GSC member from absorbing a new idea in a research application and subsequently developing it into a research project of his/her own - it is perfectly natural for any scientist to follow up on any idea that he/she finds interesting regardless of its source. The applicants whose ideas are misappropriated in this way often suffer the double whammy of being robbed of their best ideas and (as innovative "nil" awardees) of being denied any research money with which to develop their own ideas themselves.

   This fact, as well as other perilous aspects of the system outlined above, make a strong case for revoking (or at least putting under tight public control) the discretionary powers in awarding grants which are vested with the councils by the government. It seems to us that the public has a right to know exactly how its tax money is being distributed to researchers by the federal granting agencies. Another aspect of the self-preserving secretiveness of NSERC is that it refuses to publish a full summary of each grant competition (including the "nil" awardees); this prevents outsiders from independently assessing whether the results of the process appear to be fair or not. The secretiveness raises the suspicion (as we have indicated above) that the process is far from being fair (neither fiscally nor scientifically).

   Therefore, we advocate action by the Canadian government to commission an unbridled public enquiry into the operations of the three research councils (NSERC, MRC and SSHRC), and/or the creation of a public "watchdog" to monitor the process by which grants are awarded, to question the rationale behind the policies and administrative decisions of the councils, and to institute genuine (as opposed to partial and superficial) regular public accountability of NSERC and the other councils. Without such a measure we see very little hope that the operations of the granting councils will be improved on their own initiative.

CONCLUSION

University professors are by definition expected to do teaching and research. They are paid relatively high salaries for accomplishing precisely this dual task. Practically any research involves some operating costs. Hence, any system which denies competent and industrious professors the opportunity to carry out both parts of their "job descriptions" is fundamentally defective.

   NSERC awards numerous individual research grants in the range of $50,000 to $80,000 per year and at the same time routinely denies even $3,000 grants to many competent and meritorious researchers on the pretext that there is "no money" to fund them at any level. In view of this obvious absurdity, claims that the system is "underfunded" can hardly be taken seriously. Therefore, we see no reason to support the campaign for more money to NSERC until the above described flaws in the grant distribution system are constructively eliminated. According to our estimates (above: $40,000 per professor on average), there appear to be sufficient funds currently in the system to provide all qualified researchers with at least basic, no-frills operating grants so that all qualified researchers can be productive. Thus, until the policy of base grants to all active researchers is adopted by our federal granting agencies, the Canadian research community will, in our view, remain disunited, demoralized, and operating far below its true potential.

------------------------------------------------------

Alexander A. Berezin is a professor in the Department of Engineering Physics, McMaster University; Geoffrey Hunter is a professor in the Chemistry Department, York University; Joseph J. Pear is a professor in the Department of Psychology, University of Manitoba; and Chary Rangacharyulu is a professor in the Department of Physics, University of Saskatchewan.


Bill C-13. The Establishment of the CIHR

To the honourable: Lynn Myers, Reed Elley, Ovid Jackson, Yvon Charbonneau, Christine Gagnon, Gurmant Grewal, Keith Martin, Bill Matthews, Ted McWhinney, Roal Monard, Bernard Patry, Karen Redman, Paul Szabo, Greg Thompson, Rose-Marie Ur, Judy Wasylycia-Leis, Allan Rock.

I wish to thank you for hearing the important presentation made to you by Professors Richard Gordon and Bryan Poulin in December 1999, pertaining to Bill C-13. I have read the official transcript of the material they presented, and the discussion, and wish to affirm that their point of view was valid, well presented and highly relevant to a great number of scientists in Canada today.

Canada prides itself in its democratic society and on its ability to care for the full spectrum of society, including the old, disabled, sick and unfortunate. Even the homeless on the streets attract much media attention these days and have generally been offered many safety-nets such as out-of-the-cold programs, Salvation Army hostels, etc. 

    Yet, when it comes to high achieving echelons of scientists, who spend the first 35 years of their lives in intensive, expensive training and then in fierce competition  for positions, Canada's faulty research funding system seems to be geared to knocking everyone down who is deemed to be  below the scientific equivalent of  Wayne Gretzky!

    There are 2 huge problems with the present funding system.

  • It wastes the talents of, and investment in, large numbers of good scientists (Canada's shameful internal brain drain).

  • Secondly, the perceived Wayne Gretzkys of science are not necessarily the best, most cost-efficient, investment.

    You see, in hockey, you can tally up the goals and assists but in science you don't know which minds will spawn the next insulin, or when. It is frequently said in our Department of Physiology, U of Toronto (home of Banting and Best and their insulin) that those novice researchers of long ago would never have qualified for funding in today's tightly controlled  environment where those who have get more thrive, while the have-nots languish. Banting and Best wouldn't stand a chance these days. Their Vitae would be too short and unimpressive.

     To correct the present wasteful, cruel, unacceptable funding system that is generous to 20% of Canada's scientists while excluding most of the rest, the Gordon/Poulin proposal for  providing baseline funding for all research-active and eligible academic medical researchers is highly recommended. 

    These researchers will remain subject to ongoing monitoring and progress reports. Their creative talents will remain at the service of their country, university and students, and they will likely do much more for the money than those who have more than they can use effectively. Most scientists have only a few good ideas to pursue and are subject to the law of diminishing returns. The so-called 'superstars' getting $300,000/yr for research would probably deliver no less value at $150,000, and  the residual $150,000 could be spread out among 10 creative scientists so as to keep their bright ideas in play for the benefit of science and society.

    I urge you to honour and implement the Gordon/Poulin proposal, or something close to it, because they speak the truth, desire better things for Canadian science and society, and desire to release and harness  the creative talents of scientists who have been sidelined for too long. 

    The only reason you do not hear from hundreds of such scientists is that they have become discouraged, demoralized  and have given up. They also don't want to attract attention to themselves as 'losers' in a cruel system that recognizes only 'winners', as defined by these same winners who are in charge of the system!

    Please grasp this golden opportunity to bring about much-needed change to our research funding system! Thank you.

Daniel H. Osmond, Professor of Physiology and Medicine, Medical Researcher, Faculty of Medicine, U of Toronto.

(Who has been funded for my entire career at U of T,  since 1969, except for a year or 2, continuing  past retirement until mid-2002. Who has received major industrial funding, but has been deeply distressed for over 30 years over the  enormous amount of time wasted in seeking funding and at the inequities and stupidities of the system. 

    I have observed many highly funded 'superstars' come and go leaving little behind them and I remain deeply suspicious of such people who believe they should have it all while they last, leaving nothing for others. 

    The funding system must be opened up, streamlined and more evenly shared. After writing Malice in Wonderland: Research Funding and Peer Review, J Neurobiology, 14 (2) 95-112, 1983, describing the present system, I received some 600 passionate letters and cards almost all expressing approval, convincing me beyond any doubt that the problems were very real).

To: the Canadian House of Commons Committee on Bill C-13

Re: Bill C-13 - the Establishment of CIHR  (the Canadian Institutes for Health Research)

From: Professor Geoffrey Hunter, York University, Toronto;

President, Canadian Association for Responsible Research Funding

Date: 27 January 2000

The Purpose of this letter is to emphasize the need for an Amendment to this Bill that is vital to the success of CIHR. The Amendment will mandate CIHR (as a primary responsibility) to provide baseline funding to all researcher-applicants who meet its eligibility criteria. This Proposal was presented to the CIHR hearings by Professors Richard Gordon and Bryan Poulin; their dispositions are appended to this letter. These dispositions presented the results of their research into what it takes to create innovative discoveries; they have recorded the detailed basis of their inferences in a scholarly paper which I have studied (Abstract appended).

      The experience of successful innovation within the 3M company, at Bell Laboratories, and in the intramural program of the U.S. NIH (National Institutes of Health), is positive evidence that significant innovative discoveries occur when all of the researching scientists are encouraged (through funding and release time from other activities) to investigate whatever appears to be interesting (typically an unexpected, serendipitous observation as in the discovery of penicillin).

      If you believe in the practical value of scientific inference you will incorporate into Bill C-13 Professor Gordon's and Poulin's inferences about what it takes to stimulate innovation. It will take some courage to do this, because it is diametrically opposed to the traditional MRC and NSERC policies of selecting many of their applications for non-funding. This "selectivity" is politically and bureaucratically appealing because it gives MRC and NSERC officials a feeling of being in control - of keeping a tight rein on the purse-strings to convince the Minister that they are disbursing research grants responsibly. However, as John Polanyi has said many times:

          "You must give the research horse free-rein if you want it to win the race".

       The way to give the health research horse free-rein is to mandate CIHR to provide baseline grants to all eligible applicants. The details of such a mandate may be left to the CIHR Council, but the mandate must clearly define "eligible applicant" as typified by a University Professor employed to engage in research within a field within CIHR's purview, and "baseline grant" as about the amount of money required for a Professor to direct the work of one graduate student (about $10,000 per annum).

2. The CONTEXT of Bill C-13

      Scientific investigation has led to advances in knowledge which have been applied to enhance people's lives. This process began about 400 years ago and has continued at an accelerated pace during the 1900s. The prospect is that it will continue indefinitely, and the Canadian Government rightly regards the provision of public funding for scientific research and technological development as a top priority (note the distinction between R and D).

      Public funding of R&D has been a policy of the Canadian Government throughout the latter part of the 20th century, and the CIHR proposal is one of several specific initiatives in recent years. One view is that these initiatives are an appropriate response to changing needs and circumstances, but they also raise questions about why previously enacted policies have proven to be less than completely satisfactory ? It is important to find the answers to these questions so that the causes of failure in past policies can be avoided in the future.

     This scenario of the intent of a enacted policy not being fully realized in its effects, must be familiar to seasoned members of the House of Commons; it is easy to compile the objectives of new legislation – much harder to determine how to achieve those objectives.

3. The Crucial NEED for the Amendment

      The Canadian Association for Responsible Research Funding (CARRF) was formed in 1993/94 by professors of (mainly) science, engineering and medicine, who had independently realized that the selection of NSERC, MRC and SSHRC, grant applications for funding based upon peer-review was counter-productive of the research discoveries that this procedure was supposed to accomplish; a case of the effect of a policy being the opposite of its intent.

      As it stands Bill C-13 will tacitly accept this common practice of the research councils as the basis upon which CIHR will distribute its available funding, and hence members of CARRF anticipate (the proof is the scientific study by Richard Gordon and Bryan Poulin) that the CIHR (like some other recent initiatives) will be a dismal failure.

This is why our Amendment to Bill C-13 is vital to the successful accomplishment of the Government's intent in creating the CIHR.

4. Political Motivations and the Bureaucratic Prerogative

      Selectivity of grant applications for funding or non-funding is ostensibly rational: not all research work produces new knowledge with useful applications, and who else is better fitted to judge which applications are more worthy of support than others, than scientific experts engaged in research themselves ?

      Selectivity and Peer-Review are politically pleasing because they create the impression that the Granting Agency (NSERC, MRC, SSHRC) is being very responsible by selecting applications for funding, rather than simply handing out money indiscriminately; the selection process appears to be a procedure for ensuring that the taxpayer's money is well-spent.

      The selection process also gives the granting agency a sense of doing worthwhile work. Several thousand applications (in multiple copies), a score of peer-review committees (each with a dozen or so members), create the need for a substantial bureaucracy ostensibly bent upon making the right decisions about which applications to fund.

5. Why Does Selectivity by Peer-Review Fail to Yield Innovative Discoveries ?

      Selectivity by peer-review is based upon the fallacy that peers (experts doing similar research) can objectively determine the worthiness of each applicant and proposal. The reality is that what each expert deems to be worthy of support is determined mainly by what interests this particular expert; a proposal that enhances the expert's own work readily receives endorsement, whereas new ideas are disparaged with scepticism. It is, after all, a competition for survival as funded researchers; the expert privileged to determine who survives and who does not, naturally favours his own survival over that of his competitors who threaten his existence with radical ideas.

      A case in point was cited by Daniel Osmond (Professor of Physiology, University of Toronto) in the CBC Quirks and Quarks program ("The Science Game", broadcast February 15, 1992) in which a young medical researcher's application for funding to MRC was turned down because the Peer-Review Committee decided that the Proposal "wouldn't work". During the 6-months that MRC takes for its annual grant-giving exercise, the young researcher proved (in the laboratory) that his idea did work. Regardless (and without recourse) he received the death sentence of the "nil" award, leaving him without any operating funds with which to continue his research. This is just one example (of many) of the killing of creativity by peer-review.

    In that same CBC broadcast (available on cassette tape from the CBC) Richard Gordon said:

"The peer-review system is one of the last bastions of slander and libel that is still legal",

and he might have added that the victims of that slander and libel have no effective recourse against it, for the perpetrators are protected with anonymity by the granting agency, and once unfunded the victims are deemed (by their peers and by the granting agencies) to be less excellent than the highly funded grantees who sit on the peer-review committees, and hence their complaints are typically brushed aside by callous bureaucrats whose raison d'etre is the perpetuation of the agency that employs them.

     This pervasive situation has earned the agencies a reputation for promoting dishonesty and corruption (conduct incongruous with the scientific principle of objective truth); the peer-review committees are widely perceived to be part of an incestuous, self-serving network otherwise known as "the old boys club".

6. Perversion of Scientific Research by Selective Funding

     All the agency hype about "excellence" disguises the reality of selective funding; that it is a witch hunt comparable with the Spanish Inquisition, in which the conformists survive (typically with a 4-year grant) while their victims (the agency insists that there be a substantial percentage of victims) are sacrificed on the holy altar of excellence with a "nil" award. This climate of repression (peer review has been equated with repressive bigotry) distorts the whole process of doing research; the professor's primary motivation is to stay funded, and hence his/her choice of research project is directed towards producing results that are readily publishable in respectable (i.e. conservative) refereed journals - the best insurance against the "nil" award.

      Thus research becomes Grant Driven, Grant Limited (the grant is fixed for 4 years), and (most pertinently) Grant Seeking; curiosity about the nature of things becomes secondary to the motivation to maintain funding. Money becomes the end rather than the means. Fixed funding encourages routine productivity of research results - adding to the pile of data without being concerned about innovative insights.

      The graduate students and post-docs who are employed to produce this routine data attest to the boring nature of this so-called "research" – they choose to work for the professors who base "successful" careers on routine research because these are the professors who have most of the Research Grant dollars to pay their stipends. Basic survival (paying student fees and putting bread on the table) naturally takes precedence over intrinsic scientific interest. NSERC and MRC applaud (and reward) such prolific producers of routine results for training many so-called "highly qualified personnel".

7. Productivity of the Public Investment

      First dollars are the most effective; with a limited budget, each professor must decide which of his projects to pursue based upon timeliness and promise, whereas a highly-funded researcher will pursue several projects concurrently (with the aid of paid assistants). It doesn't make economic sense to allow some (highly funded) professors to pursue secondary and tertiary as well as their primary research interests, while others (the "nil" awardees) are denied the opportunity to pursue even their primary project.

     As Bryan Poulin said during the Hearings (appended) nobody (no not even the most expert and "excellent" peer-reviewer) can predict which projects are going to lead to important discoveries, and which will not, and hence overall productivity of innovations is favoured by allowing all potentially innovative ideas to be investigated [hedging one's bets].

     The climate of competition (for survival as a funded researcher) encourages "empire building"; i.e. greed for ever larger grants with which to hire more assistants (students and post-docs) some of whose efforts are likely to lead to the publishable results that are the key to further success in the competition. This game of "grantsmanship" (akin to the parlour game of Monopoly) has no ethical legitimacy; more (research, students, post-docs etc) should not be equated with "excellence", and yet in the mercenary minded milieu of Canadian society, the pedestrian measure of money (in this context research funding) is too often equated with prestige as a scientist. MRC officially measures a researcher's "excellence" by the dollar value of his/her research grants – a monstrous affront to scientific integrity.

8. Baseline Grants - an Important First Step towards Ultimate Reform

      The Proposed Amendment to C-13 (to mandate that CIHR disburse baseline grants to all eligible applicants for funding) is a first step towards putting Canadian Scientific Research on the path towards being truly innovative and productive. Yet it is only a first step, for what is ultimately needed is a flexible system of funding whereby researchers can request additional funds (above the baseline grant) at any time as needs and opportunities arise. Such a system was proposed to the Canadian Government in October 1984 under the title: "Interactive Financing of University Research" (proposal to Gordon McNabb, NSERC President and Tom Siddon, Minister of Science by Geoffrey Hunter). In 1984 the internet was still in its infancy, but it was possible even then, and today its implementation would be quite easy. Grant applications for specific line-items such a student’s stipend or an equipment purchase should be responded to within 24 hours. Such an immediately responsive funding system will be a tremendous stimulus to innovative research, because it will switch the research climate from being Grant Driven, Grant Limited and Grant Seeking, to scientific idea and opportunity driven. It will also end the huge waste of both money and talent that is endemic in grants that are fixed for 4 years.

      Grants that are fixed for (typically) 4 years waste both money and talent; inevitably some researchers have more money in their research grant budgets than is currently needed, while others have insufficient funds; the former obviously is a waste of money while the latter is a waste of motivated talent.

     Providing money to sustain research when and where it is currently needed is the key towards creating a stimulating and creative environment for the pursuit of science; e.g. in 1947/48 two electrical engineers (Tom Kilburn and Freddie Williams) were engaged in trying to build the world's first stored-program computer; they were given carte blanche by their Minister of Science (whatever money they needed to assemble the thousands of electrical components), and in June 1948 they were successful in running their first program. This event marked the beginning of the computer revolution that in the year 2000 is pervasive and ubiquitous. It happened because those engineers were freed from concern about funding, and hence they could spend their time thinking about the scientific problem that they were trying to solve.

     Bryan Poulin refers to the low morale that is widespread in Canadian University faculties of science, engineering and medicine. This low morale is generated by the climate of competition for research funding that stems directly from the Research Councils' policy of Selectivity. Even people in the same department (some of whom are funded while others are not) live in a climate of competition, enmity and jealousy; this is antagonistic to the spirit of collaboration and cooperation that is essential to the enthusiastic pursuit of scientific knowledge and understanding.

      It is plainly stupid to employ professors (with typical salaries of $50,000 - $100,000) and at the same time deny them a basic operating budget with which to pursue the Research that they are expected to do. The primary concern when a professor is hired is his/her potential to do research (teaching ability is taken for granted - no teacher training is required). As professors who were hired in the 1960s and 1970s retire, their replacements are highly selected (typically 1 appointment from 50-100 applicants). All of these highly selected researchers deserve some baseline funding to allow them to maintain their research programs.

Geoffrey Hunter, Department of Chemistry, York University, Toronto. President: Canadian Association for Responsible Research Funding. Research on "The Nature of the Photon and the Electron" currently funded by NSERC at $18,000 per annum (1999-2003). email: ghunter@yorku.ca

APPENDIX 1 - Extract From:

http://www.parl.gc.ca/InfoComDoc/36/2/HEAL/Meetings/Evidence/healev07-f.htm

We'll move on now to Dr. Richard Gordon from the Canadian Association for Responsible Research Funding, again from the University of Manitoba.  Dr. Gordon, please.

Dr. Richard Gordon (Vice-President, Canadian Association for Responsible Research Funding; Professor of Radiology and Adjunct Professor of Electrical and Computer Engineering, University of Manitoba):

Thank you.  My name is Richard Gordon. I'm vice-president of the Canadian Association for Responsible Research Funding, which is known as CARRF. I'm a full tenured professor of radiology and also an adjunct professor of electrical and computer engineering at the University of Manitoba.

      It is important for you to know with whom you are dealing, so I'm going to give you a little bit of my background. My undergraduate degree is in mathematics and my Ph.D. is in chemical physics.

     I have two careers. In the field of embryology and evolution, I have just published this two-volume tome, which you may pass around if people want to look at it. The title is The Hierarchical Genome and Differentiation Waves. It shows how much of the mystery of the growth of embryos can be explained in terms of physics combined with genetics. It also explains how evolution has produced ever-more-complex organisms, leading eventually to human beings. It explains how an embryo forms many kinds of cells from one egg cell, and how this is likely to open up a new pharmaceutical industry.

      My second career is in medical imaging. In this I have concentrated for the past 23 years on the detection of breast cancer. I'm now working with a company with which I have designed a breast scanner that should detect small breast tumours before they spread. It will then zap them with X-ray beams. If it works as planned, it will cure this disease. (I hope you will put a rider on the CIHR bill to give me the $3 million we need to build and test this prototype.)

      CARRF is a gadfly organization of about forty professors spread across Canada, including five from Quebec. The president is Geoff Hunter, a chemist at York University. Alex Berezin, a physicist at McMaster, is our executive secretary. Dan Osmond, a physiologist at the University of Toronto, is also a vice-president. About ten people in other countries, in the National Cancer Institute of Canada, and in the Canadian Federation of Biological Societies listen in to our e-mail discussions and sometimes contribute their opinions. We have published many articles and even a forthcoming book. I brought a list of our publications, if anyone wants to look at it. We have been featured on CBC Radio's Quirks and Quarks twice.

     Some of us have been haranguing MRC and NSERC since 1983. In 1995 we incorporated in preparation for a case of one our members against NSERC in the Federal Court of Canada. We lost that one, but not our hope of someday reforming the grant system.

     Our opponents will portray us as losers. I have had fourteen federal grants myself, and the cumulative value of all my grants exceeds $1 million, which is not bad for someone who is primarily a theoretician. Many CARRF members are presently federally funded.

    What do CARRF members want? Simply put, we want all university professors who are paid part of their salaries to do scientific research and want to pursue it to be able to actually do some. The present federal granting system gives all of the money to a fraction of the scientists, and the rest get nothing - literally nothing.

     A survey done by Joe Pear and me in Manitoba last January showed NSERC funded 50% of their targeted academic scientists and MRC funded only 21%. Bryan Poulin, who you'll hear from next, and I have documented how innovative organizations give all of their scientists some time and money to play around with crazy ideas. A few succeed, and we have revolutions in industry, in health, and in our daily lives.

    No one can predict in advance which scientists will have the ideas that work. That includes scientists. MRC can't make these predictions, nor can CIHR. MRC and CIHR are stuck in a rut, wanting to peer-review every idea to death, no matter how fresh and untested. They need your help to get out of this rut. Without federal legislation via an amendment to the CIHR bill, Bill C-13, nothing will change. But if Parliament were to legislate that a fraction of the CIHR budget be shared equally amongst all health scientists, we predict a very substantial increase in Canadian innovation. Thank you.

The Chair: Thank you very much for those comments. We're certainly interested in your point of view, and you've expressed it, so we're grateful for that. We'll move on to Lakehead University. Dr. Bryan Poulin is an associate professor there.

Dr. Bryan J. Poulin (Associate Professor in Strategic Management, Faculty of Business Administration, Lakehead University):

Thank you very much. Honourable members of the Standing Committee on Health, it's a pleasure to address you.

     I don't come here presenting myself as knowing much about medical research, but over the last fifteen years I have looked at the innovation process. I did a masters degree in that and followed through with a doctoral degree on effective organizations and how to improve their morale, which was another point brought up by Dr. Glavin.

     We believe morale is a very important issue in any kind of research. One of the reasons why I got together with Dr. Richard Gordon is that I told him there were instances of the best organizational practices in which the support was radically different from what he was telling me occurred at MRC and the new organization that was going to replace it.

     I told him about 3M, which was primarily a commercial organization giving some free rein to scientists, inventors, and engineers at the idea stage. We used 3M as a baseline case because 3M is primarily a commercial organization, although I accept the point that we're not really looking at commercialization as a priority.

     My definition of innovation is somewhat different from what I've heard from Dr. James Turk, but it has some parallels. In my view, the innovation process starts with an idea, tests out that idea through the feasibility stage, and eventually leads to commercialization. We're not suggesting that anything like the majority of the effort would go to commercialization. Most should be in the idea stage and in the testing stage. That's the mandate of university research, medical research, and research of any kind. To my mind, it's publicly funded. However, it seems to me that the response of the ineffective organizations is to try to simplify things beyond the complex nature that exists.

      Such simplification is actually dangerous. The world is complex, research of any type is complex, and the phenomena we deal with are complex. Even the three-stage model that we've looked at is a simplification, but it's not a gross simplification in which you treat all the stages equally and peer-review everything to death before things get started. What we're saying here is that, at the idea stage, there is no way you can tell a priori whether an idea is going to reach the commercialization stage, nor is there any way that there are peer reviews in any sense of the term.

     Somebody having a new idea usually is considered, frankly, crazy. The person who invented or looked at electric phenomena in human species was accused of trying to play with frogs to make them dance. The thing is that when we do curiosity-driven research, it is very difficult...in fact, it is impossible to see where it's going to lead. Oftentimes in innovation, a real innovation comes from somebody who's aware enough to see something different and whose mistake can lead to the proper solution. Therefore, it's easy to critique something because of supposed errors. It's those very errors and the initiative in following the curiosity that often lead to a breakthrough solution.

      This is what we're proposing. We're proposing that we follow the innovation process and set in place something that actually mirrors that process. Chaos ought to reign in the idea stage. Then, over time, as it reaches the feasibility and commercial stage, we rein in the chaos. We don't rein in the chaos before it even gets started.

     That's what I see as so fundamentally flawed with the MRC's approach to peer-reviewing everything, especially the scientists who are going to have the breakthroughs, who are not going to be part of the old boys' network, who are not going to be part of excellence defined in terms of having received grants. Excellence ought to be the impact of one's career, not a particular stage of it, and especially not the beginning stage.

    Let's go back to morale. When we do things in a way that reflects reality, people get excited and empowered. The morale is low now, but I believe it's higher because we expect changes in the new CIHR. If those changes are not made in a meaningful way, that morale will collapse to the floor again.

     Richard and I are suggesting that what we need to do is mandate a little bit of seed money to the scientists to whom we already pay $60,000 and $70,000 to sit in their chairs with no equipment. We need to give them a little bit of the resources needed to get started. We can then have the ideas come forward, and the best ones can be vetted in a little bit more formal way through the feasibility stage. Then, in the commercialization stage, which is a very small part of the effort, we can get into very rigorous peer review. In fact, it will be not only peer review; it most likely will be industrial review. Thank you very much.

The Chair: Thank you for those comments, Dr. Poulin.

APPENDIX 2 - ABSTRACT of:

"How to Organize Science Funding: The New Canadian Institutes for Health Research (CIHR), an Opportunity to Vastly Increase Innovation".

by Bryan Poulin (Faculty of Business Administration, Lakehead University, bryan.poulin@lakeheadu.ca) and Richard Gordon (Department of Radiology, U. Manitoba GordonR@cc.UManitoba.ca)

Why are Bell, 3M and the intramural program at NIH (US National Institutes of Health) so successful at inspiring innovation? How does what MRC (Medical Research Council of Canada) is doing compare and, in fact conflict with that? Is the existing and proposed structure/culture of CIHR (planned to replace MRC) doomed to produce little innovation from the outset?

     Our examination leads to recommendations for a funding system and organizational structure for CIHR: minimal structure with baseline funding at the initial, idea stage (30-40% of budget); more formal structure and competitive funding at the feasibility stage (50-60%); and matching industrial grants for the commercialization stage (10%). Developing countries inaugurating their science funding that follow our recommendations may take off in science well beyond present expectations.

 


Two for the Price of One. Inferior Research and Inferior Advice 

Opening remarks by Donald R. Forsdyke of the Department of Biochemistry, Queens University to the Parliamentary Standing Committee on Industry, Science and Technology, relating to its study of the granting processes of the three federal research granting agencies. Ottawa 29th November 2001  [2 months after the 11th September attack on the World Trade Center]

 Those who are perceived as having attained excellence in a given field are most likely to be asked for advice on that field. They become labelled as "experts," and governments, assuming their advice to be the best available, call upon them to testify. If the subject were ice-hockey or ballet, perhaps Wayne Gretsky or Evelyn Hart would be sitting here. It follows that, if the processes by which the Gretskys and Harts become labelled as experts were flawed, then government would be less likely to get the best advice. Because ice-hockey players and ballet dancers are in the public domain, their excellence can be evaluated both by their peers, and by the public at large. However, when the subject is scientific research, this does not apply. We depend entirely on some form of peer review to judge excellence, and this process alone defines those who will be considered "experts." If the peer-review process were flawed then the Canadian taxpayer would be footing the bill not only for inferior research, but also for inferior advice.

     While we might, despite inferior research, survive in the short term by exploiting the originality and creativity of scientists in other countries, it is certain that, even in the short term, we will not survive in the 21st century with inferior advice. And I am not just talking about the gross domestic product and the quality of life. I am talking about survival itself! Problems such as AIDS, bioterrorism, genetically-modified food, mad cow disease and the poisoning of the biosphere, demand advice from those best equipped to address them.

     Forty years of involvement in biomedical research in the UK and Canada have led me to believe that peer-review as currently practiced is highly flawed. Government and, yes, the research councils themselves, are not getting advice from the brightest and the best. Indeed, a person incorrectly designated as "expert" is more likely to incorrectly designate others as "experts;" these, in turn, will incorrectly designate others, and so on. Thus, there is a built-in multiplier that threatens to send the entire system spiraling out of control. From some of the events we read of daily in our newspapers (Olivieri, Koren, Grinstein, Healy, Cuticchia, etc., some of these names have become household words) it might be inferred that the system is already out of control (1). Recent happenings at the University of Toronto have alerted the general public, which is now coming to suspect that the system is broken. If there is ever to be a time for discarding the honeyed assurances of the Neville Chamberlains and listening to the Churchills, it is now!

     I experienced the blitz in London during World War II. My father was torpedoed three times in the Atlantic. I graduated in medicine at London University, and gained a Ph.D. in biochemistry at Cambridge University. The great dangers threatening our civilization have seldom been far from my consciousness. Some of my first publications in the UK were in the journal of the International Institute for Strategic Studies. Here in the 1960s I warned of covert terrorism and of the dangers of biological warfare (2). When I arrived at Queens University in 1968 I envisaged that I would be able to balance the various demands on my time and, while continuing my research program and general university duties, would be able to continue developing my expertise on strategic matters. Thus, if called upon, I would be in a position to advise government. To my amazement, I found I had, like Alice in Wonderland, landed in a strange Mad Hatters world dominated by a peer-review system which has been described by Nobelist Joshua Lederberg as having become "vicious beyond imagination" and by Nobelist Phillip Sharp as having taken on a "mask of madness" (3). It soon became evident that I would need to attend to serious immediate problems in my own back yard rather than keep myself informed on bioterrorism.

     I had anticipated that I would be writing one grant application every 5 years, so that over a research career of 40 years I might have to write 8 applications. Instead, I found that I was sometimes writing as many as 8 applications a year! This was not exceptional. In a recent article geological engineer Kurt Kyser of Queens University estimates he spends a third of his time writing grant applications and submits up to 6 proposals a year (4). The taxpayer is footing the bill for all this! If you found that people you had hired to add a garage to your house were spending a third of their time playing cards you would loudly protest. The notion that writing a grant application is a useful exercise that helps an applicant sort out his/her ideas is false. Grant writing is an exercise in marketing and politics. It is playing cards. It has nothing whatsoever to do with creative research. Canada cannot afford to waste the time of its talented people in this way.

     In fact, the first rule of writing grant applications is not to be creative. As anyone can learn by reading accounts of great discoveries in the past, novel ideas are often difficult to articulate and difficult to understand. To put an original idea on a grant application is akin to professional suicide. People suffering the affliction of originality must either bring this deviant trait to order, or get out of scientific research. It might be thought that current peer review procedures, despite their flaws, are better than simply allocating funds by tossing a coin. But coin-tossing at least gives excellence a fighting chance. In fact, the current system is worse that coin-tossing since it actively selects against excellence (Click Here).

     In general and thankfully there are some exceptions we fund those who have found a safe academic haven by seeking more of the obvious. In the 19th century safe havens were found by describing new species of plants and animals, not by supporting Mendel and Bateson who were struggling to create the science of Genetics upon which much of the progress of 20th century bioscience came to depend. Today, the struggle is the same. Safe havens are found by describing genes preferably those with some disease connection and of interest to pharmaceutical companies not by trying to understand how our genomes work through the new science of the 21st century Bioinformatics. Until such an understanding is obtained, "quick-fix" attempts at gene therapy may open unforeseen Pandoras boxes.

     I do not just carp. I have come up with an alternative to the current peer review system which I call "bicameral review." Over past decades I have communicated this in briefs to research council executives. Much of it has been published as journal articles. Several of these articles have been collected together in a book entitled Tomorrows Cures Today? How to Reform the Health Research System (5). I have had a web-page on peer-review up and running for over 2 years (6). I am a founding member of the Canadian Association for Responsible Research Funding (CARRF). Reading transcripts of previous meetings of this committee I frequently found the remark, usually from Agency executives, that "the current peer review system, like democracy, is a terrible system, but it is the best we have." This is not true. An alternative, bicameral review, has been on the table for at least a decade!

     What is bicameral review? As the name suggests, there are two reviewing bodies, not one. One body is a committee of peers, as in the present system. The other is the Grant Agency itself. The information needs of the two bodies are different. This is because the grant decision-making process has two components, relating to the person, and to the project. There are just two questions: 1. Should this person be funded? 2. How much funding does his/her project need? Under bicameral review the first decision is made by the committee of peers, who only review the applicants track record, not the applicants proposed project. The second decision is made, in house, by specialists in the funding agency who, with respect to budget justification, only review the applicants proposed project, not the applicants track record.

     The underlying premise of bicameral review is that, for reasons spelled out above, evaluation of proposed research projects is highly error-prone. As any Bay Street analyst will tell you, the golden rules for operating in error-prone environments are just two. 1. Use only the most objective parameters. 2. Hedge your bets. In the context of bicameral review these rules translate into getting the committee of peers only to evaluate track record, and the grant agency to allocate funds on a sliding scale.

     Track record is assessed as a ratio of achievement to funds received. Researchers are made accountable. From those to whom much has been given, much should be expected. Armed with the peer-review ratings, the grant agency then decides what funds are needed and, following the hedging principle, disburses them on a sliding scale. Those at the top of the scale get 100% of what they are deemed to need. Those just below the top get, say, 80%. This progresses to the bottom where the applicant may get only 10% of what he/she needs.

     That in a nutshell is bicameral review. There are many details which are dealt with elsewhere (5,6). With bicameral review there would be more justice than under the present system. Now there is a sharp cut-off line. Those whose rating is a point below the line receive the same capital sentence as those at the bottom of the rating scale. Under the sliding-scale proposal, the "punishment" fits the "crime." Although there are those who will swear that their research programs will collapse unless they receive at least $100,000 a year, in fact, quantity of funds is most likely to determine the rate of progress, not whether any progress is possible at all.

     I can attest to this. I managed for many years with one technician who resisted the lure of more stably-funded laboratories and continued to work, sometimes for only half a day a week. We always made progress, albeit sometimes slow. Despite meager funding, I have published around 100 papers relevant to AIDS, cancer, and immune system diseases. In the nineties my bioinformatic analyses of genomes led to a new understanding of the problem Charles Darwin described as "the mystery of mysteries," the origin of species. In a recent book I have identified Darwins close research associate, George Romanes, as one of Canadas unsung heroes (7).

    A final word. Events of the last few decades have shown that small groups, with an imaginary or real sense of injustice and a little technical know-how, can disrupt peace in powerful ways. As antidote to imaginary injustice we must bolster our mental health system. But the only antidote to real injustice is to ensure that such injustice does not occur. You will recall the Fabricant case in 1992 when a professor went on a shooting rampage at Concordia University. Here the level of real injustice was sufficient to push over the brink someone who, in other circumstances, might have behaved sanely. We have been foot-dragging too long. Unless the problems of peer-review injustice are properly addressed we may find that the present anthrax scare is but the grim harbinger of even worse tragedies.

(1) Thomson, J., Baird, P., Downie, J. (2001) Report of the Committee of Inquiry on the case involving Dr. Nancy Olivieri, The Hospital for Sick Children, the University of Toronto, and Apotex Inc. CAUT, Ottawa.

(2) Forsdyke, D. R. (1966) Survival 8, 30. Letter to the Editor; (1969) Survival 11, 69-70. Review of Book: We All Fall Down. The Prospect of Biological and Chemical Warfare by R. Clarke. (Click Here)

(3) Lederberg, J. (1989) Does scientific progress come from projects or people? Current Contents. Life Sciences. 32, no.48. 5-12; Sharp, P. A. (1990) The crisis in funding: a time for decision. Cell 62, 839-840.

(4) Eggertson. L. (2001) NSERC may streamline processes to ease burden on researchers. University Affairs. November. p. 31.

(5) Forsdyke, D. R. (2000) Tomorrows Cures Today? How to Reform the Health Research System. Harwood Academic, Amsterdam.

(6) Forsdyke, D. R. Web-Page:(Click Here)

(7) Forsdyke, D. R. (2001) The Origin of Species, Revisited. A Victorian who Anticipated Modern Developments in Darwins Theory. McGill-Queens University Press, Montreal. (Click Here)

Click Here for Transcript of Entire Proceedings

 

REPORT

In June 2002 the above Standing Committee on Industry, Science and Technology published a report entitled:

Canada's Innovation Strategy: Peer Review and the Allocation of Federal Research Funds

The Chairman (Walt Lastewka, M.P.) opened by noting that "A wide range of views ... was heard; the Committee believes that this report captures the essence of those views."

    While some of my views were indeed accurately captured (p. 59), the evidence was incorrectly (see main Peer Review Web-Page) dismissed as "anecdotal" (p. 59). 

    Furthermore, it was suggested (p. 60) that under the system of bicameral review there was an "absence of technical experts to evaluate the proposed research." This was untrue. The technical experts were not absent. They were very much present. Their roles were to evaluate track-record, a task demanding all that their expertise could provide. 

    Sadly omitted was my most important point, - that our entire system of government, run by non-specialist politicians, depends on input from experts, and hence, is critically dependent upon the correct working of the system which designates individuals as "experts." We need the most expert "experts." Quite predictably, the Report ended up, yet again, rubber-stamping the status quo.

 


Click Here for Poulin and Gordon (2000) on Reforming CIHR

Click Here to Return to main PEER REVIEW page

Click Here to Return to Homepage


This page was last edited on 26 October 2014 by Donald Forsdyke