1. WHAT'S IT ALL ABOUT? OR, OFF WITH THEIR HEADS!              

II. PRINCIPLE AND PRACTICE OF PEER REVIEW

III. BASIC INCONGRUITIES IN THE CURRENT SYSTEM

IV. WHAT ACTUALLY HAPPENS UNDER THE SYSTEM?

V. THINK BIG, STAY SMALL

VI. WHERE TO GO FROM HERE

VII. CONCLUSION

The players all played at once, without waiting for turns, quarreling all the while and fighting for the hedgehogs; and in a very short time the queen was in a furious passion and went stamping about, and shouting "off with his head"or "off with her head," about once a minute.

Alice began to feel very uneasy: to be sure she had not as yet had any dispute with the queen, but she knew that it might happen any minute, and then thought she, what would become of me? They're dreadfully fond of beheading people here: the great wonder is, that there's anyone left alive.

1. WHAT'S IT ALL ABOUT? OR, OFF WITH THEIR HEADS!

    Off with their heads! The decapitation principle of research funding is alive, well, and hard at work among scientists. It has been making a bloody mess of things for as long as I can remember and will go on doing so until we rise in revolt and scrap the guillotines.

    Doomsday heralds itself as hundreds of bleary-eyed researchers grunt and sweat their grant applications to funding agencies. Many have shortened or foregone vacations, sacrificed 1-3 months of productive research, and burned out themselves, as well as their families, secretaries, and photocopying machines. Courier services have frantically delivered the resulting multimillion dollar avalanche of paper to beat the witching hour deadline that chimes researchers into pumpkinhood.

    A lull now settles uneasily upon researchland. Then the colossal granting apparatus bestirs itself again and grinds its gears from "write" to "review." Not internal review, which precedes the big send-off, but external review by the granting agencies. In a twinkle, an army of scientists transforms itself Jekyll-Hyde fashion from meek overwrought writers to grim reapers slashing at the helpless paper victims before them. "Off with their heads!"

    Not a hint of this appears on the surface, of course. It's all done respectably, officially, scientifically. Even the symbolism is right -- the reviewers work in divine trinities, usually two external and one internal to the agency. As appropriate for divinities, they wield absolute power, especially the internal reviewer, who, like the Ayatollah, is more equal than the others. Such trinities make or break the fortunes of fine scientists, even those greater than themselves. They can put dedicated technicians out of work and scare the living daylights out of grant-supported fellows and graduate students.

    External appearances notwithstanding, the trinities are more often cruel than wise or gracious. Edification is not their game: it is easier to tear down than build up. Skill-testing words, like "Nonsense" or "Not enough numbers" or "Significance unclear" roll off their ballpoints with destructive ease. Such insults can be safely delivered anonymously, without necessarily even reading those long applications. Or if read, they can be read for flies in the ointment, without giving the ointment its due.

The Queen had only one way of settling all difficulties, great or small. "Off with his head!" she said without even looking around.

   This vicious process is quaintly called "peer review," evaluation of researchers by their equals. Its vile origins are murky -- few seem to know, but it's well ensconced, and the granting agencies are its major sculptors and guardians.

    At precisely defined regular intervals, tersely, and with doomsday finality, thousands of scientists learn of their financial decapitation.

    Whammo! The head rolls off. Your quivering body can apply again, but your blood will taint and haunt the next application. Or, you can cast about to private agencies in a desperate effort to regenerate. Additional precious weeks are lost in more frantic paperwork, consuming time desperately needed to make headway. Meanwhile, in the corridors of university buildings, whispering clusters of anxious technicians gather to count bodies and deliberate job prospects.

    But all is not gloom. Out there in Malice Land are Cheshire grins on researchers who didn't get chopped this time around. But the grins flicker uncertainly, then gradually fade away altogether. The heads that frame them know that the Queens' head-lust is insatiable. Their turn must come, and nervous jokes are uttered about Russian roulette. All energies bend to the careful grooming of the research -- like show dogs -- for next Doomsday. The long shadow of the guillotine haunts every grant-supported laboratory.

    Strangely enough, we have learned to live with decapitations, much as battered natives in backward countries tolerate goon squads and summary executions. The system has become a Witch Doctor before whom we are cowering natives. We believe there is no alternative. When his magic kills, it is the will of the gods -- and our own fault. Such enslavement is the ingredient for backwardness. To snap out of it demands critical analysis and courage to act decisively.

    And we must do both quickly. For the quality of research depends on proper distribution as well as availability of dollars. Availability is under government control, while distribution is primarily under ours as scientists. And peer review is the mechanism we operate ourselves. If we do it badly, every grant-supported scientist reaps the grim harvest sooner or later.

II. PRINCIPLE AND PRACTICE OF PEER REVIEW

"Let the jury consider their verdict," the king said for about the twentieth time  that day.

"No, no!" said the queen."Sentence first-verdict afterwards."

"Stuff and nonsense!"said Alice. "The idea of having the sentence first!"

"Hold your tongue!" said the queen turning purple.

"I won't!" said Alice.     

"Off with her head!" the queen shouted at the top of her voice. Nobody moved.

"Who cares for you?" said Alice (she had grown to her full size by this time)."You're nothing but a pack of cards!"

    I'm for peer review. Evaluation by one's equals is both essential and unavoidable wherever reputable science is done. Every researcher applying for grants has been evaluated since kindergarten, through doctoral and postdoctoral studies. Supervising professors, colleagues, scientific societies, students, and editorial boards have scrutinized, poked, and probed every strength and weakness of his personality and scientific endeavour. Every scientist applying for his first grant has already been reviewed for some 30 years, has jumped through umpteen flaming hoops, and has continued to do so with every manuscript submitted for publication in a reputable journal.

    The critical question is whether peer review by the funding agency adds a dimension missing from all previous reviews. If so, it is necessary. If not, it is superfluous -- at least at the level of intensity that now prevails. Over-review is as unproductive as under-review. It multiplies work and expense, setting one jury upon the next, level upon level, to the point where the final agency jury passes judgement, not only on the researcher but also on the competence of all preceding juries. All this happens without any guarantee of superior competence. Too often the evidence, as judged by their written comments, points to astonishing misunderstandings, if not incompetence, by reviewers.

    Indeed, the panels and study sections are not necessarily more expert, wise, innovative, far sighted, nor fairer than the trinities of journal editorial boards. For the critics to rule, in effect, that work of the caliber published in good peer-reviewed journals is unworthy of future funding is to set themselves above other experts – on their own say–so and without any come-back or accountability. No scientist can legitimately exercise so much influence; yet this is done all the time. A good applicant may have published 3-10 good papers in reputable journals during the past year and have much exciting work in progress but still be decapitated by colleagues working for the agencies.

    I can scarcely find words to describe such a questionable, dastardly, and potentially libelous process. The issue before us is not peer review. The issue is one particular system of peer review applied to that tiny promissory segment of a scientist's profile called "the proposal," with a heavy-handed impact that can cripple his morale and career. This system must go, and a better system of peer review must take its place before incalculable damage is done to research. We cannot excuse the status quo by pleading human frailty. Rather, we must have a system in which human frailties and their evil consequences are checked more closely. The system has sunk as low as it should go. We must now strive for superior application of the noble principle of peer review.

III. BASIC INCONGRUITIES IN THE CURRENT SYSTEM

A competition that isn't.

There was no "one, two, three, and away!" but they began running when they liked and left off when they liked, so that it was not easy to know when the race was over. However, when they had been running half an hour or so, and were quite dry again, the dodo suddenly called out "the race is over!" and they all crowded round it panting, and asking, "but who has won?"

    Various granting agency officers have made it abundantly clear over the years that their agency conducts competitions for funds. In competitions there must rightly be winners who get funds and losers who don't.

    What's missed however is that in any valid contest there must also be appropriate conditions (e.g., starting line, distance, goal, and time). The winner is the one who covers the distance to the goal in the shortest time.

An elimination that isn’t.

    It is not uncommon to observe a good scientist's promising work suffer from lack of funds, while next door a comparable scientist works overtime at year's end to try and dispose of his surplus.

Close rivals make bad judges.

The table was a large one, but the three were all crowded together at one corner of it.

"No room! No room!" they cried out when they saw Alice coming!

"There's plenty of room!"said Alice indignantly, and she sat down in a large arm chair at one end of the table.

"Have some wine,"the March Hare said in an encouraging tone. Alice looked all around the table, but there was nothing on it but tea. "I don't see any wine,"she remarked!

"There isn't any," said the March Hare.

"Then it wasn't very civil of you to offer it," said Alice angrily.

"It wasn't very civil of you to sit down without being invited," said the March Hare.

"I didn't know it was your table," said Alice, "It’s laid for a great many more than three. "

"Your hair needs cutting," replied the Hatter.

    In any valid contest the judges are on the sidelines -- not in the race itself. As Alice doubtless recognized, it's not easy to find a seat or refreshment when others at the table have themselves primarily in mind.

    Let's face it, reviewers and panel members who are expert enough to review a grant proposal are that knowledgeable because they are intimately involved in similar work. They do not relish being scooped by a rival. Or, they may strongly resist a rival's hypothesis that challenges their own.

    Moreover, fair minded as they may be, they are powerfully tempted to borrow ideas and secure advantage to themselves. Of course researchers, unlike most other mortals, would never succumb to such temptation! But still, it would seem prudent not to give them the chance!

    Peer review should not be used as a license to kill. Rivals should not be allowed to operate directly upon each other's budgets, but one or two steps removed. They may evaluate the science but not arbitrarily govern careers.

Proposals are not persons

"By-the-bye, what became of the baby?"said the cat. "I'd nearly forgotten to ask."

"It turned into a pig," Alice answered very quietly, just as if the cat had come back in a natural way.

"I thought it would," said the cat.

   A proposal is a promissory note by a well-intentioned scientist. It may convert into a great discovery, or turn into a pig. The first experiments could prove the proposal unfeasible or unworthy. In this case, the entire machinery will have travailed to deliver a pig.

    On the other hand, a scientist is a person with the training and qualities to salvage bad proposals and continually develop better ones. No one is more interested than the scientist himself in doing better science. He should not be judged and eliminated on the basis of a faulty proposal -- be that fault real or perceived -- any more than a tennis champion should be discredited for losing one serve or game.

    Everything tangible and objective in the scientist's grant application has to do with the person --education, training, character references, publications. The review process takes these into account, but only as back-up for the proposal, which it treats as the prime exhibit. This is topsy turvy. A good scientist should not be decapitated for a bad proposal. If the track record is good, with every sign of continuing performance, funding should continue. The proposal is but a mere sign post to where the person is going. Only such a reduced emphasis will eliminate the paper burden, cost, and travail of grant applications. So long as the proposal counts for almost everything, it will continue to be long, burdensome, and expensive.

IV. WHAT ACTUALLY HAPPENS UNDER THE SYSTEM?

Alice tried a little to fancy to herself what such an extraordinary way of living would be like, but it puzzled her too much: so she went on: "but why did they live at the bottom of a well?"

Take some more tea," the March Hare said to Alice, very earnestly.

"I've had nothing yet,"Alice replied in an offended tone: "so I can't take more."

"You mean you can't take less," said the Hatter, "It's very easy to take more than nothing."

    The unsympathetic reader may quibble that no human system is perfect, and that imperfect systems, like untuned cars, may still get us handily to our desired destination. May we hope to achieve more than we have? Or should we be grateful to have as much?

   Unfortunately, the most important question cannot be answered: namely, what the system has done for the quality of research. We cannot divine what would have happened under some other system. Nor can we rule out improvement by a new system. However, we do know what the day-to-day operation of the system produces, much as we can know the tragic daily tolls of war without full comprehension of its ultimate cause or effect. I'm afraid the indicators are not favorable to the agencies.

High costs of preparing and processing grant applications

    The costs of professional and secretarial salaries, photocopying, mailing, distribution and other administrative items incurred both at the universities and at the agencies, together with costs of travel, subsistence, and honoraria for members of reviewing groups, add up to a few thousands of dollars per application. The grand total is far in excess of $100 million per year for the U.S. and Canada, sufficient in itself to provide adequate funding for a large number of university laboratories that are now obliged to be inactive. It's not clear that the presumed gain in quality is worth so much money. Indications are that the bulk of unworthy applications could be sifted out by a simple in-house process.

    A month's labor per year per grant does not begin to define the professional burden of grant getting. The entire year is dominated by thoughts of preparation, and of the tragic consequences of refusal. Our graduate students observe our constant preoccupation with money and are turned off. From being scientists who enjoy doing their science, we gradually evolve into money-grubbing operators.

    Our science becomes ugly. Our life is measured by the size of grant and by the interval between applications. Relationships, even the honesty of our scientific discussions, are colored by the knowledge that everyone is a potential reviewer of our next application. We must do quick experiments, write them up fast, and publish, publish, publish. Yet the time available for reflective writing is punishingly reduced by the time given over to grant-getting. Innovative, time-consuming work must be done on the side with unbudgeted dollars to diminish the risk of rejection by an over-cautious grants committee when the work eventually surfaces. This has become standard practice.

    The "rat race" of grant-getting may paradoxically contribute to lack of productivity. Not every good scientist is psychologically equipped to live under the guillotine. Many become cynical, burn out quicker, or escape into clinical, administrative, or other avenues.

    The "rat race" creates an aura of "big science," whereas in real life the average grant obtainable most of the time is only sufficient for a modest operation. After all the fuss and bother, the average lab usually emerges with one or two technicians and a modest operating budget -- rather obvious predictable needs! Like the long-suffering citizens of troubled present-day Poland, we scramble constantly for the bare essentials of our professional lives.

   Society needs its scientific manpower to be working at full capacity and must be careful not to take away with one hand what it gives with the other. An abundance of traineeships at the intake should not be contradicted by a vicious elimination process at the output. Many first-class scientists are being eliminated completely, and numerous others discouraged into apathy.

Grantsmanship is not the same as scholarship

"Reeling and writhing, of course, to begin with," the mock turtle replied, "and then the different branches of arithmetic -- ambition, distraction, uglification and derision. "

"And how many hours a day did you do lessons?" said Alice.

"Ten hours the first day," said the mock turtle:"nine the next, and so on."

"What a curious plan!" exclaimed Alice.

"That's the reason they're called lessons," the gryphon remarked; "because they lessen from day to day. "

    Grantsmanship has become one of the games people play. There are good grant-getters who are uncreative researchers and poor grant-getters who enjoy greater respect -- as long as they last, of course! Their eventual loss to research constitutes an invisible, internal brain drain.

    Grantsmanship can become a substitute for scholarship and a bridge to academic tenure and promotion --"tenure by dollars." It is possible to "advance-by-grants," not just by one's own but also by those of a senior, productive colleague. It is also possible to advance by expensive instruments that every one else wants to use in their research. The resulting tidal wave of data washes the principal researcher onto the coveted shores of tenure-land.

   Faddism is also a common ingredient of grantsmanship. Find out who the big boys are, and what they like to hear most, and then let them hear it from you!

Grant applications are IOU's

    As previously stated, grant applications are little more than elaborate IOU'S. Their main value is the integrity of the scientist writing them, who could just as easily have charted his research without onerous applications. There is no compelling need to spell out one's plans every year like kindergarten pupils spelling their alphabet. If the scientist doesn't know what to do next, only additional experiments or peer advice will illumine him -- not the agency guillotine.

    If reviewers were geared to improvement of research proposals by generous input of fresh insights, it would be one thing. Scientists would be helped to do better science, and medical research would be enriched. There would be constructive dividends from the huge investment in peer review. But, at present, the system works destructively. The eliminated ones can write in to discover the stated reasons for the elimination, but these are more likely to be negative comments than constructive ones and would, in any case, not arrive in time to benefit the applicant.

Proposals are often history

    Because of the obsessive emphasis on what's on paper, it has become almost universal practice for researchers to "propose" what they have already done. The finesse demanded in applications can often only be achieved after the experiments have been completed. This is fundamentally dishonest. It is also extremely wasteful, because so much time and money is spent on evaluating the prospective value of work that has already been carried out!

Creation of anomalies

    The present system creates extreme anomalies. For instance, a creative, proven researcher may on the basis of a poor review be abruptly cut, or terminated. Technically, this constitutes a penalty for poor performance. Yet, the individual may have produced and published impressively during the past year. He may be onto exciting new discoveries, but all this is disregarded or not discovered in the first place. The faulted application cancels out all performance in real life. This ought not to be. An excellent person has not abruptly ceased to be excellent simply because the review says so. It makes no sense at all.

    One of the most striking anomalies in my experience is the case of a person who won a scholarship from one committee, but was denied operating funds by another of the same agency! These contradictory decisions held firm against all pleas!

    The present system undermines the morale of researchers, especially of research technicians. Very simply, the best, most ambitious technicians, with or without heavy mortgages in high-cost cities, will not entrust their futures to the whims of anonymous reviewers who can demolish their employer any year, even if he is among the best scientists in the land. Thus, the present system, willy nilly, erodes the quality of the research enterprise it is set up to safeguard. For the agencies to take the position that they do not employ technicians, only fund researchers (i.e. proposals), is incredibly short-sighted, considering that two thirds of grant monies go to pay technicians who do most of the bench work. Their quality determines the excellence of what comes out of our laboratories.

Least effectiveness where most needed

    A well-placed agency officer admitted to me recently that the system readily weeds out the very bad applications, but is questionably effective in discriminating among the good, better, and best.

   A practical question is whether the system's primary objective is merely to cull the bad from the good or to rank the good. If it is to cull the bad, all indications are that the system is unnecessarily sophisticated. If it is to rank the good, the system is not sophisticated enough, despite the huge investment in it.

    Several panelists have suggested to me that the vast majority of unfundable applications have easily recognizable characteristics. These often occur as a syndrome: sloppy presentation, poor literature review, inadequately-trained investigator, questionable environment, unrealistic or unspecified goals. poor publication record, poor experimental design and/or technique, etc. It would seem that a multidisciplinary in-house panel could easily pick out such applications. Borderline ones could be sent out for external review. This would greatly expedite the review process. The presence of an appeal mechanism would serve as an additional safety net against injustice.

    With the truly bad applications out of the way, more careful attention could be paid to ranking the good ones. The system needs to be much more effective where it is most needed, and if this cannot be achieved, then an alternate procedure should be sought along the lines suggested below.

    The present system of review excludes all local knowledge of the applicant. Even if a department is represented on a grants panel when an application from that department is under review, the representative has to absent himself. This may seem fair in one respect, yet is foolish in all others. For a grant proposal cannot tell all, anymore than a snapshot can. Local knowledge about a person's integrity, creativity, technical expertise, dedication, discipline, and other special attributes are profoundly important and must not be disregarded. Local advocacy should be included in an assessment, if only now and again, at certain critical stages, e.g. after tenure review. I know of scientists who won tenure and promotion on the basis of a stiff international review that established their credentials beyond any reasonable doubt. A few months later these same professors got rejection slips from a grant agency, which implied incompetence. Is there any objective reason for believing that the agency's assessment was more valid than the other? Not likely!

No defense

    Our legal system, as well as common everyday decency, precludes judgement of an individual without defense. Yet agency holy trinities perform character and professional assassinations by the hundreds every year without defense. Who of us hasn't ground his teeth in anger and frustration when we've read the cruel flimsy reviews upon which our applications were shipwrecked? If we could only have been there to rebut! But no, the gods have decreed otherwise, and the secret beheadings go on.

No review of the quality of review

    Year after year I've read my reviewer's reports with mounting despair. These people can write anything and get away with it! One even wrote emphatically that I should stop working with human plasma because it is too complex. How can one discover something in plasma (as I've done) without working with it? My grant was decimated that year. My discoveries have been amply confirmed and affirmed by others, worldwide, but my reviewers have not yet caught on.

    Peer review should cut both ways. I should be able to show up my reviewers when they utter abominable nonsense. We should have a system for disqualifying reviewers.

No continuity of review

    A proposal turned down one year for the wrong reasons registers unfavorably for the applicant. There can be nothing in one's favor if one was refused an award last competition -- and there could be some stigma to it. But suppose the applicant's viewpoint has since been vindicated. The appropriate thing would be for this year's panel to recognize that last year's panel had misjudged the case, and that some allowance should now be made. But no, this year's panel may have changed, the external reviewers may be different, and the error may be unfairly repeated.

   In this way, whatever memory there is in the system tends to work against grantees rather than for them.

No appeal system

    There is no appeal system. Criticisms are made, applications are turned down, the applicant and those connected with him are subjected to extreme anxiety and hardship, all as a result of a secret process carried on without defense or appeal. All this happens throughout their careers to highly selected academics who have repeatedly proved their worth. I suspect it is just a question of time before the legality of this medieval procedure is challenged. Sexual harassment in the workplace, about which so much has been heard recently, can scarcely be more damaging than unjust career harassment of men and women by unscrupulous reviewers working for inflexible, insensitive, bureaucratic masters.

The problem of plagiarism

V. THINK BIG, STAY SMALL

"Who are you?"said the caterpillar. This was not an encouraging opening for a conversation.

Alice replied rather shyly, " I -- I hardly know, sir, just at present -- at least I know who I was when I got up this morning, but I think I must have been changed several times since then.

"What do you mean by that?" said the caterpillar, sternly. "Explain yourself!"

"I can't explain myself, I'm afraid, sir," said Alice, "because I'm not myself, you see.

"I don't see," said the caterpillar.

   The granting agency system has additional undesirable side effects. Its insecurities generate inner compulsions to apply for bigger bucks each year. Big bucks afford greater protection from the guillotine. Dollars act as a security blanket, of which we crave more and more in a threatening system.

    One inevitable consequence of such cravings is the spawning of big-time operators. The system cultivates scientist-operators with the talent to capture money out of all proportion to their scientific achievements past, present, or future. Their glittering financial bandwagon rolls on triumphantly while the rest of us marvel like peasants of old at a king's procession.

    Such operators have also wielded influence to change our medical schools to suit their favor. Some schools now determine "base budgets" for their departments at least partly in terms of "peer-reviewed dollars" pulled in by their staff members. More grant dollars thus signify "better departments" deserving of bigger university operating budgets. This, in turn, enhances the department's capabilities in several areas, e.g., number of faculty members, secretaries, and office equipment.

    In other words, "peer-review dollars" are now driving the system in its administrative and teaching functions, in some cases overshadowing other valid indices of a department's capabilities and responsibilities for research and teaching. For instance, a department may embody professors who by all criteria are superb teacher-researchers carrying heavy teaching loads imposed on them by local imperatives. Their "peer-review" grants however may be modest, though proportional to the research for which they have time. In this case the department is not seen objectively for what it is, but as a "weak" department worthy of disdain, and more severe budget cuts than "strong" departments, which may do little or no teaching. Thus, a "weak" department is weakened even more, while the "strong" department is reinforced in its abrogation of teaching responsibilities. Intellectual assessment has given way to dollar assessment. Might becomes right, which is unworthy of the true university ethos.

    This situation prevails because so many of us have become king-worshippers rather than thinkers. We haven't twigged to the fact that there can only be so many chiefs for so many Indians, or that the latter may be more productive.

    Another aspect of bigness is that for every hundred, maybe thousand, aspirants, only a handful can actually become chiefs. All may hope, but the majority must remain Indians in a chief's world. We must think "chief," but remain "Indian," and as such, feel we have fallen short of some hypothetical ideal. But, in fact, the departments that hired us, set us up to work as small independent units. Tenure was awarded on the basis of independent performance. Thus, we observe a curious paradox. On the one hand, there is a wild competitive scramble for funds that will transform us from Indians to chiefs. On the other hand, our departmental circumstances, space limitations, over-zealous reviewers, and limited funds effectively keep us pruned to a modest size, creating a relative constancy on the research landscape.

    After all the dust settles following each head-rolling, nothing much really changes. Most of the grants fund 1-2 technicians' salaries plus modest operating expenses. The distribution of grants has altered somewhat, as in a game of musical chairs, without any real guarantee that current holders are superior to the dispossessed. And so the crazy game goes on from year to year.

    Note that the typical grant represents the basic needs of most researchers, most of the time. It is as silly that they should have to fight tooth and nail each year to secure these requirements as for a surgeon to have to compete for his instruments before every operation! So long as their productivity is good, researchers should get the basic resources needed for their jobs. Only beyond such basics should the competition make extra demands.

    A third aspect of thinking big and being small has to do with research groups. There prevails an enticing argument that well organized groups have a better crack at the Nobel Prize than "one-man operations." One might call this "Nobel Prize Engineering," and its resemblance to real science is purely coincidental. The rule is that money should go where the potential is for big-time achievement, and by implication, away from all those professors who make the mistake of functioning independently -- as they were set up to function by their departmental chairmen!

    In contrast, the history of Nobel Prizes suggests an uncharted road to great achievements. Inspired individuals, rather than scientific chain gangs, have accomplished most of the great things.

VI. WHERE TO GO FROM HERE

"Cheshire-Puss ,"she began rather timidly,"would you tell me, please, which way I ought to go from here?"

"That depends a good deal on where you want to get to," said the cat.

The Goal

    Where we want to get to, of course, is a quality of research that is equal, or superior to what we now have, under a system of funding that is more humane and much less expensive, burdensome, and distracting. In other words, we want more productivity with less overhead.

    The detailed road map to this goal is not easy to draw by a grantee who has no administrative experience within a granting agency. Even with such experience, the best plan devised would likely require a period of evolution before reaching maturity.

    Undoubtedly, this happened with the present system and would apply to any future system designed to replace it. So the possibility of some ambiguity and trial and error must neither surprise nor deter us.

Broad underlying principles

   The broad principles I work with are, first, that change from the present system is urgently needed. There is universal acknowledgment that the burden of writing and/or reading grant proposals (usually the same people do both) has become punishingly excessive. At this rate (and considering that most researchers additionally review a lot of journal manuscripts) we are in danger of totally preoccupying our most productive people with administrative, fundraising, and review activities. Real researchers are being degraded into paper pushers.

    Second, the destructively competitive aspects of the present system are not necessary for excellence to prevail. The decapitation policy of funding is bankrupt in principle and practice. Essentially all scientists of any worth are already highly motivated by the inner drives of scientific curiosity, humanitarianism, joy of discovery, ambition, and the desire for recognition by one's peers. We need no other monkeys on our backs or guillotine blades upon our necks. The sensory overload they produce is counterproductive. A new funding system may be governed by tangible evidence that these drives are operative without continually threatening our professional extinction.

    Third, peer review can be fully expressed in systems that differ from the present one. Modification or elimination of the present system is not the same as elimination of peer review. There is absolutely no excuse for any die-hard to oppose innovation simply because they love peer review so much. I also favor peer review, but want to see it applied sensibly.

    Fourth, scientists must take precedence over proposals. A good scientist will deliver good science overall, though his proposals would likely be a mixture of good and bad, just like a tennis champion's games. To continually sink scientists for flaws in their proposals (real or perceived) is ridiculous beyond words. The scientists themselves, or their scientific peers who see their work at meetings or on journal editorial boards, are better able to keep scientists on-track than grant reviewers.

    Fifth, a scientist's track record (not his proposals) is the surest indicator of future prospects. The track record of new scientists applying for funds for the first time resides mainly in their background, letters of recommendation, and early publications. Their research proposal(s) could be somewhat fuller in the first year or two to develop their organizing abilities and compensate for their short bibliographies. In the case of more established scientists the emphasis may shift to the quantity, quality, and special significance of publications, invitations to high level symposia, and other indicators (see Productivity Model, below).

    Sixth, research workers are selected by their institutions and not by the grant agencies. They have teaching, clinical, and administrative duties that hold them in place, such that "eliminations" are relatively uninfluential upon the system as a whole. The fundamental principle of agency competitions that the fittest survive while the unfit disappear does not regulate the system and should not be operated as though it did, and at cross purposes with university hiring. Grant winners rotate in a financial game of musical chairs based more on chance than sense. Losers neither disappear, nor improve, because even more of their time is sapped by efforts to secure funds from other sources. The new system must not be based on false premises regarding "natural selection" and must focus instead on maximizing the performance of real people, as they are and where they are, with the limited resources available!

    Seventh, the virtue upheld by some proponents of the present system that shrinking financial pies should be sliced-into fewer, larger pieces for the fittest must also be questioned. The performances of "giants" who have hogged larger pieces of pie have often been neither durable-nor excellent. Their moral and scientific right to deprive other excellent researchers of money is questionable. And the tradeoff against the lost achievements of all who have been deprived is incalculable.

What grant applications mainly try to achieve

    Consider also the fact that our university departments are already entrusted with staff recruitment, tenure and promotion, office and laboratory allocation, and provision of ancillary services (secretarial, glass washing, etc.) while animal and surgical facilities, stores, technical shops, computer facilities, etc., are provided by the Universities. All this represents a major proportion of total research costs, and it is eminently pertinent to ask why the remaining, i.e. direct, operating costs of research should not also be controlled by the departments, preferably with a degree of accountability to another body which could also provide for some additional directionality in the research effort.

    All the objections to the departmental model, especially nepotism, can also be applied to the agency models. A departmental chairman accountable to his dean, or to a senior departmental advisory committee, is no more likely to be misinformed or biased than a grants panel. They are likely to judge individual financial needs with fuller information at their disposal than do the agencies. They are better placed to shelter temporarily a promising young scientist whose list of publications has not yet matured, or a scientist of any age who experiences unexpected difficulties and needs a helping hand for a short time.

    I envisage a working partnership between departments and funding agencies. A new departmental appointee would be sponsored by the department for a 2-3 year start-up grant, which really should be quite automatic, along pre-arranged lines. It makes no sense to appoint someone on the one hand, and on the other hand deny him the tools of his job. Such support should not be difficult in times of relatively stable departments with few new appointments.

    Within 2-3 years, the new appointee has had the chance to demonstrate his productivity, and thus the level at which funding should continue. The funding agencies could graduate to a more creative scientific advisory role and reduce their bureaucratic paper shoveling. For instance, well respected retired and non-retired scientists in each region, or across regions, could be engaged at modest fees to conduct fairly regular site visits. These could be informal, and in some depth, focusing on labs, rather than the Dean's conference room.

    They should be non-threatening constructive experiences, especially for young scientists, who would welcome criticisms and suggestions. The site visit would enable the agency to fine-tune its perception of the scientist's "productivity index", the status of the department and, accordingly, the level of financial support required. For the life of me, I can't see what's wrong with being told by a respected site visitor, "Look, you're using the wrong methods," or "I've just visited 10 other scientists of your vintage who are publishing a lot more on a smaller budget -- could you please explain your difficulties?"

   Second, a signed critique is bound to be more scientifically responsible, which is surely needed when so much is at stake. Third, there is compensation for the vulnerability of self-revelation. What I lose (if anything) by signing my name, I gain back when my critics sign theirs. Fourth, the issue of signing is usually treated as a matter of personal preference, when in reality, it is not. It is a matter of what a reviewer wishes to do with someone else's future under the protection of anonymity. This is not an individual matter, like smoking, in which lung cancer remains one's own private risk. This is a matter of giving someone else lung cancer!

    Taking all factors into consideration, one could seriously argue that the innocent have nothing to fear when they sign reviews. Those who are adamantly opposed, whatever their stated reasons, may really be trying to hide carelessness or wrong-doing and should be disqualified. Scientific reputation is not the sole requirement for good reviewership. The reviewer also has to be fair, and if he is, there is no need for anonymity.

    Ideally, if the productivity index remains high, the appeal should not make the difference between funding and lack of it, but only influence the magnitude of budget cuts proposed on the basis of the criticisms.

    If a personal letter of explanation is impractical, a system of check boxes could be designed that would provide some explanation. Most of us could stomach being told that we're already well enough funded by other agencies, and that the funds we qualified for were used instead to "rescue" someone who had no other funds. In this case we would at least be able to judge whether to take the trouble to apply again next year.

    Individual sections of the application might receive a number grade, so that the applicant knows where his weaknesses are and can improve his application next time. For instance, the progress report might be very good but the description of methods might be inadequate. Or, the productivity index was low relative to other applicants. Injustice is less likely to be perceived if one finds out in what specific areas one's performance falls significantly below the funded group.

    But now the expenditure of money is under tight third-party scrutiny, and the comic pattern of frantic year-end spending is quite out of place. This is especially incongruous when next year's grant is small and a carry-over from last year would be most helpful.

    A month's cooling period has elapsed since the preceding sections were written. Valued feedback has come in. The main concern is not to permit vigorous statement of the issues to deflect attention from their validity. Have I exaggerated?

    The answer comes from the Mad Hatter. It's a matter of wasted quantity and quality of life, the enormity of which can scarcely be overstated. The productively hectic life degenerates into the wastefully frantic, which can only lead to early psychological exhaustion.

    As happened to hundreds of others, my August and September were consumed by that special frenzy that accompanies grant application. Six of us in the department fought like cornered rats against extinction-by-deadline as a prelude to possible decapitation by review. All of us are mature scholars with active programs and important discoveries to our credit. Yet none of us has a private secretary to ease the load. We share two typists with some 14 other academics who are also trying to get on with the normal work of researching and teaching some 2000 students in a busy Fall term. Many of us will face an October grant deadline as soon as this one is over.

    Quickly our two long-suffering typists are swamped; the overflow spills over into the labs, and then into the offices of hired typists. September is a total wipe-out of late nights, ruined weekends, and strained relationships. The mechanics of getting it all typed, corrected, photocopied, collated, signed by several officers, and away on time, have blanked out all creativity. Ten minute's worth of creative thought has spun out into an entrapping tangle of adrenaline-soaked make-work. More overload will come, as Winter follows Fall, when the shuffled applications return to us for review. All this will consume about one year in every four or five of our working lives.

    Meanwhile the real work piles up. It will extend the mayhem into the post-deadline weeks. There is a chapter to complete, several papers to write, and articles to review. I fend off irritated editors calling for their overdue manuscripts, and in a spare moment wonder what it would be like to think, read, or do some actual research again. I also wonder how I can possibly be trustworthy enough to review so many manuscripts and grant applications without qualifying for assured funds to carry on myself' Or, why in the next 20 years, regardless of performance, I shall still be wasting my time like this without relief in sight.

    Soon we shall have not just workshops on grantsmanship, but schools as well, churning out skillful writers. The first-prize graduate would get an agency panelship with license to decapitate. Second prize would permit the winner to do one month of research per year. Third prize would allow two months of research. The rest would be decapitated!