Despite review panels being the foremost common manner of choosing candidates for analysis funding, very little is thought concerning however alternatives square measure created. New strategies for large-scale text analysis provide review panels’ written reports to be analysed and studied for patterns. Peter van den Besselaar and Ulf Sandström show however the frequency of positive and negative analysis words correlate with applicants’ final scores, and conjointly reveal however panels’ apparent political orientation implies that the doubtless groundbreaking, high-risk/high-gain analysis is unlikely to advance through the method and be elite for funding.
Despite several studies showing the weaknesses of peer and panel review, it remains the dominant sort of choosing candidates for positions and grants. However, quite what kinds of processes occur in review panels isn’t documented. Observation studies would be the thanks to investigate this, however these square measure terribly rare and people that square measure offered square measure typically small-scale. It looks troublesome for councils to permit for large-scale observations. One might argue that senior scientists have served on several panels and then ought to have enough insight; but, perceptive a panel is totally totally different from being on a panel whereas attempting to try to to a decent job.
Fortunately, with the provision of latest strategies for large-scale text analysis, another manner of learning panel processes is to analyse the written reports. in an exceedingly study that specialize in gender bias in grant allocation at the ecu analysis Council, we tend to obtained all review reports of each flourishing and rejected candidates (in total three,030 candidates with between four and eight short reviews and a brief outline review report) for more analysis. we tend to conducted a linguistic analysis of the reviews, victimization LIWC – a frenzied tool for such analysis. The tool distinguishes a spread of linguistic classes, every consisting of an oversized set of words. For our study we tend to used linguistic classes assumed relevant for review texts, among others: negation words; negative analysis words; positive analysis words; superlatives, research words; data words. The question is whether or not we will predict the score associate application gets from the relative frequencies of the linguistic classes within the reviews, and if so, what will this tell concerning the panel process?