| Arviointilomake ja ohjeet |
University of Helsinki
Research Assessment Exercise 1999
GUIDELINES FOR EVALUATION
EVIDENCE AND CRITERIA FOR ASSESSMENT
The basic unit to be assessed and to receive a numerical rating is a department. All departments have drawn up a list of their research active staff for the period 1994-98. The list includes automatically some categories of staff (e.g. all professors, senior assistants, postdoctoral research fellows). From some other categories (e.g. full-time teachers and lecturers, junior assistants, postgraduate research fellows) the departments have been free to choose those whose research output they wished to submit for the assessment.
Each department has compiled a select bibliography of its most important research output between 1994-98. The bibliography contains one work from each person listed as research active. In addition, the department has included a selection of its best works so that their number equals the number of research active staff. Thus, the total size of the selected bibliography is normally twice the number of research active staff. In this way it reflects both the average quality of each unit's work and its best achievements. In principle, the panels will consider the work of a department as a whole to reach a view as to its overall quality profile. However, they should also compare the bibliography with the list of research active staff in order not to overlook the presence of staff who have been employed for several years during the assessment period but whose current individual output is unimpressive.
"Research" for the purpose of this evaluation is to be understood as original investigation undertaken in order to gain knowledge and understanding. It includes work of direct relevance to the needs of culture and society. The panels should base their ratings primarily on the submitted selected bibliography, but also on other submitted material, and on the impression they receive during their site visit. Among others, the following criteria should be applied:
- originality, depth, and breadth of past (1994-98) and ongoing research activity
- current importance of the research themes
- multidisciplinarity and relevance for other research areas
- education and training of researchers (postgraduate and postdoctoral)
- interaction and collaboration with foreign researchers and research units
RATING SCALE AND DESCRIPTIONS
The research activities of each assessed unit should be rated according to the following scale:
7 The majority of the submitted works are at a high international level and virtually all others at a
good international level.
6 At least one third of the submitted works are at a high international level and many others
at a good international level, these together comprising a clear majority.
5 The majority of the submitted works are at least at a good international level and virtually all others at a fair international level.
4 At least one third of the submitted works are at a good international level and many others
at a fair international level, these together comprising a clear majority.
3 The majority of the submitted works are at least at a fair international level.
2 A minority of the submitted works are at a fair international level.
1 None, or virtually none, of the submitted works are at a fair international level.
High international level means work which is apt to arouse serious interest within international academic communities and which in principle could, if offered, be published by the leading international publishers or in the leading international journals with the most rigorous editorial standard (but irrespective of where it has actually been published).
Good international level means work which is of undisputed relevance for international academic communities and which could be published by well-known international publishers or in well-known international journals (but irrespective of where it has actually been published).
Fair international level means work which is of possible relevance for international academic communities and which has been published abroad or by well-known national publishers or in well-known national journals.
The panels should not give one of the two highest ratings (7 or 6) unless they believe that the assessed department has to be ranked among the qualitatively best 10 % (rating 7) or 25 % (rating 6) European departments in its field of academic research.
The ratings refer to an absolute standard of quality in each department. They should be related only to the size of the research active staff, but independent of the conditions for research (such as teaching load or funding) within individual departments. The presence of a high proportion of young or recently established researchers may, for example, explain why the average output of a departement remains below expectation. The panels are invited to record such special circumstances in their written report, but they should not have an effect upon the rating.
For the purposes of the above definitions "international journal" refers to a journal whose main audience is an international academic community, irrespective of the country where it has been published. "National journal" refers to a journal whose main audience is the Finnish academic community. In principle, the media where the output of a unit has been published should not have an effect upon its rating, because the rating scale is based on the inherent quality of the output. However, the panels should mention in their written statement if they believe that the work of an assessed unit is regularly published in a way which does not favour a genuine dissemination of ideas.
STUDIES WITH A NATIONAL BASIS
High international level refers to a level that is reasonable to expect for the unit. This may sometimes be difficult to define, especially when assessing studies with a national basis. In the absence of current examples, standards in cognate research areas where international comparisons do exist will need to be adopted. Thus, research on Finnish law should be compared with e.g. research on German law in a German university, or a department of Finnish language with a department of French in France or a department of Swedish in Sweden.
The panels will be aware that in some fields of research much work of high quality can be done which, because of its subject matter, may not arouse active interest outside Finland. In many cases it is not feasible to publish the results of such research in languages other than Finnish or Swedish. Although these studies might not be published by leading international publishers, they may still provide evidence of international excellence if they can be compared favourably with similar studies in other countries. Thus, the choice of a nationally justified topic should not have a negative effect upon the rating. However, if a department fails to produce any serious work with a wider international interest the panel is asked to consider carefully the reasons for that and may take it into account in determining the rating for the unit. In case the limited international interest in a particular output is due to a real lack of depth and originality, it should have a detrimental effect upon the rating.
ADDITIONAL QUESTIONS TO BE DISCUSSED
In addition to the numerical rating, the panels are asked to discuss in their written report of each assessed unit the following questions (when applicable):
1. The particular strengths and weaknesses of the unit. The panels should distinguish, as much as possible, between sub-areas of research activity within the unit. This is particularly important when a department is large and/or consists of heterogeneous smaller units (laboratories, individual disciplines).
2. Adequacy of resources (personnel, libraries, equipment, technical support, funding) for the research and efficiency in their use.
3. Special reasons why the unit has not been more productive during the assessment period and/or suggestions how the unit could improve the quality of its research.
4. The success of the unit in publishing the results of its work. (How active are the staff in publishing their results? Have the best possible publishing channels been used? Are the researchers and their work widely known among colleagues?).
5. The panels have been asked to adopt an international perspective in rating the research activity of all the departments (see above). Would the adoption of a different (national) perspective change the evaluation of the assessed unit? The existence of very different scientific traditions within the field may also make it difficult to determine who exactly is the international academic audience. The panel may here state if it feels that the rating has been particularly problematic in this respect.