| Title |
Does an expert-based evaluation allow us to go beyond the Impact Factor? Experiences from building a ranking of national journals in Poland
|
|---|---|
| Published in |
Scientometrics, February 2017
|
| DOI | 10.1007/s11192-017-2261-x |
| Pubmed ID | |
| Authors |
Emanuel Kulczycki, Ewa A. Rozkosz |
| Abstract |
This article discusses the Polish Journal Ranking, which is used in the research evaluation system in Poland. In 2015, the ranking, which represents all disciplines, allocated 17,437 journals into three lists: A, B, and C. The B list constitutes a ranking of Polish journals that are indexed neither in the Web of Science nor the European Reference Index for the Humanities. This ranking was built by evaluating journals in three dimensions: formal, bibliometric, and expert-based. We have analysed data on 2035 Polish journals from the B list. Our study aims to determine how an expert-based evaluation influenced the results of final evaluation. In our study, we used structural equation modelling, which is regression based, and we designed three pairs of theoretical models for three fields of science: (1) humanities, (2) social sciences, and (3) engineering, natural sciences, and medical sciences. Each pair consisted of the full model and the reduced model (i.e., the model without the expert-based evaluation). Our analysis revealed that the multidimensional evaluation of local journals should not rely only on the bibliometric indicators, which are based on the Web of Science or Scopus. Moreover, we have shown that the expert-based evaluation plays a major role in all fields of science. We conclude with recommendations that the formal evaluation should be reduced to verifiable parameters and that the expert-based evaluation should be based on common guidelines for the experts. |
X Demographics
The data shown below were collected from the profiles of 5 X users who shared this research output. Click here to find out more about how the information was compiled.
Geographical breakdown
| Country | Count | As % |
|---|---|---|
| Poland | 3 | 60% |
| Ukraine | 1 | 20% |
| Unknown | 1 | 20% |
Demographic breakdown
| Type | Count | As % |
|---|---|---|
| Members of the public | 3 | 60% |
| Scientists | 2 | 40% |
Mendeley readers
The data shown below were compiled from readership statistics for 68 Mendeley readers of this research output. Click here to see the associated Mendeley record.
Geographical breakdown
| Country | Count | As % |
|---|---|---|
| Poland | 1 | 1% |
| Unknown | 67 | 99% |
Demographic breakdown
| Readers by professional status | Count | As % |
|---|---|---|
| Researcher | 12 | 18% |
| Student > Ph. D. Student | 12 | 18% |
| Librarian | 8 | 12% |
| Other | 7 | 10% |
| Student > Doctoral Student | 5 | 7% |
| Other | 12 | 18% |
| Unknown | 12 | 18% |
| Readers by discipline | Count | As % |
|---|---|---|
| Social Sciences | 18 | 26% |
| Computer Science | 8 | 12% |
| Engineering | 6 | 9% |
| Business, Management and Accounting | 5 | 7% |
| Arts and Humanities | 5 | 7% |
| Other | 14 | 21% |
| Unknown | 12 | 18% |
Attention Score in Context
This research output has an Altmetric Attention Score of 20. This is our high-level measure of the quality and quantity of online attention that it has received. This Attention Score, as well as the ranking and number of research outputs shown below, was calculated when the research output was last mentioned on 13 October 2017.
All research outputs
#1,630,012
of 22,950,943 outputs
Outputs from Scientometrics
#321
of 2,689 outputs
Outputs of similar age
#37,368
of 420,372 outputs
Outputs of similar age from Scientometrics
#17
of 82 outputs
Altmetric has tracked 22,950,943 research outputs across all sources so far. Compared to these this one has done particularly well and is in the 92nd percentile: it's in the top 10% of all research outputs ever tracked by Altmetric.
So far Altmetric has tracked 2,689 research outputs from this source. They typically receive a lot more attention than average, with a mean Attention Score of 10.7. This one has done well, scoring higher than 88% of its peers.
Older research outputs will score higher simply because they've had more time to accumulate mentions. To account for age we can compare this Altmetric Attention Score to the 420,372 tracked outputs that were published within six weeks on either side of this one in any source. This one has done particularly well, scoring higher than 91% of its contemporaries.
We're also able to compare this research output to 82 others from the same source and published within six weeks on either side of this one. This one has done well, scoring higher than 79% of its contemporaries.