Saturday 9 July 2016

Can I reliably determine the quality of a journal by checking citation numbers in Google Scholar?


I have recently come across this journal International Journal of Advanced Computer Science and Applications(IJACSA)


The journal provides acceptance notification within 15 days. This raised red flags, but then I performed a Scholar Search for thesai.org and seems that most of their papers have decent citations. So, how reliable is scholar search when evaluating the quality of a journal or how to interpret the results from scholar search ?


EDIT: As pointed out by Nate Eldredge in Publishing again acknowledging the original publication of oneself this journal is on Beall's List of Predatory Open-Access Publishers. But this again raises the question how reliable is scholar search then or what are ways to search reliably for a research papers.



Answer




Even in case you generally accept numerical measures such as citation counts or the h-index as a measure of quality, Google Scholar seems to be rather unreliable in them. In the case of evaluating individual authors, let me cite the Wikipedia article on Google Scholar:



Vulnerability to spam — Google Scholar is vulnerable to spam.[26] Researchers from the University of California, Berkeley and Otto-von-Guericke University Magdeburg demonstrated that citation counts on Google Scholar can be manipulated and complete non-sense articles created with SCIgen were indexed from Google Scholar.[27] They concluded that citation counts from Google Scholar should only be used with care especially when used to calculate performance metrics such as the h-index or impact factor. Google Scholar started computing an h-index in 2012 with the advent of individual Scholar pages. Several downstream packages like Harzing's Publish or Perish also use its data.[28] The practicality of manipulating h-index calculators by spoofing Google Scholar was demonstrated in 2010 by Cyril Labbe from Joseph Fourier University, who managed to rank "Ike Antkare" ahead of Albert Einstein by means of a large set of SCIgen-produced documents citing each other (effectively an academic link farm).[29]



I'm not aware of a case where this has been exploited by a predatory journal, but if a publisher wants to do that, it doesn't seem to be too difficult.


No comments:

Post a Comment

evolution - Are there any multicellular forms of life which exist without consuming other forms of life in some manner?

The title is the question. If additional specificity is needed I will add clarification here. Are there any multicellular forms of life whic...