Objectives

Research assessment is a scientific task that entails relevant social aspects, as it aims at evaluating the scientific activity of each researcher, involves the academic Departments activity and the cultural policies of the academic authorities. The new bibliometric indicator for research evaluation in SSH, Library Catalog Analysis, is a matter of importance in the KO field, as it involves library acquisition procedures and the realization of OPAC.

The evaluation of research in SSH is founded on the analysis of books as product of research and the expert peer review is the strategy currently adopted. Henk Moed (2005) reported the problems that arise using in SSH fields bibliometric counts based on citations. Daniel Torres-Salinas and Henk F. Moed (2009), Adrianus J. M. Linmans (2010), and Howard D. White (et al., 2009) autonomously proposed the Library Catalog Analysis[1]. It consists in using the number of copies of books in libraries as a new bibliometric approach in SSH, with the aim to evaluate the authors’ productivity and their scientific level. However, the literature has not offered a wide empirical analysis over a range of disciplines of the potential and limitations of that strategy. Therefore, a deeper examination of scientific fields different from those tested by the mentioned authors is required.

This paper presents the concise results of an Italian survey on the diffusion in Italian and foreign library holdings of books published by Italian scholars in two scientific fields: History of books, Bibliography, Library Science and History of political institutions. The survey, sponsored by the Italian Agency for the evaluation of the University and the research (ANVUR), was carried out in 2015 and 2016 with the purpose to verify the possibility of using the results of the analysis as an indicator in SSH research assessment.

Methods

Instead of using unit catalogues (Torres-Salinas and Moed used WorldCat), we preferred to verify the presence of books in a set of OPAC we chose as representative of different environments: 9 foreign libraries and 13 Italian, including national libraries and libraries of prestigious European and US universities. Therefore, our work was not restricted to offering only quantitative results, but rather acquired a qualitative dimension. We set up a database of monographs published between 2000 and 2014 selecting them from the Italian SBN[2] collective catalogue. 563 units: 279 in History of books, Bibliography and Library Science, and 284 in History of political institutions.

Main results

First, we underlined that key factors conditioning the inclusion of monographs in library holdings are numerous, varying according to the type of library and being often influenced by contingencies. Second, in evaluating the inclusion of books, we considered the incidence of gifts. None of the authors here mentioned (Torres-Salinas and Moed, Linmans, White), who proposed library holdings as a reliable indicator of the scientific level of monographs, considered the incidence of gifts in library holdings. Third, the analysis of the holdings of Italian and foreign libraries allowed us to have a wide view and to compare different realities and make an articulate analysis. Our examination revealed a very mixed picture and a substantial difference in the presence of the monographs in the libraries in which the surveys were completed. This paper offers Tables of distribution of presence of monographs in the Italian and foreign libraries. In our survey we found that the occurrences are not always the result of conscious choices, but are influenced by several factors, such as cooperative acquisition policies of publications, purchasing plans, approval plans and patron driven acquisition or particular forms of giving.

Moreover, the survey highlighted the inaccurate present management of identifiers of the entities registered in the bibliographic records. This paper underlines that the application of RDA and Linked Open Data technologies will enable linking to external repositories to identify the data and “qualify” the relationships between them, using specific domain ontologies (e.g., FRBR, RDA etc.).  Authority control is a crucial point for purposes of scientific evaluation. It enables to precisely identify each entity of interest (the author and the publication) ensuring the correct attribution of the works to the respective authors. The collaborative nature of the Semantic Web introduces a new distributed vision of authority control, which no longer depends on the national agencies, but involves subjects of different nature. The new perspectives of LOD are oriented in two main directions: the use of unique and persistent identifiers in authority data and the reconciliation of different identifiers.

The research also examined the nature and operation of discovery tools in order to understand if these research tools can affect the qualitative and quantitative outcomes of research into monographs. A sample of monographs of the database used for our survey was selected for testing discovery. The results showed that searches through the online catalogue are to be preferred for quantitative and qualitative assessments, in particular for the comprehensibility of the document retrieval techniques, entirely managed in the field of Library and Information Science.

Conclusions

In short, our survey found that:

1. The criterion of the presence of monographs in libraries cannot be adopted to establish the scientific value of monographs and, therefore, of their authors.

2. However, it may be claimed with reason that verifying the diffusion of monographs in prestigious libraries, selected on the basis of well-balanced criteria, can constitute a reinforcing element of a positive or negative judgement that has already been formulated, after a well-managed peer review.

3. We analysed the presence of monographs in a range of unit catalogues and OPAC belonging to national and university libraries, Italian and foreign. Thanks to a deeper analysis on different library types, it has been possible to offer a broader picture and to better verify the possibility of using that indicator.

4. We demonstrated the effect of gifts of monographs in library holdings.

References

Beyond Bibliometrics: harnessing multidimensional indicators of scholarly impact, edited by Blaise Cronin and Cassidy R. Sugimoto, Cambridge (Mass.) - London, The MIT Press, 2014.

Marshall Breeding, The future of library resource discovery, Baltimore, NISO, 2015.

Myung-Ja Han, New Discovery Services and Library Bibliographic Control, “Library trends”, 61 (2012), 1, p. 162–172.

Adrianus J. M. Linmans, Why with bibliometrics the Humanities does not need to be the weakest link. Indicators for research evaluation based on citations, library holdings, and productivi­ty measures, «Scientometrics», 83 (2010), 2, p. 337-354.

Henk F. Moed, Citation analysis in research evaluation, Dordrecht, Springer, 2005.

Francis Narin, Evaluative Bibliometrics: the use of publication and citation analysis in the evaluation of scientific activity, Computer Horizons Inc., 1976.

Daniel Torres-Salinas, Henk F. Moed, Library Catalog Analysis as a tool in studies of social sciences and humanities: an explo­ratory study of published book titles in Economics, «Journal of Informetrics», 3 (2009), p. 9-26.

Daniel Torres-Salinas [et alii], Coverage, field specialization and the impact of scientific publishers indexed in the Book Cita­tion Index, “Online Information Review”, 38 (2014), 1, p. 24-42.

Yongming Wanga, Jia Mi, Searchability and discoverability of library resources: federated search and beyond, “College & Undergraduate Libraries”, 19 (2012), 2-4, p. 229-245.

Howard D. White [et al.], Libcitations: a Measure for Compa­rative Assessment of Book Publications in the Humanities and Social Sciences, “Journal of the American Society for In­formation Science and Technology”, 60 (2009), 6, p. 1083-1096.

Alesia Zuccala, Raf Guns, Comparing book citations in humani­ties journals to library holdings: scholarly use versus “percei­ved cultural benefit”, in Proceedings of ISSI 2013 Vienna: 14th International Society of Scientometrics and Informe­trics Conference, Vienna, Austria, 15th to 20th July 2013, editors Juan Gorraiz [et al.]. Vienna, Austrian Institute of Technology, 2013, vol. I, p. 353-360.

Alesia Zuccala, Howard D. White, Correlating Libcitations and Citations in the Humanities with WorldCat and Scopus Data, in Proceedings of ISSI 2015 Istanbul: 15th Internatio­nal Society of Scientometrics and Informetrics Conference, Istanbul, Turkey, 29th June to 4th July 2015, editors Albert Ali Salah [et al.]. Istanbul, Boğaziçi University Printhouse, 2015, p. 305-316.

[1] According to Linmans, Library Holdings Analysis. Recently, Alesia Zuccala and Raf Guns (2013) adopted a similar bibliometric approach, the Libcitation count.

[2] http://opac.sbn.it/opacsbn/opac/iccu/free.jsp