Grey literature and Multivocal Literature Reviews (MLRs) in Software Engineering

  in Special Issue   Posted on May 5, 2020

Information for the Special Issue

Submission Deadline: Mon 01 Jun 2020
Journal Impact Factor : 2.694
Journal Name : Information and Software Technology
Journal Publisher:
Website for the Special Issue: https://www.journals.elsevier.com/information-and-software-technology/call-for-papers/grey-literature-and-multivocal-literature-reviews
Journal & Submission Website: https://www.journals.elsevier.com/information-and-software-technology

Special Issue Call for Papers:

Grey Literature (GL) is defined as “… literature that is not formally published in sources such as books or journal articles” [1]. Given the scale of software engineering (SE) activities in practice and the large number of SE practitioners world-wide (some estimates are around 27 million), a great scale of knowledge/experience produced by SE practitioners is shared in the GL, e.g., as blog posts, videos, discussion on question-answer sites (such as StackOverflow), and white papers. Consequently there is a considerable opportunity for SE researchers to benefit from GL knowledge and data by using it rigorously in their research.

Systematic literature studies in the form of Systematic Literature Review (SLR) and Systematic Literature Mapping (SLM), also called Systematic Mapping (SM), are well-established in SE. The principal reference (guideline) for SLRs – the technical report by Kitchenham and Charters – recognizes GL as a source of evidence for an SLR. We and many researchers share the opinion that: “[if] used with care, grey literature can open up valuable additional sources of information for researchers” [2]. One way to utilize and benefit from GL in SE research is to conduct Multivocal Literature Reviews (MLRs) [3, 4]. An MLR is a form of a Systematic Literature Review (SLR) which includes the GL (e.g., blog posts, videos and white papers) in addition to the peer-reviewed academic literature (e.g., journal and conference papers). MLRs are useful for both researchers and practitioners since they provide summaries of both the state–of–the art and the state–of–practice in a given area. MLRs are popular in other fields, e.g., medicine and social science.

Guidelines for MLRs and Grey Literature Reviews (GLRs) [4], as well as studies conducting MLRs and GLRs, are beginning to emerge in SE. According to Garousi et al. [4], at least 10 MLR/GLR papers were published in different areas of SE between 2013-2018, most of which have been published more recently.

At the same time, there are risks and concerns relating to the credibility of the GL used in MLRs and GLRs, and about the credibility of GL in general. For these reasons, we believe it is timely for the SE research community to look carefully at the value of GL to SE research, and to examine the methodological issues as well as the path forward in this important area. This Special Issue on GL, MLRs and GLRs in SE will provide the opportunity for the SE community to publicly debate this issue.

More specifically, amongst other benefits, the Special Issue will:

This CfP is seeking contributions on research, practice, and compelling new ideas pertaining, but not limited, to the following topics:

  • MLRs / GLRs in different sub-disciplines of SE. Any submitted MLR/GLR paper should include at least one section that reports experience, challenges and/or lessons learned in the MLR / GLR process
  • Papers focusing on the methodological issues in studying GL
  • Studies assessing the value and benefit of GL, MLRs and GLRs to practitioners and/or to researchers, e.g., [3]
  • Studies that examine how research using MLRs and GLRs can help close the gap between industry and academia in software engineering.
  • Studies investigating the rationale and nature of information needs and knowledge sources for practitioners and researchers, as these two communities mainly tend to refer to GL and to academic literature, respectively
  • Papers that examine how to interconnect GL and academic literature more closely: discussing the implications of and risks arising from the current situation in which practitioners and researchers write and disseminate knowledge in separate venues and channels, either in GL or academic literature, respectively. The current situation further leads to the disconnect between industry and academia. Why do the sources in GL and academic literature cite the sources in the other community, very sparsely? How can we, as the SE community, improve the situation?
  • Tool-support for collecting, storing and analyzing GL, and for designing, conducting and disseminating MLRs and GLRs. Tool papers should also include an empirical evaluation or detailed demonstration of the real application of the tool/s
  • Studies examining the effectiveness of the existing MLR guidelines, e.g., [4]. This could, for example, be undertaken using opinion surveys, studying the effectiveness of the existing MLR guidelines, with subsequent suggestions and proposals to revise and improve guidelines. As an indicative example of such studies, a recent study [5] conducted an opinion survey on the effectiveness of guidelines for “replications” in SE

Timetable:

Paper submission deadline: June 1, 2020

Notifications to the authors (initial round): October 1, 2020

Deadline to submit revised papers: December 1, 2020

Notifications to the authors (final outcome): February 15, 2021

Submission of camera-ready papers: March 30, 2021

Submissions instructions:

Manuscripts should be submitted through the EVISE system via following link: http://www.evise.com/evise/faces/pages/navigation/NavController.jspx?JRNL_ACR=INFSOF Authors must choose the “GL-MLR-in-SE” from the “Issue” dropdown list when submitting the submission, and follow the instructions in the system.

Word limit of submissions: As per the IST’s author guidelines, please also note that the maximum length for a research paper is 15,000 words with the exception for systematic literature review or systematic mapping studies where the maximum length is 20,000 words. Also, notice that figures and tables count 200 words each.

Guest editors:

Vahid Garousi
Queen’s University Belfast
Belfast, Northern Ireland, UK
v.garousi@qub.ac.uk

Austen Rainer
Queen’s University Belfast
Belfast, Northern Ireland, UK
a.rainer@qub.ac.uk

Michael Felderer
University of Innsbruck, Innsbruck, Austria & Blekinge Institute of Technology, Sweden
michael.felderer@uibk.ac.at

Mika V. Mäntylä
University of Oulu, Oulu, Finland
mika.mantyla@oulu.fi

References:

  1. C Lefebvre, E Manheimer, and J Glanville, “Searching for studies,” in Cochrane handbook for systematic reviews of interventions, J. P. T. Higgins and S. Green Eds.: Chichester: Wiley-Blackwell, 2008.
  2. Cindy Elliott, “Jinfo Blog: Garner Additional Research Sources with Grey Literature,” https://web.jinfo.com/go/blog/70203, 2019.
  3. Vahid Garousi, Michael Felderer, Mika V Mäntylä, and Austen Rainer, “Benefitting from the grey literature in software engineering research,” A book chapter of “Contemporary Empirical Methods in Software Engineering”, In Press, 2020
  4. Vahid Garousi, Michael Felderer, and Mika V. Mäntylä, “Guidelines for including grey literature and conducting multivocal literature reviews in software engineering,” Information and Software Technology, vol. 106, pp. 101-121, 2019.
  5. Jeffrey C. Carver, Natalia Juristo, Maria Teresa Baldassarre, and Sira Vegas, “Replications of software engineering experiments,” Empirical Software Engineering, vol. 19, no. 2, pp. 267-276, 2014.

Other Special Issues on this journal

Closed Special Issues