Using AI to Make Literature Reviews Smarter and More Efficient

Marin-Garcia, J. A., Martinez-Tomas, J., Juarez-Tarraga, A., & Santandreu-Mascarell, C. (2024). Protocol paper: From Chaos to Order. Augmenting Manual Article Screening with Sentence Transformers in Management Systematic Reviews. WPOM-Working Papers on Operations Management15, 172–208. https://doi.org/10.4995/wpom.22282

What is it about?

This protocol paper describes a new method to help researchers screen and classify scientific articles more efficiently during systematic literature reviews. The authors propose using AI language models called «sentence transformers» to automatically analyze article titles and abstracts, comparing them to the review’s topic of interest. This helps researchers prioritize which articles to review first, rather than working through them randomly. The method was tested with 14 different AI models on a small set of articles about workplace management practices.

Featured Image

Why is it important?

As scientific publications grow exponentially, researchers struggle to efficiently review all relevant literature. This method could: * Save significant time in the screening process * Reduce researcher fatigue and potential bias * Make systematic reviews more accessible to researchers with limited resources * Help democratize access to advanced AI tools for academic research * Support evidence-based management practices by making literature reviews more feasible The approach is particularly valuable because it’s designed to complement rather than replace human judgment, and can be implemented using free, accessible tools.

Perspectives

This protocol represents an innovative bridge between cutting-edge AI technology and traditional academic research methods. The authors’ commitment to making the tool freely available and easy to use for researchers worldwide, regardless of technical expertise or resources, is particularly noteworthy. The pilot results suggest promising potential, though more testing is needed to validate the approach at larger scales.

Professor Juan A. Marin-Garcia
Universitat Politecnica de Valencia

Read the Original

This page is a summary of: Protocol paper: From Chaos to Order. Augmenting Manual Article Screening with Sentence Transformers in Management Systematic Reviews, WPOM – Working Papers on Operations Management, December 2024, Universitat Politecnica de Valencia,
DOI: 10.4995/wpom.22282.

Visitas: 1

A Three-Step Publishing Model to Support Evidence-Based Management

Marin-Garcia, J. A. (2021). Three-stage publishing to support evidence-based management practice. WPOM-Working Papers on Operations Management12(2), 56–95. https://doi.org/10.4995/wpom.11755 

 

What is it about?

This paper proposes a new model for publishing scientific research in business and management that aims to make studies more transparent and useful for practitioners. The model includes three key stages: 1) Publishing the research protocol/plan before starting, 2) Publishing the collected data, and 3) Publishing the final results and conclusions. This approach helps ensure research is well-planned, data is accessible for verification, and findings can be confidently applied in real-world management practice.

Featured Image

Why is it important?

Currently, there’s a big gap between management research and practice – many research findings never get implemented by actual managers. This model helps bridge that gap by: * Making the research process more transparent and trustworthy * Ensuring data quality through peer review at each stage * Making it easier for practitioners to access and understand findings * Encouraging more systematic, well-planned studies * Supporting evidence-based management decisions The model could significantly improve how management research is conducted and shared, ultimately making it more valuable for both academics and practitioners.

Perspectives

This innovative proposal challenges traditional publishing approaches in management research. While similar models exist in medicine and other fields, this is one of the first attempts to systematically apply such rigorous standards to business research. This offers a practical solution to improve research quality and impact. The detailed guidelines and examples make this a valuable roadmap for researchers wanting to conduct more transparent, reproducible studies.

Professor Juan A. Marin-Garcia
Universitat Politecnica de Valencia

Visitas: 3

https://www.frontiersin.org/articles/10.3389/fpsyg.2019.02368/full

Just published Bonavia, T., & Marin-Garcia, J. A. (2019). Spanish validation of the leader empowering behavior questionnaire (lebq). Frontiers in Psychology, 10(2368). doi:10.3389/fpsyg.2019.02368
#HRM #ContinuousImprovement #TalentManagement

The concept of empowering leadership (EL) has attracted widespread academic and practical interest and different questionnaires have been developed to measure it. However, there are no instruments to measure EL in the Spanish language. This article presents the translation, adaptation, and validation of a scale to measure this construct. In addition, it analyzes the relationship between managers’ EL and employees’ job satisfaction. In turn, the study analyzes whether employees who participate in a greater number of continuous improvement (CI) programs have supervisors who favor more empowering behaviors. A total of 739 participants with various occupations from different companies that have implemented CI processes filled out the Spanish version of the Leader Empowering Behavior Questionnaire (LEBQ-sp). Two different subsamples were used to test the relationships between the LEBQ and job satisfaction and CI, by means of Pearson’s correlation coefficient and analysis of variance, making it possible to provide evidence about the validity of the Spanish LEBQ. The confirmatory factor analysis supported the original structure of the six-factor model. The factors show a high level of internal consistency, as well as sufficient convergent and discriminant validity. Moreover, the results show that the more companies invest in formal CI programs, the more important it is for their leaders to adapt their behavior by displaying more EL. The LEBQ-sp is a valid and reliable instrument for use in research and a useful tool for applied purposes in the context of Spanish-speaking countries.

Visitas: 18

Publicado: Marin-Garcia, J. (2018). What are the research focuses regarding learning in the field of operations management in higher education? The case of Spain in 2017. Journal of Industrial Engineering and Management, 11(4), 607-616. doi:http://dx.doi.org/10.3926/jiem.2550

Publicado: Marin-Garcia, J. (2018). What are the research focuses regarding learning in the field of operations management in higher education? The case of Spain in 2017. Journal of Industrial Engineering and Management, 11(4), 607-616. doi:http://dx.doi.org/10.3926/jiem.2550

Purpose: What are the current research topics being studied by higher education professors in the area of operations management in Spain with regard to the learning of their students? Are the approaches that support these investigations adequate?

Design/methodology/approach: For the analysis, we have selected 25 publications in peer-reviewed scientific journals published by Spanish authors in 2017, and we have encoded them using Atlas.ti.

Findings: Most of the research centers on a very basic type of approach to learning, which reproduces the type of research typically conducted more than 40 years ago (type 1). For this reason, we propose an example of how to convert type 1 research questions into type 2 or 3 questions.

Originality/value: This paper collects and summarizes the main works on learning research carried out by members of ACEDEDOT and published in 2017, identifying themes, methods, levels of teacher conception and focus on the type of student learning. We intend to use this information to create a map of the current situation and propose possible suggestions to implement evidence-based instruction on operations management.

Keywords

scholarship, student learning, student engagement, higher education

Visitas: 21

Publicado: Marin-Garcia, J. A., Alfalla-Luque, R., & Machuca, J. A. D. (2018). A triple-a supply chain measurement model: Validation and analysis. International Journal of Physical Distribution & Logistics Management, 48(10) doi:10.1108/IJPDLM-06-2018-0233

Marin-Garcia, J. A., Alfalla-Luque, R., & Machuca, J. A. D. (2018). A triple-a supply chain measurement model: Validation and analysis. International Journal of Physical Distribution & Logistics Management, 48(10), null. doi:10.1108/IJPDLM-06-2018-0233
 
 
 
Purpose The purpose of this paper is to establish definitions and dimensions of Triple-A supply chain (SC) variables based on a literature review and to validate a Triple-A SC measurement model using a worldwide multiple informant sample. Design/methodology/approach Following a literature review, Triple-A SC variables (agility, alignment and adaptability) are conceptualized and a list of possible items is created for their measurement. An international 309 plant sample is used to validate the convergent and criterion validities of the composites proposed to measure Triple-A SC. Findings Contributions to the literature: clarification of Triple-A SC variable concepts; identification of key dimensions of Triple-A SC variables; development of a validated Triple-A SC measurement scale for future empirical research and industrial applications. Research limitations/implications A rigorously validated instrument is needed to measure Triple-A SC variables and enable researchers to credibly test theories regarding causal links between capabilities, practices and performance. Practical implications Proposal of a scale for use by managers of different functions to analyze Triple-A SC deployment in the company. Originality/value The only Triple-A SC scale used in the previous literature has serious limitations: scales were not taken from an extended literature review; data were collected from single respondents in a single country. This is the first validated Triple-A SC measurement model to overcome these limitations

Visitas: 15

Publicado: Losilla et al (2018) Three risk of bias tools lead to opposite conclusions in observational research synthesis

Nuevo artículo publicado que me ha permitido colaborar con estupendos investigadores del área de la salud para seguir avanzando en instrumentos que nos permitan hacer mejor ciencia y, sobre todo, informarla mejor. Algo que me interesa doblemente como editor de WPOM y de JIEM. Por otra parte, sufrir los estándares de publicación de una Q1 en el área de CC de la salud también es una experiencia interesante.

Visitas: 16

Publicado-Marin-Garcia & EtAl (2016) Proposal of a Framework for Innovation Competencies Development and Assessment (FINCODA)

Marin-Garcia, J., Andreu Andres, M., Atares-Huerta, L., Aznar-Mas, L., Garcia-Carbonell, A., González-Ladrón-de-Gevara, F., Montero Fleta, B., Perez-Peñalver, M., & Watts, F. (2016). Proposal of a Framework for Innovation Competencies Development and Assessment (FINCODA). WPOM-Working Papers on Operations Management, 7(2), 119-126. doi:http://dx.doi.org/10.4995/wpom.v7i2.6472

In this article we propose an innovation competence model of the people which is based on the existing literature to integrate and complement existing models. The main contribution of this work consists in demonstrating the differences and similarities of current models and in providing a conceptual definition for each model element. In this way, both researchers and people in charge of human resources in companies obtain a framework with which to design measuring instruments to assess innovation competence, which can fulfill the twofold demand of validity and reliability.

This work has been conducted as part of a European project financed by the European Union [«FINCODA» Project 554493-EPP-1-2014-1-FI-EPPKA2-KA] (http://bit.ly/FINCODA-EUsite01). (The European Commission support for the production of this publication does not constitute an endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein); and by the Universitat Politénica de Valencia PIME/2015/A/009/A “Evaluation of innovative behavior indicators in university students”.

Cartton abstract

Keywords

competence assessment; innovation; model; literature review

Visitas: 25

Publicado- Gumaelius Et Al (2016-on-line) Outreach initiatives operated by universities for increasing interest in science and technology

Visitas: 21

Publicado-MARIN-GARCIA, Juan A.(2015) Publishing in two phases for focused research by means of «research collaborations»

MARIN-GARCIA, Juan A.. Publishing in two phases for focused research by means of «research collaborations». WPOM-Working Papers on Operations Management, [S.l.], v. 6, n. 2, p. 76-80, dec. 2015. ISSN 1989-9068. Available at: <http://polipapers.upv.es/index.php/WPOM/article/view/4459>. Date accessed: 27 dec. 2015. doi:http://dx.doi.org/10.4995/wpom.v6i2.4459.

(La publicación en dos fases para la investigación enfocada a través de “research collaborations”)

We present and justify a new way to research and publish. This proposal is not intended to substitute or replace traditional ways of doing science, but rather complement, filling a gap and providing an efficient way to achieve scientific advances. The process begins with the sending a proposal of protocol to WPOM. Proposals are evaluated in each of the collaborations, depending on the potential to fill a research niche in the area. In the case of accepted protocols, WPOM guarantees commitment to publish the article if the protocol and deadlines are met. Thus, researchers can develop their projects focusing on meeting the protocol that has been approved without the question of whether, once completed, research is relevant or if the methodology is correct.

—-

En este artículo presentamos y justificamos una nueva forma de investigar y publicar. Esta propuesta no pretende sustituir o reemplazar a los modos tradicionales de hacer ciencias, sino complementarlos, rellenando una laguna y ofreciendo un camino eficiente para lograr los avances científicos. El proceso se inicia con el envío de la propuesta de protocolo WPOM. Las propuestas se evalúan, en cada una de las collaborations, en función del potencial para cubrir un nicho de investigación en el área. En el caso de ser aceptados los protocolos, WPOM garantiza un compromiso de publicación del artículo si se cumple el protocolo y plazos prometidos en el proyecto. De este modo, los investigadores pueden desarrollar proyectos de escritura centrándose en cumplir el protocolo que ha sido aprobado, sin la incógnita de si, una vez terminada, la investigación es relevante o si la metodología es la correcta.

 

Keywords

focused research; protocol; collaboration;Investigación enfocada; protocolos

Visitas: 19

Publicado: Marin-Garcia, J. A., & Santandreu-Mascarell, C. (2015). What do we know about rubrics used in higher education? Intangible Capital, 11(1), 118-145.

Marin-Garcia, J. A., & Santandreu-Mascarell, C. (2015). What do we know about rubrics used in higher education? Intangible Capital, 11(1), 118-145. doi: 1697-9818 – Print ISSN: 2014-3214
http://dx.doi.org/10.3926/ic.538

Acceso al PDF del documento:

http://www.intangiblecapital.org/index.php/ic/article/view/538/462

Purpose: In this paper we compare the search strategies and the results of three recent literature reviews on the rubrics use in education. With all this we aim to identify what is known, what it is not known yet and if there is material to launch a new literature review right now.

Design/methodology/approach: Comparative analysis of three recen literature reviews and systematic literature review.

Findings and Originality/value: Of the 142 papers included in the three recent reviews, few focus on university learning and the results are not broken down by educational level. We believe that there is no conclusive scientific answer to how to create and validate an assessment rubric for university courses, or what purpose they serve , or if it’s worth the effort to develop them. Recommendations for the appropriate use of rubrics are based on anecdotal evidence without robust and proven methodology. Nor is easy to find what subjects, or what objects, and in what contexts have been investigated the rubrics or which validated rubrics are available.

Originality/value: we conducted a new review and we compare the selected papers with those used in previous reviews. We have located 241 new papers not included in previous reviews. Probably all this new material can be used in the future for a meta-analysis

Keywords

rating; scoring; rubric; Higher Education; systematic Literature Review; assessment instrument; educational; learning; self-assessment; peer-assessment

Visitas: 14