A Novel Approach Based on Integrating Small Language Models and Retrieval-Augmented Generation for Medical Question Answering
Abstract
This paper proposes a novel approach which com bines small language models with retrieval-augmented gener ation in medical question answering to provide accurate and comprehensible information. Our method extracts relevant evi dence from external knowledge and converts them into vector embeddings which are used for high-fidelity semantic retrieval. The small language model subsequently synthesizes the retrieved evidence into fluent, context-aware responses. Extensive experi ments conducted on the PubMed benchmark dataset, the results show competitive performance to larger language models while being far more suitable for deployment in resource-constrained environments. Moreover, the proposed method supports efficient domain knowledge updates without the need for extensive re training. Our implementation is available in the following GitHub repository: https://github.com/LeoBaoNguyen12/RagSLM-MQA.
Keywords
Small Language Models, Retrieval-Augmented Generation, Medical Question Answering, RagSLM-MQA
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
B. Nguyen and Q. Le, "A Novel Approach Based on Integrating Small Language Models and Retrieval-Augmented Generation for Medical Question Answering," in Journal of Communications Software and Systems, vol. 22, no. 3, pp. 356-366, July 2026, doi: 10.24138/jcomss-2026-0036
@article{nguyen2026novelapproach,
author = {Nguyen, Bao-Gia and Le, Quang-Hung},
title = {{A Novel Approach Based on Integrating Small Language Models and Retrieval-Augmented Generation for Medical Question Answering}},
journal = {Journal of Communications Software and Systems},
month = jul,
year = {2026},
volume = {22},
number = {3},
pages = {356--366},
doi = {10.24138/jcomss-2026-0036},
url = {https://doi.org/10.24138/jcomss-2026-0036}
}