Publications

Publications are listed below. For more details, please visit my Google Scholar page.

2026

1. SIGIR

   

   VulGen: Workshop on Vulnerabilities in Generative Systems for Information Retrieval

   Shuoqi Sun, Sara Allawati, Laura Dietz, Madhurima Khirbat, Bhaskar Mitra, Maarten de Rijke, and Damiano Spina

   In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’26), 2026

   Abs DOI Bib PDF Website

   Generative systems are rapidly transforming both academic research and industrial practices. These systems are increasingly integrated into information access and information retrieval (IR) tasks and continue to evolve at a substantial pace. Integrating these models into daily workflows exposes critical vulnerabilities, including adversarial attacks, inherent biases, and negative impacts on user behavior, which can lead to suboptimal or even detrimental outcomes. The VulGen workshop at SIGIR 2026 brings together the IR community and related disciplines (e.g., cyber security) to map this evolving landscape. Through a full day of structured discussion and engagement, we aim to synthesize the current state of research and identify new avenues for future investigation. Information about VulGen is hosted at: https://vulgen-workshop.github.io/SIGIR2026/.

   @inproceedings{sun2026vulgen,
     author = {Sun, Shuoqi and Allawati, Sara and Dietz, Laura and Khirbat, Madhurima and Mitra, Bhaskar and {de Rijke}, Maarten and Spina, Damiano},
     title = {VulGen: Workshop on Vulnerabilities in Generative Systems for Information Retrieval},
     booktitle = {Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '26)},
     year = {2026},
     doi = {10.1145/3805712.3808654},
   }

2. NeurIPS MMU-RAG

   

   RMIT-ADM+S at the MMU-RAG NeurIPS 2025 Competition

   Kun Ran, Marwah Alaofi, Danula Hettiachchi, Chenglong Ma, Khoi Nguyen Dinh Anh, Khoi Vo Nguyen, Sachin Pathiyan Cherumanal, Lida Rashidi, Falk Scholer, Damiano Spina, Shuoqi Sun, and Oleg Zendel

   2026

   Abs arXiv Bib PDF Code

   This paper presents the award-winning RMIT-ADM+S system for the Text-to-Text track of the NeurIPS 2025 MMU-RAG Competition. We introduce Routing-to-RAG (R2RAG), a research-focused retrieval-augmented generation (RAG) architecture composed of lightweight components that dynamically adapt the retrieval strategy based on inferred query complexity and evidence sufficiency. The system uses smaller LLMs, enabling operation on a single consumer-grade GPU while supporting complex research tasks. It builds on the G-RAG system, winner of the ACM SIGIR 2025 LiveRAG Challenge, and extends it with modules informed by qualitative review of outputs. R2RAG won the Best Dynamic Evaluation award in the Open Source category, demonstrating high effectiveness with careful design and efficient use of resources.

   @misc{ran2026rmitadmsmmuragneurips2025,
     title = {RMIT-ADM+S at the MMU-RAG NeurIPS 2025 Competition},
     author = {Ran, Kun and Alaofi, Marwah and Hettiachchi, Danula and Ma, Chenglong and Anh, Khoi Nguyen Dinh and Nguyen, Khoi Vo and Cherumanal, Sachin Pathiyan and Rashidi, Lida and Scholer, Falk and Spina, Damiano and Sun, Shuoqi and Zendel, Oleg},
     year = {2026},
     eprint = {2602.20735},
     archiveprefix = {arXiv},
     primaryclass = {cs.IR},
     url = {https://arxiv.org/abs/2602.20735},
   }

2025

1. SIGIR-AP

   

   ISMIE: A Framework to Characterize Information Seeking in Modern Information Environments

   Shuoqi Sun, Danula Hettiachchi, and Damiano Spina

   In Proceedings of the 2025 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP ’25), 2025

   Abs DOI arXiv Bib PDF

   The modern information environment (MIE) is increasingly complex, shaped by a wide range of techniques designed to satisfy users’ information needs. Information seeking (IS) models are effective mechanisms for characterizing user-system interactions. However, conceptualizing a model that fully captures the MIE landscape poses a challenge. We argue: Does such a model exist? To address this, we propose the Information Seeking in Modern Information Environments (ISMIE) framework as a fundamental step. ISMIE conceptualizes the information seeking process (ISP) via three key concepts: Components (e.g., Information Seeker), Intervening Variables (e.g., Interactive Variables), and Activities (e.g., Acquiring). Using ISMIE’s concepts and employing a case study based on a common scenario - misinformation dissemination - we analyze six existing IS and information retrieval (IR) models to illustrate their limitations and the necessity of ISMIE. We then show how ISMIE serves as an actionable framework for both characterization and experimental design. We characterize three pressing issues and then outline two research blueprints: a user-centric, industry-driven experimental design for the authenticity and trust crisis to AI-generated content and a system-oriented, academic-driven design for tackling dopamine-driven content consumption. Our framework offers a foundation for developing IS and IR models to advance knowledge on understanding human interactions and system design in MIEs.

   @inproceedings{sun2025ismie,
     title = {ISMIE: A Framework to Characterize Information Seeking in Modern Information Environments},
     author = {Sun, Shuoqi and Hettiachchi, Danula and Spina, Damiano},
     year = {2025},
     booktitle = {Proceedings of the 2025 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP '25)},
     doi = {10.1145/3767695.3769509},
   }

2. SIGIR LiveRAG

   

   RMIT-ADM+S at the SIGIR 2025 LiveRAG Challenge - GRAG: Generation-Retrieval-Augmented Generation

   Kun Ran, Shuoqi Sun, Khoi Nguyen Dinh Anh, Damiano Spina, and Oleg Zendel

   2025

   Abs arXiv Bib PDF Code

   This paper presents the RMIT–ADM+S participation in the SIGIR 2025 LiveRAG Challenge. Our Generation-Retrieval-Augmented Generation (GRAG) approach relies on generating a hypothetical answer that is used in the retrieval phase, alongside the original question. GRAG also incorporates a pointwise large language model (LLM)-based re-ranking step prior to final answer generation. We describe the system architecture and the rationale behind our design choices. In particular, a systematic evaluation using the Grid of Points (GoP) framework and N-way ANOVA enabled comparison across multiple configurations, including query variant generation, question decomposition, rank fusion strategies, and prompting techniques for answer generation. Our system achieved a Relevance score of 1.199 and a Faithfulness score of 0.477 on the private leaderboard, placing among the top four finalists in the LiveRAG 2025 Challenge.

   @misc{ran2025rmitadmssigir2025liverag,
     title = {RMIT-ADM+S at the SIGIR 2025 LiveRAG Challenge - GRAG: Generation-Retrieval-Augmented Generation},
     author = {Ran, Kun and Sun, Shuoqi and Anh, Khoi Nguyen Dinh and Spina, Damiano and Zendel, Oleg},
     year = {2025},
     eprint = {2506.14516},
     archiveprefix = {arXiv},
     primaryclass = {cs.IR},
     url = {https://arxiv.org/abs/2506.14516},
   }

3. SIGIR Forum

   

   Report on the 3rd Workshop on NeuroPhysiological Approaches for Interactive Information Retrieval (NeuroPhysIIR 2025) at SIGIR CHIIR 2025

   Damiano Spina, Jacek Gwizdka, Kaixin Ji, Yashar Moshfeghi, Javed Mostafa, Tuukka Ruotsalo, Min Zhang, Adnan Ahmad, Sara Fahad Dawood Al Lawati, Nattapat Boonprakong, Nishani Fernando, Jiaman He, Orland Hoeber, Gavindya Jayawardena, Boon-Giin Lee, Haiming Liu, Matthew Pike, Abbas Pirmoradi, Bahareh Nakisa, Mohammad Naim Rastgoo, Flora D. Salim, Fletcher Scott, Shuoqi Sun, Huimin Tang, Dave Towey, and Max L. Wilson

   SIGIR Forum, Oct 2025

   Abs DOI Bib PDF

   The International Workshop on NeuroPhysiological Approaches for Interactive Information Retrieval (NeuroPhysIIR’25), co-located with ACM SIGIR CHIIR 2025 in Naarm/Melbourne, Australia, included 19 participants who discussed 12 statements addressing open challenges in neurophysiological interactive IR. The report summarizes the statements presented and the discussions held at the full-day workshop.Date: 27 March 2025.Website: https://neurophysiir.github.io/chiir2025/.

   @article{10.1145/3769733.3769740,
     author = {Spina, Damiano and Gwizdka, Jacek and Ji, Kaixin and Moshfeghi, Yashar and Mostafa, Javed and Ruotsalo, Tuukka and Zhang, Min and Ahmad, Adnan and Lawati, Sara Fahad Dawood Al and Boonprakong, Nattapat and Fernando, Nishani and He, Jiaman and Hoeber, Orland and Jayawardena, Gavindya and Lee, Boon-Giin and Liu, Haiming and Pike, Matthew and Pirmoradi, Abbas and Nakisa, Bahareh and Rastgoo, Mohammad Naim and Salim, Flora D. and Scott, Fletcher and Sun, Shuoqi and Tang, Huimin and Towey, Dave and Wilson, Max L.},
     title = {Report on the 3rd Workshop on NeuroPhysiological Approaches for Interactive Information Retrieval (NeuroPhysIIR 2025) at SIGIR CHIIR 2025},
     year = {2025},
     issue_date = {June 2025},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     volume = {59},
     number = {1},
     issn = {0163-5840},
     url = {https://doi.org/10.1145/3769733.3769740},
     doi = {10.1145/3769733.3769740},
     journal = {SIGIR Forum},
     month = oct,
     pages = {1--43},
     numpages = {43},
   }

4. ECIR

   

   An Investigation of Prompt Variations for Zero-shot LLM-based Rankers

   Shuoqi Sun, Shengyao Zhuang, Shuai Wang, and Guido Zuccon

   In Advances in Information Retrieval, Oct 2025

   Abs DOI arXiv Bib PDF Code

   We provide a systematic understanding of the impact of specific components and wordings used in prompts on the effectiveness of rankers based on zero-shot Large Language Models (LLMs). Several zero-shot ranking methods based on LLMs have recently been proposed. Among many aspects, methods differ across (1) the ranking algorithm they implement, e.g., pointwise vs. listwise, (2) the backbone LLMs used, e.g., GPT3.5 vs. FLAN-T5, (3) the components and wording used in prompts, e.g., the use or not of role-definition (role-playing) and the actual words used to express this. It is currently unclear whether performance differences are due to the underlying ranking algorithm, or because of spurious factors such as better choice of words used in prompts. This confusion risks to undermine future research. Through our large-scale experimentation and analysis, we find that ranking algorithms do contribute to differences between methods for zero-shot LLM ranking. However, so do the LLM backbones – but even more importantly, the choice of prompt components and wordings affect the ranking. In fact, in our experiments, we find that, at times, these latter elements have more impact on the ranker’s effectiveness than the actual ranking algorithms, and that differences among ranking methods become more blurred when prompt variations are considered.

   @inproceedings{sun2025investigation,
     title = {An Investigation of Prompt Variations for Zero-shot LLM-based Rankers},
     author = {Sun, Shuoqi and Zhuang, Shengyao and Wang, Shuai and Zuccon, Guido},
     booktitle = {Advances in Information Retrieval},
     year = {2025},
     publisher = {Springer Nature Switzerland},
     address = {Cham},
     pages = {185--201},
     isbn = {978-3-031-88711-6},
     doi = {10.1007/978-3-031-88711-6_12},
     url = {https://doi.org/10.1007/978-3-031-88711-6_12},
   }