Urrutia Oñate, DorletaOnieva Caracuel, EnriquePerallos Ruiz, Asier2026-08-042026-08-042026Urrutia-Onate, D., Onieva, E., & Perallos, A. (2026). Deployer-side XAI instrumentation for regulated AI: a clinical case study in ICL sizing. International Journal of Advanced Computer Science and Applications, 17(6), 45-61. https://doi.org/10.14569/IJACSA.2026.01706052158-107X10.14569/IJACSA.2026.0170605https://hdl.handle.net/20.500.14454/6463Regulated AI creates a monitoring problem for deployers who must organise human oversight, log-retention and post-market surveillance while often having access only to the prediction interface. This study specifies a deployer-side XAI instrumentation protocol for the output→action boundary, where a model output becomes a reason for action. The protocol reorganises KernelSHAP, nearest-neighbour envelope checks, bounded perturbation, and rank-order stability into a per-decision evidence record computed from predict() calls. We instantiate the protocol in a clinical case of phakic Implantable Collamer Lens sizing, using a 55-eye held-out cohort and an Extra Trees regressor for post-operative vault prediction. The record contains five signals: score_margin, constraint_enforcement, envelope_validity, decision_robustness, and record_integrity, plus two cohort-level oversight aggregates. The case study shows how the same record can support decision-time human oversight, later audit and post-market surveillance under the EU AI Act and the Medical Devices Regulation.engEU AI ActExplainable AIHuman oversightICL sizingMedical Device SoftwareXAI instrumentationjournal article2026-08-042156-5570