A TRUST-AWARE TRANSFORMER ARCHITECTURE WITH UNCERTAINTY ESTIMATION FOR HIGH-STAKES LANGUAGE-BASED DECISION SYSTEMS

Abstract

This study proposes a trust-aware transformer with integrated uncertainty (TATUi). In high-stakes applications such as healthcare, law, and finance, standard transformers do not provide any indicators to account for the reliability of their outputs. TATIUs uses Interpretable Attention, Monte Carlo Dropout (MCD) to estimate uncertainty, and Calibration Layers (CL) that are integrated into the architecture of the TATU transformer. When tested against MIMIC-CXR, ECHR, and FiQA, TATIU produced comparable levels of performance to other transformers, with an additional benefit in terms of superior quantification of uncertainty and the offering of a greater volume of interpretative resources that facilitate the effective deployment of AI into safe circumstances