A SYSTEMATIC COMPARATIVE ANALYSIS OF PRIVACY-PRESERVING TRANSFORMER PREDICTION UNDER ADVERSARIAL CHALLENGES

Abstract

The growing deployment of large language models (LLMs) based on transformers in Machine Learning-as-a-Service (MLaaS) contexts, which poses significant risks of adversarial exploitation and privacy leakage. While models are still vulnerable to backdoors, jailbreak prompts and poisoning, user queries are accessible to membership inference and data extraction. Cryptographic secure inference, differential privacy, verified robustness and adversarial prompt filtering are some of the defense techniques now in use, however they are insufficient to completely prevent some threat classes. This study presents a unified comparative evaluation of privacy-preserving transformer inference techniques, analyzing data from twenty-one fundamental research studies and a new empirical analysis across five transformer models (DistilBERT, BERT-base, RoBERTa-base, RoBERTa-Large and a DeBERTa-v3 + RoBERTa ensemble) .Furthermore, we employ Adaptive Privacy-Preserving Transformer Inference (APPTI), a hybrid defense that combines adversarial prompt sanitization, representation perturbation, inference-time differential privacy and logit obfuscation. The Ensemble and RoBERTa-Large models show the best privacy-utility balance across MIA, ROC-AUC, backdoor ASR and latency metrics. APPTI introduces <15% latency overhead, halves backdoor ASR in multiple circumstances and reduces MIA success by 7 to 10%. Adaptive hybrid defenses offer the most useful trade-offs for real-world MLaaS deployment, according to comparative testing with DP-SGD, secure simultaneous inference, authorized smoothing and jailbreak detection.