TITLE: BRIO: A Bias and Risk Assessment Formal Methodology and Tool

Prof. Giuseppe Primiero, University of Milan, Italy

ABSTRACT:

Phenomena of bias by AI systems based on machine learning methods are well known and largely discussed in the literature. A variety of tools are being developed to assess these undesirable behaviours. In this talk I present a bias and risk assessment formal methodology and tool developed within the BRIO Research Project (https://sites.unimi.it/brio/).The tool is based on various formal logics and works on the I/O data of a ML system remaining agnostic on the model itself. The user can choose one of two distinct modules to evaluate either the difference in behaviour that the model displays on outputs produced by subclasses of inputs, or to evaluate against a desirable output. The type of distance and the threshold for admissible distance from the target distribution can also be selected. The result is a set of all the features and combinations thereof that produce violations with respect to the target distribution. These features can be fed into a risk function which computes an overall value weighting them on parameters such as size of the population and number of features involved, mapping naturally into notions of group and individual fairness

Reliable and Efficient hardware for Trustworthy Deep Neural Networks

Prof. Alberto Bosio, University of Lyon - Lyon Institute of Nanotechnology

ABSTRACT:

Deep Neural Networks (DNNs) are amongst the most intensively and widely used predictive models in machine learning. Nonetheless, increased computation speed and memory resources, along with significant energy consumption, are required to achieve the full potentials of DNNs. To be able to run DNNs algorithms out of the cloud and onto distributed Internet-of-Things (IoT) devices, customized HardWare platforms for Artificial Intelligence (HW-AI) are required. However, similar to traditional computing hardware, HW-AI is subject to hardware faults, occurring due to process, aging and environmental reliability threats. Although HW-AI comes with some inherent fault resilience, faults can lead to prediction failures seriously affecting the application execution. Typical reliability approaches, such as on-line testing and hardware redundancy, or even retraining, are less appropriate for HW-AI due to prohibited overhead; DNNs are large architectures with important memory requirements, coming along with an immense training set. This talk will address these limitations by exploiting the particularities of HW-AI architectures to develop low-cost and efficient reliability strategies.  

Title: Component-level Explanation and Validation of AI Models

Prof. Wojciech Samek, Technical University of Berlin & Fraunhofer Heinrich Hertz Institute (HHI), Berlin, Germany.

ABSTRACT:

Human-designed systems are constructed step by step, with each component serving a clear and well-defined purpose. For instance, the functions of an airplane’s wings and wheels are explicitly understood and independently verifiable. In contrast, modern AI systems are developed holistically through optimization, leaving their internal processes opaque and making verification and trust more difficult. This talk explores how explanation methods can uncover the inner workings of AI, revealing what knowledge models encode, how they use it to make predictions, and where this knowledge originates in the training data. It presents SemanticLens, a novel approach that maps hidden neural network knowledge into the semantically rich space of foundation models like CLIP. This mapping enables effective model debugging, comparison, validation, and alignment with reasoning expectations. The talk concludes by demonstrating how SemanticLens can help in identifying flaws in medical AI models, enhancing robustness and safety, and ultimately bridging the “trust gap” between AI systems and traditional engineering.

Title: A tale of adversarial attacks & out-of-distribution detection stories in the activation space

Dr. Celia Cintas, Research Scientist, IBM Research Africa – Nairobi

ABSTRACT:

Most deep learning models assume ideal conditions and rely on the assumption that test/production data comes from the in-distribution samples from the training data. However, this assumption is not satisfied in most real-world applications. Test data could differ from the training data either due to adversarial perturbations, different hardware devices, diverse patient populations with different or unknown conditions samples, generated content, noise, or other distribution changes. These shifts in the input data can lead to classifying unknown types, classes that do not appear during training, as known with high confidence. On the other hand, adversarial perturbations in the input data can cause a sample to be incorrectly classified. In this talk, we will discuss approaches based on subset scanning methods from the anomalous pattern detection domain and how they can be applied over off-the-shelf DL models.

Title: Explainable AI from Classification to Generation

Dr. Amit Dhurandhars, Principle Research Staff Member, IBM T.J. Watson, Yorktown Heights NY. USA.

ABSTRACT:

Over the past decade XAI has focused mainly on supervised learning problems. However, with the explosion of Large Language Models in the last couple of years explainability for generative tasks has become an important problem. In this talk, I will present two directions that we have been pushing extending approaches previously built for classification to now generation. One is a post-hoc explainability approach based on popular paradigm of contrastive/counterfactual explanations. The other is a directly interpretable approach generalizing CoFrNets a novel continued fraction inspired architecture that we had proposed few years back. Both of these complimentary directions aim at making the black-box nature of these ever more opaque models increasingly transparent.

Title: Human-Agent Explainability Architecture: Application of Remote Robots

Dr. Yazan Mualla, University of Technology of Belfort-Montébliard (UTBM), France.

ABSTRACT:

Recent studies in the goal-driven eXplainable AI (XAI) domain have confirmed that explaining the agent’s behavior to humans fosters the latter’s understandability of the agent and increases its acceptability. However, providing overwhelming information may also confuse human users and cause misunderstandings. For these reasons, the parsimony of explanations has been outlined as one of the key features facilitating successful human-agent interaction with a parsimonious explanation defined as the simplest explanation that describes the situation adequately. This keynote discusses a context-aware and adaptive formulation of parsimonious explanations and proposes a Human-Agent Explainability Architecture (HAExA). The latter relies first on generating normal and contrastive explanations and second on updating and filtering them before communicating them to the human. To evaluate HAExA, we design and conduct empirical human-computer interaction studies employing agent-based simulation of remote robots.

Title: Why Explainability Matters

Dr. Mourad Zerai, ESPRIT School of Engineering, Tunis, Tunisia.

ABSTRACT:

In a world increasingly shaped by AI, explainability is no longer optional-it’s essential. Regulations like the EU’s GDPR and AI Act, or the U.S. Equal Credit Opportunity Act, require AI systems to be transparent, especially in sensitive industries like healthcare, finance, and law enforcement. These laws aim to ensure fairness, accountability, and trust by demanding that AI decisions are understandable to humansThis talk breaks down what explainability means in practical terms, why regulators insist on it, and how it impacts both companies and individuals. From protecting rights to improving trust in technology, we’ll explore why making AI less of a “black box” benefits everyone.

Title: Trustworthy AI at NVIDIA

Michael Boone, Manager, Trustworthy AI Product, NVIDIA, USA.

ABSTRACT:

This presentation discusses what Trustworthy AI (TAI) means, what we are doing, and how we are deploying trustworthiness as an embedded property of technology for us, our customers and partners, and the greater ecosystem.

Title: The impact of Explainable AI models in enhancing interpretability and transparency for Cybersecurity and Digital Forensics

Pr. Farkhund Iqbal, College of Technological Innovation, Zayed University, UAE.

ABSTRACT:

Many computational and machine learning models operate as ‘black boxes,’ often delivering outputs—like classification accuracies—that are difficult for even experts to interpret. Cybersecurity professionals, vulnerability analysts, and risk analysis teams frequently struggle to trace how these models arrive at their conclusions, complicating efforts to predict and prepare for future attacks. This challenge also extends to the courtroom, where judges and police officers must understand and trust digital evidence to make informed decisions, yet often find themselves at a loss. Explainable AI (XAI) has become an invaluable tool in addressing these challenges. By enhancing the transparency and interpretability of AI models, XAI demystifies the results these technologies produce. This presentation will explore the critical roles that both supervised and unsupervised XAI models play in digital forensics. Supervised learning models, which are trained with labeled data, are particularly effective in areas like malware detection and email phishing identification. They provide detailed explanations of their predictions, making the evidence accessible and understandable for non-technical audiences. We will also discuss the tools and techniques used to implement XAI in cybersecurity, including popular frameworks like LIME and SHAP for supervised models, and feature important methods for unsupervised models. Ultimately, the use of explainable models in digital forensics does more than improve interpretability; they are essential in ensuring that justice is effectively served in our increasingly digital world.

TITLE: Interpretable Image Classification Through an Argumentative Dialog Between Encoders

Prof. Wassila Ouerdane, CentraleSupélec, Paris-Saclay University, France UAE.

ABSTRACT:

We address the problem of designing interpretable algorithms for image classification. Modern computer vision algorithms implement classification in two phases: feature extraction – the encoding – that relies on deep neural networks (DNN), followed by a task-oriented decision – the decoding – often also using a DNN. We propose to formulate this last phase as an argumentative DialoguE Between two agents relying on visual ATtributEs and Similarity to prototypes (DEBATES). DEBATES represents the combination of information provided by two encoders in a transparent and interpretable way. It relies on a dual process that combines similarity to prototypes and visual attributes, each extracted from an encoder. DEBATES makes explicit the agreements and conflicts between the two encoders managed by the two agents, reveals the causes of unintended behaviors, and helps identify potential corrective actions to improve performance. The approach is demonstrated on two problems of fine-grained image classification.