Qianying Liao

As generative AI (GenAI) systems become increasingly prevalent across various technological stacks, the question of how such systems handle sensitive and personal data flows becomes increasingly important. Specifically, both the ability to harness and process large swaths of information as well as their stochastic nature raise key concerns related to both security and privacy. Unfortunately, while some of the traditional security threat modeling can effectively identify certain violations, privacy-related issues are often overlooked. To respond to these challenges, we introduce a novel domain-specific privacy threat modeling framework to support the privacy threat analysis of GenAI-based applications. This framework is constructed through a two-pronged approach: (i) a systematic review of the emerging literature on GenAI privacy threats, and (ii) a case-driven application to a representative Chatbot system. These efforts yield a foundational GenAI privacy threat modeling framework built on LINDDUN. The new framework extends three out of the seven privacy threat types of LINDDUN and introduces 100 GenAI examples to the entire knowledge base. Its effectiveness is validated on an AI Agent system, which demonstrates that a comprehensive privacy analysis can be supported by the new framework.

Privacy-preserving data publication (PPDP) aims to reduce privacy risks before releasing datasets. As alternative techniques, (i) data de-identification alters original data values through generalization or suppression to weaken their link to data subject identities, whereas (ii) synthetic data generation produces artificial datasets that do not contain any original data yet preserve statistical similarity to the original data. Because their outcomes lack direct comparability, it remains unclear which technique provides better privacy protection while retaining data utility. To address this gap, we present: (i) a novel quantitative evaluation framework for comparing privacy and utility across synthetic and de-identified datasets, and (ii) a comparative study based on a location optimisation task and a predictive machine learning application. Our results reveal a nuanced trade-off. De-identified datasets perform better on traditional statistical privacy metrics but are more vulnerable to adversarial attacks such as attribute inference. Conversely, utility outcomes vary widely by use case, indicating that the selection of a PPDP technique remains highly dependent on the application context.

Generative Artificial Intelligence (GenAI) has rapidly evolved from an experimental technology into a foundational component of modern software systems. However, as its adoption grows, protecting sensitive personal data becomes increasingly challenging. Specifically, GenAI systems not only amplify traditional privacy threats but also introduce new inference-based risks, such as constructing detailed user profiles from seemingly harmless inputs. In response, privacy threat modeling frameworks are beginning to capture GenAI-specific privacy threats with finer granularity. At the same time, a growing number of mitigation techniques have been proposed to address these threats. However, although knowledge of both threats and mitigations continues to mature, the problem- and solution-space have developed largely independently. This position paper argues that the primary challenge in GenAI privacy engineering is not the lack of knowledge about privacy threats or mitigation techniques, but the missing bridge between them. We decompose this gap into three sub-problems: (i) lack of fine-grained threat-to-mitigation mapping for GenAI systems, (ii) inapplicable solution-space assumptions in the GenAI context, and (iii) prioritization difficulty under GenAI constraints. We derive four recommendations for future mitigation-selection approaches, and outline a suggested approach that extends established threat-to-mitigation mapping methods to GenAI-specific threat characteristics. We propose a research agenda toward more systematic privacy mitigation selection for GenAI-based systems.

The trend to support healthcare outside of the premises and control of health institutions has given rise to a diversity of devices (e.g. wearables), services, and players contributing to the ecosystem. In consequence, health-related and medical data are frequently dispersed across multiple institutions and devices, posing a significant challenge to comprehensive data analysis and exploration for AI-driven research on new medicines or treatments. Centralizing such sensitive data is often unachievable due to the rigorous security and privacy demands imposed by various stakeholders, including patients, general practitioners, and pharmaceutical researchers. The decentralized and dispersed nature of healthcare data, in combination with stringent regulatory constraints such as HIPAA and GDPR, necessitate innovative approaches to collaborative healthcare research. Federated learning is a promising class of solutions to harness the collective intelligence of diverse datasets by allowing collaborative model training across decentralized data sources. Despite the many privacy-enhancing techniques proposed to ensure the confidentiality of data and intellectual property, this chapter discusses and illustrates the challenges of leveraging shared representations and knowledge transfer between smart health applications while addressing the intricacies of data fragmentation, security, and privacy.

The data economy thrives on data-centric collaboration between organizations. However, open data sharing remains a pipe dream without addressing pragmatic, regulatory, and strategic concerns – which include data protection and confidentiality. Generative artificial intelligence supports the production of realistic synthetic datasets on demand, and is a promising technology to alleviate such concerns. However, the replacement of an original dataset with a synthetic dataset incurs a specific trade-off between utility and privacy, and the appropriateness of this trade-off is highly context- and application-dependent. Manually establishing and managing different synthetic generators that have diverging properties is error-prone and time-consuming, lacks flexibility, and thus is costly and impractical. This paper introduces Data Chameleon, a novel self-adaptive data management architecture for different synthetic data generators. Data Chameleon adaptively samples from different synthetic data generators in function of the data request at hand. Furthermore, in a longer-term adaptation loop, Data Chameleon monitors and evaluates the overall suitability of the available generators, to monitor evolutions in data demand, or possible concept drifts. Based on this, the Chameleon autonomously decides to re-train existing generators, or instantiate additional ones in a self-adaptive manner. The Data Chameleon architecture enhances the practical applicability of synthetic data generation, enabling more efficient and secure data sharing in real-world scenarios.

Distributed Collaborative Machine Learning (DCML) enables collaborative model training without the need to reveal or share datasets. However, its implementation incurs complexities in balancing performance, efficiency, and security and privacy. Different architectural approaches in how the learning can be distributed and coordinated over different parties have emerged, leading to significant diversity. Current academic work often lacks a comprehensive evaluation of the diverse architectural DCML approaches, which in turn hinders objective comparison of different DCML approaches and impedes broader adoption. This paper highlights the need for a general and holistic evaluation framework for DCML approaches, and proposes a number of relevant evaluation metrics and criteria.

Building a Machine Learning model requires the use of large amounts of data. Due to privacy and regulatory concerns, these data might be owned by multiple sites and are often not mutually shareable. Our work deals with private learning and inference for the Weighted Random Forest model when data records are vertically distributed among multiple sites. Previous privacy-preserving vertical tree-based frameworks either adapt Secure Multi-party Computation or share intermediate results and are hard to generalize or scale. In contrast, our proposal contains efficient collaborative calculation algorithms of the Gini Index and Entropy for computing the impurity of decision tree nodes while protecting all intermediate values and disclosing minimal information. We offer a learning protocol based on the Paillier Cryptosystem and Digital Envelope. Also, we provide an inference protocol found on the Look-up Table. Our experiments show that the proposed protocols do not cause predictive performance loss while still establishing and utilizing the model within a reasonable time. The results imply that practitioners can overcome the barrier of data sharing and produce random forest models for data-heavy domains with strict privacy requirements, such as Health Prediction, Fraud Detection, and Risk Evaluation.