Privacy protection in transformer-based neural network

Jiaqi Lang, Linjing Li, Weiyun Chen, Daniel Zeng

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

With the great success of neural networks, it is important to improve the information security of application systems based on them. This paper investigates a scenario where an attacker eavesdrops the intermediate representation computed by the encoder layers and tries to recover the private information of the input text. We propose a new metric to evaluate the encoder's ability to protect privacy and evaluate the Transformer-based encoder, which is the first privacy research conducted on Transformer-based neural networks. We also propose an adversarial training method to enhance the privacy of Transformer-based neural networks.

Original languageEnglish (US)
Title of host publication2019 IEEE International Conference on Intelligence and Security Informatics, ISI 2019
EditorsXiaolong Zheng, Ahmed Abbasi, Michael Chau, Alan Wang, Lina Zhou
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages182-184
Number of pages3
ISBN (Electronic)9781728125046
DOIs
StatePublished - Jul 2019
Externally publishedYes
Event17th IEEE International Conference on Intelligence and Security Informatics, ISI 2019 - Shenzhen, China
Duration: Jul 1 2019Jul 3 2019

Publication series

Name2019 IEEE International Conference on Intelligence and Security Informatics, ISI 2019

Conference

Conference17th IEEE International Conference on Intelligence and Security Informatics, ISI 2019
CountryChina
CityShenzhen
Period7/1/197/3/19

Keywords

  • Neural network
  • Privacy protection
  • Representation learning
  • Transformer

ASJC Scopus subject areas

  • Safety, Risk, Reliability and Quality
  • Computer Vision and Pattern Recognition
  • Information Systems and Management
  • Information Systems

Fingerprint Dive into the research topics of 'Privacy protection in transformer-based neural network'. Together they form a unique fingerprint.

  • Cite this

    Lang, J., Li, L., Chen, W., & Zeng, D. (2019). Privacy protection in transformer-based neural network. In X. Zheng, A. Abbasi, M. Chau, A. Wang, & L. Zhou (Eds.), 2019 IEEE International Conference on Intelligence and Security Informatics, ISI 2019 (pp. 182-184). [8823346] (2019 IEEE International Conference on Intelligence and Security Informatics, ISI 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISI.2019.8823346