Jialiang Xu

Selected Publications

(For full publication list please refer to my Google Scholar page. "*" denotes equal contribution)

1. ACL 2024Outstanding

   Word Embeddings Are Steers for Language Models

   Chi Han, Jialiang Xu, Manling Li, Yi Fung, Chenkai Sun, and 3 more authors

   (ACL 2024 Outstanding Paper Award)

   2024

   Abs Bib arXiv PDF

   Language models (LMs) automatically learn word embeddings during pre-training on language corpora. Although word embeddings are usually interpreted as feature vectors for individual words, their roles in language model generation remain underexplored. In this work, we theoretically and empirically revisit output word embeddings and find that their linear transformations are equivalent to steering language model generation styles. We name such steers LM-Steers and find them existing in LMs of all sizes. It requires learning parameters equal to 0.2% of the original LMs’ size for steering each style. On tasks such as language model detoxification and sentiment control, LM-Steers can achieve comparable or superior performance compared with state-of-the-art controlled generation methods while maintaining a better balance with generation quality. The learned LM-Steer serves as a lens in text styles: it reveals that word embeddings are interpretable when associated with language model generations and can highlight text spans that most indicate the style differences. An LM-Steer is transferrable between different language models by an explicit form calculation. One can also continuously steer LMs simply by scaling the LM-Steer or compose multiple LM-Steers by adding their transformations.

   @misc{han2024word,
     title = {Word Embeddings Are Steers for Language Models},
     author = {Han, Chi and Xu, Jialiang and Li, Manling and Fung, Yi and Sun, Chenkai and Jiang, Nan and Abdelzaher, Tarek and Ji, Heng},
     year = {2024},
     eprint = {2305.12798},
     archiveprefix = {arXiv},
     primaryclass = {id='cs.CL' full_name='Computation and Language' is_active=True alt_name='cmp-lg' in_archive='cs' is_general=False description='Covers natural language processing. Roughly includes material in ACM Subject Class I.2.7. Note that work on artificial languages (programming languages, logics, formal systems) that does not explicitly address natural-language issues broadly construed (natural-language processing, computational linguistics, speech, text retrieval, etc.) is not appropriate for this area.'},
     acc_type = {Outstanding},
     award = {ACL 2024 Outstanding Paper Award}
   }

2. NAACL 2024Findings

   SUQL: Conversational Search over Structured and Unstructured Data with Large Language Models

   Shicheng Liu, Jialiang Xu, Wesley Tjangnaka, Sina Semnani, Chen Jie Yu, and 1 more author

   2024

   Abs Bib arXiv PDF

   Many knowledge sources consist of both structured information such as relational databases as well as unstructured free text. Building a conversational interface to such data sources is challenging. This paper introduces SUQL, Structured and Unstructured Query Language, the first formal executable representation that naturally covers compositions of structured and unstructured data queries. Specifically, it augments SQL with several free-text primitives to form a precise, succinct, and expressive representation. This paper also presents a conversational search agent based on large language models, including a few-shot contextual semantic parser for SUQL. To validate our approach, we introduce a dataset consisting of crowdsourced questions and conversations about real restaurants. Over 51% of the questions in the dataset require both structured and unstructured data, suggesting that it is a common phenomenon. We show that our few-shot conversational agent based on SUQL finds an entity satisfying all user requirements 89.3% of the time, compared to just 65.0% for a strong and commonly used baseline.

   @misc{liu2024suql,
     title = {{SUQL}: Conversational Search over Structured and Unstructured Data with Large Language Models},
     author = {Liu, Shicheng and Xu, Jialiang and Tjangnaka, Wesley and Semnani, Sina and Yu, Chen Jie and Lam, Monica},
     year = {2024},
     url = {https://openreview.net/forum?id=DoSQeeVlUO},
     acc_type = {Findings}
   }

3. EMNLP 2022

   Towards Robust Numerical Question Answering: Diagnosing Numerical Capabilities of NLP Systems

   Jialiang Xu, Mengyu Zhou, Xinyi He, Shi Han, and Dongmei Zhang

   2022

   Abs Bib arXiv PDF Poster Slides

   Numerical Question Answering is the task of answering questions that require numerical capabilities. Previous works introduce general adversarial attacks to Numerical Question Answering, while not systematically exploring numerical capabilities specific to the topic. In this paper, we propose to conduct numerical capability diagnosis on a series of Numerical Question Answering systems and datasets. A series of numerical capabilities are highlighted, and corresponding dataset perturbations are designed. Empirical results indicate that existing systems are severely challenged by these perturbations. E.g., Graph2Tree experienced a 53.83% absolute accuracy drop against the “Extra” perturbation on ASDiv-a, and BART experienced 13.80% accuracy drop against the “Language” perturbation on the numerical subset of DROP. As a counteracting approach, we also investigate the effectiveness of applying perturbations as data augmentation to relieve systems’ lack of robust numerical capabilities. With experiment analysis and empirical studies, it is demonstrated that Numerical Question Answering with robust numerical capabilities is still to a large extent an open question. We discuss future directions of Numerical Question Answering and summarize guidelines on future dataset collection and system design.

     doi = {10.48550/ARXIV.2211.07455},
     author = {Xu, Jialiang and Zhou, Mengyu and He, Xinyi and Han, Shi and Zhang, Dongmei},
     keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
     title = {Towards Robust Numerical Question Answering: Diagnosing Numerical Capabilities of NLP Systems},
     publisher = {arXiv},
     year = {2022},
     copyright = {arXiv.org perpetual, non-exclusive license},
   }