Unseen Word Representation by Aligning Heterogeneous Lexical Semantic Spaces

Authors

  • Victor Prokhorov University of Cambridge
  • Mohammad Taher Pilehvar University of Cambridge
  • Dimitri Kartsaklis University of Cambridge
  • Pietro Lio University of Cambridge
  • Nigel Collier University of Cambridge

DOI:

https://doi.org/10.1609/aaai.v33i01.33016900

Abstract

Word embedding techniques heavily rely on the abundance of training data for individual words. Given the Zipfian distribution of words in natural language texts, a large number of words do not usually appear frequently or at all in the training data. In this paper we put forward a technique that exploits the knowledge encoded in lexical resources, such as WordNet, to induce embeddings for unseen words. Our approach adapts graph embedding and cross-lingual vector space transformation techniques in order to merge lexical knowledge encoded in ontologies with that derived from corpus statistics. We show that the approach can provide consistent performance improvements across multiple evaluation benchmarks: in-vitro, on multiple rare word similarity datasets, and invivo, in two downstream text classification tasks.

Downloads

Published

2019-07-17

How to Cite

Prokhorov, V., Pilehvar, M. T., Kartsaklis, D., Lio, P., & Collier, N. (2019). Unseen Word Representation by Aligning Heterogeneous Lexical Semantic Spaces. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 6900-6907. https://doi.org/10.1609/aaai.v33i01.33016900

Issue

Section

AAAI Technical Track: Natural Language Processing