Files and links (1)
Preprint (Author's original)arXiv.org - Non-exclusive license to distribute, Open
Preprint
A Comparative Study on Structural and Semantic Properties of Sentence Embeddings
23 Sep 2020
Abstract
Sentence embeddings encode natural language sentences as low-dimensional
dense vectors. A great deal of effort has been put into using sentence
embeddings to improve several important natural language processing tasks.
Relation extraction is such an NLP task that aims at identifying structured
relations defined in a knowledge base from unstructured text. A promising and
more efficient approach would be to embed both the text and structured
knowledge in low-dimensional spaces and discover semantic alignments or
mappings between them. Although a number of techniques have been proposed in
the literature for embedding both sentences and knowledge graphs, little is
known about the structural and semantic properties of these embedding spaces in
terms of relation extraction. In this paper, we investigate the aforementioned
properties by evaluating the extent to which sentences carrying similar senses
are embedded in close proximity sub-spaces, and if we can exploit that
structure to align sentences to a knowledge graph. We propose a set of
experiments using a widely-used large-scale data set for relation extraction
and focusing on a set of key sentence embedding methods. We additionally
provide the code for reproducing these experiments at
https://github.com/akalino/semantic-structural-sentences. These embedding
methods cover a wide variety of techniques ranging from simple word embedding
combination to transformer-based BERT-style model. Our experimental results
show that different embedding spaces have different degrees of strength for the
structural and semantic properties. These results provide useful information
for developing embedding-based relation extraction methods.
Metrics
7 Record Views
Details
- Title
- A Comparative Study on Structural and Semantic Properties of Sentence Embeddings
- Creators
- Alexander KalinowskiYuan An
- Resource Type
- Preprint
- Language
- English
- Academic Unit
- Information Science (Informatics); Decision Sciences (and Management Information Systems)
- Other Identifier
- 991020547792704721