Accession Number : AD1028485


Title :   Consistent Alignment of World Embedding Models


Descriptive Note : Technical Report


Corporate Author : MIT Lincoln Laboratory Lexington United States


Personal Author(s) : Caceres,Rajmonda S ; Sahin,Cem S ; Oselio,Brandon ; Campbell,William M


Full Text : https://apps.dtic.mil/dtic/tr/fulltext/u2/1028485.pdf


Report Date : 02 Mar 2017


Pagination or Media Count : 4


Abstract : Word embedding models offer continuous vector representations that can capture rich contextual semantics based on their word co-occurrence patterns. While these word vectors can provide very effective features used in many NLP tasks such as clustering similar words and inferring learning relationships, many challenges and open research questions remain. In this paper, we propose a solution that aligns variations of the same model (or different models) in a joint low-dimensional latent space leveraging carefully generated synthetic data points. This generative process is inspired by the observation that a variety of linguistic relationships is captured by simple linear operations in embedded space. We demonstrate that our approach can lead to substantial improvements in recovering quality embeddings of local neighborhoods aligned and fused across different input word models.


Descriptors :   models , computational linguistics , embedding , information processing , semantics , neural networks , words(language)


Subject Categories : Linguistics


Distribution Statement : APPROVED FOR PUBLIC RELEASE