Major Vincent, Surkis Alisa, Aphinyanaphongs Yindalon
NYU Langone Health, New York, NY, USA.
AMIA Annu Symp Proc. 2018 Dec 5;2018:1405-1414. eCollection 2018.
Conventional text classification models make a bag-of-words assumption reducing text into word occurrence counts per document. Recent algorithms such as word2vec are capable of learning semantic meaning and similarity between words in an entirely unsupervised manner using a contextual window and doing so much faster than previous methods. Each word is projected into vector space such that similar meaning words such as "strong" and "powerful" are projected into the same general Euclidean space. Open questions about these embeddings include their utility across classification tasks and the optimal properties and source of documents to construct broadly functional embeddings. In this work, we demonstrate the usefulness of pre-trained embeddings for classification in our task and demonstrate that custom word embeddings, built in the domain and for the tasks, can improve performance over word embeddings learnt on more general data including news articles or Wikipedia.
传统的文本分类模型采用词袋假设，将文本简化为每个文档中单词的出现次数。最近的算法，如word2vec，能够使用上下文窗口以完全无监督的方式学习单词之间的语义含义和相似度，并且比以前的方法快得多。每个单词都被投影到向量空间中，使得具有相似含义的单词，如“strong”和“powerful”，被投影到同一个一般的欧几里得空间中。关于这些嵌入的开放性问题包括它们在分类任务中的效用，以及构建广泛功能嵌入的文档的最佳属性和来源。在这项工作中，我们展示了预训练嵌入在我们的任务中的分类有用性，并证明了在领域内和任务中构建的自定义词嵌入可以比在包括新闻文章或维基百科在内的更通用数据上学习的词嵌入提高性能。
