python3.X下使用skip-thought获取句子向量

作者:luozhipeng   发表日期:2018-1-9  浏览:21次

  • 按照需要的包:

TensorFlow

NumPy

scikit-learn

Natural Language Toolkit (NLTK)

 

  • 下载解压bidirectional model:

wget "http://download.tensorflow.org/models/skip_thoughts_bi_2017_02_16.tar.gz"

tar -xvf skip_thoughts_bi_2017_02_16.tar.gz        

rm skip_thoughts_bi_2017_02_16.tar.gz

 

 

  • 修改encoder_manager.py

 

  • 加载模型

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import numpy as np
import os.path
import scipy.spatial.distance as sd
from skip_thoughts import configuration
from skip_thoughts import encoder_manager
VOCAB_FILE = "path/to/vocab.txt"
EMBEDDING_MATRIX_FILE = "path/to/embeddings.npy"
#CHECKPOINT_PATH = "path/to/model.ckpt-9999"
CHECKPOINT_PATH = "path/to/model.ckpt-500008"

encoder = encoder_manager.EncoderManager()

unidirectional model


encoder.load_model(configuration.model_config(),
vocabulary_file=VOCAB_FILE,
embedding_matrix_file=EMBEDDING_MATRIX_FILE,
 checkpoint_path=CHECKPOINT_PATH)

 

bidirectional model


encoder.load_model(configuration.model_config(bidirectional_encoder=True),
vocabulary_file=VOCAB_FILE,
embedding_matrix_file=EMBEDDING_MATRIX_FILE,
checkpoint_path=CHECKPOINT_PATH)

 

本文固定链接: http://www.luozhipeng.com/?p=1319
转载请注明: luozhipeng 2018-1-9 于 罗志鹏的BLOG 发表

上一篇: :下一篇
返回顶部