论文阅读-IRGAN_A_Minimax_Game_for_Unifying_Generative_and_Discriminative_Information_Retrieval_Models

参考文献

IRGAN: A Minimax Game for Unifying Generative and Discriminative Information Retrieval Models

Information retrieval GAN

再读IRGAN,聊聊Code与Formulation的差异

IRGAN SIGIR paper experimental code

关键问题

  1. 研究对象?

    Information retrieval(IR) is to provide a list of documents(e.g., textual documents, information items, answers) given a query. There are two major schools of thinking when coming to IR theory and modeling.

    Generative school: focused on describing how a document is generated from a given information need: q → d.

    Discriminative(Classification) school: Consider documents and queries jointly as features and predicts their relevancy from a large amount of training data: q + d → r.

  2. 生成式检索模型?

    $p_θ (d|q, r)$ tries to generate (or select) relevant documents from the candidate pool for the given query $q$, whose goal is to approximate the true relevance distribution over documents $p_{true} (d|q, r)$ as much as possible.

    在推荐系统中,作者使用的是矩阵分解,即下图中的函数$g$是右侧的函数$s$。

  3. 判别式检索模型?

    $f_ϕ (q,d)$ tries to discriminate well-matched query-document tuples $(q,d)$ from ill-matched ones, which is in fact simply a binary classifier, and we could use 1 for truly match pair while 0 for those that do not really match.

  4. 总体框架?

    $p_θ (d|q, r)$ tries to approximate the true relevance distribution over documents $p_{true} (d|q, r)$ as much as possible.

    $f_ϕ (q,d)$ tries to discriminate well-matched query-document tuples (q,d) from ill-matched ones.

  5. 推广到pair-wise的情况?

    we have a set of labeled document pairs $R_n=\{│d_i≻d_j \}$. The generator $G$ would try to generate document pairs that are similar to those in $R_n$, i.e., the correct ranking. The discriminator $D $ would try to distinguish such generated document pairs from those real document pairs.

  6. 优化算法?

    判别器使用的是随机梯度下降,而生成器为了处理自然语言处理中的问题,所以使用的策略梯度下降:

实验复现(1)

实验的环境很奇怪,用的是python2的语法和tensorflow,这让我这个新时代的程序员很为难,因为tensorflow是和python3兼容的。改了一圈两者不兼容的地方,一个是print函数的使用,一个是把dict.key()对象转化为list对象,还有一个是sort函数的使用,然而还是卡住了,,卡住的原因是之前训练好的生成器参数用pkl的格式存储的,怎么读取都出错,换了各种打开和编码方式,,,求大佬指导啊

目前的解决方案是去掉参数读取,把用到的地方换为None,然而跑了一天一夜,限于调参的水平,NDCG只跑到了二十几,,,

实验复现(2)

经过幸运的尝试,终于解决了上面的问题,代码如下

1
2
3
4
param = pickle.load(
open(os.path.join(workdir, "model_dns_ori.pkl"), 'rb'),
encoding='latin1'
)

最后基本达到了论文中的结果!