Luận án Tiến sĩ Khoa học máy tính: Advanced deep learning models and applications in semantic relation extraction

Furthermore, experiments on the task of RE proved that data representation is one of the most influential factors to the model’s performance but still has many limitations. We propose a compositional embedding that combines several dominant linguistic as well as architectural features and dependency tree normalization techniques for generating rich representations for both words and dependency relations in the SDP
Nội dung trích xuất từ tài liệu:
Luận án Tiến sĩ Khoa học máy tính: Advanced deep learning models and applications in semantic relation extraction VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF ENGINEERING AND TECHNOLOGY CAN DUY CATADVANCED DEEP LEARNING MODELS AND APPLICATIONS IN SEMANTIC RELATION EXTRACTION MASTER THESIS Major: Computer Science HA NOI - 2019 VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF ENGINEERING AND TECHNOLOGY Can Duy Cat ADVANCED DEEP LEARNING MODELS AND APPLICATIONS IN SEMANTIC RELATION EXTRACTION MASTER THESIS Major: Computer ScienceSupervisor: Assoc.Prof. Ha Quang Thuy Assoc.Prof. Chng Eng Siong HA NOI - 2019AbstractRelation Extraction (RE) is one of the most fundamental task of Natural Language Pro-cessing (NLP) and Information Extraction (IE). To extract the relationship between twoentities in a sentence, two common approaches are (1) using their shortest dependencypath (SDP) and (2) using an attention model to capture a context-based representationof the sentence. Each approach suffers from its own disadvantage of either missing orredundant information. In this work, we propose a novel model that combines the ad-vantages of these two approaches. This is based on the basic information in the SDPenhanced with information selected by several attention mechanisms with kernel filters,namely RbSP (Richer-but-Smarter SDP). To exploit the representation behind the RbSPstructure effectively, we develop a combined Deep Neural Network (DNN) with a LongShort-Term Memory (LSTM) network on word sequences and a Convolutional NeuralNetwork (CNN) on RbSP. Furthermore, experiments on the task of RE proved that data representation is oneof the most influential factors to the model’s performance but still has many limitations.We propose (i) a compositional embedding that combines several dominant linguisticas well as architectural features and (ii) dependency tree normalization techniques forgenerating rich representations for both words and dependency relations in the SDP. Experimental results on both general data (SemEval-2010 Task 8) and biomedicaldata (BioCreative V Track 3 CDR) demonstrate the out-performance of our proposedmodel over all compared models.Keywords: Relation Extraction, Shortest Dependency Path, Convolutional Neural Net-work, Long Short-Term Memory, Attention Mechanism. iiiAcknowledgementsI would first like to thank my thesis supervisor Assoc.Prof. Ha Quang Thuy of theData Science and Knowledge Technology Laboratory at University of Engineering andTechnology. He consistently allowed this paper to be my own work, but steered me inthe right the direction whenever he thought I needed it. I also want to acknowledge my co-supervisor Assoc.Prof Chng Eng Siong fromNanyang Technological University, Singapore for offering me the internship opportuni-ties at NTU, Singapore and leading me working on diverse exciting projects. Furthermore, I am very grateful to my external advisor MSc. Le Hoang Quynh, forinsightful comments both in my work and in this thesis, for her support, and for manymotivating discussions. In addition, I have been very privileged to get to know and to collaborate withmany other great collaborators. I would like to thank BSc. Nguyen Minh Trang andBSc. Nguyen Duc Canh for inspiring discussion, and for all the fun we have had overthe last two years. I thank to MSc. Ho Thi Nga and MSc. Vu Thi Ly for continuoussupport during the time in Singapore. Finally, I must express my very profound gratitude to my family for providing mewith unfailing support and continuous encouragement throughout my years of study andthrough the process of researching and writing this thesis. This accomplishment wouldnot have been possible without them. ivDeclarationI declare that the thesis has been composed by myself and that the work has not besubmitted for any other degree or professional qualification. I confirm that the worksubmitted is my own, except where work which has formed part of jointly-authoredpublications has been included. My contribution and those of the other authors to thiswork have been explicitly indicated below. I confirm that appropriate credit has beengiven within this thesis where reference has been made to the work of others. The model presented in Chapter 3 and the results presented in Chapter 4 was pre-viously published in the Proceedings of ACIIDS 2019 as “Improving Semantic RelationExtraction System with Compositional Dependency Unit on Enriched Shortest Depen-dency Path” and NAACL-HTL 2019 as “A Richer-but-Smarter Shortest DependencyPath with Attentive Augme ...