Hitachi announced today that it has developed a basic artificial intelligence (AI) technology that analyzes huge volumes of Japanese text data on issues that are subject to debate, and presents in Japanese both affirmative and negative opinions on those issues together with reasons and grounds.
In this research, Hitachi applied deep learning to the process of distinguishing sentences representing reasons and grounds for opinions, eliminating the need for a dedicated program to be prepared for each language and thus enabling the creation of a general-purpose system analyzing text data in any language. Previously, Hitachi developed a basic AI technology which analyzed huge volumes of English text data and presented opinions in English. This time, Hitachi incorporated this technology into a new AI technology for the Japanese language to meet the needs of Japanese enterprises.
Hitachi has been developing a basic AI technology that analyzes huge volumes of English text data and presents opinions in English to help enterprises make business decisions. The original technology required rules of grammar specific to the English language to be programmed, to extract sentences representing reasons and grounds for opinions.
Hitachi added an “attention mechanism” which support deep learning to estimate which words and phrases are worthy of attention in texts like news articles and research reports. The “attention mechanism” helps the system to grasp the points that require attention, including words and phrases related to topics and values.
This method enables the system to distinguish sentences which have a high probability of being reasons and grounds from text data in any language. The technology developed will be core technology in achieving a multi-lingual AI system capable of offering opinion, according to Hitachi.