[Objective/Significance] Considering that static topic models are difficult to meet users' dynamic analysis needs, in order to solve the problems of high computational costs or deep influence from subjective factors in existing dynamic topic models, this study proposes a text time window partitioning algorithm based on the LDA model, starting from time window similarity. [Method/Process] This study constructs a time window similarity index that integrates differences between time windows and consistency within time windows. This study constructs a time window partitioning algorithm based on this indicator and conducts empirical research using the innovation research field as an example. [Results/Conclusions] By analyzing the average JS divergence between topics under the optimal number of topics within each time window, as well as the average JS divergence between different topics between adjacent time windows, the partitioning results obtained by the algorithm proposed in this study are significantly better than those obtained by multiple fixed time window length partitioning methods, verifying the effectiveness of using the improved LDA model proposed in this study for text time window partitioning. The algorithm proposed in this study to some extent solves the shortcomings of existing dynamic topic models such as high computational costs and strong subjectivity, increases the objectivity and accuracy of text time window partitioning results, and can provide technical support for related research such as theme evolution.
Long Yixuan, Wang Xiaogang, Zhou Ziwei, Wang Rongsheng, Yi Huifang.
Research on Text Time Window Partition Based on LDA Model[J].
Science Focus, 2024, 19(2): 34-45 DOI:10.15978/j.cnki.1673-5668.202402004
连续时间分布划分中，比较有代表性的是TOT（Topic over Time）主题模型[4]和cDTM（Continuous Time Dynamic Topic Models）模型[5]，通过分析可知，该类模型存在以下问题：首先，该类模型规定每个时间窗内的主题数量是恒定的，因此该模型只能揭示主题强度随连续时间的变化情况，并不能深入到主题内容层面[6]；其次，因为该模型将时间视为随机变量，因此时间变化十分频繁，当需要建模的数据量较大时，计算成本非常高，效率很低；再次，不同阶段虽然数据量不同，但由于默认主题数量恒定，会在一定程度影响主题发现的合理性[7]。
本文用时间窗口差异性指标度量不同时间窗口之间主题内容的差异性。具体计算过程如下：首先，在相邻时间窗口使用LDA模型开展主题建模。最优主题数目依据JS散度和困惑度综合判定，本文采用的是关鹏等（2016）提出的LDA最优主题个数改进方法，该方法从理论和实践两个层面证实可以在一定程度上缓解困惑度带来的主题数量过多问题，给出的最优主题个数更加合理[21]。之后，获取每个主题模型下的主题-词汇概率，得到每个主题对应的前N个主题词的词向量为$\left(\operatorname{word}_{i 1}^{l d a_{j}}, \operatorname{word}_{i 2}^{l d a_{j}}, \ldots, \operatorname{word}_{i N}^{l d a_{j}}\right) $，其中，N=1,2,3,…,N代表前N个主题词，i=1,2,…,K代表最优主题个数，j=t，t+∆t代表相邻时间窗口。最终获得两个时间窗口中各自的主题向量：
$V_{l d a_{j}}=\left[\begin{array}{l} \left(\operatorname{word}_{11}^{l d a_{j}}, \operatorname{word}_{12}^{l d a_{j}}, \ldots, \operatorname{word}_{1 N}^{l d a_{j}}\right) \\ \left(\operatorname{word}_{21}^{l d a_{j}}, \operatorname{word}_{22}^{l d a_{j}}, \ldots, \operatorname{word}_{2 N}^{l d a_{j}}\right) \\ \ldots \\ \left(\operatorname{word}_{K 1}^{l d a_{j}}, \operatorname{word}_{K 2}^{l d d_{j}}, \ldots, \operatorname{word}_{K_{l d a_{t}}N}^{l d a_{j}}\right) \end{array}\right] $
依据主题-词向量，分别计算两个时间窗口各自主题两两之间的JS散度。定义时间窗口ldat的主题数量为Kldat，时间窗口ldat+∆t的主题数量为Kldat+∆t 。具体步骤如下：在时间窗口ldat中选取任意主题的词向量与时间窗口ldat+∆t中任意主题的词向量进行主题相似度计算，记做 $J S_{K_{m} K_{n}}^{l d a_{l} d a_{t+\Delta t}}\left[m \epsilon\left(1 \ldots K_{l d a+\Delta t s}\right)\right] $，所有搭配情况穷举后求和，记做 $\sum_{m=1}^{K_{l d a_{t}}}\left(\sum_{n=1}^{K_{l d a_{t+\Delta t}}} J S_{K_{m} K_{n}}^{l d a_{t} l d a_{t+\Delta t}}\right) $。取相似度之和的倒数为差异性衡量指标，最后定义ldat与ldat+∆t之间的差异性指标如下所示：
$F_{ {difference }}=\frac{C_{K_{l d a_{t}}}^{1} C_{K_{l d a_{t+\Delta t}}}^{1}+1}{\sum_{m=1}^{K_{l d a_{t}}}\left(\sum_{n=1}^{K_{l d a_{t+\Delta t}}} J S_{K_{m} K_{n}}^{l d a_{t} l d a_{t+\Delta t}}\right)+1}$
JS散度的取值范围是0到1，JS为0时相似度最高。所以分母累加之和的取值范围为1到$K_{l d a_{t}} \times K_{l d a_{t+\Delta t}}+1$，这里分子分母均加1是为了防止出现分母为0的情况，$F_{{difference }}$的取值范围为$\left(1, K_{l d a_{t}} \times K_{l d a_{t+\Delta t}}+1\right) $，$F_{{difference }}$越小（越趋近于1）差异性越大，时间窗口划分效果越好。
（2）时间窗口内一致性指标构建
本文使用时间窗口内主题一致性指标度量同一时间窗口内主题内容的相似度。
首先，获得时间窗口中的主题向量：
$V_{l d a_{j}}=\left[\begin{array}{l} \left(\operatorname{word}_{11}^{l d a_{j}}, \operatorname{word}_{12}^{l d a_{j}}, \ldots, \operatorname{word}_{1 N}^{l d a_{j}}\right) \\ \left(\operatorname{word}_{21}^{l d a_{j}}, \operatorname{word}_{22}^{l d a_{j}}, \ldots, \operatorname{word}_{2 N}^{l d a_{j}}\right) \\ \ldots \\ \left(\operatorname{word}_{K 1}^{l d a_{j}}, \operatorname{word}_{K 2}^{l d a_{j}}, \ldots, \operatorname{word}_{K_{l d a_{t} N}^{l d a_{j}}}\right) \end{array}\right]$
本文选择Web of Science核心合集（简称WoS核心合集）作为来源数据库。通过梳理已有研究发现，目前有关创新研究领域的综述研究在构建检索式时大多采用TS=“innovat*”或TS=“innovation”，即仅用“创新”单一名词构建检索式。但在创新研究发展进程中，表征该领域的通用名词不是一成不变的，不同的研究者随着领域的发展以不同的名称对创新研究相关的研究活动进行了标记。从最开始20世纪60年代普遍使用的“科学政策”“研究政策”或“科学技术研究”，到20世纪70年代和80年代广泛使用的科学、技术和创新三个词的多种组合，再到20世纪90年代，“创新”逐渐成为创新研究的主流名词，“创新”逐渐包容了“科学”“技术”“政策”等多个术语。从表征创新研究的通用名词变化过程来看，本文认为仅基于“创新”单一名词构建检索式是远远不够的。为构建更加全面的检索式，本文基于前人对创新研究定义的梳理以及范围的界定，在确定“innovat*”检索词和相关研究领域界定的前提之下，进一步补充不同领域的核心检索关键词。最终确定的检索策略表如表1所示，检索时间是2022年12月15日，检索到145072条，之后对检索结果根据标题和关键词人工进行过滤，最终清洗后的数据为130130条。
基于时间窗口划分与主题建模结果分析可知，创新研究领域至少已有70年的发展历史。从创新研究领域整体研究主题数量来看，按照时间顺序呈现“多—少—多”这一规律。自20世纪50年代，研究人员开始尝试系统研究与创新、技术或科学相关的问题，并希望可以借此推进经济发展，这些研究人员来自不同的学科，所研究的基础概念和使用的科学方法大相径庭，因此形成了众多的学科知识研究主题，但他们整体研究目标趋同，有着共同的利益，渴望并且尝试互相“接触”，这也为后来创新研究学科知识主题的融合奠定了基础。之后的20世纪70年代至20世纪末研究人员开始逐渐尝试合作文献、共享方法和概念，创新研究领域逐步走向成熟，研究方向也逐渐向经济发展与技术改变（economic development and technological change）、产业技术创新与创新政策（industrial technology innovation and innovation policy）、技术采纳与创新扩散（technology adoption and innovation diffusion）、产业创新与新产品的研发过程（industrial innovation and development process of new products）、产品创新的战略管理与知识转移（strategic management of product innovation and knowledge transfer）等方向靠拢。进入21世纪之后，创新研究开始呈现多元化道路，在技术研发相关的经济学（economics of R&D）、产业创新管理与企业资源观（management of industrial innovation and the resource-based view of the firm）、组织与创新（organizations and innovation）、创新系统（innovation systems）等几个大的研究方向的基础上逐渐分化。如今，技术研发相关的经济学逐渐分化为创新与进化经济学、技术和创新经济学、内生增长理论等；产业创新管理与企业资源观逐渐分化为创新管理与创新过程的互动模式、企业资源观等；组织与创新逐渐分化为组织创新、组织学习与知识管理、技术创新与组织制度之间的交互、组织合作与开放创新等；创新系统逐渐分化为国家创新系统、区域创新体系、部门创新体系等。
(JiangZ R, ChenY, GaoL C, et al. A supervised dynamic topic model[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2015, 51(2): 367-376.)
[6]
WangX R, McCallumA. Topics over time: a non-markov continuous-time model of topical trends[C]. Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, NY, USA, ACM, 2006, 424-433.
DingW, ChenC. Dynamic topic detection and tracking: a comparison of HDP, C-word, and cocitation methods[J]. Journal of the Association for Information Science and Technology, 2014, 65(10): 2084-2097.
(GuiX Q, ZhangJ, ZhangX M, et al. Survey on temporal topic model methods and application[J]. Computer Science, 2017, 44(2): 46-55.)
[11]
DerntlM, GünnemannN, KlammaR. A dynamic topic model of learning analytics research[C]. Proceedings of International Symposium on Instrumentation and Measurement, Sensor Network and Automation. Germany: IEEE, 2013: 436-439.
[12]
HaT, BeijnonB, KimS, et al. Examining user perceptions of smartwatch through dynamic topic modeling[J]. Telematics and Informatics, 2017, 34(7): 1262-1273.
[13]
NallapatiR M, DitmoreS, LaffertyJ D, et al. Multiscale topic tomography[C]. Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. San Jose, California, USA, ACM, 2007: 520-529.
[14]
KimE, ChoY, KimW. Dynamic patterns of technological convergence in printed electronics technologies: Patent citation network[J]. Scientometrics, 2014, 98(2): 975-998.
(WuF F, ZhangY R, HuangL C, et al. Multi-dimension dynamic evolution analysis of technology topics based on AToT by taking grapheme technology as an example[J]. Library and Information Service, 2017, 61(5): 95-102.)
(YiH F, WuH, LiC, et al. Technology topics evolution of graphene based on topic life cycle and technology entropy[J]. Journal of Intelligence, 2019, 38(2): 64-70.)
(YanD W, SuQ, ZhangX Y. Research on frontier detection in scientific field based on sequential topic association evolution[J]. Information Studies (Theory & Application), 2019, 42(7): 144-150.)
(ZhangX L, ChenY. Analysis of the evolutionary path of international open access research theme and its enlightenment[J]. Jiangsu Science and Technology Information, 2023, 40(35): 8-14.)
[23]
FangD, YangH, GaoB, et al. Discovering research topics from library electronic references using latent Dirichlet allocation[J]. Library Hi Tech, 2018, 36(3): 400-410.
(WangY F, FuZ, ChenB K. Topic identification of scientific literature based on LDA topic model: comparative analysis of two views of global and discipline[J]. Information Studies (Theory & Application), 2016, 39(7): 121-126.)
(GuanP, WangY F. Topic mining in scientific literature based on LDA topic model and life cycle theory[J]. Journal of the China Society for Scientific and Technical Information, 2015, 34(3): 286-299.)
(GuanP, WangY F. Identifying optimal topic numbers from sci-tech information with LDA model[J]. New Technology of Library and Information Service, 2016(9): 42-50.)
FagerbergJ, VerspagenB. Innovation studies—the emerging structure of a new scientific field[J]. Research Policy, 2009, 38(2): 218-233.
[40]
FagerbergJ, FosaasM, SapprasertK. Innovation: exploring the knowledge base[J]. Research Policy, 2012, 41(7): 1132-1153.
[41]
MartinB R. The evolution of science policy and innovation studies[J]. Research Policy, 2012, 41(7): 1219-1239.
[42]
ShafiqueM. Thinking inside the box? Intellectual structure of the knowledge base of innovation research (1988-2008)[J]. Strategic Management Journal, 2013, 34(1): 62-93.
PDF
(4173KB)
