2019/11/03 by Xiaolei Lu, Bin Ni, Lu, Xiaolei +1
Business, Management and Accounting · Computer Science · Materials Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Intellectual Property and Patents #Machine Learning (cs.LG) #Machine Learning in Materials Science #Topic Modeling
paper · pdf · doi:10.48550/arxiv.1911.06241
openalex publication_date 2019/11/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The automatic classification is a process of automatically assigning text documents to predefined categories. An accurate automatic patent classifier is crucial to patent inventors and patent examiners in terms of intellectual property protection, patent management, and patent information retrieval. We present BERT-CNN, a hierarchical patent classifier based on pre-trained language model by training the national patent application documents collected from the State Information Center, China. The experimental results show that BERT-CNN achieves 84.3% accuracy, which is far better than the two compared baseline methods, Convolutional Neural Networks and Recurrent Neural Networks. We didn't apply our model to the third and fourth hierarchical level of the International Patent Classification - "subclass" and "group".The visualization of the Attention Mechanism shows that BERT-CNN obtains new state-of-the-art results in representing vocabularies and semantics. This article demonstrates the practicality and effectiveness of BERT-CNN in the field of automatic patent classification.