Will the tagger see a reply on another wall
For example, the entities of person names such as “ Xavier” can be recognized by predicting a common name (label word) such as “ John”.īenefiting from the reuse of the entire architecture of PLMs, our method can directly utilize plugin vectors to switch tasks without making any modifications to the model, as shown on the right side of Figure 1. We take high-frequency words predicted by PLM to serve as the label words for corresponding labels. The change to the label set leads to an unavoidable modification of the model classifier, but we overcome this problem by reformulating sequence labeling to the task of predicting the label words. The proposed method allows a frozen PLM to perform different tasks without modifying the model but instead by switching plugin vectors on the input. To overcome this limitation, in this paper, we propose plug-tagger, a plug-and-play framework for various sequence labeling tasks. When performing different tasks, the fine-tuning method needs to switch the model, while the plug-and-play method only needs to switch the plugin on input. Figure 1: Comparison of fine-tuning (left) and plug-and-play method (right). Thus, generation tasks performed by the PLM can be switched by simply switching the prefix vectors at the inputs. These vectors work as a prompt for the model, manipulating it to generate outputs required by different tasks. Prefix-tuning does not modify the model architecture and instead prepends a sequence of continuous task-specific vectors to the inputs.
#Will the tagger see a reply on another wall full
By inserting a task-specific adapter layer between layers of the PLM, adapter-tuning merely needs to store the parameters of one shared PLM and multiple task-specific layers to perform different tasks, rather than storing a full copy of the PLM for each task. These approaches focus on keeping parameters of the PLMs fixed and optimizing only a few parameters for each task. ( 2019) and prefix-tuning Li and Liang ( 2021) are popular methods for achieving pluggability. READ FULL TEXT VIEW PDFĪdapter-tuning Raffel et al. Times faster than non-plug-and-play methods while switching different tasks Proposed method can achieve comparable performance with standard fine-tuning Results on three sequence labeling tasks show that the performance of the Pre-trained language model is allowed to perform different tasks. Result, by simply switching the plugin vectors on the input, a frozen Manipulate the model predictions towards the corresponding label words. Then, task-specific vectors are inserted into the inputs and optimized to Specifically, for each task, a label word set is firstĬonstructed by selecting a high-frequency word for each class respectively, and Of classification to totally reuse the architecture of pre-trained models for
In this work, we propose the use of label word prediction instead Since different label sets demand changes to the architecture of the modelĬlassifier. However, sequence labeling tasks invalidate existing plug-and-play methods Tasks by simply inserting corresponding continuous vectors into the inputs. Prefix-tuning was shown to be a plug-and-play method on various text generation Plug-and-play functionality allows deep learning models to adapt well toĭifferent tasks without requiring any parameters modified.