Bert use cases
Bert use cases
Recently, I found this amazing article from Manu Cohen Yashar that talks about application of BERT. I would like to summarize few important things from the article. Original Link: https://medium.com/swlh/nlu-for-everyone-with-bert-7bedaa609a61
Bert use cases
So what can we do with BERT? The NLU community, over the years, defined several standard benchmarks (e.g., GLUE General Language Understanding Evaluation benchmark ) and developed many specific datasets to test the performance on typical NLU tasks. Google announced that BERT “Demonstrated state-of-the-art results on 11 NLP tasks, including the very competitive Stanford Question Answering Dataset (SQuAD v1.1).” So if you are asking: Where can I use BERT? Here is a list of NLU tasks that BERT can help you implement.
- Question answering. (SQUAD 2.0 question answering, Google natural questions task)
- Named Entity Recognition. (GLUE multi-task learning)
- Automatic summarization.
- Sentiment Analysis.
- Text classification.
- Textual entailment & next sentence prediction.
- Coreference resolution. (finding all expressions that refer to the same entity in a text)
- Polysemy resolution and Word sense disambiguation (selection of valid dictionary definitions in context).
- Biomedical domain-specific tasks (BioBERT) Other tasks such as Machine Translation fit less for the fine-tuning. But there is research about methods to leverage BERT in machine translation.
Fine-tuning
To use BERT for a specific NLU task such as question answering an extra layer, specific to that task is put on top of the original BERT network. Then the whole network is trained together on a dataset, specific for that task, with a task-specific loss function. For example: For the Question and Answering task, we can use SQUAD 2.0 as a fine-tuning dataset. The SQUAD (Stanford Question Answering Dataset) is a dataset consisting of questions posed on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.