Understanding BigBird | Is Google BigBird gonna be the new leader in the NLP domain?
In 2018, Google released a big fish in the market termed BERT(Bidirectional Encoder Representations from Transformers). It brought state-of-the-art accuracy on NLP and NLU tasks. The distinguishing factor was “Bidirectional Training” which was proved to be holding great potential in giving a deeper sense of language context. Before BERT, models used to analyze text sequences either from left to right or combined left-to-right and right-to-left training.