Quadrillion-parameter transformers and how to train them(qlabs.sh)
60 points by frozenseven | 48 comments
Q Labs research post presenting a data-constrained scaling law with ensembling that prescribes far larger transformer models, up to quadrillion parameters, for fixed datasets.