I have tried a number linear regressions, and though the standard ones are all good (e.g., lm.fit is really nicely fast), the fastLM in https://www.rdocumentation.org/packages/RcppEigen/versions/0.3.4.0.2/topics/fastLm gets the cake. It is truly amazingly fast. Thanks Doug, Dirk, Romain, and Yixuan.