跳转到主要内容

参考文献

  1. Bentley, J., 1999. The Back of the Envelope（信封背面的估算）. IEEE Software 16(5), 121.
  2. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., et al., 2020. Language Models Are Few-Shot Learners（语言模型是少样本学习者）. Advances in Neural Information Processing Systems 33, 1877–1901.
  3. Choquette, J., Giroux, O., Foley, D., 2018. Volta: Performance and Programmability（Volta：性能与可编程性）. IEEE Micro 38(2), 42–52.
  4. Dean, J., 2022. A Golden Decade of Deep Learning: Computing Systems & Applications（深度学习的黄金十年：计算系统与应用）. Daedalus 151(2), 58–74.
  5. Devlin, J., Chang, M.W., Lee, K., Toutanova, K., 2018. BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding（BERT：面向语言理解的深度双向 Transformer 预训练）. Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 4171–4186.
  6. Fedus, W., Zoph, B., Shazeer, N., 2022. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity（Switch Transformer：通过简单高效的稀疏机制扩展至万亿参数模型）. Journal of Machine Learning Research 23(120), 1–39.
  7. Jouppi, N.P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., et al., 2017. In-Datacenter Performance Analysis of a Tensor Processing Unit（张量处理单元在数据中心中的性能分析）. Proceedings of the 44th Annual International Symposium on Computer Architecture, 1–12.
  8. Jouppi, N.P., Yoon, D.H., Kurian, G., Li, S., Patil, N., Laudon, J., et al., 2020. A Domain-Specific Supercomputer for Training Deep Neural Networks（用于训练深度神经网络的领域专用超级计算机）. Communications of the ACM 63(7), 67–78.
  9. Jouppi, N.P., Yoon, D.H., Ashcraft, M., Gottscho, M., Jablin, T.B., Kurian, G., et al., 2021. Ten Lessons from Three Generations Shaped Google’s TPU v4（三代产品的十项经验塑造了 Google TPU v4）. 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA), 1–14.
  10. Jouppi, N., Kurian, G., Li, S., Ma, P., Nagarajan, R., Nai, L., et al., 2023. TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings（TPU v4：提供嵌入硬件支持、可光学重配置的机器学习超级计算机）. Proceedings of the 50th Annual International Symposium on Computer Architecture, 1–14.
  11. Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., et al., 2021. Highly Accurate Protein Structure Prediction with AlphaFold（使用 AlphaFold 高精度预测蛋白质结构）. Nature 596(7873), 583–589.
  12. Krizhevsky, A., Sutskever, I., Hinton, G.E., 2012. ImageNet Classification with Deep Convolutional Neural Networks（使用深度卷积神经网络进行 ImageNet 分类）. Advances in Neural Information Processing Systems 25.
  1. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., et al., 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library（PyTorch：命令式风格的高性能深度学习库）. Advances in Neural Information Processing Systems 32.
  2. Patterson, D., Gonzalez, J., Hölzle, U., Le, Q., Liang, C., Munguia, L.M., et al., 2022. The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink（机器学习训练的碳足迹将先趋于平稳，再逐步缩小）. Computer 55(7), 18–28.
  3. Raihan, M.A., Goli, N., Aamodt, T.M., 2019. Modeling Deep Learning Accelerator Enabled GPUs（支持深度学习加速器的 GPU 建模）. 2019 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). IEEE, 79–92.
  4. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B., 2022. High-Resolution Image Synthesis with Latent Diffusion Models（使用潜在扩散模型合成高分辨率图像）. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10684–10695.
  5. So, D., Le, Q., Liang, C., 2019. The Evolved Transformer（进化得到的 Transformer）. International Conference on Machine Learning, 5877–5886.
  6. So, D., Mańke, W., Liu, H., Dai, Z., Shazeer, N., Le, Q.V., 2021. Searching for Efficient Transformers for Language Modeling（搜索用于语言建模的高效 Transformer）. Advances in Neural Information Processing Systems 34, 6010–6022.
  7. Strubell, E., Ganesh, A., McCallum, A., 2019. Energy and Policy Considerations for Deep Learning in NLP（自然语言处理深度学习的能源与政策考量）. Annual Meeting of the Association for Computational Linguistics.
  8. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., et al., 2017. Attention Is All You Need（注意力机制即可满足所需）. Advances in Neural Information Processing Systems 30.
  9. Zhao, M., Agarwal, N., Basant, A., Gedik, B., Pan, S., Ozdal, M., et al., 2022. Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Training: Industrial Product（理解大规模深度推荐模型训练的数据存储与摄取：工业产品）. Proceedings of the 49th Annual International Symposium on Computer Architecture, 1042–1057.