ChatGPT’s Underlying AI Transformer Attention Model
The ChatGPT transformer is neural-network based with self-attention layers, allowing the model to weigh different words/phrases. See Vaswani et al. 2017 paper “Attention is All You Need.”
Krishna Bala, Ph.D.
I am an engineering leader creating technology solutions that accelerate cancer research and diagnosis.
You may also enjoy
Founder Mode
Paul Graham’s blog discusses Founder mode versus Manager mode: What it takes to build a successful scalable company!
Google DeepMind’s new AlphaFold can model a much larger slice of biological life
The already amazing AlphaFold 2 AI system only knew about amino acids, but now, AlphaFold 3 can in principle predict where a drug binds to a protein!
Engineers and Tech Leaders Looking to Learn And Grow
Great perspective and advice for engineers and tech leaders on growing, learning and contributing!
How to Create a Culture Of Performance in Teams
A great episode on significantly accelerating innovation using “Tiger Teams”. An OKRs focus is not sufficient!