大局和重点 这波AI浪潮特别是关于建立在生成式AI之上——它在GPT-3之后开始，并在ChatGPT（2022年 11月）之后迅速增长。 我们只关注建立在LLMs之上的应用程序。因此，目前我们将忽略图像生成、语音处理、机 器人等领域。 如果我们看当前的生成式AI堆栈，有4层：基础设施，（基础）模型开发，应用开发工 具，最终应用。我们将只关注应用程序。 在AI应用程序中，有横向应用程序（LLMs之上的一层）、企业应用程序（营销、销售、客 户支持等）和垂直应用程序（法律、保险、编码等）。我们将只关注垂直应用程序。 AI可以帮助解决的问题类型 任务媒介是自然语言 任务的许多示例出现在训练数据中 不需要超过90%的准确率 AI的价值主张 所 …

Read More

The big picture and the focus This wave of AI is specifically about building on top of Generative AI - it got started after GPT-3, and the explosive growth really started after ChatGPT (Nov 2022). We are only looking into building on top of LLMs. With that, at this moment we will ignore fields like image generation, …

Read More

Why did the teddy bear say no to dessert? Because she was stuffed! What do you call a dinosaur that is sleeping? A dino-snore! Why did the math book look sad? Because it had too many problems! What do you call a bear with no teeth? A gummy bear! Why did the student eat his homework? Because his teacher told him it was …

Read More

FT with LoRA RAG vs long context window Models/things to explore FT mixtral PyTorch internals Transformer as a computer Andrej Karpathy Transformer converged Andrej Karpathy AI big ideas LeCun FT with LoRA If you have ever fine-tuned LLMs using Lora and felt disappointed after merging the adapter with the base model, …

Read More

Lightning Inheritance Cashu

Read More

key repeat on Mac To enable key repeat on mac for VSCode + Vim, run the following commands in the Terminal: 1# For VSCode: 2defaults write com.microsoft.VSCode ApplePressAndHoldEnabled -bool false 3 4# To enable global key-repeat 5defaults write -g ApplePressAndHoldEnabled -bool false Certificate self-signed: …

Read More

Push a local git repo to a server 1## Create a bare repo on the server 2git init --bare 3## Add the remote origin on the local server 4git remote add origin shu@linode:git-repos/docs 5## Push to the server with update option 6git push -u origin master git-annex workflow for me Still setup a bare git repo in the cloud: …

Read More

03-09-2018 Signatures of RWS monads Reader Monad: 1ask :: Reader r r 2asks :: (r -> a) -> Reader r a 3local :: (r -> r) -> Reader r a -> Reader r a 4runReader :: Reader r a -> r -> a Writer Monad: 1tell :: w -> Writer w () 2execWriter :: Writer w a -> w 3runWriter :: Writer w a -> (a, w) …

Read More