Let's welcome our new Baby AGI overlords

Welcome to the first issue of The Inference Times. Think of this humble newsletter as your source for news from the frontiers of AI.

Front page:

• AutoGPT is the next frontier in prompt engineering, at least according to Karpathy.
  
  The only thing more eerie than chatting with a language model is seeing a language model take action and ‘think’ on its own, creating, prioritizing and executing tasks towards a goal. Apart from being quite eerie, AutoGPT effectively provides GPT more state, stages of execution and iterative discovery than otherwise possible. BabyAGI, and AutoGPT are two examples, but there are a bunch of others based on these - HustleGPT, etc.
  
  Carnegie Mellon even dropped a paper earlier this week, in which they taught an autonomous AI agent to perform autonomous scientific research, designing, planning and executing chemical experiments.
  
  I suspect these approaches will be native to future models, much like some of OpenAIs plugin capabilities have obviated LangChain connectors. But for now, the coolest tools belong to the hackers and tinkerers.
  
  Less wrong has a good overview with some decent speculation on future directions.
  

• Databricks releases Dolly 2.0, based on an open model and instruction-following dataset. The last part is key, since previous scrappy ‘open’ GPT clones models cribbed the instruction-following capabilities from OpenAI responses — technically a no-no. (scrappy clones… and also, the $1.3T giants that rhymes with oogle).
  
  Strategically, a future dominated by closed-source LLMs on hosted APIs is an existential threat to Databricks whose bread-and-butter is making it easier for companies to run models on their own (cloud) infra. Apart from Databricks competitive realities, this will help companies that want LLM magic without shipping sensitive data to MSFT / OpenAI.

• GPT-4 massively reduces the cost of data labeling, saving researchers $500k while dramatically increasing quality.
  
  ML research obviously requires massive amounts of labeling and annotation, but all sorts of research in economics, political science and public health requires similar, painstaking ‘coding’ (not the engineering sort). This will be a boon to researchers everywhere.

🔧 Tool Time:

• Compare language models from most providers. nat.dev provides a dead-simple UI to easily compare the output from all major text models. repl.it provided a quick summary of the backstory.

• HyperWrite released an AI-driven browser. Goes beyond Bing in Microsoft’s Edge browser to actually drive the browser’s actions. Godspeed to the startups staying ahead of Microsoft’s Copilot integrations and OpenAI’s plugin ecosystem.
  

• Rahul (of Ligma fame) and a few friends created a natural language front-end to query and visualize San Francisco’s public datasets. Helpfully, their example question quantifying public defecation reminds me not to move to SF./

• GPT-4 integrated into a slick spreadsheet tool. We’ve seen a few of these as plugins and formula explainers and Microsoft has announced their intensions with Excel, but Equals pulls it all together in a shiny command bar interface.
  

• Build LangchainJS workflows in a GUI, no config files needed.

🧪 Research:

• Have a conversation about a video. VideoChatCaptioner allows users to have a conversation about a video, answering questions about the actions, people and surroundings in a video. More progress towards our multi-modal future.
  

• More multi-modal research, this time doing a roundtrip from Image→Paragraph description→Image. Combines Segment Anything Model and existing captioning models, then feeds it into ChatGPT and uses the description to reconstruct the image.