Bigger LLMs and Adobe's Generative Filler

published15 days ago
1 min read

The Progress Report

Hi Reader,

We may be coming to a state near you. After our presentation in San Diego, we are lining up a few things in the coming weeks and months. I’ll let you know where and when as things are solidified.

Meanwhile work with our beta group is progressing nicely. We’ve got some really neat things happening.

Here are a few links worth your time.

Worth Reading

Meta recently released Megabyte, which was tested with 1.2M tokens. Reddit user ShotgunProxy breaks it down in detail here, but he says:

  • Instead of using individual tokens, the researchers break a sequence into "patches." Patch size can vary, but a patch can contain the equivalent of many tokens. Think of the traditional approach like assembling a 1000-piece puzzle vs. a 10-piece puzzle. Now the researchers are breaking that 1000-piece puzzle into 10-piece mini-puzzles again.
  • The patches are then individually handled by a smaller model, while a larger global model coordinates the overall output across all patches.

Worth Seeing

Adobe just released their generative filler and this demo video is pretty incredible. You really have to see it to believe it!

Worth Hearing

It seems from all the news that artificial intelligence is new, but it’s been around for decades in different forms. Here’s a video from more than a decade ago about robotic limbs, using an earlier version of AI to control them. I love this quote from Jan Sherman who suffers from spino-cerebellar degeneration, she can only move her face and a few muscles in her neck:

“I've always believed there's a purpose to my illness. I didn't think I would ever find out what it was in my lifetime, and here came this study where they needed me. You know, they couldn't just pick any tom, dick or harry off the street. And in a few years, the quadriplegics and the amputees this is just going to help.”


Share the newsletter…

Here’s a link to today’s newsletter

Sign up for the newsletter…

If you got this from a friend, you can sign up to get the newsletter for free every week morning here.

The Progress Report w/ Caleb Hicks

👋 Hi there! I'm Caleb Hicks. This is The Progress Report, a weekly newsletter on the future of education – where we look at AI as a tool, a learning outcome, and a disrupting agent in education. Join 1000+ education futurists free.

Read more from The Progress Report w/ Caleb Hicks

Minecraft controlled by AI

about 9 hours ago
1 min read

6 tools and travel update

7 days ago
2 min read

OpenAI Open Source

22 days ago
1 min read