Hashbrown v0.4 served fresh with a side of streaming Markdown, local models, threads, and new adapters

Today we are releasing Hashbrown v0.4, with streaming Markdown parsing, experimental support for browser local models in Chrome & Edge, threads, new adapters, and a whole host of improvements and fixes to our Angular and React libraries.

Hashbrown is an agent framework that runs in the browser. It implements tool calling, threads, structured outputs, streaming, and code execution all in the client. Where an agent built in LangChain might be used to perform research or make changes to a system, a Hashbrown agent is all about the interactivity layer of apps. In their simplest form, Hashbrown agents can speed users up through forms or create dynamic shortcuts to new destinations. At their most complex, they can generate user interfaces using your existing Angular or React components, have long conversations with users, and even generate and execute JavaScript on the fly.

With v0.4, we're continuing to focus on the set of primitives we think developers will need to build agents in the browser.

Release at a glance

Magic Text: streaming Markdown parser with inline citations.
Local models: experimental Chrome/Edge on-device runtimes with structured outputs.
Threads: opt-in thread mode that restores history and sends deltas.
Adapters: Ollama, AWS Bedrock, and Anthropic.

Parse Streaming Markdown with Magic Text

The first special of the day is Magic Text, Hashbrown's headless streaming Markdown parser.

LLMs are commonly reinforced to generate valid Markdown in their outputs. Unfortunately, most Markdown parsers are designed to parse complete Markdown chunks. The result when using these parsers on LLM-streamed Markdown is the appearance of broken Markdown syntax when rendering intermediate chunks. Additionally, most Markdown parsers don't have support for resolving and rendering linkable citations.

Magic Text is designed to be a cross-framework solution to this problem, and is heavily inspired by Streamdown. It optimistically parses Markdown, handles inline citations, and gives developers complete control over the rendering pipeline. It also leverages the Intl.Segmenter API to gracefully segment incoming text, letting developers add subtle animations to text as it arrives in the application.

Use the slider below to simulate streaming Markdown through Hashbrown's Magic Text parser:

We have a full recipe written up for Angular developers to start implementing Magic Text in their apps. We'll be bringing Magic Text to React soon, and in the interim we encourage React developers to take advantage of Streamdown.

Local Models

Next up on the menu is experimental support for local models. I often get asked, "Why have an agent framework in the browser in the first place?" and I often point to the emergence of local models as the main driving force. Chrome and Edge both ship a Prompt API behind a few feature flags. When enabled, web apps can take advantage of a local "small" language model. It can handle text and structured outputs. Even better, since it's on-device it feels really fast compared to streaming responses from a remote model provider.

To try it out, first you'll need to enable the right flags in Chrome or Edge. Then, you can use to connect a Hashbrown hook or resource to the local model:

const { output } = useStructuredCompletion({
  input: 'Portland, OR',
  model: experimental_local(),
  system: `You are a travel assistant returning JSON that matches the schema strictly.`,
  schema: s.object('Weekend itinerary', {
    title: s.streaming.string('Short title for the plan'),
    summary: s.streaming.string('One sentence summary'),
    stops: s.streaming.array(
      'Stops on the trip',
      s.object('Destination', {
        name: s.streaming.string('Name of the stop'),
        description: s.streaming.string('What happens here'),
        durationHours: s.number('Approximate time spent at the stop'),
      }),
    ),
  }),
});

I recommend checking out the full recipe to understand how to build with them. My hope is that, in the years to come, more and more of Hashbrown is happening client-side. Local models in Chrome and Edge feel like an exciting first step, and I'm eager to see if Apple will expose their Foundation Models to web developers in similar ways.

Threads

Hashbrown v0.4 comes with our initial implementation of threads. In Hashbrown, threads accomplish two goals:

Restore message history & state for a given chat or completion
Change Hashbrown's HTTP behavior to only send deltas for a thread instead of the entire message history

Threads are completely opt-in. To take advantage of them, first, you add saveThread and loadThread methods to the adapter of your choice:

HashbrownWriter.stream.text({
  apiKey: WRITER_API_KEY,
  request,
  loadThread: async (threadId: string) => {
    // Return messages for the thread from your DB
  },
  saveThread: async (
    thread: Chat.Api.Message[],
    threadId: string = uuidv4(),
  ) => {
    // Save messages to your DB
    return threadId;
  },
});

Note the behavior of saveThread: it reuses the threadId if present, or generates a new one. When your adapter generates a thread ID, the consuming Angular Resource or React Hook enters thread mode, at which point it only starts sending message deltas to your backend.

Alternatively, you can start a hook or resource in thread mode by supplying the thread ID upfront:

const chat = useChat({
  threadId: 'some-thread-id',
});

On initialization, the hook fetches the thread's message history, giving you a way to rehydrate chats.

All of the hooks and resources expose thread state to you for you to build a nice UI:

const { isSavingThread, isLoadingThread, saveThreadError, loadThreadError } =
  useUiChat(...);

As mentioned, this is totally opt-in. If you don't supply loadThread/saveThread methods to your Hashbrown adapter, it will continue to work like it does today.

New Adapters and Framework Improvements

Hashbrown v0.4 is packed with additional new features and polish:

Our Ollama adapter is now ready to use, letting you tap into open-weight models with Hashbrown.
We've added support for AWS Bedrock for developers building Hashbrown-powered apps in AWS.
Anthropic joins Hashbrown's family of adapters letting you connect Claude to your user experience.
We added and to our React and Angular SDKs, letting you create UI completions without having to set up an instance of chat.
Our React provider now lets you configure the emulateStructuredOutput flag for model providers that do not support structured outputs.

What’s Next

If you’re on v0.3, this release is a low-friction upgrade with few breaking changes. Of particular note, if you've built your own custom adapter, you'll want to read through the source code of one of our existing adapters to see the updates we've made to streaming to support threads.

We're now turning our attention to v0.5. Expect to see the first public APIs around audio support, better agent composition, and adapters for languages other than JavaScript. As always, if you're interested in contributing to open source, we'd love to have your help. Shoot me an email at mike@liveloveapp.com and I'll personally get you onboarded.

Thank you to our community for all of the help with this release! ✌️