Case Study: A Keybase Live Chat Feed for Chia Network

TL;DR: In 2 weeks, we built an open-source live feed of Chia Network’s Keybase community. Check it out here!

Overview

Chia Network is a “decentralized open source global blockchain which is less wasteful, more decentralized, and more secure”. It was founded by Bram Cohen, the creator of BitTorrent.

Building a strong developer community is key to the success of a cryptocurrency like Chia. Their community was on Keybase Teams, an end-to-end encrypted instant-messaging application similar to Slack.

But without a Keybase account, visitors to their site could not view the latest updates about Chia. We worked with Chia to swiftly develop and launch a live chat feed for their website.

The Challenge

Chia wanted visitors to keep coming back to their website, and it was important that any visitor could stay up-to-date on news about Chia.

However, visitors without a Keybase account would need to:

Those visitors also had no visibility into how active Chia’s community was, nor its size. This resulted in friction joining the community and getting the latest news.

Results

Reception from Chia’s community was overwhelmingly positive! Visitors could now see a real-time preview of the discussion right in their browser, and without requiring a Keybase account.

It’s the first of its kind and has been open-sourced here. We hope that it enables other developers to easily integrate a Keybase feed with their own website.

The Project

To reduce this friction, we built a custom integration that previews Chia’s #general Keybase channel. It’s robust, mobile-friendly, and the message feed updates in real-time (including emoji reacts!).

Design

  • Frontend: React, React-Markdown, Bootstrap, Socket.IO
  • Backend: Node, Express, Socket.IO, and Keybase.
Architecture diagram of the Keybase Live Feed.

Frontend

We wanted the code to be simple to use, update, and maintain. We went with a time-tested, minimal React stack — only 8 dependencies in total!

The chat UI is implemented with React for view layer, and Bootstrap to help with layout.

Chat messages stream in via Socket.IO, and are immediately shown on screen — either appended to the messages list, or rewriting existing messages. There’s a separate message type for metadata, such as the number of people in the conversation.

Because Keybase Teams supports a small subset of Markdown in chat messages, we implemented this using the React-Markdown library. It’s important to use escapeHtml: false here to prevent XSS vulnerabilities.

Emoji rendering in text messages was a bit of a challenge. To match the Keybase behaviour as much as possible, we wrote our own spritesheet display component. This uses regular expressions to find emoji.

Some of the interesting behaviours it handles are:

Backend

As of this writing, Keybase doesn’t offer a complete API for their Teams product. It does have endpoints for basic functionality such as identity management.

However, the keybase command-line tool offers a semi-documented way to directly interact with their Teams product, via various subcommands such as keybase chat api and keybase chat api-listen.

We looked into duplicating what the command-line tool does, but in the end, the command-line tool’s API surface looked more stable than their undocumented HTTP/Websockets API surface. To use it, we decided to simply run the tool in the background and capture its standard output.

There were only a few things we needed from the API:

After automating the command-line Keybase tool, we had a reliable, consistent way of obtaining all of these. We then abstracted it into the keybaseChatApi library.

On top of this, we built keybaseChatHistory, a scrollback-management library that keeps track of created/edited/deleted messages and reactions, as well as providing an up-to-date view of the current chat history. This allows us to easily retrieve messages with their related content, such as reactions and edit markers, and synchronize that state across all clients.

The Live Feed itself now becomes a simple Express/Socket.IO server. There are three listeners running in parallel in each Node process:

Finally, the entire application was written to be stateless — all data is stored using in-memory data structures and can be restarted at any time without side effects. Upon start, the chat history and the various caches are rebuilt automatically, and the application is immediately ready to serve users.

Deployment

The live feed was deployed to AWS EC2 instances with Debian images. We set them up with the usual best practices, such as:

For larger use cases, Keybase Live Feed also supports running within Google Kubernetes Engine, StackDriver for logging, and its own K8S-specific runbook.

Hypotenuse Labs is an elite team of software consultants. Hailing from Facebook, Amazon, Uber, and Snap, we specialize in delivering web and AI software products for startups and SMBs.

If custom integration struggles are keeping you up at night, contact us at hello@hypotenuse.ca.

Building incredible web, AI, and blockchain solutions since 2018.