I'm thrilled to announce a new project I've been putting hours into the last couple of days! Building data-driven products often means that you want to be able to react to changes as soon as they come in. This means, that you'll expect ways to hook into the constant flow of changes, consuming and processing them in real-time. Data might flow through many services until it reaches its target. So naturally, we'll put our focus on the final destination, the data store.
lode is a brand-new toolkit for Go applications, allowing everybody to build change-streaming pipelines on top
of Postgres. Whether you're building an audit log, backup solution, or any other application requiring real-time capabilities from your primary Postgres data store,
lode will lower the entry barrier for realizing your use case substantially.
Built on top of Postgres' logical decoding feature, lode continuously receives new transactions sent by the database instance and allows you to hook into the processing part, while handling everything around it like health checks and replication slot management, so you don't have to. The light abstraction layer lode offers, also allows you to be confident about putting it to use, while maintenance and improvement tasks under the hood remain possible for future endeavors.
Please check out the 🔗 GitHub repository, which includes two examples for building synchronous and asynchronous stream-processing applications on top of lode.
I hope you'll try it out and build a cool piece of software with lode! If you've got any questions, suggestions or feedback about the project, please reach out via GitHub issues in the project, so we'll be able to improve it together 👍