`rmmbr` — Super Simple Persistent Caching in Python

Here’s a familiar scenario: you’re working on a project that involves making requests to an API like OpenAI, which can quickly rack up your bill if you’re not careful.

The solution? Caching.

Caching allows avoiding paying for repeated calls by storing the results and retrieving them as needed. But typically caching works in memory. This means that when your your cache is gone, not to mention your collegues who always start with an empty cache.

Implementing a caching solution which is persistent across runs and devices can be nontrivial. This is where rmmbr comes in: https://github.com/uriva/rmmbr.

rmmbr is a lightweight service with client libraries in Python, Javascript or -TypeScript. It provides a decorator you can place around your asynchronous functions to cache their results, either locally or in the cloud.

With just a few lines of code, you can have a cross device persistent caching service reduce the number of API requests you make, saving you time and money.

To use rmmbr, simply install it via pip (or npm):

pip install rmmbr

Let’s take a look at an example using OpenAI’s GPT-3 API:

from local_cache import rmmbr
from openai import openai

@local_cache("chat-completions")
async def call_open_ai(params):
  result = await openai.create_chat_completion(params);
  return result.data

async def do_prompt(prompt):
  result = await call_open_ai({
    "model": "gpt-3.5-turbo",
    "messages": [{ "role": "user", "content": prompt }],
  })
  return result["choices"][0]["message"]["content"]

Here we are using the local_cache function to cache the result of an asynchronous function that generates text using OpenAI’s GPT-3 API. The cacher stores the requests in a file in a local directory.

If you want to persist across devices, you can also store the results in the cloud with e2e encryption:

from cloud_cache import rmmbr
from openai import openai

@cloud_cache(
  "https://rmmbr.net",
  "<service token>",
  "chat-completions",
  // Time to live, in seconds, use `None` for max time.
  24 * 60 * 60,
  // e2ee for sensitive data, use `None` for no encryption.
  "<encryption key>",
})
async def call_open_ai(params):
  result = await openai.create_chat_completion(params);
  return result.data

rmmbr has a free tier up to a quota, and the backend is open source as well, so you can use your own instance if you’d like.

We’d love to hear about your use cases and desired features, so feel free to post issues here.

We hope you find rmmbr useful and that it saves you time and money.

More content at PlainEnglish.io.

Sign up for our free weekly newsletter. Follow us on Twitter, LinkedIn, YouTube, and Discord.