Evaluating usage-based billing providers: what engineers need to know (part 1)

At Knock, we provide a notifications-as-a-service platform that uses a usage-based billing model to charge our customers for the volume of notifications sent each month.

In the spirit of doing things that don't scale as an early-stage startup, for the first 9 months of our company we were billing our customers manually each month using our internal dashboard to see their usage and Stripe Invoicing to charge them.

As we entered Q2 of this year one of our key projects was shipping a refreshed pricing model and self-service billing. Being keen observers of the developer-tools ecosystem, and firm believers in outsourcing anything that's not core to our business, we saw this as an opportunity to potentially utilize one of the new services on the block for powering usage-based billing such as Metronome, Orb, Kable, or Lotus.

In this two-part post series, we'll look into our experience buying and integrating a usage-based billing provider, starting with a look at what a usage-based billing provider is, and why you might want to consider buying one.

What is usage-based billing?

In a usage-based pricing model, you charge the customer based on their usage of your product. At Knock, our usage unit is the number of notifications sent from our platform each month. Each of our plans includes a bucket of notifications, with any overages charged on a unit basis.

Here are a few other usage-based billing models you might encounter:

• API request volume
• Storage per GB in an bucket, and egress per GB out of that bucket
• Database read and write operation volume
• SMS sending volume

How do usage-based billing providers fit into the picture?

A usage-based billing provider sits between your application and a payment processor to process subscriptions and charge your customers based on usage accrued during the billing period. It acts as the system of record for customer subscription and billing data but outsources payment to third processors like Stripe, bill.com, or cloud marketplaces like AWS or Azure.

A usage-based billing provider stores information about:

• Usage data: the amount of usage the customer has used in the current period calculated by the events emitted by your service.
• Plans and products: the plans offered and what rules govern how your customers get charged for your services.
• Customers and subscriptions: basic information about your customers (name, email), the subscription data for that customer (what plan they are on), as well as any special pricing rules for the customer.
• Invoices: the amount charged to the customer each period and the status of the payment.

Why outsource usage-based billing?

My co-founder Sam and I both have extensive experience working with a complex billing and pricing model at Frame.io and our willingness to outsource this part of our offering was largely driven by our experience there. Beyond that, we have a deep belief in outsourcing any part of our product that isn’t core to the experience. WorkOS for SSO, Algolia for search, and LaunchDarkly for feature flagging are good examples of this.

You may also be reading this thinking “can’t you just use Stripe to power that?”. You are technically correct and you can use Stripe’s usage-based billing solution, however, in practice there are areas where this falls short:

• Handling per-customer adjustments: sales negotiations can result in custom contract prices and volume discounts, which are usually accounted for at the plan level or as discounts applied to a subscription. Both of these approaches have drawbacks: structuring as plans means you'll likely end up creating bespoke plans per customer in Stripe which can get messy quickly and is difficult to migrate from, and structuring as discounts means your customer's invoices will always include the base prices with discounts shown.
• Supporting different billing providers: early on at Knock, we started to see demand for using the venerable (and favorite of the CFO) bill.com as the billing layer rather than paying with ACH or Credit Card via Stripe. Additionally, an abstraction layer here opens up the potential for integrating other billing providers like cloud marketplaces (AWS, Azure, GCP).
• Having a single source of truth for subscription data: related to the above as soon as multiple providers are handling billing the source of truth quickly becomes the application database. That means your likely going to have to do additional work to consolidate this data to understand billing metrics like revenue retention, churn, billing history, ARR, etc.
• Flexibility to model complex product offerings: Stripe has great coverage for lots of different subscription models, but things can get complex quickly when you want to introduce plans that mix multiple models (e.g. seats in addition to usage-based consumption).

Additionally, we saw some other opportunities in outsourcing our usage billing:

• Usage data API: exposing a real-time usage API means we can easily build usage graphs in our dashboard, without needing to expose analytical data in our services.
• Power alerting and notifications: we wanted to make use of the alerting capabilities to power common alerting use-cases like sending a notification when the customer has used some percentage of their notifications, as well as driving internal notifications for upgrade opportunities.
• Self-service internal upgrades: we haven't yet built out an internal tool, so having the ability for our team to set customer subscriptions and cost structures from a pre-built admin tool would be a nice win for us.

Tradeoffs and considerations when buying a usage-based billing provider

As with evaluating any software buying decision, there are tradeoffs to consider in adopting a usage-based billing platform over integrating directly with a provider like Stripe:

• Building checkout and billing management: building on top of Stripe Subscriptions grants access to the excellent pre-built Billing Portal and Checkout offerings for powering common subscription management tasks. At the time of writing, none of the usage-based billing providers offer this functionality out-of-the-box meaning that your team will need to absorb the non-trivial cost of building out subscription management and checkout.
• Outsourcing subscription management: Stripe and other incumbents in the space are well-tried and tested providers that have long since ironed out any of the issues surrounding recurring subscription management. Going with new on-the-market providers like Orb or Metronome presents a risk with the immaturity of their offerings.
• Additional integration cost: as the usage-based billing providers still depend on Stripe for payment processing you're still going to need to do some integration there in addition to the integration with the usage-based billing provider itself, increasing the surface area of the billing build.
• Cost of the service: in addition to the fees that a provider like Stripe charges per transaction, a usage-based billing provider will charge a fee to utilize their service that scales with the revenue being charged through the platform.
• Migration effort: in the event that there was a decision to migrate from the provider, the migration effort to move customers and subscription data to a new provider would be considerable given the need to handle active subscriptions.

We decided that the benefits outweighed the tradeoffs and that the flexibility we gained from adopting a provider now and not building this on top of Stripe or a similar solution would be a worthwhile medium to long-term investment, even with the increased upfront work and the inherent risks of going with relatively young companies to power this service.

Choosing our usage-based billing partner

We seriously evaluated Metronome and Orb as well as offerings from Kable and Lotus. We found that the base offerings were largely comparable for each provider, with Orb having the most well-developed UI interface and dashboard. We quickly ruled out Lotus as they don't yet have a cloud offering and we didn't want to manage and operate this service ourselves.

Ultimately we chose to partner with Orb as our usage-based billing provider. We found their team incredibly helpful during the buying process and very aligned with how we operate as a team. Beyond that, we found their documentation well-written, and their API and primitives well thought through.

Given how early we are as a business, we also deeply value partnering with teams at similar stages in the journey to us. We felt Orb did a great job here and provided exceptional support throughout the process of working with them.

Looking ahead

The effort to go from founder-driven, manual billing each month to an automated, self-service, usage-based billing system represented a significant investment for our team in Q2 of this year. We spent 10+ weeks of engineering time split across 2 engineers to complete this project. In an early-stage company that is an eternity of resources being tied up on a set of features that—let’s face it—don’t directly benefit our customers. So was it worth the investment?

Weighing the alternative of building this all on top of Stripe, we would have made a similar investment in terms of engineering time on a lot of the tedious integrations work, although we would have saved some time by integrating the Billing Portal and offloading much of that experience.

We believe the investment here was worthwhile, but it’s too early to give a definitive answer. Early signs point to “yes” — every time we land an enterprise customer with a slightly different discounted rate card, that we can easily set in a portal we didn’t build I think we’ve done the right thing. Similarly, seeing a customer self-service upgrade onto a paid plan is still a magical experience for our team, and one less task for us as founders to do.

Getting pricing right is an extension of the product-market fit challenge that requires iteration, testing, and ultimately time. Knowing we have a platform behind us where we can more easily iterate and experiment with different models feels like time well-spent. As with many technical decisions though, time will tell, but we’re already planning some tweaks to our pricing model and we’re not dreading it. Yet.

Stay tuned for the next post in this series where we break down what the integration looked like in practice as well as the challenges we had along the way.