How to Design Software — Rules Engines
Walk through the components of a minimal rules engine and understand some of the thought processes behind approaching its design.
Walk through the components of a minimal rules engine and understand some of the thought processes behind approaching its design.
Imagine this scenario
You’re a busy engineer working in a company with too much to build and too few resources to do it. Departments constantly ask for changes only an engineer can make. Marketing continuously asks for email on-boarding changes. Operations wants more and more batch tooling. Sales wants to test new pricing structures every other week. You and your team are overwhelmed with requests.
What are your options?
Stop progress on planned work to make these requested changes? A constant stream of interruptions is terrible way to get any real work done, and who knows if these asks are even worth interrupting the real work for.
Ignore it? You’ll make progress on your planned work, but you’ll also have a bunch of external stakeholders now suddenly very angry at your department. Besides, isn’t technology supposed to help the business? Is it really helpful to have the business miss out on potentially significant opportunities just because you couldn’t spare an hour or two?
Triage the asks around a planned schedule? Planned interruptions are better than unplanned ones, but you’re still potentially limiting the progress of other departments and forcing them to work around your schedule — not quite as collaborative as one could be.
Imagine if there was a way to prevent these requirements from every reaching you, and to give the other departments tools to pursue these opportunities without your involvement. They’d be able to make internal pivots without having to rely on engineering to perform the work.
If such a solution existed, you’d save a lot of time and be able to focus on more important problems.
Enter the rules engine.
What is a rules engine?
A rules engine is a system that performs a set of actions based on specific conditions which can be configured during runtime. This means that an effectively set up rules engine would not require engineers to change a system’s business logic.
Engineers build rules-based systems all the time, albeit unconsciously. Every time you code an if-else statement, you are effectively creating a hardcoded rule that is followed by the system.
You can go a step further and make these systems configurable dynamically. When I say dynamic, I mean that the behavior of the system is not determined by flows coded by engineers, but by configuration through data.
These so-called “codeless” systems are not new. Software like Zapier, Hubspot, and IFTTT operate off of the same concepts, as do more technical implementations like Boomi or Drools.
A rules engine is a system that performs a set of actions based on specific conditions which can be configured during runtime.
The fragility of concrete
Why are rules engines even needed?
By default, most engineers concern themselves on the details of what the rules are of the code they are writing. They were concrete implementations specific to the business use case they are encountering.
This makes sense — code does exactly what it says it does, so engineers need to know what needs to be done to write the code properly.
However, this coupling to the details of the use case makes the code they write churn significantly when the details of the use case changes. While this approach works in many cases, sometimes the rate of change can become overwhelming, especially on smaller teams or more dynamic environments like post-investment startups.
It’s easy to see how these things can change. Business logic is implemented to fulfill a requirement in what behavior the software should exhibit. These requirements can come from many sources — business demands, improvements in UX, regulatory needs, technical drivers, etc.
Examples include:
A “welcome” email must be sent 2 hours after a user signs up
If a payment account holds more than $100,000, over 3 consecutive days, the entire amount must be disbursed immediately
Employees that are a member of this union should accrue vacation time at double the standard rate for 2019, but not 2021.
All of these can also vary in levels of stability, or how often the details change.
For example, a requirement based on data obtained through user analytics may change weekly, whereas a requirement based on a financial regulation may change once every couple of years. A requirement based on a union contract may only change when the contract is renegotiated.
When it does change, the implementation must change alongside of it.
The evolution of an implementation
Let’s examine a hypothetical implementation of the “welcome” email requirement as it evolves over time.
A “welcome” email must be sent 2 hours after a user signs up
The first stage
Developers can be relied on to do the easiest thing. Most engineers boil business logic down to an if-this-do-that: if this happens, perform this action.
The first (and unfortunately often last) implementation most developers take is the direct approach.
It’s so easy — a single line of code can fulfill this requirement:
def on_user_create
WelcomeEmail.send(in: 2.hours)
end
The second stage
As time passes, the business often demands changes to details that seemed concrete and set in stone in the past.
Change is a constant, and software is SOFTware for a reason — it has to be malleable to fit the situation and circumstances.
What if we discover that 2 hours is too long, and we want to change it to 15 minutes? If we do, an engineer needs to go and change it.
def after_user_create
WelcomeEmail.send(in: 15.minutes)
end
Change is a constant, and software is SOFTware for a reason — it has to be malleable to fit the situation and circumstances.
The third stage
If the business is performing experiments to see what kind of on-boarding leads to the best retention and conversion rates, it’s likely this value will change a lot.
If this happens enough times, the developer will hopefully get smart and move this detail out of the code so it can be configured during runtime.
def after_user_create
WelcomeEmail.send(in: Configuration.get('welcome_email_wait'))
end
Configuration.get
could access the database, read a configuration file, look at an environment variable, or perhaps even randomly decide. The important factor is that it is now determined at runtime instead of hard-coded.
The fourth stage
Each individual requirement taken in isolation often ends at the third stage.
Keep reading with a 7-day free trial
Subscribe to Joseph Gefroh to keep reading this post and get 7 days of free access to the full post archives.