Return to Answer

I assume there is a distribution over entity activity, perhaps Zipfian, where some entities are far more popular or active than other entities. In addition to quota values, each entity should also store a recently seen rate of redemption events, to help us assess risk of imminent quota violation.

I assume there is a distribution over entity activity, perhaps Zipfian, where some entities are far more popular or active than other entities. In addition to quota values, each entity should also store a recently seen rate of redemption events, to help us assess risk of imminent quota violation. The fact that some offers are more popular than others, and can be handled differently, is essential to keeping the "at risk" query rate down.

I assume there is a distribution over entity activity, perhaps Zipfian, where some entities are far more popular or active than other entities.

I assume there is a distribution over entity activity, perhaps Zipfian, where some entities are far more popular or active than other entities. In addition to quota values, each entity should also store a recently seen rate of redemption events, to help us assess risk of imminent quota violation.

Independent of the backend technology you choose for this (kafka, redis, rdbms), I recommend you apply a boolean label to each entity: "fast" vs "slow" path for processing. The fast-path entities are "boring"; they are far from their limit so there is little danger of exceeding their quotas.

Each redemption transaction event shall be appended to a common event log. This is easy to do, cheaply and reliably, even during network partition events. We append to a local log, which later will show up in an eventually consistent log. Thanks to transaction guids, logging an event is idempotent. Such logging is highly available.

Independent of the backend technology you choose for this (kafka, redis, rdbms), I recommend you apply a boolean label to each entity: "fast" vs "slow" path for processing. The fast-path entities are "boring"; they are far from their limit so there is little danger of soon exceeding their quotas.

Each redemption transaction event shall be appended to a common event log. This is easy to do, cheaply and reliably, even during a network partition. We append to a local log, which later will show up in an eventually consistent log. Thanks to transaction guids, logging an event is idempotent. Such logging is highly available.