-
Spreadsheets
AWS
Eventual Consistency for an Event Sourced Spreadsheet
Last time we looked at general approaches to ensuring eventual consistency in the cloud. Now it’s time to apply what we’ve learnt to the case of my Event Sourced Cloud Spreadsheet. Previously, I went into some detail on how to implement an Event Log using DynamoDB. Long story short, there are some operations that involve multiple writes and some that need to trigger side effects.
-
Cloud Architecture
Ensuring Eventual Consistency
Remember how we used to build web apps? A database, some app servers and a load balancer. What we’d disparagingly call a Monolith today.
-
Spreadsheets
Databases
AWS
Implementing a Spreadsheet Event Log on DynamoDB
In the distant past, before I got sucked into a seemingly never ending series on databases, I said that I was going to start formalizing the format for my cloud based, serverless, event sourced spreadsheet. I realize now that I’ve said very little on how I’m going to implement the central component of my spreadsheet, the event log.
-
Databases
DynamoDB Database Grid View
DynamoDB is AWS’s flagship, serverless, horizontally scalable, NoSQL database. The work that ultimately led to the release of DynamoDB in 2012, started in 2004. Amazon was growing rapidly and finding it hard to scale their relational databases. The development of DynamoDB was driven by the observation that 70% of Amazon’s queries were key-value lookups using a primary key to return a single row. Another 20% returned a set of rows from a single table.
-
Databases
MongoDB Database Grid View
Last time we dipped our toes into the waters of schemaless databases by using a JSONB column in Postgres to store a set of custom fields. After a few attempts we got it working. However, it didn’t offer many benefits compared with the denormalized relational database optimizations we previously tried, while bringing in plenty of additional friction of its own.
-
Databases
JSON Relational Database Grid View
When we aggresively denormalized our Grid View relational database, we were able to achieve our aim of a single table solution. However, it came with a lot of unintended consequences. In order to support up to 100 custom fields of each type, we added 400 custom field columns to our issue table, most of which were NULL. In theory the overhead should have been minimal. In practice, we ran into all kinds of implementation and operational problems.
-
Databases
Denormalized Relational Database Grid View
We’ve been good. We’ve followed the rules. Our database is fully normalized and we have all the referential integrity checks we need in place. And yet. Our queries seem overly complex. There’s a constant battle to try and keep queries scalable. Despite all that, performance is not what we’d like.
-
Spreadsheets
Merging and Importing Spreadsheet Snapshots
Last time, we looked at the added complexity that comes when you start inserting and deleting rows and columns from your spreadsheet. Spreadsheet snapshots are made up of multiple segments. Once you start inserting and deleting things, those segments are in different coordinate spaces. You need to transform the earlier segments into the coordinate space of the most recent as you load them.
-
Databases
Custom Fields with a Normalized Relational Database
Last time we discovered that it’s relatively easy to build a Grid View application using a Normalized Relational Database. True, it was a toy example with a small number of fixed fields. However, given some reasonable functional limitations, we showed how it could scale to manage large collections with 100,000 items or more.
-
Databases
Normalized Relational Database Grid View
Let me take you back to a time before NoSQL, when E.F. Codd’s relational rules and normal forms were the last word in database design. Data was modelled logically, without redundant duplication, with integrity enforced by the database.