The priceless experience that comes as a Software Architect working with teams that use databases. All the fun of schema and query design with none of the operational headaches of figuring out why the database has stopped responding.
-
Cloud Architecture
Databases
ACID! - Atomicity, Consistency, Isolation, Durability
There are lots of things you need to worry about when building a cloud application. Often the hardest problems only appear as you scale up, when multiple clients interact with the same data at the same time.
-
Databases
Front End
Event Sourced Database Grid View
When we started on this journey, I told you that there was an easy way and a hard way to implement a database backed grid view. I then spent the next six posts in this series taking you through different variations of the hard way.
-
Spreadsheets
Databases
AWS
Implementing a Spreadsheet Event Log on DynamoDB
In the distant past, before I got sucked into a seemingly never ending series on databases, I said that I was going to start formalizing the format for my cloud based, serverless, event sourced spreadsheet. I realize now that I’ve said very little on how I’m going to implement the central component of my spreadsheet, the event log.
-
Databases
DynamoDB Database Grid View
DynamoDB is AWS’s flagship, serverless, horizontally scalable, NoSQL database. The work that ultimately led to the release of DynamoDB in 2012, started in 2004. Amazon was growing rapidly and finding it hard to scale their relational databases. The development of DynamoDB was driven by the observation that 70% of Amazon’s queries were key-value lookups using a primary key to return a single row. Another 20% returned a set of rows from a single table.
-
Databases
MongoDB Database Grid View
Last time we dipped our toes into the waters of schemaless databases by using a JSONB column in Postgres to store a set of custom fields. After a few attempts we got it working. However, it didn’t offer many benefits compared with the denormalized relational database optimizations we previously tried, while bringing in plenty of additional friction of its own.
-
Databases
JSON Relational Database Grid View
When we aggresively denormalized our Grid View relational database, we were able to achieve our aim of a single table solution. However, it came with a lot of unintended consequences. In order to support up to 100 custom fields of each type, we added 400 custom field columns to our issue table, most of which were NULL. In theory the overhead should have been minimal. In practice, we ran into all kinds of implementation and operational problems.
-
Databases
Denormalized Relational Database Grid View
We’ve been good. We’ve followed the rules. Our database is fully normalized and we have all the referential integrity checks we need in place. And yet. Our queries seem overly complex. There’s a constant battle to try and keep queries scalable. Despite all that, performance is not what we’d like.
-
Databases
Custom Fields with a Normalized Relational Database
Last time we discovered that it’s relatively easy to build a Grid View application using a Normalized Relational Database. True, it was a toy example with a small number of fixed fields. However, given some reasonable functional limitations, we showed how it could scale to manage large collections with 100,000 items or more.
-
Databases
Normalized Relational Database Grid View
Let me take you back to a time before NoSQL, when E.F. Codd’s relational rules and normal forms were the last word in database design. Data was modelled logically, without redundant duplication, with integrity enforced by the database.
-
Databases
Cloud Architecture
The Ubiquitous Database Grid View
You’re building a full stack application that enables teams of people to do … something. In order to manage the process of doing something, the teams collaborate by creating, filling in, finding, sorting and archiving forms. There will be some industry specific bits of workflow that let you convince yourself that you’re not building the same thing over and over again.