Episode #34: Advanced NoSQL Data Modeling in DynamoDB with Rick Houlihan (Part 1)

About Rick Houlihan:

Rick has 30+ years of software and IT expertise and holds nine patents in Cloud Virtualization, Complex Event Processing, Root Cause Analysis, Microprocessor Architecture, and NoSQL Database technology. He currently runs the NoSQL Blackbelt team at AWS and for the last 5 years have been responsible for consulting with and on boarding the largest and most strategic customers our business supports. His role spans technology sectors and as part of his engagements he routinely provide guidance on industry best practices, distributed systems implementation, cloud migration, and more. He led the architecture and design effort at Amazon for migrating thousands of relational workloads from Oracle to NoSQL and built the center of excellence team responsible for defining the best practices and design patterns used today by thousands of Amazon internal service teams and AWS customers. He currently work on the DynamoDB service team as a Principal Technologist focused on building the market for NoSQL services through design consultations, content creation, evangelism, and training.

• Twitter: @houlihan_rick
• LinkedIN: https://www.linkedin.com/in/rickhoulihan/
• Best Practices for DynamoDB: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/best-practices.html
• 2017 re:Invent Talk: https://www.youtube.com/watch?v=jzeKPKpucS0
• 2018 re:Invent Talk: https://www.youtube.com/watch?v=HaEPXoXVf2k
• 2019 re:Invent Talk: https://www.youtube.com/watch?v=6yqfmXiZTlM

Transcript:

Jeremy: Hi everyone, I'm Jeremy Daly and you're listening to Serverless Chats. This week, I'm chatting with Rick Houlihan. Hey Rick, thanks for joining me.

Rick: Hey Jeremy. Thanks for having me on.

Jeremy: So you are a principal technologist for NoSQL at AWS. So why don't you tell the listeners a little bit about yourself and what it is that you do at AWS?

Rick: Yeah, sure. So I've been at AWS almost six years now, I guess five and a half years. My primary focus in life since joining AWS has really been NoSQL technologies. Shortly after joining organization, I joined the specialist team and then for about two years, I spent a large amount of my time focused on the migration of Amazon's internal application services from a relational database, specifically Oracle to a NoSQL technology which was DynamoDB, of course.

So that was my mission in life and the last two years or so, I've been more focused externally taking the learnings that we gained from that exercise out to our customers and helping them solve similar problems.

Jeremy: Awesome, and I love the fact that you are actually working with customers and working with these big data problems, actually solving these problems as opposed to just sort of advocating for ...

Rick: ... thinking about them, right?

Jeremy: Exactly. Exactly.

Rick: We got called out. Yeah.

Jeremy: And that's, I mean, again, obviously this firsthand experience and all this work you do, you see all these different permutations and different ways that you can use DynamoDB and NoSQL. So I think if anybody is in the sort of AWS ecosystem, if they've ever sort of thought about using DynamoDB, your name has probably come up. You've become somewhat of a legend at AWS re:Invent with your NoSQL talks on data modeling in NoSQL.

So there are a million different things that we could talk about obviously, and I could probably talk to you for quite some time. I don't want to spend a lot of time rehashing the things that are in your presentations and I will put these in the show notes and people can go and spend some time looking at these things. But I actually, I want to be a little bit selfish here because I have all these questions that I've sort of come across and some people have asked me and now that I've got you here, I would love to sort of ask you those and see if we can dig a little bit deeper in them.

But what I do want to do is be fair to people who are not overly familiar with DynamoDB or NoSQL in general. So maybe we start quickly and just kind of explain or if you could explain to us what's the difference between NoSQL and relational databases or RDBMS.

Rick: Sure. Yeah, great place to start. So if you think about the relational database today, it's about normalized data. We're all very familiar with the idea of a normalized data model where you have multiple tables, we have all these relationships, parent child relationships and many to many relationships. And so we built these tables that contain this data and then we have this ad hoc query engine that we write called SQL. We write queries in SQL to return the data that our application needs.

So the server, the database actually restructures the data and reformats the data on the fly whenever we need it to satisfy a request. Well, NoSQL on the other hand eliminates that CPU overhead and that's really what the cost of the relational database is and the reason why it can't scale because it takes so much CPU to reformat that data.

So with NoSQL, what we're going to do is we're going to actually denormalize the data somewhat and we're going to tune it to what we call the access pattern, tune it to the access pattern to create an environment that allows the server to satisfy the request with simple queries.

So we don't actually have to join the data together. So we talk about the modeling and whatnot in my sessions, we can get into how do we do that, but the fundamental crux of the issue here is that the relational database burns a lot of CPU to join the data and produce these materialized views whereas the NoSQL database kind of stores the data that way and makes it easier for the application to use it.

Jeremy: All right, and then one of the things I think that you see all the time when people are sort of migrating or trying to figure out that NoSQL mindset is they think about access patterns and one of their access patterns is something like list all customers.

Rick: Right.

Jeremy: And it's one of those things where I think it's really hard for people to make that jump from just being able to say select star from customers and understand that data will get back and we can add limits and things like that. It's not quite the same with something like with NoSQL, so when do you suggest people not use NoSQL?

Rick: Okay, so that's actually a really good question. So NoSQL is really suited and as we talked about, we have to denormalize the data, right? Which does that means I have to structure it and tune it to the access pattern. So if I don't really understand those access patterns, if they're not really well-defined, then maybe what we're looking at is a different type of application that's not necessarily so well-suited for NoSQL, right?

And that's really what it comes down to. There's two types of applications out there. There's no OLTP or online transaction processing application which is really built using well-defined access patterns. You're going to have a limited number of queries that are going to execute against the data, they're g...