Mobycast

Hands On AWS - Massively Scalable Image Hosting Using S3 and CloudFront - Part 2

In this episode, we cover the following topics:We discuss the features and limitations of serving files directly from S3.We then talk about how CloudFront can address many of S3's limitations. In particular, CloudFront is performant, inexpensive and allows us to use custom CNAMEs with TLS encryption.How to create a secure CloudFront distribution for files hosted in S3.What is OAI (Origin Access Identity), why we need it and how to set it up.We show how you can configure your CloudFront distribution to use TLS and redirect HTTP to HTTPS.We finish up by discussing "byte-range requests" and how to enable them for our image hosting solution.Detailed Show NotesWant the complete episode outline with detailed notes? Sign up here: https://mobycast.fm/show-notes/End SongBeauty in Rhythm by Roy EnglandMore InfoFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at:Web: https://mobycast.fmVoicemail: 844-818-0993Email: ask@mobycast.fmTwitter: https://twitter.com/hashtag/mobycastReddit: https://reddit.com/r/mobycast

Hands On AWS - Massively Scalable Image Hosting Using S3 and CloudFront - Part 1

In this episode, we cover the following topics:A common feature for web apps is image upload. And we all know the "best practices" for how to build this feature. But getting it right can be tricky.We start off by discussing the problem space, and what we want to solve. A key goal is to have a solution that is massively scalable while being cost-effective.We outline the general architecture of the solution, with separate techniques for handling image uploading and downloading.We then dive deep into how to handle image uploading, highlighting various techniques for controlling access over who can perform uploads.Two common techniques for securing uploads when using AWS are presigned URLs and presigned POSTs. We discuss how each works and when to use one over the other.We finish up by putting everything together and detailing the steps involved with uploading an image.Detailed Show NotesWant the complete episode outline with detailed notes? Sign up here: https://mobycast.fm/show-notes/Support Mobycasthttps://glow.fm/mobycastEnd SongLazy Sunday by Roy EnglandMore InfoFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at:Web: https://mobycast.fmVoicemail: 844-818-0993Email: ask@mobycast.fmTwitter: https://twitter.com/hashtag/mobycastReddit: https://reddit.com/r/mobycast

Replay of Ep 43 - The Birth of NoSQL and DynamoDb – Part 5

Show DetailsJon Christensen and Chris Hickman of Kelsus and Rich Staats of Secret Stache conclude their series on the birth of NoSQL and DynamoDB. They compare the NoSQL database, Leviathan, created by Chris’s startup in the late 1990s to today’s DynamoDB. A lot of things haven’t changed, even though technology has evolved. It’s cyclical. There are patterns and problems that continue to dominate.  Some of the highlights of the show include:Reason for Creation of NoSQL Database: How to scale database with Internet-scale applications to have a virtual pool of infinite storage that can be scaled outMain Architecture Components of Leviathan:API clientUpdate distributor (UD)Base server (storage node)Shepherd (housekeeping management system)  Additional core components included smart IP and storage abstraction layer (SAL)Leviathan mostly used C code and minimal Java code to support usersBig difference between DynamoDB and Leviathan is request router and partition metadata system living on the server vs. living on the edgeLeviathan was a closed system with an instance for every network or data center; not designed to run as a software as a service, like DynamoDBLeviathan was strongly consistent, unlike DynamoDB’s eventually consistent modelDefinition and Different Types of TransactionsShepherd was used to identify and address consistency, synchronous, and timing issues Rather than using a file system, Leviathan used relational databases Links and ResourcesDynamoDBMicrosoft SQLOracle DBAWS IoT GreengrassKelsusSecret Stache Media Quotes:“We had the same kind of problems that DynamoDB had - how do you scale your database dealing with Internet-scale applications and have this virtual pool of infinite storage that can be scaled out.” Chris Hickman “This system and this technology went through many iterations.” Chris Hickman “You can’t have a 100% consistent state across everything. It’s just impossible. How do you do the right thing?” Chris Hickman “The big difference between DynamoDB and Leviathan...is the request router and partition metadata system living on the server vs. living out at the edge.” Jon Christen 

Replay of Ep 42 - The Birth of NoSQL and DynamoDb – Part 4

Show DetailsWhat’s under the hood of Amazon’s DynamoDB? Jon Christensen and Chris Hickman of Kelsus continue their discussion on DynamoDB, specifically about it’s architecture and components. They utilize a presentation from re:Invent titled, Amazon DynamoDB Under the Hood: How we built a hyper-scale database.  Some of the highlights of the show include:Partition keys and global secondary indexes determine how data is partitioned across a storage node; allows you to scale out, instead of upUnderstand how a database is built to make architecture/component definitions less abstractDynamoDB has four components:          1.   Request Router: Frontline service that receives and handles requests          2.   Storage Node: Services responsible for persisting and retrieving data          3.   Partition Metadata System: Keeps track of where data is located          4.   Auto Admin: Handles housekeeping aspects to manage systemWhat level of uptime availability do you want to guarantee?Replication: Strongly Consistent vs. Eventually ConsistentWalkthrough of Workflow: What happens when, what does it mean when…DynamoDB architecture and components are designed to improve speed and scalabilitySplit Partitions: Longer times that your database is up and the more data you put into it, the more likely you’re going to get a hot partition or partitions that are too big Links and ResourcesDynamoDBre:InventAmazon DynamoDB Under the Hood: How we built a hyper-scale databasePaxos AlgorithmAmazon S3Amazon Relational Database Service (RDS)MongoDBJSONKelsusSecret Stache MediaQuotes:“Keep in mind that data is partitioned across storage node, and that’s a key feature of being able to scale out, as opposed to scaling up.” Jon Christensen“Amazon was opening up the kimono...how DynamoDB has been architected and constructed and how it works.” Chris Hickman“Managed Service - they get to decide how it’s architected...because they also have to keep it up and live up to their SLA.” Chris Hickman“The longer the time that your database is up and the more data you put into it, the more likely that you’re going to get a hot partition or partitions are just going to get too big.” Chris Hickman

Replay of Ep 41 - The Birth of NoSQL and DynamoDb – Part 3

Show DetailsJon Christensen and Chris Hickman of Kelsus and Rich Staats of Secret Stache continue their discussion on the birth of NoSQL and DynamoDB. They examine DynamoDB’s architecture and popularity as a solution for Internet-scale databases. Some of the highlights of the show include:Challenges, evolution, and reasons associated with Internet-scale dataDynamoDB has been around a long time, but people are finally using itDynamoDB and MongoDB are document or key value stores that offer scalability and event-driven programming to reduce complexityTechniques for keeping NoSQL database’s replicated data in syncImportance of indexes to understand query patternsDynamoDB’s Table Concept: Collection of documents/key value items; must have partition key to uniquely identify items in table and determine data distributionSort keys create indexes (i.e. global/local secondary index) to support items within partitioning Query a DynamoDB database based on what’s being stored and using keys; conduct analysis on queries to determine their effectivenessLinks and ResourcesAWSre:InventDynamoDBNoSQLMongoDBGrouponJSONPostgreSQLKelsusSecret Stache MediaQuotes:“Kind of what drove this evolution from SQL to NoSQL - realizing that the constraints were now different, the economics of the resources that were being used.” Chris Hickman“People are realizing that Dynamo is not an ugly stepchild.” Jon Christensen“Event-driven programming...it’s very popular, and it’s going to become even more popular.” Chris HickmanEnd SongBenirrás Nights by Roy England ft. Dovetracks