Episode 96: An AWS private cloud strategy, kubernetes aplenty, microservices by yaml, & detailed hot-dog creature analysis

The cat-nip of Mary Meeker's Internet Trends report is out this week so we discuss the highlights which leads to a sudden discussion of what an Amazon private cloud product would look like. Then, with a raft of new container related news we sort out what CoreOS is doing with their Tectonic managed service, what Heptio is (the Mirantis of Kubernetes?), and then a deep dive into the newly announced Istio which seems to be looking to create a yaml-based(!) standard for microservices configuration and policy and, then, the actual code for managing it all. Also, an extensive analysis of a hot-dog display, which is either basting itself or putting on some condiment-hair. Alternate Titles I've seen this hot-dog before. I’ve been doing this since dickity-4 I’m sticking with the Mary Meeker slides, you nerds go figure it out Mid-roll Pivotal Cloud-native workshop in DC, June 7th (http://connect.pivotal.io/Cloud-Native-Strategy-Workshop-DC.html). LOOK, MA! I PUT IN DATES! DevOpsDays Minneapolis, July 25 to 26th: get 20% off registration with the code SDT (https://devopsdays-minneapolis-2017.eventbrite.com?discount=SDT) (Thanks, Bridget!). Coté: CF Summit June 13 to 15, 2017 (https://www.cloudfoundry.org/event/summit-silicon-valley-2017/). 20% off registration code: cfsv17cote Coté: Want 2 days of Spring knowledge? Check out SpringDays (https://www.springdays.io/ehome/index.php?eventid=228094&) SpringDays.io Get half-off with the code SpringDays_HalfOff Chicago (May 30th to 31st) (https://www.springdays.io/ehome/spring-days/chicago) New York (June 20th to 21st) (https://www.springdays.io/ehome/spring-days/new-york) Atlanta (July 18th to 19th) (https://www.springdays.io/ehome/spring-days/atlanta) Hot-dog guy in Japan Zoom in on that little fellow (https://www.flickr.com/photos/cote/35012640896/). Internet Trends 2017 300 plus slides of charts (http://www.kpcb.com/internet-trends) Computes! Coté’s notebook (https://content.pivotal.io/blog/analysis-of-mary-meeker-s-internet-trends), summary of summary: Google and Facebook make a lot of ad money. The Kids like using smart phones, the olds like using traditional telephones. One of them will die sooner. Voice, image recognition, etc. China is pretty much a mature market, and it’s huge. India has potential, but doing business there is hard and you need more Internet in a pocket rollout. The public/private cloud debate is still far from over. But, AWS, Microsoft, and Google have pretty much won. Bonus: there’s surprisingly little funding and exits this year. Would Amazon sell some private clouds? Isotoner and Hephaestus - All the new container orchestration poop Coté: Catching up on all this week's container poop & as always, my first reaction is “oh, I thought the existing stuff did all that already..so." Managed service for Tectonic as a Service (https://thenewstack.io/coreos-takes-cloud-portability-tectonic-release/) - so, keeping your Kubernates cluster software updated? Presumably enforcing config, etc? However, not all done, still working on the complete solution. But, there’s an etcd thing ‘As a first step, Tectonic 1.6.4 will offer the distributed etcd key-value data store as a fully managed cloud service. “It’s the logical one to offer first because it is everything else gets built on it,” Polvi explained. The data store “guarantees that data is in a consistent state for very specific operations,” he said, referring to how etcd can be essential for operations such as database migrations.’ Another etcd description (https://blog.heptio.com/core-kubernetes-jazz-improv-over-orchestration-a7903ea92ca): “etcd is a clustered database that prizes consistency above partition tolerance… Interestingly, at Google, chubby is most frequently accessed using an abstracted File interface that works across local files, object stores, etc. The highly consistent nature, however, provides for strict ordering of writes and allows clients to do atomic updates of a set of values. So, you need locks for - dun-dun-dun! - transactions! Queue JP lecturing me in 2002. Then there’s Istio (http://blog.kubernetes.io/2017/05/managing-microservices-with-istio-service-mesh.html): Istio (https://istio.io/)?! Whao! Check out the exec-pitch (https://istio.io/blog/istio-service-mesh-for-microservices.html): “ Istio gives CIOs a powerful tool to enforce security, policy and compliance requirements across the enterprise.” And Google (https://cloudplatform.googleblog.com/2017/05/istio-modern-approach-to-developing-and.html): “Through the Open Service Broker model CIOs can define a catalog of services which may be used within their enterprise and auditing tools to enforce compliance.” I love their idea of what a CIO does. “An open platform to connect, manage, and secure microservices“ SDN++ overlay for container orchestrators from Google, IBM & Lyft - once you control the network with the “data plane,” you add in the “control plane” (https://istio.io/docs/concepts/what-is-istio/overview.html#architecture) which allows you to control the flow and shit of the actual microservices. Tackling the “new problems emerge due to the sheer number of services that exist in a larger system. Problems that had to be solved once for a monolith, like security, load balancing, monitoring, and rate limiting need to be handled for each service.” And, you know, all the agnostic, multi-cloud, open stuff. Thankfully, they didn’t use a bunch of garbage, nonsense names for things. Let’s look at the docs (https://istio.io/docs/concepts/what-is-istio/overview.html) (BTW, can you kids start just putting out PDFs instead of only these auto-generated from markdown web pages?): First of all, these are good docs. Monkey-patching for the container era: “You add Istio support to services by deploying a special sidecar proxy throughout your environment that intercepts all network communication between microservices, configured and managed using Istio’s control plane functionality.” The future! Where we all shall live! “Istio currently only supports service deployment on Kubernetes, though other environments will be supported in future versions.” Problems being solved, aka, “ways you must be this tall to ride the microservices ride”: “Its requirements can include discovery, load balancing, failure recovery, metrics, and monitoring, and often more complex operational requirements such as A/B testing, canary releases, rate limiting, access control, and end-to-end authentication.” Also: Traffic Management (https://istio.io/docs/concepts/traffic-management/overview.html), Observability, Policy Enforcement, Service Identity and Security. Does it have the part where it reboots/fixes failed services for you? So: you monkey-patch all this shit in (er, sorry, “sidecar”), which controls the network with SDN shit, Istio-Manager + Envoy (https://istio.io/docs/concepts/traffic-management/overview.html) does all your load-balancing/circuit breaker (https://istio.io/docs/concepts/traffic-management/handling-failures.html)/canary/AB shit, service discovery/registry, service versioning (https://istio.io/docs/concepts/traffic-management/request-routing.html#service-model-and-service-versions) (i.e., running n+1 different versions of code - always a pretty cool feature), configuring “routes,” what connects to what (https://istio.io/docs/concepts/traffic-management/rules-configuration.html), I don’t think it provides a service registry/discover service (https://istio.io/docs/concepts/traffic-management/load-balancing.html)? Maybe just a waffer thin API (“a platform-agnostic service discovery interface”)? Question: what does this look like in your code? The (https://istio.io/docs/concepts/policy-and-control/mixer.html) thing 12 factor-style passes a configuration into your actual code. Here, you’re adding a bunch of name/value pairs (which can be nested) and also translating them to the name/value pairs that your code is expecting...on an HTTP call? Executing a command in your container? As ENV vars? And then, I think you finally get ahold of the network to reply back with some HTML, JSON, or some sort of HTTP request by (https://istio.io/docs/tasks/ingress.html)., So, big questions, aka, Coté mental breakdown that only Matt Ray can cure: Er...so this all really is a replacement for the VMware stack, right? And OpenStack? Or do you still need those. What the fuck is all this stuff? It just installs the Docker image on a server? And then handles multi-zone replication, and making sure config drift is handles (bringing up failed nodes, too)? So, it’s just cheaper and more transparent than VMware? What’s the set of shit one needs? Ubuntu, Moby Engine (?), Moby command line tools, etcd? Actuality kubernetes code? What’s Swarm do? And then there’s monitoring, which according to Whiskey Charity, is all shit, right? Where’ my fucking chart on this shit? Please write two page memo for the BoD by 2pm today. Meanwhile: Oracle’s cool with it (https://thenewstack.io/oracle-joins-kubernetes-fray/), “WTF is a microservice” (https://news.ycombinator.com/item?id=14414031), compared to SOA/ESB and RESTful (https://news.ycombinator.com/item?id=1441150), and James Governor tries to explain it all (http://redmonk.com/jgovernor/2017/05/31/so-what-even-is-a-service-mesh-hot-take-on-istio-and-linkerd/). BONUS LINKS! Not covered in episode. Rackspace Buys Enterprise Apps Management TriCore Link (http://www.enterprisecloudnews.com/author.asp?doc_id=733171§ion_id=571) New CEO and biggest acquisition, I thought they were quieting down with the PE Red Hat buys Codenvy Codenvy sets up your developer environments (https://codenvy.com/developers/), and has team stuff. Red Hat is really after the developer market. TaskTop has a good chance of being acquired in this climate. Pour one out from BMC/StreamStep. Notes from Carl Lehmann report at 451 (https://451research.com/report-short?entityId=92575): In-browser IDE and devtool chain(?) for OpenShift.io, based on Eclipse Che “Founded in 2013, San Francisco-based Codenvy raised $10m in January of that year, and used a portion of its funds to buy its initial codebase from eXo Platform, which had developed the eXo Cloud IDE in-browser coding suite to support its social and collaboration applications.” “The company's suite works with developer tools like subversion and git, CloudBees, Jenkins, Docker, MongoDB, Cloud Foundry, Maven and ant, as well as PaaS and IaaS offerings such as Heroku, Google AppEngine, Red Hat OpenShift and AWS.” Check out the Dell Sputnik call-out: “Rivals to Codenvy include cloud-based development suites Eclipse Orion (open source), Cloud9 IDE and Nitrous.IO. There are other 'cloud IDEs,' including Codeanywhere, CodeRun Studio, Neutron Drive and ShiftEdit. On the developer environment configuration front, Pivotal created and open-sourced a developer and OS X laptop configuration tool called Workstation, and now Sprout. Dell's Project Sputnik is seeking to address similar build environment standup productivity challenges.” Uber back in Austin Is that a thing? (https://twitter.com/Uber_ATX/status/867781159178051584) Amazon Hiring Old Folks (Like Me) Anecdotes are the singular of data (https://redmonk.com/jgovernor/2017/05/23/how-aws-cloud-is-demolishing-the-cult-of-youth/)? More Tech Against Texas’ Discriminatory Laws Lords of Tech sign a thing (https://www.dallasnews.com/news/texas-legislature/2017/05/28/mark-zuckerberg-tim-cook-texas-gov-abbott-pass-discriminatory-laws) “In addition to Facebook founder Mark Zuckerberg and Apple CEO Tim Cook, the letter was signed by Amazon CEO Jeff Wilke, IBM Chairman Ginni Rometty, Microsoft Corp. President Brad Smith and Google CEO Sundar Pichai. The leaders of Dell Technologies, Hewlett Packard Enterprise, Cisco, Silicon Labs, Celanese Corp., GSD&M, Salesforce and Gearbox Software also signed the letter.” “Peeing is not political” (https://www.texastribune.org/2017/05/28/bathroom-bill-showdown-has-been-building-years/) - recap of the history of the bathroom bill. Still doesn’t really address “is there actually a problem here, backed up with citations.” Without such coverage, it’s hard to understand (and therefore figure out and react to) the hillbilly’s side on this beyond: "It's just common sense and common decency — we don't want men in women's, ladies' rooms." It also highlights the huge, social divide between “city folk” and the hillbillies. A lot more from TheNewStack (https://thenewstack.io/tech-leaders-ask-texas-governor-halt-discriminatory-legislation/). ChefConf Retrospective ICYMI (https://www.youtube.com/watch?v=HtF3oScoYqk) Competing in Public Cloud is Crazy Expensive Link (http://www.platformonomics.com/2017/04/follow-the-capex-cloud-table-stakes/) Tracks the CAPEX spend over the years for MS, Google and Amazon A Year of Google & Apple Maps Link (https://www.justinobeirne.com/a-year-of-google-maps-and-apple-maps) Comprehensive drill-down into the mapping changes made by Google and the smaller moves by Apple. Probably not content for conversation, but whoa. FAA Flight Delay Tracking Check the map, fool (http://www.fly.faa.gov/flyfaa/usmap.jsp) Recommendations Brandon: Beauty of A Bad Idea — with Walker & Company's Tristan (http://www.stitcher.com/podcast/stitcher/masters-of-scale/e/beauty-of-a-bad-idea-with-walker-companys-tristan-walker-50186227) Matt: Arrested DevOps #84 (https://www.arresteddevops.com/yelling-at-cloud/) Old Geeks Yell At Cloud With Andrew Clay Shafer & Bryan Cantrill Epic rants. Also, Bryan Cantrill sounds like Bob Odenkirk Enjoying Westworld and everything Brandon recommended months ago Coté: Butternut-squash hash (http://www.paleorunningmomma.com/butternut-squash-hash-paleo-whole30/).