DevOps Industry Updates #1
It’s been only two weeks since the last issue and there’s already so much to cover! Besides the usual tech updates, we’ve seen some increased M&A activity and a growing trend of companies doubling-down/going all-in on their remote working cultures. Without a doubt, our industry is currently undergoing a major transformation and I personally can’t wait to see what type of efficiencies can be enacted with the new societal norms that form.
🔥 Top Cream
This issue’s top 3 stories:
- WeChat Surveillance Explained
- FairwindsOps/goldilocks: get your Kubernetes resource requests “Just Right”
- EC2 Price Reduction – For EC2 Instance Saving Plans and Standard Reserved Instances
- Zoom Acquires Keybase by Keybase: our single top priority is helping to make Zoom even more secure. There are no specific plans for the Keybase app yet, but ultimately Keybase’s future is in Zoom’s hands.
Facebook to buy Giphy for $400 million by Dan Primack, Kia Kokalitcheva & Sara Fischer: Facebook has agreed to buy Giphy, the popular platform of sharable animated images, Axios has learned from multiple sources.
Scott Forstall tells story about Steve Jobs, Microsoft, and a dead fish by Dave Mark: Scott Forstall, via Zoom, tells audience at Code Break about meeting Steve Jobs for the first time.
- The pandemic is bringing us closer to our robot takeout future by Timothy Lee: “We saw that business double overnight,” startup says of UK grocery deliveries.
🏠 The permanence of WFH
- Google Will Let Employees Work From Home Until The End Of 2020
- Facebook Starts Planning for Permanent Remote Workers
- Twitter Will Allow Employees To Work At Home Forever
- Square announces permanent work-from-home policy
- Shopify permanently moves to work-from-home model
- Box embraces a remote/office hybrid culture
- What’s new in Grafana v7.0 by Grafana Labs: this release is “the most monumental release for us in the company’s history,” said Raj Dutt, co-founder and CEO of Grafana Labs. Grafana 7.0 brings tracing to the observability platform, rounding out support for the three pillars of observability — logging, metrics, and tracing. The update also allows users to transform data on the fly and create plugins for new data sources in any language.
- Announcing HashiCorp Consul 1.8 by Neena Pemmaraju: Consul 1.8 adds features that lower the barrier to entry for adopting a service mesh in heterogeneous environments. These include 3 new Gateways: Ingress, Terminating and WAN Federation over Mesh.
- Quay.io’s outage was so bad, Ambassador switched to Docker Hub by Richard Li: given the mission critical nature of Ambassador for our customers, we’ve switched our container registry from Quay to Docker Hub, effective immediately.
- Apache Kafka Needs No Keeper: Removing the Apache ZooKeeper Dependency by Colin McCabe: currently, Apache Kafka uses Apache ZooKeeper to store its metadata. Data such as the location of partitions and the configuration of topics are stored outside of Kafka itself, in a separate ZooKeeper cluster. This initiative will break this dependency and bring metadata management into Kafka itself.
Speaking of Kafka:
What Every Software Engineer Should Know about Apache Kafka: Events, Streams, Tables, Storage, Processing, And More by Michael Noll: to help fellow engineers wrap their head around Apache Kafka and event streaming, I wrote a 4-part series on the Confluent blog on Kafka’s core fundamentals. In the series, we explore Kafka’s storage and processing layers and how they interrelate, featuring Kafka Streams and ksqlDB.
Jepsen Disputes MongoDB’s Data Consistency Claims by Jonathan Allen: MongoDB claimed that their database passed “the industry’s toughest data safety, correctness, and consistency Tests”. In response, Jepsen published an article stating that MongoDB 3.6.4 had in fact failed their tests and that the newer MongoDB 4.2.6 has even more problems.
- A multi-node, elastic, petabyte scale, time-series database on Postgres for free by Ajay Kulkarni: we’re officially making multi-node TimescaleDB, a petabyte-scale distributed time-series database on PostgreSQL available for free.
- Hashicorp Learn: Learn how to provision, secure, connect, and run any infrastructure for any application.
🛠️ DevOps Tools
- goldilocks by FairwindOps: get your resource requests “Just Right”. By using the kubernetes
vertical-pod-autoscalerin recommendation mode, we can see a suggestion for resource requests on each of our apps. This tool creates a VPA for each deployment in a namespace and then queries them for information.
- subspacecommunity/subspace: A simple WireGuard VPN server GUI.
salesforce/cloudsplaining by Salesforce: Cloudsplaining is an AWS IAM Security Assessment tool that identifies violations of least privilege and generates a risk-prioritized HTML report with a triage worksheet.
Supporting the HashiCorp Terraform Extension for Visual Studio Code by Paul Tyng: the original creator (Mikael Olenfalk) of the VS Code extension has transferred ownership of the extension to HashiCorp. HashiCorp is working internally to update the VS Code extension to fully support the 0.12 syntax and use the HashiCorp Language Server by default.
- WSL+Docker: Kubernetes on the Windows Desktop by Nuno do Carmo Docker & Ihor Dvoretskyi: new to Windows 10 and WSL2, or new to Docker and Kubernetes? Welcome to this blog post where we will install from scratch Kubernetes in Docker KinD and Minikube.
6 Ways Salesforce Gets Things Done with Python by Laura Lindeman: Salesforce Engineering puts Python to work across many areas of their business, including machine learning, security, internal DevOps teams and more.
Electron 9.0.0 Released by Verte Dinde: the popular cross platform desktop app framework gets more dependency bumps and is now running on Chromium 83, V8 8.3 and Node.js 12.14.
State of routing in Rust by Pavan Kumar Sunkara: there are many micro frameworks in Rust and we have seen many blog posts comparing their performances and middleware capabilities. But what we haven’t seen is an article comparing their routing functionality and capabilities - until now.
Learning Rust in 2020 by pretzelhammer: reviews of free online resources a rust beginner can use to practice writing small simple Rust programs.
Five Years of Rust by The Rust Core Team: it has been five years since we released 1.0! Rust has changed a lot these past five years, so we wanted to reflect back on all of our contributors’ work since the stabilization of the language.
(sorry for all the Rust articles!)
SSH Agent Explained by Carl Tashian: the SSH agent is a central part of OpenSSH. In this post, I’ll explain what the agent is, how to use it, and how it works to keep your keys safe. I’ll also describe agent forwarding and how it works.
What Is Nix by Burke Libbey: a crash course in what Nix is, how to think about it, and why it’s such a valuable and paradigm-shifting piece of technology.
- systemd, 10 years later: a historical and technical retrospective by V.R.: 10 years ago, systemd was announced and swiftly rose to become one of the most persistently controversial and polarizing pieces of software in recent history, especially in the GNU/Linux world.
- Not every container has an operating system inside by Ivan Velichko: …but every one of them needs your Linux kernel. At great deep-dive into process isolation with Linux containers and their virtualization capabilities.
- Nintendo 64 Architecture by Rodrigo Copetti: a fantastic overview of how Nintendo implemented 3D graphics, audio and anti-piracy/region-lock features for the N64 console.
- Victory! ICANN Rejects .ORG Sale to Private Equity Firm Ethos Capital by Karen Gullo and Mitch Stoltz: in a stunning victory for nonprofits and NGOs around the world working in the public interest, ICANN roundly rejected Ethos Capital’s plan to transform the .ORG domain registry into a for-profit entity.
- U.S. Secret Service: “Massive Fraud” Against State Unemployment Insurance Programs by Brian Krebs: a well-organized Nigerian crime ring is exploiting the COVID-19 crisis by committing large-scale fraud against multiple state unemployment insurance programs, with potential losses in the hundreds of millions of dollars, according to a new alert issued by the U.S. Secret Service.
WeChat Surveillance Explained by Miles Kenyon: how WeChat (the most popular social app in China) conducts surveillance of images and files shared on the platform and uses the monitored content to train censorship algorithms.
The Confessions of Marcus Hutchins, the Hacker Who Saved the Internet by Andy Greeenberg: At 22, he single-handedly put a stop to the worst cyberattack the world had ever seen. Then he was arrested by the FBI. This is his untold story.
A passwordless server run by spyware maker NSO sparks contact-tracing privacy concerns by Zack Whittaker: security researcher Bob Diachenko discovered one of NSO’s contact-tracing systems on the internet, unprotected and without a password, for anyone to access.
Unmaintained Dependencies and Other Ways to Measure CI/CD Security by Lawrence E Hecht: looks at five recent studies on software supply chain security, with a focus on CI/CD and open source code.
- Memory safety by Chromium Security: the Chromium project finds that around 70% of our serious security bugs are memory safety problems.
Amazon Web Services
- Scaling FTP and NFS in AWS to handle millions of files by David Christ: how a ops team migrated their legacy on-prem FTP service to AWS. That was not exactly straightforward and many lessons were learnt - this article shares those key take-aways.
- AWS Networking 101 by Ivan Pepelnjak: a high-level overview of AWS VPC and a look at VPC Packet Forwarding.
- Introducing CDK for Kubernetes by Elad Ben-Israel and Nathan Taber:
cdk8sis a new open-source project that lets you define Kubernetes applications and reusable components using familiar programming languages. cdk8s (pronounced “cd kates“) lets you use programming languages like TypeScript or Python to generate standard Kubernetes YAML – which means that you can use it to define applications for any Kubernetes cluster running anywhere, both on-premises and the cloud.
AWS service updates
- Easily control the naming of individual IAM role sessions by Derrick Oigiagbe: AWS Identity and Access Management (IAM) now has a new sts:RoleSessionName condition element for the AWS Security Token Service (AWS STS), that makes it easy for AWS account administrators to control the naming of individual IAM role sessions.
AWS CloudFormation now supports blue/green deployments for Amazon ECS: AWS CloudFormation is now integrated with AWS CodeDeploy to allow ECS customers with application or network load balancers to invoke blue/green and canary style deployments when performing application updates.
EC2 Price Reduction – For EC2 Instance Saving Plans and Standard Reserved Instances by Martin Beeby: a price reduction for EC2 customers who plan to use Standard Reserved Instances or EC2 Instance Saving Plans. The price changes are already in effect, and so anyone buying new RIs or a new EC2 Instance Saving Plan will be able to take advantage of the lower prices.
- 90%+ price reduction for AWS IoT Jobs by Alejandra Quetzalli: I have good news for AWS customers using the AWS IoT Device Management service. There has been a 90%+ price reduction for AWS IoT Device Jobs!
- General Availability of UltraWarm for Amazon Elasticsearch Service by Martin Beeby: this new low-cost storage tier provides fast, interactive analytics on up to three petabytes of log data at one-tenth of the cost of the current Amazon Elasticsearch Service storage tier.
AWS Trusted Advisor adds 5 Cost Optimization checks: AWS Trusted Advisor has launched 5 new Cost Optimization checks in Trusted Advisor, including a check for Savings Plan recommendations and 4 checks that provide cost savings recommendations for ElastiCache, RedShift, ElasticSearch and RDS.
Amazon EC2 now supports aliases for Amazon Machine Images (AMIs): Amazon Elastic Compute Cloud (EC2) now supports the use of custom identifiers to reference Amazon Machine Images (AMIs) during instance launch.
New query monitoring capabilities in the Amazon Redshift console: the Amazon Redshift console simplifies isolating and fixing expensive queries with a newly-redesigned query monitoring page. For a deeper dive, checkout this article.
Amazon CloudWatch now monitors Prometheus metrics: you can now use Amazon CloudWatch to monitor Prometheus metrics from Amazon Elastic Kubernetes Service (EKS) and Kubernetes clusters, available now in beta.
- Amazon EC2 M6g instances powered by AWS Graviton2 processors are now generally available: Amazon EC2 M6g instances powered by Arm-based AWS Graviton2 processors are generally available. Amazon EC2 M6g instances deliver up to 40% better price performance over the current generation x86-based Amazon EC2 M5 instances for a broad set of general-purpose workloads.
- ‘Azure appears to be full’: UK punters complain of capacity issues on Microsoft’s cloud by Tim Anderson: customers are reporting capacity issues such as the inability to create resources and associated reliability issues.
Which ironically brings us to:
- Announcing the general availability of Azure Spot Virtual Machines by Varun Shandilya: similar to AWS’s EC2 spot instances, Azure Spot VMs provide access to unused Azure compute capacity at deep discounts. Spot pricing is available on single VMs in addition to VM scale sets (VMSS).
9 commandments of building a strong remote team culture by Renee Fleck: remote is the new normal. Or is it? Four years into building a remote company, Indiez, has taught me a lot.
I Led Companies Through 2 Downturns. Here’s What I Learned by David Cancel: In times of uncertainty, you must rapidly move from the mindset of a peacetime CEO to that of a wartime CEO.
3 Guiding Principles for Building New SaaS Products on AWS by Jared Short: based on what I have seen and learned over the years, I want to talk about how I would (and in fact have) set up new organizations from day one for future success.
Article version: 1.0.0