DevOps / SRE — Top Links Last Week

Week 16 Issue #23

10 GitHub Repositories every Developer should know

GitHub is home to some of the most interesting Open Source Projects on the internet. GitHub is also a great place for sharing resources of all kinds, from free books to APIs, from roadmaps to project ideas and to the projects. With sheer mass it becomes difficult to find the most useful repositories which you may think doesn't exist. So I have curated this list of ten fabulous repositories that provide a great value for all web and software developers. All of them will add value to you and help you to become a better web or software developer.

Pulumi 3.0

Pulumi 3.0 is the next major version of the Pulumi open source project, and the foundation for Pulumi’s Cloud Engineering Platform. Pulumi offers the most complete infrastructure as code platform for building, deploying and managing modern cloud infrastructure and applications. This release includes dozens of significant new features and hundreds of improvements that build on this foundation. New Pulumi Providers for Azure (GA) and Google Cloud (Preview) include 100% platform coverage, same-day support for new features.

Please consider supporting the Weekly DevOps / SRE Report. Subscribe to the phpops Newsletter on our website!

MLOps vs DevOps

MLOps is a practice used by individuals and teams when developing software systems. DevOps involves continuous integration (CI), continuous delivery (CD) and continuous training (CT) to machine learning systems. MLOps aims to fuse together the machine learning system development (ML) and machine learning operation (Ops) together. This article goes through the similarities and differences between DevOps and MLOps as well as platforms that help enable MLOps. In MLOps there is a third concept that does not exist in DevOps which is Continuous Training (CT). This step is all about automatically identifying scenarios/events that requires a model to be re-trained and re-deployed into production due to a performance degradation.

Airbnb Deploys 125,000 Times per Year with Multicluster Kubernetes

Airbnb is one of the largest online marketplaces for lodging. Founded in 2007, the platform currently has more than 5.5 million listings across more than 100,000 cities and 220 countries. Airbnb has moved to a service-oriented architecture (SOA) in Amazon Elastic Compute Cloud. To address scaling, the company began migrating to Kubernetes in 2017. Airbnb’s main production cluster had reached 450 nodes. By December of the same year, this had doubled to 900 nodes.

Is Low-Code Development a Security Risk?

According to Gartner, the worldwide low-code development market is projected to be $13.8 billion in 2021, a whopping increase of 22.6% from 2020. The advantage is that it enables users from a non-traditional development background to participate in the app development process. It also narrows the demand-supply gap that arose as organizations catered to ever-rising digitization needs with limited resources and developer scarcity. Low-code is a visual paradigm of application development that involves drag-and-drop of pre-built components and integrations.

Agile Or DevOps — The Process. The Purpose. The Result.

Agile and DevOps are the two buzz concepts of the same modern software development practices that many organizations are eager to employ. Though the two are NOT essentially the same they can often create mix-up or overlap while trying to comprehend them. Both are aimed at providing a framework to produce a new product or a part of a product, a launch, or a release. DevOps is a software development practice that brings people, processes and technology together to deliver continuous value. The core attributes of Agile revolves around Transparency, Inspection and Adaptability, keeping pace with customer needs.

A Deep Diven Into Microservices vs. Monolith Architecture

In this article, we will discuss what teams and projects should use what type of architecture. Exploring the opposite and similar worlds of both concepts. Each has its merit above against other, and not leaving out its flaws also. The architectural design of microservices architecture requires more work — we will need to deploy each microservice independently, put into consideration the orchestration tools, such as Kubernetes, Docker Swarm, Mesosphere, or other similar tools that would help us manage infrastructure with a lot of moving parts.

Kubernetes Security with Kube-bench and Kube-hunter

Kube-bench is a Go application that checks whether Kubernetes is deployed securely by running the checks documented in the CIS Benchmarks. The tool was developed to increase awareness and visibility for security issues in the cluster. In this article, we will understand how to use Kube.bench and Kube.hunter to. secure your. cluster using the tools that help you identify vulnerabilities in your cluster. The. tools were developed by the Center for Internet Security (CIS), a non-profit organization that identifies best cybersecurity practices.

The horror of microservices in small teams

Amazon and Netflix were one the big first companies mentioning their use of microservices. Amazon has coined the term of pizza-sized teams and says that “individual teams shouldn’t be larger than what two pizzas can feed” One team should fully own a microservice — it should never be split between 2 teams. With microservices being decoupled from everything else, it’s a perfect way to scale up the number of development teams. The same goes for microservices if your team handles all microservices anyway and is full control of the application.

A Beginner’s Guide To The Developing RESTful APIs with Lumen Microservices

Lumen is an open-source PHP micro-framework created by Taylor Otwell as an alternative to Laravel to meet the demand of lightweight installations that are faster than existing micro-frameworks such as Slim and Silex. With Lumen, you can build lightning-fast microservices and APIs that can support your Laravel applications. Lumen uses the Illuminate components that power the Laravel framework. The Lumen API is provided out of the box in Lumen. You can create rest based microservices with lumen after finishing this article.

CI/CD on Serverless with Google Cloud Platform

Create a Dockerfile or a yaml configuration file and keep it in the repo you want the CI/CD to perform with. Those files will be automatically detected and will trigger the build process. When cloud build starts the build, it copies the full repository and put it under the directory /workspace/. When we use an image to build, the first directory it uses by default is /workpace/. The first directory Cloud Build uses is the directory called ‘workspace’ by default.

The Balancing Act of Infrastructure Security

Infrastructure Security provides seamless security guard rails for application developers while balancing complexity and costs. Netflix calls this “secure by default” approach The Paved Road. Best controls will be transparent to developers and users while also having a high ratio of security value compared to ongoing maintenance costs. Additional maintenance and complexity increases the likelihood that the system of security controls will fail and should be avoided where possible. An InfraSec team can’t implement and maintain an effective, balanced set of controls without first having a basic control catalog in place.

A True Easy Way to Run Kubernetes on AWS

AWS EKS Base is a repo containing a set of infrastructure and system configuration files. It uses Terraform to provision resources, deploys to AWS (EKS is a managed Kubernetes), and runs several supporting systems in K8s, such as Loki, Prometheus, build system, and so on. The best bit — how you run it is that you provision a network for the different types of resources. It leverages a mix of Spots and On-Demands instances to achieve the best pricing in the AWS region you run your infrastructure.

Enhancing Terraform with Terragrunt

Terragrunt is a thin wrapper that overlays Terraform that allows you to run commands using run-all from root to run all of your modules at the same time. When you add a terragrunt.hcl file to each module, you can run commands against all of the modules in the same way. TerragRunt recursively looks through each of your directories for the presence of that file, and if it finds it, will run the command you've included.

A Recipe for Enterprise DevOps With Kubernetes

DevOps answers the question, “How do software companies deliver value efficiently in this quickly changing world, at speed and with quality, while maintaining the trust of their customers?” The five principles below are all closely related and will help you reach the end goal of delivering value to your customers. Services are jointly owned end-to-end; no more throwing code over the wall for Ops to deal with after it is written. Everything that happens to deliver software after a commit is made should be automated. There should be a short feedback cycle and quick adaptation based on that feedback.

The Central Tenets of Serverless Infrastructure

Many enterprises are building their IT modernization, digital transformation and microservices initiatives on serverless platforms. Serverless platforms are well-suited to workloads that are on-demand, event-driven, asynchronous, stateless and scalable. Common examples of workloads suited to serverless include low-latency processing of real-time data streams, batch processing of incoming image files, application of incoming changes to a database, and execution of business process workflows. Public cloud providers play a pivotal role in the deployment of serverless applications for enterprise customers.

Automated Canary Deployment with HashiCorp Consul and Spinnaker

When you update an application or system, you likely push the changes to production and manually verify whether or not it works. Canary deployments often require someone to analyze application metrics and manually increase the percentage of traffic to the new application. This post will show how to automate a canary analysis and deployment of an application with Spinnaker analysis, Prometheus metrics, and HashiCorp Consul service mesh. You can use Consul to generate proxy metrics and Prometheus to collect them. Consul uses proxies to manage and shape traffic between services. The Consul chart creates a Consul server, clients, controller, injector, and UI.

Methods for Developers to Test Their Application

Developers need to make sure that the application behaves the way they programmed it at every step of the way and ensure that there are no security vulnerabilities within the code. There are many different methods of testing that developers can implement as well as many techniques. Unit testing is a software testing technique in which individual components or units are tested. SAST is performed early on in the software development life cycle and should be used as often as possible on all files containing source code. Dynamic Application Security Testing (DAST) is another method used to test how secure a code is.

How to Achieve DevOps Nirvana

DevOps is supposed to increase speed and collaboration, but it does create some friction or pinch points requiring human intervention that may slow things down. 75% of DevOps initiatives will fail to meet expectations due to organizational learning and change issues. DevOps tools should enable and encourage better human to computer interaction like an episode of Star Trek. The top DevOps challenge is a lack of communication and responsiveness amongst your development, security, and operations teams. The answers often reside inside unstructured data like in email, Jira, Slack.

IaC — Automate your infrastructure with AWS and CDK

IaC stands for Infrastructure as Code, and as the name says, it represents the process of defining the infrastructure of a service by the instruction contained in an executable file. In Zigbang's architecture, we treat each service individually, with a separate environment of its own. The idea is very powerful because it truly enables programmability and reusability inside of your infrastructure code. The true capacity of the development kit shines when you work with many services that use similar infrastructure, says Zigbang.

Comparing the New Generation of Build Tools

A bunch of new developer tools have landed in the past year and they are biting at the heels of the tools that have dominated front-end development over the last few years. Despite their differences, these tools do share a common goal: improve the developer experience. I’d like to evaluate each one, outlining what they do, why we need them, and their use cases. I think these tools are arriving as a reaction to JavaScript tooling fatigue. They fill a missing middle ground between writing a single vanilla JavaScript file, and having to download 200 megabytes of tooling dependencies before writing a line of your own code.

The Benefits of Moving from Terraform to Pulumi

The trusty infrastructure as code provider is starting to show some gray hairs. Pulumi is conceptually similar to Terraform. It’s a better alternative than Terraform, but Terraform's strict code structure is also a downfall. I’ll dive into the unwritten “between-the-lines” benefits of Pulumi over Terraform over the web-based code provider. The Pulumi/Go version of Terraform is written in Go, Node.js, Python,NET Core, Go.