DevOps tools and resources

Stella Sebastian
Posted by Stella Sebastian February 9, 2022

superior_hosting_service

DevOps

A curated list of awesome DevOps tools, technologies, platforms and resources. Things to help/grow you in DevOps or SRE area.

Cloud Platforms

Public and Private Cloud Platforms.

  • Amazon Web Services (AWS) – Cloud Computing Services.
  • Google Cloud Platform (GCP) – Cloud Computing Services.
  • Azure – Cloud Computing Platform & Services.
  • Alibaba Cloud – integrated suite of cloud products and services.
  • Oracle Cloud – comprehensive and fully integrated stack of cloud applications and platform services.
  • DigitalOcean – helping developers easily build, test, manage, and scale applications of any size.
  • Scaleway – single way to create, deploy and scale your infrastructure in the cloud.
  • Vultr – easily deploy cloud servers, bare metal, and storage worldwide.
  • VMware Cloud – run, manage, connect and protect all of your apps on any cloud.
  • IBM Cloud – tools, data & APIs to make AI real now.
  • Stackpath – platform of computing infrastructure and services built at the edge of the cloud.
  • Linode – accelerate innovation in the cloud, virtual computing must be more accessible, affordable, and simple.

Open Source Cloud Platforms

Private, Public and Hybrid open source Cloud Platforms.

  • Openstack – open source software for creating private and public clouds.
  • Apache CloudStack – designed to deploy and manage large networks of virtual machines.
  • OpenNebula – build Private Clouds and manage Data Center virtualization based on KVM, LXD and VMware.
  • Eucalyptus – building AWS-compatible private and hybrid clouds.
  • DC/OS – distributed operating system based on the Apache Mesos distributed systems kernel.
  • Apache Mesos – program against your datacenter like it’s a single pool of resources.
  • Localstack – fully functional local AWS cloud stack. Develop and test your cloud & Serverless apps offline.

Operating Systems

Operating Systems – Server Platform.

  • Ubuntu
  • CentOS
  • CoreOS – the pioneering lightweight container host.
  • OSv – versatile modular unikernel designed to run unmodified Linux applications securely on micro-VMs in the cloud.
  • Rancher OS – a lightweight, secure Linux distribution, built from containers to run containers well.
  • Atomic – use immutable infrastructure to deploy and scale your containerized applications.
  • Photon – Linux container host optimized for cloud-native applications, cloud platforms, and VMware infrastructure.

Distributed Filesystems

Network distributed filesystems.

  • Ceph – highly scalable object, block and file-based storage under one whole system.
  • Gluster – free and open source software scalable network filesystem.
  • LINBIT – create, remove, and replicate block storage devices for datacenter scale environments.
  • XtreemFS – fault-tolerant distributed file system for all storage needs.
  • min.io – high performance, distributed object storage system.

Applications Platforms

Applications management platforms, Containers platform and Containers management.

  • Openshift – the Kubernetes platform for big ideas.
  • Dokku – helps you build and manage the lifecycle of applications.
  • Flynn – open source platform (PaaS) for running applications in production.
  • Docker – create, deploy, and run applications by using containers.
  • Docker Compose – define and run multi-container applications with Docker.
  • Docker Swarm – Docker-native clustering system.
  • Kubernetes – automating deployment, scaling, and management of containerized applications.
  • LXC – lets Linux users easily create and manage system or application containers.
  • Rancher – lets you deliver Kubernetes-as-a-Service.
  • OpenVz – container-based virtualization for Linux.
  • Singularity – run the application from the local environment to the cloud.
  • AppScale – easy-to-manage serverless platform for building and running scalable web and mobile applications.
  • Kata Containers – building lightweight virtual machines that seamlessly plug into the containers ecosystem.
  • K3S – The certified Kubernetes distribution built for IoT and Edge computing.
  • Podman – a tool for managing OCI containers and pods.

Container Image Registry

Container Image registry.

  • Quay – container image registry that enables you to build, organize, distribute, and deploy containers.
  • Dockyard – Container & Artifact Repository.
  • Harbor – an open source trusted cloud native registry project that stores, signs, and scans content.

Cloud Platforms

Public and Private Cloud Platforms.

  • Amazon Web Services (AWS) – Cloud Computing Services.
  • Google Cloud Platform (GCP) – Cloud Computing Services.
  • Azure – Cloud Computing Platform & Services.
  • Alibaba Cloud – integrated suite of cloud products and services.
  • Oracle Cloud – comprehensive and fully integrated stack of cloud applications and platform services.
  • DigitalOcean – helping developers easily build, test, manage, and scale applications of any size.
  • Scaleway – single way to create, deploy and scale your infrastructure in the cloud.
  • Vultr – easily deploy cloud servers, bare metal, and storage worldwide.
  • VMware Cloud – run, manage, connect and protect all of your apps on any cloud.
  • IBM Cloud – tools, data & APIs to make AI real now.
  • Stackpath – platform of computing infrastructure and services built at the edge of the cloud.
  • Linode – accelerate innovation in the cloud, virtual computing must be more accessible, affordable, and simple.

Open Source Cloud Platforms

Private, Public and Hybrid open source Cloud Platforms.

  • Openstack – open source software for creating private and public clouds.
  • Apache CloudStack – designed to deploy and manage large networks of virtual machines.
  • OpenNebula – build Private Clouds and manage Data Center virtualization based on KVM, LXD and VMware.
  • Eucalyptus – building AWS-compatible private and hybrid clouds.
  • DC/OS – distributed operating system based on the Apache Mesos distributed systems kernel.
  • Apache Mesos – program against your datacenter like it’s a single pool of resources.
  • Localstack – fully functional local AWS cloud stack. Develop and test your cloud & Serverless apps offline.

Operating Systems

Operating Systems – Server Platform.

  • Ubuntu
  • CentOS
  • CoreOS – the pioneering lightweight container host.
  • OSv – versatile modular unikernel designed to run unmodified Linux applications securely on micro-VMs in the cloud.
  • Rancher OS – a lightweight, secure Linux distribution, built from containers to run containers well.
  • Atomic – use immutable infrastructure to deploy and scale your containerized applications.
  • Photon – Linux container host optimized for cloud-native applications, cloud platforms, and VMware infrastructure.

Distributed Filesystems

Network distributed filesystems.

  • Ceph – highly scalable object, block and file-based storage under one whole system.
  • Gluster – free and open source software scalable network filesystem.
  • LINBIT – create, remove, and replicate block storage devices for datacenter scale environments.
  • XtreemFS – fault-tolerant distributed file system for all storage needs.
  • min.io – high performance, distributed object storage system.

Applications Platforms

Applications management platforms, Containers platform and Containers management.

  • Openshift – the Kubernetes platform for big ideas.
  • Dokku – helps you build and manage the lifecycle of applications.
  • Flynn – open source platform (PaaS) for running applications in production.
  • Docker – create, deploy, and run applications by using containers.
  • Docker Compose – define and run multi-container applications with Docker.
  • Docker Swarm – Docker-native clustering system.
  • Kubernetes – automating deployment, scaling, and management of containerized applications.
  • LXC – lets Linux users easily create and manage system or application containers.
  • Rancher – lets you deliver Kubernetes-as-a-Service.
  • OpenVz – container-based virtualization for Linux.
  • Singularity – run the application from the local environment to the cloud.
  • AppScale – easy-to-manage serverless platform for building and running scalable web and mobile applications.
  • Kata Containers – building lightweight virtual machines that seamlessly plug into the containers ecosystem.
  • K3S – The certified Kubernetes distribution built for IoT and Edge computing.
  • Podman – a tool for managing OCI containers and pods.

Container Image Registry

Container Image registry.

  • Quay – container image registry that enables you to build, organize, distribute, and deploy containers.
  • Dockyard – Container & Artifact Repository.
  • Harbor – an open source trusted cloud native registry project that stores, signs, and scans content.

Automation & Orchestration

Tools for automation, orchestration, deployment, provisioning and configuration management.

  • Ansible – simple IT automation platform that makes your applications and systems easier to deploy.
  • Salt – automate the management and configuration of any infrastructure or application at scale.
  • Puppet – unparalleled infrastructure automation and delivery.
  • Chef – automate infrastructure and applications.
  • Juju – simplifies how you configure, scale and operate today’s complex software.
  • Rundeck – Runbook Automation For Modernizing Your Operations.
  • StackStorm – connects all your apps, services, and workflows. Automate DevOps your way.
  • Bosh – release engineering, deployment, and lifecycle management of complex distributed systems.
  • Cloudify – Connect, Control, & Automate from core to edge: unlimited locations, clouds and devices.
  • Tsuru – an extensible and open source Platform as a Service software.
  • Fabric – high level Python library designed to execute shell commands remotely over SSH.
  • Capistrano – A remote server automation and deployment tool.
  • Mina – really fast deployer and server automation tool.
  • Terraform – use Infrastructure as Code to provision and manage any cloud, infrastructure, or service.
  • Pulumi – modern infrastructure as code platform that allows you to use familiar programming languages and tools to build, deploy, and manage cloud infrastructure.
  • Packer – build Automated Machine Images.
  • Vagrant – Development Environments Made Easy.
  • Foreman – complete lifecycle management tool for physical and virtual servers.
  • Nomad – deploy and Manage Any Containerized, Legacy, or Batch Application.
  • Marathon – a production-grade container orchestration platform for DC/OS and Apache Mesos.
  • Atlantis – Terraform Pull Request Automation.
  • OctoDNS – managing DNS across multiple providers. DNS as code.
  • ManageIQ – Manage containers, virtual machines, networks, and storage from a single platform.
  • Ignite – Open Source Virtual Machine (VM) manager with a container UX and built-in GitOps management.
  • Spacelift – Flexible orchestration solution for IaC development.
  • Atlantis – Terraform Pull Request Automation.

Continuous Integration & Delivery

Continuous Integration, Continuous Delivery and Continuous Delivery. GitOps

  • On premises
    • Buildbot – automate all aspects of the software development cycle.
    • Gitlab CI – pipelines build, test, deploy, and monitor your code as part of a single, integrated workflow.
    • Jenkins – automation server for building, deploying and automating any project.
    • Drone – a Container-Native, Continuous Delivery Platform.
    • Concourse – pipeline-based continuous thing-doer.
    • Spinnaker – fast, safe, repeatable deployments for every Enterprise.
    • goCD – Delivery and Release Automation server.
    • Teamcity – enterprise-level CI and CD.
    • Bamboo – tie automated builds, tests, and releases together in a single workflow.
    • Integrity – Continuous Integration server.
    • Zuul – drives continuous integration, delivery, and deployment systems with a focus on project gating.
    • Argo – Open Source Kubernetes native workflows, events, CI and CD.
    • Strider – Continuous Deployment/Continuous Integration platform.
    • Evergreen – A Distributed Continuous Integration System from MongoDB.
    • werf – Open Source CI/CD tool for building Docker images & deploying them to Kubernetes using a GitOps approach.
    • Flux – automatically ensures that the state of your Kubernetes cluster matches the configuration you’ve supplied in Git.
    • Flagger – progressive delivery Kubernetes operator (Canary, A/B Testing and Blue/Green deployments).
    • Tekton – powerful and flexible open-source framework for creating CI/CD systems.
    • PipeCD – Continuous Delivery for Declarative Kubernetes, Serverless and Infrastructure Applications.
    • Gitploy – Build the deployment system around GitHub in minutes.
  • Public Services
    • Travis CI – easily sync your projects, you’ll be testing your code in minutes.
    • Circle CI – powerful CI/CD pipelines that keep code moving.
    • Bitrise – CI/CD for mobile applications.
    • Buildkite – run fast, secure, and scalable continuous integration pipelines on your own infrastructure.
    • Cirrus CI – continuous integration system built for the era of cloud computing.
    • Codefresh – GitOps automation platform for Kubernetes apps.
    • Github actions – GitHub Actions makes it easy to automate all your software workflows, now with world-class CI/CD.

Source Code Management

Source Code management, Git-repository manager, Version Control. Some of them include Code review.

  • Github – helps developers store and manage their code, as well as track and control changes to their code.
  • Gitlab – entire DevOps lifecycle in one application.
  • Bitbucket – gives teams one place to plan projects, collaborate on code, test, and deploy
  • Phabricator – a collection of web applications which help software companies build better software.
  • Gogs – a painless self-hosted Git service.
  • Gitea – a painless self-hosted Git service.
  • Gitblit – pure Java Git solution for managing, viewing, and serving Git repositories.

Web Servers

Web servers and reverse proxy.

  • Nginx – high performance load balancer, web server and reverse proxy.
  • Apache – web server and reverse proxy.
  • Caddy – web server with automatic HTTPS.
  • Cherokee – highly concurrent secured web applications.
  • Lighttpd – optimized for speed-critical environments while remaining standards-compliant, secure and flexible.
  • Uwsgi – application server container.

SSL

Tools for automating the management of SSL certificates.

  • Certbot – automate using Let’s Encrypt certificates on manually-managed websites to enable HTTPS.
  • Let’s Encrypt – free, automated, and open Certificate Authority.
  • Cert Manager – K8S add-on to automate the management and issuance of TLS certificates from various issuing sources.

Databases

Relational (SQL) and non-relational (NoSQL) databases.

  • Relational (SQL)
    • PostgreSQL – powerful, open source object-relational database system.
    • MySQL – open-source relational database management system.
    • MariaDB – fast, scalable and robust, with a rich ecosystem of storage engines, plugins and many other tools.
    • SQLite – small, fast, self-contained, high-reliability, full-featured, SQL database engine.
  • Non-relational (NoSQL)
    • Casandra – manage massive amounts of data, fast, without losing sleep.
    • Apache HBase – distributed, versioned, non-relational database.
    • Couchdb – database that completely embraces the web.
    • Elasticsearch – distributed, RESTful search and analytics engine capable of addressing a growing number of use cases.
    • MongoDB – general purpose, document-based, distributed database built for modern applications.
    • Rethinkdb – open-source database for the realtime web.
    • Key-Value
      • Couchbase – distributed multi-model NoSQL document-oriented database that is optimized for interactive applications.
      • Leveldb – fast key-value storage library.
      • Redis – in-memory data structure store, used as a database, cache and message broker.
      • RocksDB – a library that provides an embeddable, persistent key-value store for fast storage.
      • Etcd – distributed reliable key-value store for the most critical data of a distributed system.

Observability & Monitoring

Observability, Monitoring, Metrics/Metrics collection and Alerting tools.

  • Sensu – Simple. Scalable. Multi-cloud monitoring.
  • Alerta – scalable, minimal configuration and visualization monitoring system.
  • Cabot – self-hosted, easily-deployable monitoring and alerts service.
  • Amon – modern server monitoring platform.
  • Flapjack – monitoring notification routing + event processing system.
  • Icinga – monitors availability and performance, gives you simple access to relevant data and raises alerts.
  • Monit – managing and monitoring Unix systems.
  • Naemon – fast, stable and innovative while giving you a clear view of the state of your network and applications.
  • Nagios – computer-software application that monitors systems, networks and infrastructure.
  • Sentry – error monitoring that helps all software teams discover, triage, and prioritize errors in real-time.
  • Shinken – monitoring framework.
  • Zabbix – mature and effortless monitoring solution for network monitoring and application monitoring.
  • Glances – monitoring information through a curses or Web based interface.
  • Healthchecks – cron monitoring tool.
  • Bolo – building distributed, scalable monitoring systems.
  • cAdvisor – analyzes resource usage and performance characteristics of running containers.
  • ElastiFlow – network flow monitoring (Netflow, sFlow and IPFIX) with the Elastic Stack.
  • Co-Pilot – system performance analysis toolkit.
  • Metrics/Metrics collection
    • Thundra Foresight – visibility into CI pipeline by spotting test failures in no time.
    • Prometheus – power your metrics and alerting with a leading open-source monitoring solution.
    • Collectd – the system statistics collection daemon.
    • Facette – time series data visualization software.
    • Grafana – analytics & monitoring solution for every database.
    • Graphite – store numeric time-series data and render graphs of this data on demand.
    • Influxdata – time series database.
    • Netdata – instantly diagnose slowdowns and anomalies in your infrastructure.
    • Freeboard – real-time dashboard builder for IOT and other web mashups.
  • Logs Management
    • Anthracite – an event/change logging/management app.
    • Graylog – free and open source log management.
    • Logstash – collect, parse, transform logs.
    • Fluentd – data collector for unified logging layer.
    • Flume – distributed, reliable, and available service for efficiently collecting, aggregating, and moving logs.
    • Heka – stream processing software system.
    • Kibana – explore, visualize, discover data.
    • Loki – horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus.
  • Status
    • Cachet – beautiful and powerful open source status page system.

Service Discovery & Service Mesh

Service Discovery, Service Mesh and Failure detection tools.

  • Consul – connect and secure any service.
  • Serf – decentralized cluster membership, failure detection, and orchestration.
  • Doozerd – a consistent distributed data store.
  • Zookeeper – centralized service for configuration, naming, providing distributed synchronization, and more.
  • Etcd – distributed, reliable key-value store for the most critical data of a distributed system.
  • Istio – connect, secure, control, and observe services.
  • Kong – deliver performance needed for microservices, service mesh, and cloud native deployments.
  • Linkerd – service mesh for Kubernetes and beyond.

Chaos Engineering

The discipline of experimenting on a distributed system in order to build confidence in the system’s capability to withstand turbulent conditions in production.

  • Chaos Toolkit – the Open Source Platform for Chaos Engineering.
  • Chaos Monkey – a resiliency tool that helps applications tolerate random instance failures.
  • Toxiproxy – simulate network and system conditions for chaos and resiliency testing.
  • Pumba – chaos testing, network emulation and stress testing tool for containers.

API Gateway

API Gateway, Service Proxy and Service Management tools.

  • API Umbrella – proxy that sits in front of your APIs.
  • Ambassador – Kubernetes-Native API Gateway built on the Envoy Proxy.
  • Kong – connect all your microservices and APIs with the industry’s most performant, scalable and flexible API platform.
  • Tyk – API and service management platform.
  • Cilium – API aware networking and security using BPF and XDP.
  • Gloo – feature-rich, Kubernetes-native ingress controller, and next-generation API gateway.
  • Envoy – cloud-native high-performance edge/middle/service proxy.
  • Traefik – reverse proxy and load balancer for HTTP and TCP-based applications.
  • Nginx – high performance reverse proxy.

Code review

Code review. A few of the Source Code Management tools have built-in code review features.

  • Gerrit – web-based team code collaboration tool.
  • Review Board – web-based collaborative code review tool.

Distributed messaging

Distributed messaging platforms and Queues software.

  • Rabbitmq – message broker.
  • Kafka – building real-time data pipelines and streaming apps.
  • Activemq – Multi-Protocol messaging.
  • Beanstalkd – simple, fast work queue.
  • NSQ – realtime distributed messaging platform.
  • Celery – asynchronous task queue/job queue based on distributed message passing.
  • Faktory – repository for background jobs within your application.
  • Nats – simple, secure and high performance open source messaging system.
  • RestMQ – message queue which uses HTTP as transport.
  • Dkron – distributed, fault tolerant job scheduling system.

Programming Languages

Programming languages.

  • Python – programming language that lets you work quickly and integrate systems more effectively.
  • Ruby – a dynamic, open source programming language with a focus on simplicity and productivity.
  • Go – an open source programming language that makes it easy to build simple, reliable, and efficient software.

Chat and ChatOps

Chat and ChatOps.

  • Rocket – open source team communication.
  • Mattermost – messaging platform that enables secure team collaboration.
  • Zulip – real-time chat with an email threading model.
  • Riot – a universal secure chat app entirely under your control.
  • ChatOps:
    • CloudBot – simple, fast, expandable, open-source Python IRC Bot.
    • Hubot – a customizable life embetterment robot.
    • Lita – a robot companion for your company’s chat room.

Secret Management

Security as code, sensitive credentials and secrets need to be managed, security, maintained and rotated using automation.

  • Sops – simple and flexible tool for managing secrets.
  • Vault – manage secrets and protect sensitive data.
  • Keybase – end-to-end encrypted chat and cloud storage system.
  • Vault Secrets Operator – create Kubernetes secrets from Vault for a secure GitOps based workflow.
  • Git Secret – a bash-tool to store your private data inside a git repository.

Sharing

A collection of tools to help with sharing knowledge and telling the story.

  • Gitbook – modern documentation format and toolchain using Git and Markdown.
  • Docusaurus – easy to maintain open source documentation websites.
  • Docsify – a magical documentation site generator.
  • MkDocs – project documentation with Markdown.

VPN

VPN, routing and firewall.

  • OpenVPN – flexible VPN solutions to secure your data communications, whether it’s for Internet privacy,
  • Pritunl – enterprise Distributed OpenVPN and IPsec Server.
  • VyOS – open source network OS that runs on a wide range of hardware, virtual machines, and cloud providers.
  • Algo – set up a personal VPN in the cloud.
  • Streisand – sets up a new VPN service nearly automatically.
  • Freelan – a peer-to-peer, secure, easy-to-setup, multi-platform, open-source, highly-configurable VPN software.
  • Sshuttle – transparent proxy server that works as a poor man’s VPN.

Resources

Books

Books focused on DevOps, DevSecOps and Site Reliability Engineering.

Conferences

The Awesome DevOps is a github repository by Marius Stanca

Source from
AWS Security Tools
Tags: