Cloud

Debian Template on Proxmox

Cloud Image with SSH Key

on the Proxmox Host

Create Installer Key

# cd /your/working/dir
ssh-keygen -o -a 100 -t ed25519 -C "User: installer, $(date '+%Y-%m-%d %H:%m')" -f installer

Build Template

apt install libguestfs-tools -y;
wget https://cloud.debian.org/images/cloud/bookworm/latest/debian-12-generic-amd64.qcow2

virt-customize --install qemu-guest-agent -a debian-12-generic-amd64.qcow2
qm create 9001 --name debian-12-generic --memory 2048 --cores 2 --net0 virtio,bridge=vmbr0 --agent 1 
qm importdisk 9001 debian-12-generic-amd64.qcow2 local-lvm

qm set 9001 --scsihw virtio-scsi-pci --scsi0 local-lvm:vm-9001-disk-0
qm set 9001 --ide2 local-lvm:cloudinit
qm set 9001 --boot c --bootdisk scsi0
qm set 9001 --serial0 socket
qm set 9001 --ipconfig0 ip=dhcp
qm set 9001 --cpu cputype=x86-64-v2-AES

qm set 9001 --sshkeys installer.pub
qm set 9001 --ciuser installer

qm resize 9001 scsi0 32G
qm template 9001

Create VM

in the GUI, you can create an new VM based on the Template 9001. You have to login with user “installer” and it’s private key!

Deploy Sampleapp on Kuberentes …

Prerequisite

• Domain “kubbi.xyz”, ns1.digitalocean.com, ns2, ns3 …
• Digital Ocean Login

Doku

https://docs.digitalocean.com/products/kubernetes/getting-started/operational-readiness/

Build Kubbi Cluster

Build Cluster with WebGUI or CLI

via CLI

time doctl kubernetes cluster create prod001 --region fra1 --node-pool "size=s-2vcpu-2gb;auto-scale=true;min-nodes=3;max-nodes=5"

• FRA1
• 3 Nodes
• 2 CPU
• 2 GB RAM
• 60 GB Disk
• Costs: 54 USD/Mt!

Connecting and managing this cluster

doctl kubernetes cluster kubeconfig save 4375b470-ebe8-4ccb-925a-345df364dfbd

user@mac % doctl kubernetes cluster kubeconfig save 4375b470-ebe8-4ccb-925a-345df364dfbd


Notice: Adding cluster credentials to kubeconfig file found in "/Users/user/.kube/config"
Notice: Setting current-context to do-fra1-k8s-1-28-2-do-0-fra1-1702031438694

kubectl config get-contexts

kubectl config get-contexts

user@mac % kubectl config get-contexts

CURRENT   NAME                                         CLUSTER                                      AUTHINFO                                           NAMESPACE
*         do-fra1-prod-cluster-01                      do-fra1-prod-cluster-01                      do-fra1-prod-cluster-01-admin

kubectl cluster-info

kubectl cluster-info

user@mac % kubectl cluster-info

Kubernetes control plane is running at https://f179692f-aeac-4f9c-af3b-2422897ea578.k8s.ondigitalocean.com
CoreDNS is running at https://f179692f-aeac-4f9c-af3b-2422897ea578.k8s.ondigitalocean.com/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

kubectl version

kubectl version

user@mac % kubectl version

Client Version: v1.28.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.2

kubectl get nodes

kubectl get nodes

user@mac % kubectl get nodes

NAME                    STATUS   ROLES    AGE     VERSION
prod-cluster-01-xa856   Ready    <none>   3m59s   v1.28.2
prod-cluster-01-xa85a   Ready    <none>   4m7s    v1.28.2

Install Nginx Ingress Controller

• via GUI / Marketplace / NGINX Ingress Controller

show lbl status

doctl compute load-balancer list --format IP,ID,Name,Status

user@mac % doctl compute load-balancer list --format IP,ID,Name,Status

IP    ID                                      Name                                Status
      55d7381d-9bcd-4f8f-905e-e90efd09f03e    a6554a9aff1a340e986d48431e19cca9    new

no ip yet … wait …

Source

https://www.youtube.com/watch?v=dEAtD9PVr_Q

get Host

Build a VM on Hetzner, Ubuntu 22.04

Patch

Patch the Box

apt-get update
apt-get --fix-broken install
apt-get -y upgrade
apt-get install lynx uptimed

Kurl Small

give a try with kurl. the Open Source Kubernetes Installer ;)

time curl https://kurl.sh/ccedeec |bash |tee -a install.md

real	4m5.149s
user	1m26.425s
sys	0m22.249s

Kurl Full

same as small, but with most options enabled

curl https://kurl.sh/13609c3 | sudo bash

Ressource Warning

got a warning as i took a bit a small VM (CX21). Just ignore it.

Install Debian 12

or install Debian 11.7 and Upgrade to 12

Setup

3 Nodes

192.168.100.151     k8s-master
192.168.100.152     k8s-worker1
192.168.100.153     k8s-worker2

Locale

export LC_CTYPE=en_US.UTF-8
export LC_ALL=en_US.UTF-8

Kubernetes

https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"

Swap Off

swapoff -a
sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

Install FW

apt-get install ufw
ufw enable

Master

ufw allow 22/tcp
ufw allow 6443/tcp
ufw allow 2379/tcp
ufw allow 2380/tcp
ufw allow 10250/tcp
ufw allow 10251/tcp
ufw allow 10252/tcp
ufw allow 10255/tcp
ufw reload

Worker

ufw allow 22/tcp
ufw allow 10250/tcp
ufw allow 30000:32767/tcp
ufw reload

Containerd

cat << EOF >> /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF

modprobe overlay
modprobe br_netfilter

cat << EOF >> /etc/sysctl.d/99-kubernetes-k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

sysctl --system

Containerd

apt update
apt -y install containerd

Adapt Containerd to Kubernetes

containerd config default > /etc/containerd/config.toml >/dev/null 2>&1

Update config.toml

sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml

Restart Containerd

systemctl enable containerd
systemctl restart containerd

add Kubernetes

apt install gnupg gnupg2 curl software-properties-common -y
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg |gpg --dearmour -o /etc/apt/trusted.gpg.d/cgoogle.gpg
apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"

Kubectl

apt update
apt install kubelet kubeadm kubectl -y
apt-mark hold kubelet kubeadm kubectl

Kube Init on MASTER

kubeadm init --control-plane-endpoint=k8s-master

Downgrade to 1.26

Version 1.27 seems not production ready, so, you may have to downgrade it :(

Show Contexts

kubectl config get-contexts
CURRENT   NAME                           CLUSTER                        AUTHINFO                             NAMESPACE
*         do-fra1-k8s-1-24-xxxxxxxxxxx   do-fra1-k8s-1-24-xxxxxxxxxxx   do-fra1-k8s-1-24-xxxxxxxxxxxxxxxxx
          do-fra1-k8s-1-25-xxxxxxxxxxx   do-fra1-k8s-1-25-xxxxxxxxxxx   do-fra1-k8s-1-25-xxxxxxxxxxxxxxxxx
          do-fra1-k8s-xxxxx              do-fra1-k8s-xxxxx              do-fra1-k8s-xxxxxxxxxxx

Delete old Contexts

kubectl config delete-context do-fra1-k8s-xxxxx

sha256: f12aa09936d175b3e23f2eebc1c152675d68dce6096b9ec02d7f8429b954b4fe

Let’s give a Try with Kubernetes Mini, K3s

Source

• https://k3s.io/
• https://docs.k3s.io/

Overview

TestSetup

Booting a few VM’s on my ESX Host. All of them got 4 CPU’s, 32 GB RAM, 100G Disk

1 MasterNode

• Master

3 WorkerNodes

• Worker01
• Worker02
• Worker03

All Maschines are Running Debian Latest, that’s Version 11.6 at the Moment

Setup Master

curl -sfL https://get.k3s.io | sh -

root@master:~/bin/test_kubernetes# curl -sfL https://get.k3s.io | sh -
[INFO]  Finding release for channel stable
[INFO]  Using v1.25.4+k3s1 as release
[INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.25.4+k3s1/sha256sum-amd64.txt
[INFO]  Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.25.4+k3s1/k3s
[INFO]  Verifying binary download
[INFO]  Installing k3s to /usr/local/bin/k3s
[INFO]  Skipping installation of SELinux RPM
[INFO]  Creating /usr/local/bin/kubectl symlink to k3s
[INFO]  Creating /usr/local/bin/crictl symlink to k3s
[INFO]  Skipping /usr/local/bin/ctr symlink to k3s, command exists in PATH at /usr/bin/ctr
[INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
[INFO]  systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO]  systemd: Starting k3s

Show Nodes

kubectl get nodes

root@master:~/bin/test_kubernetes# kubectl get nodes
NAME     STATUS   ROLES                  AGE   VERSION
master   Ready    control-plane,master   70s   v1.25.4+k3s1

Show Token

cat /var/lib/rancher/k3s/server/node-token 
K10032f55153f52072a1e41f80f06551078dece476a44217e5a06facdfa6fd0f985::server:a70b4452634b7d2c4f9d33ab8808eb19

on all Worker Nodes

update hosts as root

I’ll give a try running an Application on Google Cloud. Not with great sucess :(

Source

• https://codelabs.developers.google.com/codelabs/cloud-run-django?hl=en#0
• https://codelabs.developers.google.com/codelabs/cloud-run-hello-python3#1

Get Cloud List, Active Account

gcloud auth list

gcloud config set account '[email protected]'

List Projects

gcloud config list project

[core]
project = cloud-run-372113

Your active configuration is: [cloudshell-6045]
username@cloudshell:~ (cloud-run-372113)$

Set Project ID

gcloud config set project cloud-run-372113

username@cloudshell:~ (cloud-run-372113)$ gcloud config set project cloud-run-372113
Updated property [core/project].

Enable API

gcloud services enable \
  artifactregistry.googleapis.com \
  cloudbuild.googleapis.com \
  run.googleapis.com

username@cloudshell:~ (cloud-run-372113)$ gcloud services enable \
  artifactregistry.googleapis.com \
  cloudbuild.googleapis.com \
  run.googleapis.com
ERROR: (gcloud.services.enable) User [[email protected]] does not have permission to access projects instance [cloud-run-372113] (or it may not exist): Project 'cloud-run-372113' not found or permission denied.
Help Token: AXyIxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
- '@type': type.googleapis.com/google.rpc.PreconditionFailure
  violations:
  - subject: ?error_code=210002&type=Project&resource_id=cloud-run-372113
    type: googleapis.com
- '@type': type.googleapis.com/google.rpc.ErrorInfo
  domain: serviceusage.googleapis.com
  metadata:
    resource_id: cloud-run-372113
    type: Project
  reason: RESOURCES_NOT_FOUND
username@cloudshell:~ (cloud-run-372113)$

sha256: bbf813a31c1281e47acfdca36d3b92b75b87039c93d4f295cd9a5825953e8628

If you ever wanna run a Multiplayer OWASP Juice Shop CTF on your own, here are some Notes and Info for bloody beginners

References

• https://github.com/iteratec/multi-juicer/
• https://www.digitalocean.com/
• https://kubernetes.io/de/docs/concepts/overview/what-is-kubernetes/

Prerequisite

you’ve got

• a Digital Ocean Account (or some other Cloud Provider)
• a spare domain and set the NS of DigitalOcean
• Digital Ocean CMD Line Tools installed and configured
• helm tools (kubernetes package manager -> brew install helm)
• some budget (~2 CHF/Day)
• 30min for Setup

btw. all this commands should run on macOS. linux may needs some adjustments …