Linux

Debian - Cloudimage

Debian Template on Proxmox

Cloud Image with SSH Key

on the Proxmox Host

Create Installer Key

# cd /your/working/dir
ssh-keygen -o -a 100 -t ed25519 -C "User: installer, $(date '+%Y-%m-%d %H:%m')" -f installer

Build Template

apt install libguestfs-tools -y;
wget https://cloud.debian.org/images/cloud/bookworm/latest/debian-12-generic-amd64.qcow2

virt-customize --install qemu-guest-agent -a debian-12-generic-amd64.qcow2
qm create 9001 --name debian-12-generic --memory 2048 --cores 2 --net0 virtio,bridge=vmbr0 --agent 1 
qm importdisk 9001 debian-12-generic-amd64.qcow2 local-lvm

qm set 9001 --scsihw virtio-scsi-pci --scsi0 local-lvm:vm-9001-disk-0
qm set 9001 --ide2 local-lvm:cloudinit
qm set 9001 --boot c --bootdisk scsi0
qm set 9001 --serial0 socket
qm set 9001 --ipconfig0 ip=dhcp
qm set 9001 --cpu cputype=x86-64-v2-AES

qm set 9001 --sshkeys installer.pub
qm set 9001 --ciuser installer

qm resize 9001 scsi0 32G
qm template 9001

Create VM

in the GUI, you can create an new VM based on the Template 9001. You have to login with user “installer” and it’s private key!

Borgbackup

Prerequisite

  • you need a remote Borg Server (Unix/Linux Machine with Borg installed)
  • valid User and Key for SCP Transfer
  • SSH Key -> /backup/id_ed25519

Create Local Folder

test -d /backup || (mkdir /backup; chmod 700 /backup)

Borg Backup Script

cat << 'EOF2' > /backup/borg.sh
#!/usr/bin/env bash

# BorgBackup Script, v1.0, 2024-04-09, by @stoege

# Remote server details
REMOTE_USER="borguser"
REMOTE_HOST="your.remote.borg.server"
REMOTE_REPO="mysamplerepo"

# Local directory to backup
LOCAL_DIR="/"

# List of directories to exclude
EXCLUDE_DIRS=(
    "*/.cache/"
    "/tmp"
    "/restore"
)

# Set PassPhrase for the Backup Encryption (run: pwgen 32 1)
export BORG_PASSPHRASE="your-super-strong-password"
export BORG_RSH='ssh -i /backup/id_ed25519'


# Function to perform full backup
perform_full_backup() {

    # Construct exclude options
    exclude_opts=""
    for dir in "${EXCLUDE_DIRS[@]}"; do
        exclude_opts+="--exclude $dir "
    done

    # Run BorgBackup command
    borg create \
        --verbose \
        --stats \
        --progress \
        --compression lz4 \
        $REMOTE_USER@$REMOTE_HOST:$REMOTE_REPO::'{hostname}-{now:%Y-%m-%d_%H:%M:%S}' \
        $LOCAL_DIR \
        $exclude_opts \
    || borg init -e repokey-blake2 $REMOTE_USER@$REMOTE_HOST:$REMOTE_REPO
}

# Function to perform restore
perform_restore() {

    # Create Restore
    test -d /restore || mkdir /restore

    # Change Dir
    cd /restore
 
    # Run BorgBackup command to restore specific directory
    borg extract \
        --verbose \
        --progress \
        $REMOTE_USER@$REMOTE_HOST:$REMOTE_REPO::$1 \
        $2
}

# Function to list all backups
list_backups() {

    # Run BorgBackup command to list all archives
    borg list $REMOTE_USER@$REMOTE_HOST:$REMOTE_REPO

}


# Function to rotate backups
rotate_backups() {
    # Run BorgBackup command to prune archives based on retention policy
    borg prune \
        --verbose \
        --list \
        --glob-archives "${hostname}*" \
        --keep-hourly=2 \
        --keep-daily=2 \
        --keep-weekly=2 \
        --keep-monthly=2 \
        $REMOTE_USER@$REMOTE_HOST:$REMOTE_REPO
}

# Install on Host
install_myself() {

  # Folder
  f="/backup"

  # Create Directory, copy File
  test -d ${f} || (mkdir ${f}; chmod 700 ${f})
  cp $0 ${f}/
  
  # Inform User
  cat << EOF

# to install in Crontab:
crontab -e
5 6,12,18 * * * cd ${f}; $0 backup >> /var/log/borgbackup.log 2>&1

EOF

}

# Help
show_help() {
  echo "$0 [ backup | list | rotate | install | restore BACKUPNAME /etc ]"
  exit 1
}


# Main script
echo "Starting BorgBackup..."

# Check if BorgBackup is installed
if ! command -v borg &> /dev/null; then
    echo "Error: BorgBackup is not installed. Please install BorgBackup."
    exit 1
fi

# Check if parameter is provided
if [ $# -eq 0 ]; then
  show_help
fi

# Perform action based on parameter
case "$1" in
    "backup")
        echo "Performing full backup..."
        perform_full_backup
        rotate_backups
        ;;
    "restore")
        if [ $# -lt 3 ]; then
            echo "Error: Please specify a Backup Set and directory to restore."
            echo "$0 BACKUP_SET /FOLDER/TO/RESTORE"
            exit 1
        fi
        echo "Performing restore for directory '$3' on set '$2'"
        perform_restore $2 $3
        ;;
    "list")
        echo "Listing all backups..."
        list_backups
        ;;
    "rotate")
        echo "Rotating backups..."
        rotate_backups
        ;;
    "install")
        echo "Install Scripts"
        install_myself
        ;;
    *)
        show_help
        ;;
esac

# Check backup status
if [ $? -eq 0 ]; then
    echo "Action completed successfully."
else
    echo "Action failed."
fi

# Finally done
exit 0

EOF2
chmod 700 /backup/borg.sh

Execute It

Create Backup

/backup/borg.sh backup

List Backup

/backup/borg.sh list

Restore Folder

/backup/borg.sh restore hostname-date /etc"

Any Comments ?

sha256: 3adc039f17d2b87ef48b8e9d200c53675b430603c048d4879aacb2dabb3ce37f

Debian - MinIO

Minio on Debian

Need some S3 Storage for Reasons ? Here a few Lines, how to Setup and enable TLS.

Install Minio

login as root for the whole installation. Or use sudo/doas if preferred.

Upgrade you Box

apt update && apt upgrade -y

reboot if needed

add User

Let’s add User as we don’t wanna run it as root

useradd -r minio-user -s /sbin/nologin

Get Minio

Download, set execute permission and move it

Debian behind TLS Proxy

Behind Corp Proxy

let’s assume you’re behing a Corp Proxy which enforce TLS Inspection, you don’t have the Proxy Cert and you want to Upgrade your Boxes …

… and of course, you do this in the LAB and for Research only and not your Productiv Environment!

TLS Inspection enabled

apt-get upate
W: Failed to fetch https://packages.sury.org/php/dists/bookworm/InRelease  Certificate verification failed: The certificate is NOT trusted.
The certificate issuer is unknown.  Could not handshake: Error in the certificate verification. [IP: xx.xx.xx.xx yyyy]

Disable TLS Check

touch /etc/apt/apt.conf.d/99verify-peer.conf
echo >>/etc/apt/apt.conf.d/99verify-peer.conf "Acquire { https::Verify-Peer false }"

Update

apt-get update
apt-get upgrade

Any Comments ?

sha256: 40c39ed441b4690a8644cd63bfd2e6987f06a70c4f922eca14de5dcc27d4fb35

Debian 12

Debian 12 is here !

you may find some of my Posts about Debian useful

Upgrade Script

assuming you have Debian 11.x running

cat << 'EOF' > /root/upgrade_to_v12.sh
#!/usr/bin/env bash

# set Version
sed -i 's/bullseye/bookworm/g' /etc/apt/sources.list
sed -i 's/bullseye/bookworm/g' /etc/apt/sources.list.d/*

# Update
apt-get -y update
apt-get -y upgrade
apt-get -y dist-upgrade
apt autoclean
apt autoremove
apt --purge autoremove

# you may add/update -> /etc/apt/sources.list
# deb http://security.debian.org/debian-security/ bookworm-security main
# deb-src http://security.debian.org/debian-security/ bookworm-security main

exit 0
EOF

chmod u+x /root/upgrade_to_v12.sh

Run

run the script and wait a few minutes …

Docker - Swarm

Setup

Let’s assume we have 3 Nodes in a Docker Swarm Setup.

  • Tick
  • Trick
  • Track

Track is the Leader/Manager. All Machines runs Debian 12.0 (RC, as it is not yet released)

Show Nodes

docker node ls
root@track:~# docker node ls
ID                            HOSTNAME    STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
b2zvkc374v08q2rpocljhkg5n     tick        Ready     Active                          23.0.2
khom3cx05bxqxktjq1k5j16gk *   track       Ready     Active         Leader           23.0.2
lq53s6mhuzvqrehks0j68sr3e     trick       Ready     Active                          23.0.2

Create Simple Service

Run a simple Docker service that uses an alpine-based filesystem, and isolates a ping to 8.8.8.8

Wireguard on Debian

Wireguard with Debian

Grab a Fresh Debian which has Public Internet Access. Target is to build a WG Tunnel and assign a Public IP to the Server.

Debian 11.6

apt-get install -y wireguard wireguard-tools

Gen Key

cd /etc/wireguard
umask 077; wg genkey | tee privatekey | wg pubkey > publickey

Set Vars

myprivkey=$(cat privatekey)
mypublicaddress="45.xx.xx.xx/28, 2a0e:xxxx:xxx::xxx/64"
yourpubkey="3XK8xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx="
yourpubip="45.xxx.xxx.xxx"
yourpubport="443"

Config

cat << EOF > wg0.conf
[Interface]
PrivateKey = ${myprivkey}
Address    = PUBLIC_IP_V4/xx, PUBLIC_IP_V6/xx
 
[Peer]
PublicKey  = ${yourpubkey}
Endpoint   = ${yourpubip}:${yourpubport}
AllowedIPs = 0.0.0.0/0, ::/0
PersistentKeepalive = 60
EOF

Tunnel UP

systemctl enable wg-quick@wg0
systemctl start  wg-quick@wg0
systemctl status wg-quick@wg0

Check IP

i3

Any Comments ?

sha256: c4d87bfca434aca32d6a8869720220b6ea4abe1ff534dd6e715cbb73d4f7025e

Docker - Container

Stuff for the running Containers

List running Containers

$ docker ps
CONTAINER ID  IMAGE   COMMAND                  CREATED          STATUS          PORTS                    NAMES
f99ad3355bae  blog    "/home/docker/init_a…"   14 minutes ago   Up 14 minutes   0.0.0.0:3031->3031/tcp   quizzical_bardeen

Shell into Containter

docker exec -it f99ad3355bae bash

first build cache

apt-get update

install Tools

  • netstat
  • ps
  • tcpdump
apt-get install -y net-tools procps tcpdump telnet netcat

Any Comments ?

sha256: 27a1368fbcb11db26404131aeb2b0e15d07bc32f61df6389a7c685df61bfc5aa

Alpine - Pandas on Docker Image

How to install Pandas on Alpine Linux

Run Alpine Container

docker run -it alpine

add packages

apk update
apk add python3 py3-pip gcc python3-dev g++

add / build pandas

time pip install pandas
real 26m 13.14s
user 30m 46.40s
sys   3m 27.51s

Happy Pandas !


Any Comments ?

sha256: afb99c7e3ed003bee48b65795a153c4fe7835fe3dae0759b70ab2bfb5adc4fd5

Docker - Traefik - Wildcard Subdomain

Intro

I was wondering if you can have Wildcart Certs for certain Subdomain. Idea is to provide a Service with “myservice.auth.your.domain” which automatically requests Authentication, while the same Service “myservice.whitelist.your.domain” is reachable through some Whitelisted IP only.

As Traefik can Chain Middleware, but not implements some logic (If Whitelist -> ok, else do Basic Auth …), i have to build another solution.

let’s have a look

Prepare Folders

cd /your/traffic/rootfolder
mkdir -p config/dynamic

.env File

we need two variables, so let’s put them in the .env File