OpenBSD - Nornir

Build Nornir on OpenBSD - Failed :(

Any Idea how to build it … ?

(cisco-nornir-py3.11) puffy$ RUST_BACKTRACE=1 poetry add nornir-netmiko 

...
Compiling asn1 v0.13.0
       Running `rustc --crate-name asn1 --edition=2018 /home/stoege/.cargo/registry/src/github.com-1ecc6299db9ec823/asn1-0.13.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C linker-plugin-lto -C overflow-checks=on -C metadata=e01d853059c772ba -C extra-filename=-e01d853059c772ba --out-dir /tmp/tmpyhzdyoqa/cryptography-40.0.2/src/rust/target/release/deps -L dependency=/tmp/tmpyhzdyoqa/cryptography-40.0.2/src/rust/target/release/deps --extern asn1_derive=/tmp/tmpyhzdyoqa/cryptography-40.0.2/src/rust/target/release/deps/libasn1_derive-dbeffd4d85568b7f.so --extern chrono=/tmp/tmpyhzdyoqa/cryptography-40.0.2/src/rust/target/release/deps/libchrono-994a1164a0b8cdf4.rmeta --cap-lints allow`
       Running `rustc --crate-name pyo3 --edition=2018 /home/stoege/.cargo/registry/src/github.com-1ecc6299db9ec823/pyo3-0.15.2/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C linker-plugin-lto -C overflow-checks=on --cfg 'feature="abi3"' --cfg 'feature="abi3-py310"' --cfg 'feature="abi3-py36"' --cfg 'feature="abi3-py37"' --cfg 'feature="abi3-py38"' --cfg 'feature="abi3-py39"' --cfg 'feature="default"' --cfg 'feature="extension-module"' --cfg 'feature="indoc"' --cfg 'feature="macros"' --cfg 'feature="paste"' --cfg 'feature="pyo3-macros"' --cfg 'feature="unindent"' -C metadata=001198fe1619b05c -C extra-filename=-001198fe1619b05c --out-dir /tmp/tmpyhzdyoqa/cryptography-40.0.2/src/rust/target/release/deps -L dependency=/tmp/tmpyhzdyoqa/cryptography-40.0.2/src/rust/target/release/deps --extern cfg_if=/tmp/tmpyhzdyoqa/cryptography-40.0.2/src/rust/target/release/deps/libcfg_if-4852c268cf497cfc.rmeta --extern indoc=/tmp/tmpyhzdyoqa/cryptography-40.0.2/src/rust/target/release/deps/libindoc-8cef8f8418f89ebd.rmeta --extern libc=/tmp/tmpyhzdyoqa/cryptography-40.0.2/src/rust/target/release/deps/liblibc-e2bfee5c7507c149.rmeta --extern parking_lot=/tmp/tmpyhzdyoqa/cryptography-40.0.2/src/rust/target/release/deps/libparking_lot-dbbcb49cdfedf28d.rmeta --extern paste=/tmp/tmpyhzdyoqa/cryptography-40.0.2/src/rust/target/release/deps/libpaste-b236df9534aa35a4.rmeta --extern pyo3_macros=/tmp/tmpyhzdyoqa/cryptography-40.0.2/src/rust/target/release/deps/libpyo3_macros-1c568e083598ae83.so --extern unindent=/tmp/tmpyhzdyoqa/cryptography-40.0.2/src/rust/target/release/deps/libunindent-321bc6f9dc7b8458.rmeta --cap-lints allow --cfg Py_3_6 --cfg Py_LIMITED_API --cfg 'py_sys_config="WITH_THREAD"' --cfg track_caller --cfg min_const_generics --cfg addr_of`
       Running `/tmp/tmpyhzdyoqa/cryptography-40.0.2/src/rust/target/release/build/cryptography-rust-2d57360915c33af1/build-script-build`
  error: failed to run custom build command for `cryptography-rust v0.1.0 (/tmp/tmpyhzdyoqa/cryptography-40.0.2/src/rust)`
  
  Caused by:
    process didn't exit successfully: `/tmp/tmpyhzdyoqa/cryptography-40.0.2/src/rust/target/release/build/cryptography-rust-2d57360915c33af1/build-script-build` (exit status: 101)
    --- stdout
    cargo:rerun-if-changed=../_cffi_src/
    cargo:rustc-cfg=python_implementation="CPython"
  
    --- stderr
    thread 'main' panicked at 'unable to find openssl include path', build.rs:52:49
    stack backtrace:
       0: rust_begin_unwind
       1: core::panicking::panic_fmt
       2: core::panicking::panic_display
       3: core::panicking::panic_str
       4: core::option::expect_failed
       5: core::option::Option<T>::expect
       6: build_script_build::main
       7: core::ops::function::FnOnce::call_once
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
  warning: build failed, waiting for other jobs to finish...
  
      =============================DEBUG ASSISTANCE=============================
      If you are seeing a compilation error please try the following steps to
      successfully install cryptography:
      1) Upgrade to the latest pip and try again. This will fix errors for most
         users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip
      2) Read https://cryptography.io/en/latest/installation/ for specific
         instructions for your platform.
      3) Check our frequently asked questions for more information:
         https://cryptography.io/en/latest/faq/
      4) Ensure you have a recent Rust toolchain installed:
         https://cryptography.io/en/latest/installation/#rust
      
      Python: 3.11.3
      platform: OpenBSD-7.3-amd64-64bit-ELF
      pip: n/a
      setuptools: 67.7.2
      setuptools_rust: 1.6.0
      rustc: 1.68.0 (2c8cc3432 2023-03-06) (built from a source tarball)
      =============================DEBUG ASSISTANCE=============================
      
  error: `cargo rustc --lib --message-format=json-render-diagnostics --manifest-path src/rust/Cargo.toml --release -v --features 'pyo3/abi3-py36 pyo3/extension-module' --crate-type cdylib --` failed with code 101
  

  at ~/.local/lib/python3.10/site-packages/poetry/installation/chef.py:152 in _prepare
      148│ 
      149│                 error = ChefBuildError("\n\n".join(message_parts))
      150│ 
      151│             if error is not None:
    → 152│                 raise error from None
      153│ 
      154│             return path
      155│ 
      156│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with cryptography (40.0.2) not supporting PEP 517 builds. You can verify this by running 'pip wheel --use-pep517 "cryptography (==40.0.2)"'.

Fix -> Downgrade urllib

Downgrade urllib from 2.0.2 to 1.26.6 seems working. Discussion on Github

iperf3 - rdomain

how do i run a iperf3 server in a rdomain ? if you don’t know what a rdomain is, just have a look at the man pages

show interface with rdomain

-> rdomain 1 show that this Interface is not running in the Default RDomain 0

ifconfig em3 
em3: flags=808843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,AUTOCONF4> rdomain 1 mtu 1500
	lladdr de:ad:be:ef:be:ef
	description: connected to Switch (Gi0/1)
	index 4 priority 0 llprio 3
	media: Ethernet autoselect (1000baseT full-duplex)
	status: active
	inet 192.168.1.100 netmask 0xffffff00 broadcast 192.168.1.255

Install iperf3

install Package from Repository

OpenBSD - ansible-pylibssh

wanna build ansible-pylibssh on OpenBSD 7.3 ?

Build failed ?

$ pip install ansible-pylibssh
Defaulting to user installation because normal site-packages is not writeable
Collecting ansible-pylibssh
  Using cached ansible-pylibssh-1.1.0.tar.gz (106 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: ansible-pylibssh
  Building wheel for ansible-pylibssh (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for ansible-pylibssh (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      [1/1] Cythonizing /tmp/pip-install-3inpi4en/ansible-pylibssh_04d3883cfd7d49ecb34d03dc90702e66/src/pylibsshext/_libssh_version.pyx
      /tmp/pip-install-3inpi4en/ansible-pylibssh_04d3883cfd7d49ecb34d03dc90702e66/src/pylibsshext/_libssh_version.c:757:10: fatal error: 'libssh/libssh.h' file not found
      #include "libssh/libssh.h"
               ^~~~~~~~~~~~~~~~~
      1 error generated.
      error: command '/usr/bin/cc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for ansible-pylibssh
Failed to build ansible-pylibssh
ERROR: Could not build wheels for ansible-pylibssh, which is required to install pyproject.toml-based projects

Build sucessfully !

$ CFLAGS=-I/usr/local/include pip install ansible-pylibssh
Defaulting to user installation because normal site-packages is not writeable
Collecting ansible-pylibssh
  Downloading ansible-pylibssh-1.1.0.tar.gz (106 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 106.2/106.2 kB 687.2 kB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: ansible-pylibssh
  Building wheel for ansible-pylibssh (pyproject.toml) ... done
  Created wheel for ansible-pylibssh: filename=ansible_pylibssh-1.1.0-cp310-cp310-openbsd_7_3_amd64.whl size=651397 sha256=76e37e982c0902f3a846347577ab5dc371ef97ccbea145eb9c2820fad1501d9d
  Stored in directory: /home/stoege/.cache/pip/wheels/60/74/37/a1ba5b525d4f4e67e90e1ae862e66eeedc08147fb09d82b5d8
Successfully built ansible-pylibssh
Installing collected packages: ansible-pylibssh
Successfully installed ansible-pylibssh-1.1.0

Any Comments ?

sha256: 162600846baa1e3d7e35801c67e5c7737aaf2a5dd8cc09952111c626bf942e4c

Multiprocessing

Parallel Processing

i recently read an article about parallel processing. i remembered my domain checker service which checks a lot of domains for their availablitly, and this script runs sequentiel and needs around 30 seconds.

initially i worked on a caching mechanism to speed up results. but if a service is not used that often (nobody is useing my domain checker…), there is not much you can gain with caching.

so, i gave a try with Multiprocessing and i have to admit i’m prettyhappy with the result !

SOA Checker

Intro

this is a little script which reads the Name Servers for a given Domain, and then asks the NameServer for the SOA of this Domain.

Script

cat << 'EOF' > soachecker.sh
#!/usr/bin/env bash

# Little SOA & Serial Checker, v0.1, @stoege

tmpfile=$(mktemp)

# Check Args
echo
if [ $# -eq 0 ]; then
  d="stoege.net"
  echo -e "No argument provided. use \033[1m'${d}'\033[0m"
elif [ $# -eq 1 ]; then
  d="$1"
  echo -e "Domain \033[1m'${d}'\033[0m provided"
else
  echo -e "\033[1mmore than one arguments provided. Exit 1.\033[0m"
  exit 1
fi

# Build File
for i in $(dig +short NS ${d} |tr '\n' ' '); do
  echo -e "\ndig +short SOA \033[1m@${i}\033[0m ${d}"
  dig +short SOA @${i} ${d} |tee -a ${tmpfile}
done

# uniq & count
echo
cat ${tmpfile} |sort |uniq -c |awk '{ printf "%d x Serial: %s\n", $1,$4 }'

# cleanup
rm ${tmpfile}

echo
exit 0
EOF
chmod u+x soachecker.sh

Run it

$ ./soachecker.sh stoege.net

Domain 'stoege.net' provided

echo dig +short SOA @ns3.noflow.ch. stoege.net
ns1.noflow.ch. hostmaster.noflow.ch. 2023050124 3600 900 1209600 1800

echo dig +short SOA @ns1.noflow.ch. stoege.net
ns1.noflow.ch. hostmaster.noflow.ch. 2023050124 3600 900 1209600 1800

echo dig +short SOA @ns2.nolink.ch. stoege.net
ns1.noflow.ch. hostmaster.noflow.ch. 2023050124 3600 900 1209600 1800

3 x Serial: 2023050124

let me know if you like this !

acme.sh

Certificate Management with ‘acme.sh’

I like to manage my certificates on my own. If you work with Wildcard Certs, acme.sh is a nice and flexible ACME Client, purely written in Shell.

It’s probably the easiest & smartest shell script to automatically issue & renew the free certificates.

Basic Handling

Get Version

acme.sh --version

run it

# acme.sh --version
https://github.com/acmesh-official/acme.sh
v3.0.6

Upgrade Self

are we up2date ?

acme.sh --upgrade

run it

# acme.sh --upgrade
[Mon May  1 11:35:55 CEST 2023] Already uptodate!
[Mon May  1 11:35:55 CEST 2023] Upgrade success!

Info

General Info about the Setup

Python - Redirector

Redirector App

wrote a little redirector app and tought i will explain and share it here. it’s a bit like a url shortener, but you can define the “shortcut” of the URL.

how does it work

it basically consists of a Text File wir Redirection URL’s.

redi.txt

stoege,https://www.stoege.net
blog,https://blog.stoege.net
test,https://www.test.com

Call it

so, when you open a Browser and Request the URL: https://your.domain.de/blog, you get redirected to https://blog.stoege.net

main.app

from flask import Flask, redirect, request
import datetime
import os
import random

# Vars
redirect_file="redi.txt"

app = Flask(__name__)

# Load redirection data from a text file
def get_redirections():
    redirections = {}
    with open(redirect_file,"r") as file:
        for line in file:
            path, url = line.strip().split(',')
            redirections[path] = url
    return redirections

# Main
@app.route('/')
def index():
    return 'Hello, World!'

# Redirect to Random URL
@app.route('/random')
def random_path():
    redirections = get_redirections()

    # Get Random Path
    random_url = random.choice(list(redirections.values()))
    return redirect(random_url)

# Redirect
@app.route('/<path:path>')
def redirect_path(path):

    # File Changed ?
    redirections = get_redirections()

    # Check if the path exists in the redirections dictionary
    if path in redirections:

        url = redirections[path]
        return redirect(url)

    # If the path does not exist, return a 404 Not Found error
    return 'Not Found', 404

if __name__ == '__main__':
    app.run()

get it running

you need the ususal Stuff

HTMX & Nginx

Little Test with HTMX & Nginx

recently, i saw the Keynote - “Full-Stack Python” (Andy “Pandy” Knight) and i read an article about html & websockets. So I thought why not give it a try?

Preview

Requirements

the usual stuff:

  • Virtual Machine (here: OpenBSD VM)
  • FQDN Pointing to your Box
  • SSL Cert

Webroot

on your webserver, create a new webroot wherever you have your pages located.

su - webmaster
mkdir -p /var/www/virtual/your.page.de
cd /var/www/virtual/your.page.de/

main.py

the main part of the python code …

OpenBSD - Smokeping

How to Install Smokeping on OpenBSD

Requiremens

  • running Server with OpenBSD
  • Root Permission
  • FQDN with Cert

Install Software

pkg_add smokeping

Update Config

cat << 'EOF' >/etc/smokeping/config
*** General ***

owner    = YOUR NAME
contact  = [email protected]
mailhost = localhost
sendmail = /usr/sbin/sendmail
# NOTE: do not put the Image Cache below cgi-bin
# since all files under cgi-bin will be executed ... this is not
# good for images.
imgcache = /var/www/htdocs/smokeping/cache
imgurl   = cache
datadir  = /var/db/smokeping
piddir  = /var/run
cgiurl   = https://YOUR.URL.NET/smokeping/smokeping.cgi
smokemail = /etc/smokeping/smokemail
tmail = /etc/smokeping/tmail
# specify this to get syslog logging
syslogfacility = local0
# each probe is now run in its own process
# disable this to revert to the old behaviour
# concurrentprobes = no

*** Alerts ***
to = [email protected]
from = smokealert@localhost

+someloss
type = loss
# in percent
pattern = >0%,*12*,>0%,*12*,>0%
comment = loss 3 times  in a row

*** Database ***

step     = 300
pings    = 20

# consfn mrhb steps total

AVERAGE  0.5   1  28800
AVERAGE  0.5  12   9600
    MIN  0.5  12   9600
    MAX  0.5  12   9600
AVERAGE  0.5 144   2400
    MAX  0.5 144   2400
    MIN  0.5 144   2400

*** Presentation ***

template = /etc/smokeping/basepage.html

htmltitle = yes
graphborders = no

+ charts

menu = Charts
title = The most interesting destinations

++ stddev
sorter = StdDev(entries=>4)
title = Top Standard Deviation
menu = Std Deviation
format = Standard Deviation %f

++ max
sorter = Max(entries=>5)
title = Top Max Roundtrip Time
menu = by Max
format = Max Roundtrip Time %f seconds

++ loss
sorter = Loss(entries=>5)
title = Top Packet Loss
menu = Loss
format = Packets Lost %f

++ median
sorter = Median(entries=>5)
title = Top Median Roundtrip Time
menu = by Median
format = Median RTT %f seconds

+ overview 

width = 600
height = 50
range = 10h

+ detail

width = 600
height = 200
unison_tolerance = 2

"Last 3 Hours"    3h
"Last 30 Hours"   30h
"Last 10 Days"    10d
"Last 360 Days"   360d

#+ hierarchies
#++ owner
#title = Host Owner
#++ location
#title = Location

*** Probes ***

+ FPing

binary = /usr/local/sbin/fping

*** Targets ***

probe = FPing

menu = Top
title = Network Latency Grapher
remark = Welcome to the SmokePing

+ Remote
menu= Remote
title= Remote hosts

+ DNS
menu = DNS
title = DNS

++ cf
menu = 1.1.1.1
title = 1.1.1.1
host = 1.1.1.1

++ google
menu = 8.8.8.8
title = 8.8.8.8
host = 8.8.8.8

++ quad9
menu = 9.9.9.9
title = 9.9.9.9
host = 9.9.9.9
EOF

Create Folders

don’t like to set permission to 777. but it does not work without :(

Anycast IPv6 - YourSelf

How to Build your own AnyCast Network

you’re using anycast every day. all public resolvers (1.1.1.1, 8.8.8.8, 9.9.9.9) for example are anycast ip’s and hence, many servers distributed around the world which listen and announce the same ip address.

we can build a proof of concept, how to build such a network, for a few $.

Requirements

AS Number

Anycast heavily depends on BGP. So, you need a own AS Number. I got my for 15 CHF/Year from Securebit.ch. You should also keep in mind that you will not get any IPv4 addresses without buying them for the market. So, your Setup will be based on IPv6 only.