Python

Let’s play with Packages and Libraries

References

Switch to Root Folder

cd /some/path/you/want

Create a new Package

poetry new mypackage

add some libraries

poetry add requests

… add some code …

cat << 'EOF' > mypackage/__init__.py
print("importing", __name__)
EOF

cat << 'EOF' > mypackage/main.py
print("importing", __name__)

def test1():
  print("test1")

def test2(name: str):
  print("hello", name)

def test3(name: str, age:int):
  print(f"Hello {name} at age {age}")

if __name__ == "__main__":
  print("This is a Library or Package. You should import it into your Code and not run it directly ...")
EOF

Build Package

poetry build

List Tree

(mypackage-py3.11) stoege@host:~/git/demo/mypackage> tree
.
├── README.md
├── dist
│   ├── mypackage-0.1.0-py3-none-any.whl
│   └── mypackage-0.1.0.tar.gz
├── mypackage
│   ├── __init__.py
│   └── main.py
├── poetry.lock
├── pyproject.toml
└── tests
    └── __init__.py

4 directories, 8 files

you have a package called “mypackage-0.1.0” created. As ’tar.gz and ‘.whl’ File

Storing Data in JSON - TinyDB

Small Example how to Store Data in JSON, and Query them afterwards like a NOSQL DB. Have a look at TinyDB if you wanna see more.

Code

from tinydb import TinyDB, Query
from pprint import pprint

# Create or load a database file
db = TinyDB('db.json')

# insert some sample data
def insert():
    # Insert data
    db.insert({'name': 'John', 'age': 30})
    db.insert({'name': 'Alice', 'age': 25, 'hobbies': 'sleep'})
    db.insert({'name': 'Max', 'age': 20, 'hobbies': ['sleep', 'play', 'eat']})

# show all entries
def show_all():
    all_records = db.all()
    pprint(all_records)

# entries with hobbies
def show_entries_with_hobbies():
    User = Query()
    result = db.search(User.hobbies.exists())
    pprint(result)

# entries without hobbies
def show_entries_without_hobbies():
    User = Query()
    result = db.search(~User.hobbies.exists())
    pprint(result)

# show entries with hobbies and older than 22 years
def show_entries_with_hobbies_and_older_than_22():
    User = Query()
    result =  db.search((User.hobbies.exists()) & (User.age > 22))
    pprint(result)

if __name__ == "__main__":
  # Add
  insert()

  # show
  print("\n-- ALL --")
  show_all()

  print("\n-- with Hobbies --")
  show_entries_with_hobbies()

  print("\n-- without Hobbies --")
  show_entries_without_hobbies()


  print("\n-- with Hobbies and older than 22 --")
  show_entries_with_hobbies_and_older_than_22()

Run

you need to install tinydb. use a virtual env like .venv, poetry or whatever you like

Project Template for FastAPI

gave a try with a FastAPI Template, https://github.com/rochacbruno/fastapi-project-template.git

Projectname: gugus1234

clone the repo

git clone https://github.com/rochacbruno/fastapi-project-template.git gugus1234
cd gugus1234

Switch Poetry

i’d like to have poetry as virtual env manager

make switch-to-poetry

Rename some Stuff

had to rename some string in pyproject and different files …

mv project_name gugug1234
gsed -i 's/a-flask-test/gugus1234/' pyproject.toml
gsed -i 's/project_name/gugus1234/' pyproject.toml
gsed -i 's/project_name/gugus1234/g' gugus1234/cli.py gugus1234/app.py gugus1234/config.py gugus1234/security.py

Run Poetry once

poetry shell
poetry lock
poetry install

Admin User

let’s create admin user

How to Protect your App with Simple Security

Let’s build a small API Endpoint with FastAPI and protect it with SimpleSecurity.

API key based security package for FastAPI, focused on simplicity of use:

Full functionality out of the box, no configuration required
API key security with local sqlite backend, working with both header and query parameters
Default 15 days deprecation for generated API keys
Key creation, revocation, renewing, and usage logs handled through administrator endpoints
No dependencies, only requiring FastAPI and the python standard library

Build new App

and show the Directory Structure

Need a Litte Ping Function ?

Test

cat <<'EOF'> ping.py
import argparse
from ping3 import ping, verbose_ping

def do_ping(host: str, timeout: int = 3, size: int = 1500, output: str = "json"):
    # output: json|txt
    # '21.54 ms'
    if size > 1500:
        size = 1500
    result = (
        str(
            round(
                ping(dest_addr=host, timeout=timeout, size=size, unit="ms"),
                2,
            )
        )
        + " ms"
    )
    if output.lower() == "json":
        return {"host": host, "timeout": timeout, "size": size, "result": result}
    if output.lower() == "txt":
        return result
    else:
        return f"output format '{output} unknown! use 'json|txt'"


def do_multiple_ping(host: str, count: int = 3, interval: float = 0):
    # ping 'www.stoege.net' ... 23ms
    # ping 'www.stoege.net' ... 24ms
    # ping 'www.stoege.net' ... 20ms
    verbose_ping(
        dest_addr=host,
        count=count,
        interval=interval,
    )


def main():
    # Create the argument parser
    parser = argparse.ArgumentParser(description="Ping a domain or IP address.")

    # Add the host argument
    parser.add_argument(
        "host",
        metavar="HOST",
        type=str,
        nargs="?",
        default="www.stoege.net",
        help="the domain or IP address to ping",
    )

    # Parse the command-line arguments
    args = parser.parse_args()

    # Call the ping function
    output = do_ping(host=args.host, output="json")

    # Print the ping output
    print(f"\n{output}\n")

    # Call the ping function. No return Value !
    do_multiple_ping(host=args.host, count=10, interval=0.1)


if __name__ == "__main__":
    main()
EOF

add module

poetry, venv, whatever you like

a custom logger for Python

let’s tune the default logger a bit so he write nice and colored messages.

Screenshot

config.py

a little config File …

cat <<'EOF'> config.py
LOGGER_MAX_FILE_LENGTH = 10
EOF

src/logger.py

the logger code in the ‘src’ Folder

mkdir src
cat <<'EOF'> src/logger.py
import logging
import datetime
import sys

from config import *

if isinstance(LOGGER_MAX_FILE_LENGTH, int):
    LOGGER_MAX_FILE_LENGTH = str(LOGGER_MAX_FILE_LENGTH)


def get_now() -> str:
    #
    # choose your format
    #
    current_time = datetime.datetime.now()

    # 2023-07-16 22:16:15.958
    formatted_time_1 = current_time.strftime("%Y-%m-%d %H:%M:%S.%f")[:-3]

    # 22:16:54.471
    formatted_time_2 = current_time.strftime("%H:%M:%S.%f")[:-3]

    # 22:17:21
    formatted_time_3 = current_time.strftime("%H:%M:%S")

    return formatted_time_2


class ExitOnCriticalHandler(logging.Handler):
    def emit(self, record):
        # Unset Color
        COLOR = RESET = "\033[0m"

        # Debug -> Black
        if record.levelno == logging.DEBUG:
            COLOR = "\033[0m"
        # Info -> Green
        elif record.levelno == logging.INFO:
            COLOR = "\033[92m"
        # Warn -> Blue
        elif record.levelno == logging.WARNING:
            COLOR = "\033[94m"
        # Error -> Orange
        elif record.levelno == logging.ERROR:
            COLOR = "\033[38;5;208m"
        # Critical -> Red
        elif record.levelno >= logging.CRITICAL:
            COLOR = "\033[91m"

        # Custom Line
        print(
            COLOR
            + "{:} {:} {:04} {:{w}} {:}".format(
                get_now(),
                record.levelname,
                record.lineno,
                record.filename,
                record.msg,
                w=LOGGER_MAX_FILE_LENGTH,
            )
            + RESET
        )

        # Exit on Critical
        if record.levelno >= logging.CRITICAL:
            logging.shutdown()
            print("\033[91m" + "GOT A CRITICAL -> EXIT HERE!" + "\033[0m")
            sys.exit()


# Init Logger
logger = logging.getLogger()
logger.setLevel(logging.INFO)

# Create and add the custom handler
exit_handler = ExitOnCriticalHandler()
logger.addHandler(exit_handler)


# Set Log Level
def set_log_level(level: str):
    # Parse/ Set LogLevel
    if level.lower() in ["d", "debug"]:
        logger.setLevel(logging.DEBUG)
    elif level.lower() in ["i", "info"]:
        logger.setLevel(logging.INFO)
    elif level.lower() in ["w", "warning"]:
        logger.setLevel(logging.WARNING)
    elif level.lower() in ["e", "error"]:
        logger.setLevel(logging.ERROR)
    elif level.lower() in ["c", "critical"]:
        logger.setLevel(logging.CRITICAL)

    # Custom level name mappings, Debug -> D
    logging.addLevelName(logging.DEBUG, "D")
    logging.addLevelName(logging.INFO, "I")
    logging.addLevelName(logging.WARNING, "W")
    logging.addLevelName(logging.ERROR, "E")
    logging.addLevelName(logging.CRITICAL, "C")


# Functions to Call
def ldebug(msg: str):
    logger.debug(msg)


def linfo(msg: str):
    logger.info(msg)


def lwarning(msg: str):
    logger.warning(msg)


def lerror(msg: str):
    logger.error(msg)


def lcritical(msg: str):
    logger.critical(msg)
EOF

main.py

the Main File with Argparse to set the Logging Level on Startup

wanna convert a script to a executable ?

Build a Sample Script

cat << EOF > main.py
a = "top"
b = "secret"
print("This is", a, b)
EOF

python3 main.py
This is top secret

update poetry ?

doas poetry self update
poetry self update

or

pip install poetry -U

add pyinstaller

poetry init
poetry add pyinstaller

build Binary

poetry run pyinstaller main.py --onefile

check Binary

ls -la dist/
file dist/main

ls -la dist/
4735533 Jun 26 22:17 main

```.sh
file dist/main

file dist/main
dist/main: ELF 64-bit LSB shared object, x86-64, version 1

run Binary

./dist/main

./dist/main
Hello World! geheim

find “Strings”

strings dist/main |sort

strings dist/main |sort
...
unsetenv
vfprintf
vsnprintf
w3M9
waitpid
wcsncpy
|$8H
~$E1

find Keywords in Strings

strings dist/main |grep -E "top|secret"

strings dist/main |grep -E "top|secret"
# -> nothing found

Object Dump

objdump -x dist/main

objdump -x dist/main

dist/main:     file format elf64-x86-64
dist/main
architecture: i386:x86-64, flags 0x00000150:
HAS_SYMS, DYNAMIC, D_PAGED
start address 0x0000000000005ed0

Program Header:
    PHDR off    0x0000000000000040 vaddr 0x0000000000000040 paddr 0x0000000000000040 align 2**3
         filesz 0x00000000000002a0 memsz 0x00000000000002a0 flags r--
  INTERP off    0x00000000000002e0 vaddr 0x00000000000002e0 paddr 0x00000000000002e0 align 2**0
         filesz 0x0000000000000013 memsz 0x0000000000000013 flags r--
    LOAD off    0x0000000000000000 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**12
         filesz 0x0000000000004ecc memsz 0x0000000000004ecc flags r--
    LOAD off    0x0000000000004ed0 vaddr 0x0000000000005ed0 paddr 0x0000000000005ed0 align 2**12
         filesz 0x0000000000007700 memsz 0x0000000000007700 flags --x
    LOAD off    0x000000000000c5d0 vaddr 0x000000000000e5d0 paddr 0x000000000000e5d0 align 2**12
         filesz 0x0000000000000918 memsz 0x0000000000000918 flags rw-
    LOAD off    0x000000000000cee8 vaddr 0x000000000000fee8 paddr 0x000000000000fee8 align 2**12
         filesz 0x0000000000000004 memsz 0x000000000000602c flags rw-
 DYNAMIC off    0x000000000000c900 vaddr 0x000000000000e900 paddr 0x000000000000e900 align 2**3
         filesz 0x0000000000000140 memsz 0x0000000000000140 flags rw-
   RELRO off    0x000000000000c5d0 vaddr 0x000000000000e5d0 paddr 0x000000000000e5d0 align 2**0
         filesz 0x0000000000000918 memsz 0x0000000000000a30 flags r--
EH_FRAME off    0x0000000000003c1c vaddr 0x0000000000003c1c paddr 0x0000000000003c1c align 2**2
         filesz 0x000000000000030c memsz 0x000000000000030c flags r--
OPENBSD_RANDOMIZE off    0x000000000000c5d0 vaddr 0x000000000000e5d0 paddr 0x000000000000e5d0 align 2**3
         filesz 0x0000000000000308 memsz 0x0000000000000308 flags rw-
   STACK off    0x0000000000000000 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**0
         filesz 0x0000000000000000 memsz 0x0000000000000000 flags rw-
    NOTE off    0x00000000000002f4 vaddr 0x00000000000002f4 paddr 0x00000000000002f4 align 2**2
         filesz 0x0000000000000018 memsz 0x0000000000000018 flags r--

Dynamic Section:
  NEEDED      libm.so.10.1
  NEEDED      libz.so.7.0
  NEEDED      libc.so.97.0
  FLAGS_1     0x8000000
  DEBUG       0x0
  RELA        0xf50
  RELASZ      0x6d8
  RELAENT     0x18
  RELACOUNT   0x46
  JMPREL      0x1628
  PLTRELSZ    0x6d8
  PLTGOT      0xec88
  PLTREL      0x7
  SYMTAB      0x310
  SYMENT      0x18
  STRTAB      0xcd8
  STRSZ       0x272
  GNU_HASH    0xa48
  HASH        0xa68

Sections:
Idx Name          Size      VMA               LMA               File off  Algn
  0 .interp       00000013  00000000000002e0  00000000000002e0  000002e0  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  1 .note.openbsd.ident 00000018  00000000000002f4  00000000000002f4  000002f4  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  2 .dynsym       00000738  0000000000000310  0000000000000310  00000310  2**3
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  3 .gnu.hash     00000020  0000000000000a48  0000000000000a48  00000a48  2**3
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  4 .hash         00000270  0000000000000a68  0000000000000a68  00000a68  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  5 .dynstr       00000272  0000000000000cd8  0000000000000cd8  00000cd8  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  6 .rela.dyn     000006d8  0000000000000f50  0000000000000f50  00000f50  2**3
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  7 .rela.plt     000006d8  0000000000001628  0000000000001628  00001628  2**3
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  8 .rodata       00001f1c  0000000000001d00  0000000000001d00  00001d00  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  9 .eh_frame_hdr 0000030c  0000000000003c1c  0000000000003c1c  00003c1c  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
 10 .eh_frame     00000fa4  0000000000003f28  0000000000003f28  00003f28  2**3
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
 11 .text         00006d90  0000000000005ed0  0000000000005ed0  00004ed0  2**4
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
 12 .init         0000000e  000000000000cc60  000000000000cc60  0000bc60  2**4
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
 13 .fini         0000000e  000000000000cc70  000000000000cc70  0000bc70  2**4
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
 14 .plt          00000950  000000000000cc80  000000000000cc80  0000bc80  2**4
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
 15 .openbsd.randomdata 00000308  000000000000e5d0  000000000000e5d0  0000c5d0  2**3
                  CONTENTS, ALLOC, LOAD, DATA
 16 .jcr          00000008  000000000000e8d8  000000000000e8d8  0000c8d8  2**3
                  CONTENTS, ALLOC, LOAD, DATA
 17 .ctors        00000010  000000000000e8e0  000000000000e8e0  0000c8e0  2**3
                  CONTENTS, ALLOC, LOAD, DATA
 18 .dtors        00000010  000000000000e8f0  000000000000e8f0  0000c8f0  2**3
                  CONTENTS, ALLOC, LOAD, DATA
 19 .dynamic      00000140  000000000000e900  000000000000e900  0000c900  2**3
                  CONTENTS, ALLOC, LOAD, DATA
 20 .got          00000248  000000000000ea40  000000000000ea40  0000ca40  2**3
                  CONTENTS, ALLOC, LOAD, DATA
 21 .got.plt      00000260  000000000000ec88  000000000000ec88  0000cc88  2**3
                  CONTENTS, ALLOC, LOAD, DATA
 22 .data         00000004  000000000000fee8  000000000000fee8  0000cee8  2**2
                  CONTENTS, ALLOC, LOAD, DATA
 23 .bss          00006024  000000000000fef0  000000000000fef0  0000ceec  2**4
                  ALLOC
SYMBOL TABLE:
no symbols

Disassemble

objdump -d dist/main

objdump -d dist/main |head -100

dist/main:     file format elf64-x86-64

Disassembly of section .text:

0000000000005ed0 <.text>:
    5ed0:	48 89 d1             	mov    %rdx,%rcx
    5ed3:	48 8b 3c 24          	mov    (%rsp),%rdi
    5ed7:	48 8d 54 fc 10       	lea    0x10(%rsp,%rdi,8),%rdx
    5edc:	48 8d 74 24 08       	lea    0x8(%rsp),%rsi
    5ee1:	48 83 ec 08          	sub    $0x8,%rsp
    5ee5:	48 83 e4 f0          	and    $0xfffffffffffffff0,%rsp
    5ee9:	48 83 c4 08          	add    $0x8,%rsp
    5eed:	eb 01                	jmp    5ef0 <_csu_finish@plt-0x6da0>
    5eef:	cc                   	int3
    5ef0:	55                   	push   %rbp
    5ef1:	48 89 e5             	mov    %rsp,%rbp
    5ef4:	41 57                	push   %r15
    5ef6:	41 56                	push   %r14
    5ef8:	41 55                	push   %r13
    5efa:	41 54                	push   %r12
    5efc:	53                   	push   %rbx
    5efd:	50                   	push   %rax
    5efe:	49 89 cd             	mov    %rcx,%r13
    5f01:	49 89 d4             	mov    %rdx,%r12
    5f04:	49 89 f6             	mov    %rsi,%r14
    5f07:	41 89 ff             	mov    %edi,%r15d
    5f0a:	48 89 f7             	mov    %rsi,%rdi
    5f0d:	48 89 d6             	mov    %rdx,%rsi
    5f10:	48 8b d1             	mov    %rcx,%rdx
    5f13:	e8 98 6d 00 00       	callq  ccb0 <_Jv_RegisterClasses@plt>
    5f18:	48 8b d8             	mov    %rax,%rbx
    5f1b:	4d 85 ed             	test   %r13,%r13
    5f1e:	0f 85 db 00 00 00    	jne    5fff <_csu_finish@plt-0x6c91>
    5f24:	48 89 5d d0          	mov    %rbx,0xffffffffffffffd0(%rbp)
    5f28:	48 8d 1d a1 ff ff ff 	lea    -95(%rip),%rbx        # 5ed0 <_csu_finish@plt-0x6dc0>
    5f2f:	48 8d 0d 9a ff ff ff 	lea    -102(%rip),%rcx        # 5ed0 <_csu_finish@plt-0x6dc0>
    5f36:	48 29 d9             	sub    %rbx,%rcx
    5f39:	48 8d 41 07          	lea    0x7(%rcx),%rax
    5f3d:	48 85 c9             	test   %rcx,%rcx
    5f40:	48 8b d1             	mov    %rcx,%rdx
    5f43:	48 0f 48 d0          	cmovs  %rax,%rdx
    5f47:	48 c1 fa 03          	sar    $0x3,%rdx
    5f4b:	74 46                	je     5f93 <_csu_finish@plt-0x6cfd>
    5f4d:	48 85 c9             	test   %rcx,%rcx
    5f50:	48 0f 49 c1          	cmovns %rcx,%rax
    5f54:	48 c1 f8 03          	sar    $0x3,%rax
    5f58:	48 83 f8 02          	cmp    $0x2,%rax
    5f5c:	41 bd 01 00 00 00    	mov    $0x1,%r13d
    5f62:	4c 0f 43 e8          	cmovae %rax,%r13
    5f66:	eb 08                	jmp    5f70 <_csu_finish@plt-0x6d20>
    5f68:	cc                   	int3
    5f69:	cc                   	int3
    5f6a:	cc                   	int3
    5f6b:	cc                   	int3
    5f6c:	cc                   	int3
    5f6d:	cc                   	int3
    5f6e:	cc                   	int3
    5f6f:	cc                   	int3
    5f70:	4c 8b 1b             	mov    (%rbx),%r11
    5f73:	44 89 ff             	mov    %r15d,%edi
    5f76:	4c 89 f6             	mov    %r14,%rsi
    5f79:	4c 89 e2             	mov    %r12,%rdx
    5f7c:	31 c9                	xor    %ecx,%ecx
    5f7e:	e8 9d 00 00 00       	callq  6020 <_csu_finish@plt-0x6c70>
    5f83:	48 87 d8             	xchg   %rbx,%rax
    5f86:	48 83 c0 08          	add    $0x8,%rax
    5f8a:	48 87 d8             	xchg   %rbx,%rax
    5f8d:	49 83 c5 ff          	add    $0xffffffffffffffff,%r13
    5f91:	75 dd                	jne    5f70 <_csu_finish@plt-0x6d20>
    5f93:	4c 8d 2d 36 ff ff ff 	lea    -202(%rip),%r13        # 5ed0 <_csu_finish@plt-0x6dc0>
    5f9a:	48 8d 0d 2f ff ff ff 	lea    -209(%rip),%rcx        # 5ed0 <_csu_finish@plt-0x6dc0>
    5fa1:	4c 29 e9             	sub    %r13,%rcx
    5fa4:	48 8d 41 07          	lea    0x7(%rcx),%rax
    5fa8:	48 85 c9             	test   %rcx,%rcx
    5fab:	48 8b d1             	mov    %rcx,%rdx
    5fae:	48 0f 48 d0          	cmovs  %rax,%rdx
    5fb2:	48 c1 fa 03          	sar    $0x3,%rdx
    5fb6:	74 3c                	je     5ff4 <_csu_finish@plt-0x6c9c>
    5fb8:	48 85 c9             	test   %rcx,%rcx
    5fbb:	48 0f 49 c1          	cmovns %rcx,%rax
    5fbf:	48 c1 f8 03          	sar    $0x3,%rax
    5fc3:	48 83 f8 02          	cmp    $0x2,%rax
    5fc7:	bb 01 00 00 00       	mov    $0x1,%ebx
    5fcc:	48 0f 43 d8          	cmovae %rax,%rbx
    5fd0:	4d 8b 5d 00          	mov    0x0(%r13),%r11
    5fd4:	44 89 ff             	mov    %r15d,%edi
    5fd7:	4c 89 f6             	mov    %r14,%rsi
    5fda:	4c 89 e2             	mov    %r12,%rdx
    5fdd:	31 c9                	xor    %ecx,%ecx
    5fdf:	e8 3c 00 00 00       	callq  6020 <_csu_finish@plt-0x6c70>
    5fe4:	49 83 c5 08          	add    $0x8,%r13
    5fe8:	48 87 d8             	xchg   %rbx,%rax
    5feb:	48 83 c0 ff          	add    $0xffffffffffffffff,%rax
    5fef:	48 87 d8             	xchg   %rbx,%rax
    5ff2:	75 dc                	jne    5fd0 <_csu_finish@plt-0x6cc0>
    5ff4:	c6 05 45 9f 00 00 01 	movb   $0x1,40773(%rip)        # ff40 <memcmp@plt+0x2e30>
    5ffb:	48 8b 5d d0          	mov    0xffffffffffffffd0(%rbp),%rbx
    5fff:	e8 5c 6c 00 00       	callq  cc60 <_csu_finish@plt-0x30>
    6004:	48 8b 13             	mov    (%rbx),%rdx
...

-> at least the variables are not obviously found in the binary. but don’t want to know what ghidra says about it :(

wanna build ansible-pylibssh on OpenBSD 7.3 ?

Build failed ?

$ pip install ansible-pylibssh
Defaulting to user installation because normal site-packages is not writeable
Collecting ansible-pylibssh
  Using cached ansible-pylibssh-1.1.0.tar.gz (106 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: ansible-pylibssh
  Building wheel for ansible-pylibssh (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for ansible-pylibssh (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      [1/1] Cythonizing /tmp/pip-install-3inpi4en/ansible-pylibssh_04d3883cfd7d49ecb34d03dc90702e66/src/pylibsshext/_libssh_version.pyx
      /tmp/pip-install-3inpi4en/ansible-pylibssh_04d3883cfd7d49ecb34d03dc90702e66/src/pylibsshext/_libssh_version.c:757:10: fatal error: 'libssh/libssh.h' file not found
      #include "libssh/libssh.h"
               ^~~~~~~~~~~~~~~~~
      1 error generated.
      error: command '/usr/bin/cc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for ansible-pylibssh
Failed to build ansible-pylibssh
ERROR: Could not build wheels for ansible-pylibssh, which is required to install pyproject.toml-based projects

Build sucessfully !

$ CFLAGS=-I/usr/local/include pip install ansible-pylibssh
Defaulting to user installation because normal site-packages is not writeable
Collecting ansible-pylibssh
  Downloading ansible-pylibssh-1.1.0.tar.gz (106 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 106.2/106.2 kB 687.2 kB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: ansible-pylibssh
  Building wheel for ansible-pylibssh (pyproject.toml) ... done
  Created wheel for ansible-pylibssh: filename=ansible_pylibssh-1.1.0-cp310-cp310-openbsd_7_3_amd64.whl size=651397 sha256=76e37e982c0902f3a846347577ab5dc371ef97ccbea145eb9c2820fad1501d9d
  Stored in directory: /home/stoege/.cache/pip/wheels/60/74/37/a1ba5b525d4f4e67e90e1ae862e66eeedc08147fb09d82b5d8
Successfully built ansible-pylibssh
Installing collected packages: ansible-pylibssh
Successfully installed ansible-pylibssh-1.1.0

Any Comments ?

sha256: 162600846baa1e3d7e35801c67e5c7737aaf2a5dd8cc09952111c626bf942e4c

Parallel Processing

i recently read an article about parallel processing. i remembered my domain checker service which checks a lot of domains for their availablitly, and this script runs sequentiel and needs around 30 seconds.

initially i worked on a caching mechanism to speed up results. but if a service is not used that often (nobody is useing my domain checker…), there is not much you can gain with caching.

so, i gave a try with Multiprocessing and i have to admit i’m prettyhappy with the result !

Redirector App

wrote a little redirector app and tought i will explain and share it here. it’s a bit like a url shortener, but you can define the “shortcut” of the URL.

how does it work

it basically consists of a Text File wir Redirection URL’s.

redi.txt

stoege,https://www.stoege.net
blog,https://blog.stoege.net
test,https://www.test.com

Call it

so, when you open a Browser and Request the URL: https://your.domain.de/blog, you get redirected to https://blog.stoege.net

main.app

from flask import Flask, redirect, request
import datetime
import os
import random

# Vars
redirect_file="redi.txt"

app = Flask(__name__)

# Load redirection data from a text file
def get_redirections():
    redirections = {}
    with open(redirect_file,"r") as file:
        for line in file:
            path, url = line.strip().split(',')
            redirections[path] = url
    return redirections

# Main
@app.route('/')
def index():
    return 'Hello, World!'

# Redirect to Random URL
@app.route('/random')
def random_path():
    redirections = get_redirections()

    # Get Random Path
    random_url = random.choice(list(redirections.values()))
    return redirect(random_url)

# Redirect
@app.route('/<path:path>')
def redirect_path(path):

    # File Changed ?
    redirections = get_redirections()

    # Check if the path exists in the redirections dictionary
    if path in redirections:

        url = redirections[path]
        return redirect(url)

    # If the path does not exist, return a 404 Not Found error
    return 'Not Found', 404

if __name__ == '__main__':
    app.run()

get it running

you need the ususal Stuff