Poetry
1. Intro
Poetry is a tool for dependency management and packaging in Python. It allows you to declare the libraries your project depends on and it will manage (install/update) them for you.
Links
- https://python-poetry.org/
- https://www.infoworld.com/article/3527850/how-to-manage-python-projects-with-poetry.html
Install on OpenBSD
Wanna Install on OpenBSD ?
# get root
doas su -
Install as Root
# Install Poetry
pip3 install --user poetry
# on error PEP 668:
#pip3 install --user poetry --break-system-packages
# Update Poetry
pip3 install poetry -U
# Update Profile
cat << EOF >> ~/.profile
# Poetry, added $(date)
export PATH=\$PATH:/root/.local/bin
EOF
. ~/.profile
Install as User poetry
# Update Poetry
pip3 install poetry -U
# Update Profile
cat << EOF >> ~/.profile
# Poetry, added $(date)
export PATH=\$PATH:~/.local/bin
EOF
. ~/.profile
Install on Debian
Run as User or Root
sudo apt-get install -y python3 python3-distutils
curl -sSL https://install.python-poetry.org/ | python3 -
cat << EOF >> ~/.bashrc
# Poetry, added $(date)
export PATH=\$PATH:~/.local/bin
EOF
source ~/.bashrc
Basic Usage
$ poetry --version
Poetry version 1.1.11
$ poetry self update
You are using the latest version
$ poetry self update --preview
Updating to 1.2.0a2
RuntimeError
Could not find poetry-1.2.0a2-linux.sha256sum file
at ~/.poetry/lib/poetry/console/commands/self/update.py:260 in _update
256│ try:
257│ r = urlopen(base_url + "/{}/{}".format(version, checksum))
258│ except HTTPError as e:
259│ if e.code == 404:
→ 260│ raise RuntimeError("Could not find {} file".format(checksum))
261│
262│ raise
263│
264│ checksum = r.read().decode().strip()
# Downgrade
$ poetry self update 0.8.0
poetry new
$ poetry new poetry-demo
Created package poetry_demo in poetry-demo
$ tree poetry-demo/
poetry-demo/
├── poetry_demo
│ └── __init__.py
├── pyproject.toml
├── README.rst
└── tests
├── __init__.py
└── test_poetry_demo.py
2 directories, 5 files
or if you have an existing project, you can do the poetry init
poetry init
$ poetry init
This command will guide you through creating your pyproject.toml config.
Package name [test]:
Version [0.1.0]:
Description []: just a little test
Author [None, n to skip]: That's Me
License []:
Compatible Python versions [^3.9]:
Would you like to define your main dependencies interactively? (yes/no) [yes]
You can specify a package in the following forms:
- A single name (requests)
- A name and a constraint (requests@^2.23.0)
- A git url (git+https://github.com/python-poetry/poetry.git)
- A git url with a revision (git+https://github.com/python-poetry/poetry.git#develop)
- A file path (../my-package/my-package.whl)
- A directory (../my-package/)
- A url (https://example.com/packages/my-package-0.1.0.tar.gz)
Search for package to add (or leave blank to continue):
Would you like to define your development dependencies interactively? (yes/no) [yes]
Search for package to add (or leave blank to continue):
Generated file
[tool.poetry]
name = "test"
version = "0.1.0"
description = "just a little test"
authors = ["That's Me"]
[tool.poetry.dependencies]
python = "^3.9"
[tool.poetry.dev-dependencies]
[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"
Do you confirm generation? (yes/no) [yes]
poetry add
$ poetry add pendulum
Creating virtualenv test-nWsx-QRW-py3.9 in /home/stoege/.cache/pypoetry/virtualenvs
Using version ^2.1.2 for pendulum
Updating dependencies
Resolving dependencies... (0.5s)
Writing lock file
Package operations: 4 installs, 0 updates, 0 removals
• Installing six (1.16.0)
• Installing python-dateutil (2.8.2)
• Installing pytzdata (2020.1)
• Installing pendulum (2.1.2)
poetry show tree
$ poetry show --tree
pendulum 2.1.2 Python datetimes made easy
├── python-dateutil >=2.6,<3.0
│ └── six >=1.5
└── pytzdata >=2020.1
poetry show –latest
$ poetry show --latest
pendulum 2.1.2 2.1.2 Python datetimes made easy
python-dateutil 2.8.2 2.8.2 Extensions to the standard Python datetime module
pytzdata 2020.1 2020.1 The Olson timezone database for Python.
six 1.16.0 1.16.0 Python 2 and 3 compatibility utilities
poetry build
$ poetry build
Building test (0.1.0)
ModuleOrPackageNotFound
No file/folder found for package test
at ~/.poetry/lib/poetry/_vendor/py3.9/poetry/core/masonry/utils/module.py:63 in __init__
59│ "from": str(src.relative_to(self._path)),
60│ }
61│ ]
62│ else:
→ 63│ raise ModuleOrPackageNotFound(
64│ "No file/folder found for package {}".format(name)
65│ )
66│
67│ for package in packages:
poetry publish
$ poetry publish
No files to publish. Run poetry build first or use the --build option.
set python version
poetry env use $(which python3.9)
-> pyproject.toml -> python = "^3.9"
set python version 3.9
poetry env use $(which python3.9)
set python version 3.10
poetry env use $(which python3.10)
set python version 3.11
poetry env use $(which python3.11)
Use an Existing requirements.txt File
$ cat requirements.txt
beautifulsoup4==4.9.3
certifi==2020.12.5
chardet==4.0.0
$ poetry add `cat requirements.txt`
Creating virtualenv rp-require-0ubvZ-py3.9 in ~/Library/Caches/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies... (6.2s)
Writing lock file
Package operations: 7 installs, 0 updates, 0 removals
• Installing certifi (2020.12.5)
• Installing chardet (4.0.0)
• Installing idna (2.10)
...
or
awk -F '==' '!/sha256/{print $1}' requirements.txt |xargs -n1 poetry add
or
poetry add $(awk -F '==' '!/sha256/{print $1}' requirements.txt |tr '\n' ' ')
Create requirements.txt From poetry.lock
poetry export --output requirements.txt
Command Reference
$ poetry --version Show the version of your Poetry installation.
$ poetry new Create a new Poetry project.
$ poetry init Add Poetry to an existing project.
$ poetry run Execute the given command with Poetry.
$ poetry add Add a package to pyproject.toml and install it.
$ poetry update Update your project’s dependencies.
$ poetry install Install the dependencies.
$ poetry show List installed packages.
$ poetry lock Pin the latest version of your dependencies into poetry.lock.
$ poetry lock --no-update Refresh the poetry.lock file without updating any dependency version.
$ poetry check Validate pyproject.toml.
$ poetry config --list Show the Poetry configuration.
$ poetry env list List the virtual environments of your project.
$ poetry export Export poetry.lock to other formats.
Venv
check in-project settings
poetry config virtualenvs.in-project
poetry config virtualenvs.create
set in-project settings
# In Project
poetry config virtualenvs.in-project true
poetry config virtualenvs.in-project
# Create Virtual
poetry config virtualenvs.create true
poetry config virtualenvs
2. Move Project to .venv
you have your Virtual Env somewhere in your Home Folder and like to move it to the Project Folder ?
List Virtual Env
poetry env list
app-rBXXtrXX-py3.10 (Activated)
Remove Virtual Env
poetry env remove app-rBXXtrXX-py3.10
Deleted virtualenv: /Users/XXXXXXXX/Library/Caches/pypoetry/virtualenvs/app-rBXXtrXX-py3.10
Set in-project true
poetry config virtualenvs.in-project true
Check Config
poetry config -vvv
# poetry config -vvv
Loading configuration file /home/USERNAME/.config/pypoetry/config.toml
# cat /home/USERNAME/.config/pypoetry/config.toml
[virtualenvs]
in-project = true
Poetry Lock (why ?)
poetry lock
Creating virtualenv app in /Users/XXXXXXXX/YOUR_REPO_NAME/.venv
Updating dependencies
Resolving dependencies... (1.7s)
Poetry Install
poetry install
poetry install
Installing dependencies from lock file
Package operations: 65 installs, 0 updates, 0 removals
• Installing idna (3.4)
• Installing sniffio (1.3.0)
• Installing anyio (3.6.2)
• Installing pycparser (2.21)
• Installing certifi (2022.9.24)
• Installing cffi (1.15.1)
Update Poetry on OpenBSD 7.2
before
poetry --version
Poetry version 1.1.13
upgrade
pip install poetry -U
oder
poetry self update
Update Poetry on OpenBSD
doas su -
poetry --version
poetry self update
poetry --version
then switch to the appropriate User and check Virtual Env / Update if needed
su - webmaster
cd /home/webmaster/web_domain_stoege_net
poetry shell
poetry install
after upgrade …
poetry --version
Poetry (version 1.2.2)
Poetry (version 1.3.1)
Poetry (version 1.3.2)
Poetry (version 1.4.0)
Poetry (version 1.5.0), Mai 2023
...
Downgrade Poetry
Poetry v.1.4.1 had some issues with debugpy: “Installing debugpy (1.6.6): Failed”.
poetry self update 1.4.0
3. Customization
Command Prompt
show current config
poetry config --list
...
virtualenvs.prompt = "{project_name}-py{python_version}"
show default prompt
user@host /tmp/gugus1234 (main) $ poetry shell
Spawning shell within /tmp/gugus1234/.venv
(gugus1234-py3.11) user@host /tmp/gugus1234 (main) $
update prompt
poetry config virtualenvs.prompt "hello-from-py{python_version}"
kill .venv
rm -rf .venv
build new .venv
user@host /tmp/gugus1234 (main) $ poetry shell
Creating virtualenv gugus1234 in /tmp/gugus12344/.venv
Spawning shell within /tmp/gugus1234/.venv
. /tmp/gugus1234/.venv/bin/activate
user@host /tmp/gugus1234 (main) $ . /tmp/ugus1234/.venv/bin/activate
(hello-from-py3.11) user@host /tmp/gugus1234 (main) $
4. Strange Stuff
Collection of strange behaviours & errors. hope i can keep this section small ;)
Install Error Fastapi
(gugus2-py3.10) puffy202$ poetry add [email protected]
Updating dependencies
Resolving dependencies... (0.9s)
Package operations: 10 installs, 0 updates, 0 removals
• Installing exceptiongroup (1.1.3)
• Installing idna (3.4)
• Installing sniffio (1.3.0)
• Installing typing-extensions (4.7.1)
• Installing annotated-types (0.5.0)
• Installing anyio (3.7.1)
• Installing pydantic-core (2.6.3): Failed
ChefBuildError
Backend subprocess exited when trying to invoke build_wheel
Running `maturin pep517 build-wheel -i /tmp/tmp4eqst_ob/.venv/bin/python --compatibility off`
📦 Including license file "/tmp/tmpb2vbe9ix/pydantic_core-2.6.3/LICENSE"
🍹 Building a mixed python/rust project
🔗 Found pyo3 bindings
🐍 Found CPython 3.10 at /tmp/tmp4eqst_ob/.venv/bin/python
📡 Using build options features, bindings from pyproject.toml
Compiling autocfg v1.1.0
Compiling proc-macro2 v1.0.64
Traceback (most recent call last):
File "/home/stoege/.local/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/home/stoege/.local/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/home/stoege/.local/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
File "/tmp/tmp4eqst_ob/.venv/lib/python3.10/site-packages/maturin/__init__.py", line 111, in build_wheel
return _build_wheel(wheel_directory, config_settings, metadata_directory)
File "/tmp/tmp4eqst_ob/.venv/lib/python3.10/site-packages/maturin/__init__.py", line 90, in _build_wheel
result = subprocess.run(command, stdout=subprocess.PIPE)
File "/usr/local/lib/python3.10/subprocess.py", line 505, in run
stdout, stderr = process.communicate(input, timeout=timeout)
File "/usr/local/lib/python3.10/subprocess.py", line 1141, in communicate
stdout = self.stdout.read()
KeyboardInterrupt
at ~/.local/lib/python3.10/site-packages/poetry/installation/chef.py:147 in _prepare
143│
144│ error = ChefBuildError("\n\n".join(message_parts))
145│
146│ if error is not None:
→ 147│ raise error from None
148│
149│ return path
150│
151│ def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:
Note: This error originates from the build backend, and is likely not a problem with poetry but with pydantic-core (2.6.3) not supporting PEP 517 builds. You can verify this by running 'pip wheel --use-pep517 "pydantic-core (==2.6.3)"'.
Solution
- install lower Version First (0.85.0)
- upgrade afterwars
(gugus2-py3.10) puffy202$ poetry add [email protected]
Updating dependencies
Resolving dependencies... (0.4s)
Package operations: 3 installs, 0 updates, 0 removals
• Installing pydantic (1.10.12)
• Installing starlette (0.20.4)
• Installing fastapi (0.85.0)
Writing lock file
(gugus2-py3.10) puffy202$ poetry add [email protected]
Updating dependencies
Resolving dependencies... (0.1s)
Package operations: 0 installs, 2 updates, 0 removals
• Updating starlette (0.20.4 -> 0.27.0)
• Updating fastapi (0.85.0 -> 0.103.0)
Writing lock file
do i have to know why? no does it work ? yes
KeyRing Backend
on Day, i couldn’t use poetry anymore. whatever i did, i mostly got such an error:
(poetry-py3.10) $ poetry add package
Using virtualenv: /var/www/virtual/.../.venv
Error
g-io-error-quark: Cannot autolaunch D-Bus without X11 $DISPLAY (0)
and the persistent fix
cat << EOF >> ~/.profile
# fix strange poetry error, $(date), https://blog.stoege.net/posts/poetry/#keyring-backend
export PYTHON_KEYRING_BACKEND=keyring.backends.fail.Keyring >> ~/.profile
EOF
Full Error Log, Fix & confirmation
user@host$ poetry install
Installing dependencies from lock file
Package operations: 1 install, 0 updates, 0 removals
• Installing config (0.5.1): Failed
Error
g-io-error-quark: Cannot autolaunch D-Bus without X11 $DISPLAY (0)
at ~/.local/lib/python3.9/site-packages/keyring/backends/libsecret.py:134 in get_credential
130│ Otherwise, it will return the first username and password combo that it finds.
131│ """
132│ query = self._query(service, username)
133│ try:
→ 134│ items = Secret.password_search_sync(
135│ self.schema, query, Secret.SearchFlags.UNLOCK, None
136│ )
137│ except GLib.Error as error:
138│ quark = GLib.quark_try_string('g-io-error-quark')
The following error occurred when trying to handle this error:
KeyringLocked
Failed to unlock the item!
at ~/.local/lib/python3.9/site-packages/keyring/backends/libsecret.py:140 in get_credential
136│ )
137│ except GLib.Error as error:
138│ quark = GLib.quark_try_string('g-io-error-quark')
139│ if error.matches(quark, Gio.IOErrorEnum.FAILED):
→ 140│ raise KeyringLocked('Failed to unlock the item!') from error
141│ raise
142│ for item in items:
143│ username = item.get_attributes().get("username")
144│ try:
user@host$ export PYTHON_KEYRING_BACKEND=keyring.backends.fail.Keyring
user@host$ poetry install
Installing dependencies from lock file
Package operations: 1 install, 0 updates, 0 removals
• Installing config (0.5.1)
Installing the current project: tutorial-flaskr (0.1.0)
user@host$
[Errno 2] No such file or directory: ‘python’
$ poetry shell
$ rm -rf $(poetry config virtualenvs.path)/*
$ rm -rf .venv
Alternative: venv
Create Venv
python3 -m venv env
Activate venv
source ./env/bin/activate
Install Packages
pip3 install fastapi
Deactivate venv
deactivate
Cleanup
rm -rf venv
Install Special Version of a Package
# Allow >=2.0.5, <3.0.0 versions
poetry add pendulum@^2.0.5
# Allow >=2.0.5, <2.1.0 versions
poetry add pendulum@~2.0.5
# Allow >=2.0.5 versions, without upper bound
poetry add "pendulum>=2.0.5"
# Allow only 2.0.5 version
poetry add pendulum==2.0.5
Any Comments ?
sha256: dd082caac3ae7b25baf8894df57d2e1da976d8c940d3d53da116c78f1cb716a4