Compare commits

...

3 Commits

Author SHA1 Message Date
Ferdinand Thiessen 5038582406 feat(cli): Added CLI command for handling plugins
ci/woodpecker/push/lint Pipeline was successful Details
ci/woodpecker/push/test Pipeline was successful Details
ci/woodpecker/pr/lint Pipeline was successful Details
ci/woodpecker/pr/test Pipeline was successful Details
* Install / Uninstall plugins
* List plugins
2022-02-23 15:40:28 +01:00
Ferdinand Thiessen 0a7ffc29d1 feat(docs): Add documentation on how to install tables
Also various documentation fixed and improvments
2022-02-23 15:40:26 +01:00
Ferdinand Thiessen 047e807fc0 feat(db): Add database migration support, implements #19
Migrations allow us to keep track of database changes and upgrading databases if needed.

* Add initial migrations for core Flaschengeist
* Add migrations to balance
* Add migrations to pricelist

* Skip plugins with not satisfied dependencies.
2022-02-23 15:40:01 +01:00
24 changed files with 803 additions and 94 deletions

View File

@ -31,18 +31,35 @@ or if you want to also run the tests:
pip3 install --user ".[ldap,tests]" pip3 install --user ".[ldap,tests]"
You will also need a MySQL driver, recommended drivers are You will also need a MySQL driver, by default one of this is installed:
- `mysqlclient` - `mysqlclient` (non Windows)
- `PyMySQL` - `PyMySQL` (on Windows)
`setup.py` will try to install a matching driver. #### Hint on MySQL driver on Windows:
If you want to use `mysqlclient` instead of `PyMySQL` (performance?) you have to follow [this guide](https://www.radishlogic.com/coding/python-3/installing-mysqldb-for-python-3-in-windows/)
#### Windows ### Install database
Same as above, but if you want to use `mysqlclient` instead of `PyMySQL` (performance?) you have to follow this guide: The user needs to have full permissions to the database.
If not you need to create user and database manually do (or similar on Windows):
https://www.radishlogic.com/coding/python-3/installing-mysqldb-for-python-3-in-windows/ (
echo "CREATE DATABASE flaschengeist;"
echo "CREATE USER 'flaschengeist'@'localhost' IDENTIFIED BY 'flaschengeist';"
echo "GRANT ALL PRIVILEGES ON flaschengeist.* TO 'flaschengeist'@'localhost';"
echo "FLUSH PRIVILEGES;"
) | sudo mysql
### Configuration Then you can install the database tables, this will update all tables from core + all enabled plugins.
*Hint:* The same command can be later used to upgrade the database after plugins or core are updated.
$ flaschengeist db upgrade heads
## Plugins
To only upgrade one plugin (for example the `events` plugin):
$ flaschengeist db upgrade events@head
## Configuration
Configuration is done within the a `flaschengeist.toml`file, you can copy the one located inside the module path Configuration is done within the a `flaschengeist.toml`file, you can copy the one located inside the module path
(where flaschegeist is installed) or create an empty one and place it inside either: (where flaschegeist is installed) or create an empty one and place it inside either:
1. `~/.config/` 1. `~/.config/`
@ -63,21 +80,6 @@ So you have to configure one of the following options to call flaschengeists CRO
- Pros: Guaranteed execution interval, no impact on user experience (at least if you do not limit wsgi worker threads) - Pros: Guaranteed execution interval, no impact on user experience (at least if you do not limit wsgi worker threads)
- Cons: Uses one of the webserver threads while executing - Cons: Uses one of the webserver threads while executing
### Database installation
The user needs to have full permissions to the database.
If not you need to create user and database manually do (or similar on Windows):
(
echo "CREATE DATABASE flaschengeist;"
echo "CREATE USER 'flaschengeist'@'localhost' IDENTIFIED BY 'flaschengeist';"
echo "GRANT ALL PRIVILEGES ON flaschengeist.* TO 'flaschengeist'@'localhost';"
echo "FLUSH PRIVILEGES;"
) | sudo mysql
Then you can install the database tables and initial entries:
$ flaschengeist install
### Run ### Run
Flaschengeist provides a CLI, based on the flask CLI, respectivly called `flaschengeist`. Flaschengeist provides a CLI, based on the flask CLI, respectivly called `flaschengeist`.

View File

@ -5,9 +5,53 @@
- your_plugin/ - your_plugin/
- __init__.py - __init__.py
- ... - ...
- migrations/ (optional)
- ... - ...
- setup.cfg - setup.cfg
The basic layout of a plugin is quite simple, you will only need the `setup.cfg` or `setup.py` and The basic layout of a plugin is quite simple, you will only need the `setup.cfg` or `setup.py` and
the package containing your plugin code, at lease a `__init__.py` file with your `Plugin` class. the package containing your plugin code, at lease a `__init__.py` file with your `Plugin` class.
If you use custom database tables you need to provide a `migrations` directory within your package,
see next section.
## Database Tables / Migrations
To allow upgrades of installed plugins, the database is versioned and handled
through [Alembic](https://alembic.sqlalchemy.org/en/latest/index.html) migrations.
Each plugin, which uses custom database tables, is represented as an other base.
So you could simply follow the Alembic tutorial on [how to work with multiple bases](https://alembic.sqlalchemy.org/en/latest/branches.html#creating-a-labeled-base-revision).
A quick overview on how to work with migrations for your plugin:
$ flaschengeist db revision -m "Create my super plugin" \
--head=base --branch-label=myplugin_name --version-path=your/plugin/migrations
This would add a new base named `myplugin_name`, which should be the same as the pypi name of you plugin.
If your tables depend on an other plugin or a specific base version you could of cause add
--depends-on=VERSION
or
--depends-on=other_plugin
### Plugin Removal and Database Tables
As generic downgrades are most often hard to write, your plugin is not required to provide such functionallity.
For Flaschengeist only instable versions provide meaningful downgrade migrations down to the latest stable version.
So this means if you do not provide downgrades you must at lease provide a series of migrations toward removal of
the database tables in case the users wants to delete the plugin.
(base) ----> 1.0 <----> 1.1 <----> 1.2
|
--> removal
After the removal step the database is stamped to to "remove" your
## Useful Hooks
There are some predefined hooks, which might get handy for you.
For more information, please refer to
- `flaschengeist.utils.hook.HookBefore` and
- `flaschengeist.utils.hook.HookAfter`

View File

View File

@ -0,0 +1,53 @@
# A generic, single database configuration.
# No used by flaschengeist
[alembic]
# template used to generate migration files
# file_template = %%(rev)s_%%(slug)s
# set to 'true' to run the environment during
# the 'revision' command, regardless of autogenerate
# revision_environment = false
version_path_separator = os
version_locations = %(here)s/migrations
# Logging configuration
[loggers]
keys = root,sqlalchemy,alembic,flask_migrate
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARN
handlers = console
qualname =
[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[logger_flask_migrate]
level = INFO
handlers =
qualname = flask_migrate
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S

View File

@ -0,0 +1,74 @@
import logging
from logging.config import fileConfig
from pathlib import Path
from flask import current_app
from alembic import context
# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config
# Interpret the config file for Python logging.
# This line sets up loggers basically.
fileConfig(Path(config.get_main_option("script_location")) / config.config_file_name.split("/")[-1])
logger = logging.getLogger("alembic.env")
config.set_main_option("sqlalchemy.url", str(current_app.extensions["migrate"].db.get_engine().url).replace("%", "%%"))
target_metadata = current_app.extensions["migrate"].db.metadata
def run_migrations_offline():
"""Run migrations in 'offline' mode.
This configures the context with just a URL
and not an Engine, though an Engine is acceptable
here as well. By skipping the Engine creation
we don't even need a DBAPI to be available.
Calls to context.execute() here emit the given string to the
script output.
"""
url = config.get_main_option("sqlalchemy.url")
context.configure(url=url, target_metadata=target_metadata, literal_binds=True)
with context.begin_transaction():
context.run_migrations()
def run_migrations_online():
"""Run migrations in 'online' mode.
In this scenario we need to create an Engine
and associate a connection with the context.
"""
# this callback is used to prevent an auto-migration from being generated
# when there are no changes to the schema
# reference: http://alembic.zzzcomputing.com/en/latest/cookbook.html
def process_revision_directives(context, revision, directives):
if getattr(config.cmd_opts, "autogenerate", False):
script = directives[0]
if script.upgrade_ops.is_empty():
directives[:] = []
logger.info("No changes in schema detected.")
connectable = current_app.extensions["migrate"].db.get_engine()
with connectable.connect() as connection:
context.configure(
connection=connection,
target_metadata=target_metadata,
process_revision_directives=process_revision_directives,
**current_app.extensions["migrate"].configure_args,
)
with context.begin_transaction():
context.run_migrations()
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()

View File

@ -0,0 +1,132 @@
"""Flaschengeist: Initial
Revision ID: 255b93b6beed
Revises:
Create Date: 2022-02-23 14:33:02.851388
"""
from alembic import op
import sqlalchemy as sa
import flaschengeist
# revision identifiers, used by Alembic.
revision = "255b93b6beed"
down_revision = None
branch_labels = ("flaschengeist",)
depends_on = None
def upgrade():
# ### commands auto generated by Alembic - please adjust! ###
op.create_table(
"image",
sa.Column("id", flaschengeist.models.Serial(), nullable=False),
sa.Column("filename_", sa.String(length=127), nullable=False),
sa.Column("mimetype_", sa.String(length=30), nullable=False),
sa.Column("thumbnail_", sa.String(length=127), nullable=True),
sa.Column("path_", sa.String(length=127), nullable=True),
sa.PrimaryKeyConstraint("id", name=op.f("pk_image")),
)
op.create_table(
"permission",
sa.Column("name", sa.String(length=30), nullable=True),
sa.Column("id", flaschengeist.models.Serial(), nullable=False),
sa.PrimaryKeyConstraint("id", name=op.f("pk_permission")),
sa.UniqueConstraint("name", name=op.f("uq_permission_name")),
)
op.create_table(
"role",
sa.Column("id", flaschengeist.models.Serial(), nullable=False),
sa.Column("name", sa.String(length=30), nullable=True),
sa.PrimaryKeyConstraint("id", name=op.f("pk_role")),
sa.UniqueConstraint("name", name=op.f("uq_role_name")),
)
op.create_table(
"role_x_permission",
sa.Column("role_id", flaschengeist.models.Serial(), nullable=True),
sa.Column("permission_id", flaschengeist.models.Serial(), nullable=True),
sa.ForeignKeyConstraint(
["permission_id"], ["permission.id"], name=op.f("fk_role_x_permission_permission_id_permission")
),
sa.ForeignKeyConstraint(["role_id"], ["role.id"], name=op.f("fk_role_x_permission_role_id_role")),
)
op.create_table(
"user",
sa.Column("userid", sa.String(length=30), nullable=False),
sa.Column("display_name", sa.String(length=30), nullable=True),
sa.Column("firstname", sa.String(length=50), nullable=False),
sa.Column("lastname", sa.String(length=50), nullable=False),
sa.Column("deleted", sa.Boolean(), nullable=True),
sa.Column("birthday", sa.Date(), nullable=True),
sa.Column("mail", sa.String(length=60), nullable=True),
sa.Column("id", flaschengeist.models.Serial(), nullable=False),
sa.Column("avatar", flaschengeist.models.Serial(), nullable=True),
sa.ForeignKeyConstraint(["avatar"], ["image.id"], name=op.f("fk_user_avatar_image")),
sa.PrimaryKeyConstraint("id", name=op.f("pk_user")),
sa.UniqueConstraint("userid", name=op.f("uq_user_userid")),
)
op.create_table(
"notification",
sa.Column("id", flaschengeist.models.Serial(), nullable=False),
sa.Column("plugin", sa.String(length=127), nullable=False),
sa.Column("text", sa.Text(), nullable=True),
sa.Column("data", sa.PickleType(), nullable=True),
sa.Column("time", flaschengeist.models.UtcDateTime(), nullable=False),
sa.Column("user_id", flaschengeist.models.Serial(), nullable=False),
sa.ForeignKeyConstraint(["user_id"], ["user.id"], name=op.f("fk_notification_user_id_user")),
sa.PrimaryKeyConstraint("id", name=op.f("pk_notification")),
)
op.create_table(
"password_reset",
sa.Column("user", flaschengeist.models.Serial(), nullable=False),
sa.Column("token", sa.String(length=32), nullable=True),
sa.Column("expires", flaschengeist.models.UtcDateTime(), nullable=True),
sa.ForeignKeyConstraint(["user"], ["user.id"], name=op.f("fk_password_reset_user_user")),
sa.PrimaryKeyConstraint("user", name=op.f("pk_password_reset")),
)
op.create_table(
"session",
sa.Column("expires", flaschengeist.models.UtcDateTime(), nullable=True),
sa.Column("token", sa.String(length=32), nullable=True),
sa.Column("lifetime", sa.Integer(), nullable=True),
sa.Column("browser", sa.String(length=30), nullable=True),
sa.Column("platform", sa.String(length=30), nullable=True),
sa.Column("id", flaschengeist.models.Serial(), nullable=False),
sa.Column("user_id", flaschengeist.models.Serial(), nullable=True),
sa.ForeignKeyConstraint(["user_id"], ["user.id"], name=op.f("fk_session_user_id_user")),
sa.PrimaryKeyConstraint("id", name=op.f("pk_session")),
sa.UniqueConstraint("token", name=op.f("uq_session_token")),
)
op.create_table(
"user_attribute",
sa.Column("id", flaschengeist.models.Serial(), nullable=False),
sa.Column("user", flaschengeist.models.Serial(), nullable=False),
sa.Column("name", sa.String(length=30), nullable=True),
sa.Column("value", sa.PickleType(), nullable=True),
sa.ForeignKeyConstraint(["user"], ["user.id"], name=op.f("fk_user_attribute_user_user")),
sa.PrimaryKeyConstraint("id", name=op.f("pk_user_attribute")),
)
op.create_table(
"user_x_role",
sa.Column("user_id", flaschengeist.models.Serial(), nullable=True),
sa.Column("role_id", flaschengeist.models.Serial(), nullable=True),
sa.ForeignKeyConstraint(["role_id"], ["role.id"], name=op.f("fk_user_x_role_role_id_role")),
sa.ForeignKeyConstraint(["user_id"], ["user.id"], name=op.f("fk_user_x_role_user_id_user")),
)
# ### end Alembic commands ###
def downgrade():
# ### commands auto generated by Alembic - please adjust! ###
op.drop_table("user_x_role")
op.drop_table("user_attribute")
op.drop_table("session")
op.drop_table("password_reset")
op.drop_table("notification")
op.drop_table("user")
op.drop_table("role_x_permission")
op.drop_table("role")
op.drop_table("permission")
op.drop_table("image")
# ### end Alembic commands ###

View File

@ -0,0 +1,25 @@
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
from alembic import op
import sqlalchemy as sa
import flaschengeist
${imports if imports else ""}
# revision identifiers, used by Alembic.
revision = ${repr(up_revision)}
down_revision = ${repr(down_revision)}
branch_labels = ${repr(branch_labels)}
depends_on = ${repr(depends_on)}
def upgrade():
${upgrades if upgrades else "pass"}
def downgrade():
${downgrades if downgrades else "pass"}

View File

@ -1,6 +1,6 @@
import enum import enum
from flask import Flask, current_app from flask import Flask
from flask_cors import CORS from flask_cors import CORS
from datetime import datetime, date from datetime import datetime, date
from flask.json import JSONEncoder, jsonify from flask.json import JSONEncoder, jsonify
@ -11,7 +11,6 @@ from werkzeug.exceptions import HTTPException
from flaschengeist import logger from flaschengeist import logger
from flaschengeist.utils.hook import Hook from flaschengeist.utils.hook import Hook
from flaschengeist.plugins import AuthPlugin from flaschengeist.plugins import AuthPlugin
from flaschengeist.controller import roleController
from flaschengeist.config import config, configure_app from flaschengeist.config import config, configure_app
@ -37,10 +36,9 @@ class CustomJSONEncoder(JSONEncoder):
@Hook("plugins.loaded") @Hook("plugins.loaded")
def __load_plugins(app): def load_plugins(app: Flask):
logger.debug("Search for plugins")
app.config["FG_PLUGINS"] = {} app.config["FG_PLUGINS"] = {}
for entry_point in entry_points(group="flaschengeist.plugins"): for entry_point in entry_points(group="flaschengeist.plugins"):
logger.debug(f"Found plugin: {entry_point.name} ({entry_point.dist.version})") logger.debug(f"Found plugin: {entry_point.name} ({entry_point.dist.version})")
@ -72,37 +70,22 @@ def __load_plugins(app):
else: else:
logger.debug(f"Skip disabled plugin {entry_point.name}") logger.debug(f"Skip disabled plugin {entry_point.name}")
if "FG_AUTH_BACKEND" not in app.config: if "FG_AUTH_BACKEND" not in app.config:
logger.error("No authentication plugin configured or authentication plugin not found") logger.fatal("No authentication plugin configured or authentication plugin not found")
raise RuntimeError("No authentication plugin configured or authentication plugin not found") raise RuntimeError("No authentication plugin configured or authentication plugin not found")
@Hook("plugins.installed") def create_app(test_config=None, cli=False):
def install_all():
from flaschengeist.database import db
db.create_all()
db.session.commit()
for name, plugin in current_app.config["FG_PLUGINS"].items():
if not plugin:
logger.debug(f"Skip disabled plugin: {name}")
continue
logger.info(f"Install plugin {name}")
plugin.install()
if plugin.permissions:
roleController.create_permissions(plugin.permissions)
def create_app(test_config=None):
app = Flask("flaschengeist") app = Flask("flaschengeist")
app.json_encoder = CustomJSONEncoder app.json_encoder = CustomJSONEncoder
CORS(app) CORS(app)
with app.app_context(): with app.app_context():
from flaschengeist.database import db from flaschengeist.database import db, migrate
configure_app(app, test_config) configure_app(app, test_config)
db.init_app(app) db.init_app(app)
__load_plugins(app) migrate.init_app(app, db, compare_type=True)
load_plugins(app)
@app.route("/", methods=["GET"]) @app.route("/", methods=["GET"])
def __get_state(): def __get_state():

View File

@ -84,7 +84,7 @@ def cli():
def main(*args, **kwargs): def main(*args, **kwargs):
# from .plugin_cmd import plugin from .plugin_cmd import plugin
from .export_cmd import export from .export_cmd import export
from .docs_cmd import docs from .docs_cmd import docs
from .run_cmd import run from .run_cmd import run
@ -92,8 +92,8 @@ def main(*args, **kwargs):
# Override logging level # Override logging level
environ.setdefault("FG_LOGGING", logging.getLevelName(LOGGING_MAX)) environ.setdefault("FG_LOGGING", logging.getLevelName(LOGGING_MAX))
# cli.add_command(plugin)
cli.add_command(export) cli.add_command(export)
cli.add_command(docs) cli.add_command(docs)
cli.add_command(plugin)
cli.add_command(run) cli.add_command(run)
cli(*args, **kwargs) cli(*args, **kwargs)

View File

@ -0,0 +1,96 @@
import click
from click.decorators import pass_context
from flask import current_app
from flask.cli import with_appcontext
from importlib_metadata import EntryPoint, entry_points
from flaschengeist.config import config
@click.group()
def plugin():
pass
@plugin.command()
@click.argument("plugin", nargs=-1, type=str)
@click.option("--all", help="Install all enabled plugins", is_flag=True)
@with_appcontext
@pass_context
def install(ctx, plugin, all):
"""Install one or more plugins"""
if not all and len(plugin) == 0:
ctx.fail("At least one plugin must be specified, or use `--all` flag.")
if all:
plugins = current_app.config["FG_PLUGINS"].values()
else:
try:
plugins = [current_app.config["FG_PLUGINS"][p] for p in plugin]
except KeyError as e:
ctx.fail(f"Invalid plugin ID, could not find >{e.args[0]}<")
for p in plugins:
name = p.id.split(".")[-1]
click.echo(f"Installing {name}{'.'*(20-len(name))}", nl=False)
p.install()
click.secho(" ok", fg="green")
@plugin.command()
@click.argument("plugin", nargs=-1, type=str)
@with_appcontext
@pass_context
def uninstall(ctx: click.Context, plugin):
"""Uninstall one or more plugins"""
if len(plugin) == 0:
ctx.fail("At least one plugin must be specified")
try:
plugins = [current_app.config["FG_PLUGINS"][p] for p in plugin]
except KeyError as e:
ctx.fail(f"Invalid plugin ID, could not find >{e.args[0]}<")
if (
click.prompt(
"You are going to uninstall:\n\n"
f"\t{', '.join([p.id.split('.')[-1] for p in plugins])}\n\n"
"Are you sure?",
default="n",
show_choices=True,
type=click.Choice(["y", "N"], False),
).lower()
!= "y"
):
ctx.exit()
for p in plugins:
name = p.id.split(".")[-1]
click.echo(f"Uninstalling {name}{'.'*(20-len(name))}", nl=False)
p.uninstall()
click.secho(" ok", fg="green")
@plugin.command()
@click.option("--enabled", "-e", help="List only enabled plugins", is_flag=True)
@click.option("--no-header", "-n", help="Do not show header", is_flag=True)
@with_appcontext
def ls(enabled, no_header):
def plugin_version(p):
if isinstance(p, EntryPoint):
return p.dist.version
return p.version
plugins = entry_points(group="flaschengeist.plugins")
enabled_plugins = [key for key, value in config.items() if "enabled" in value] + [config["FLASCHENGEIST"]["auth"]]
loaded_plugins = current_app.config["FG_PLUGINS"].keys()
if not no_header:
print(f"{' '*13}{'name': <20}|{'version': >10}")
print("-" * 46)
for plugin in plugins:
if enabled and plugin.name not in enabled_plugins:
continue
print(
f"{plugin.name: <33}|{plugin_version(plugin): >12}"
f"{click.style(' (enabled)', fg='green') if plugin.name in enabled_plugins else ' (disabled)'}"
)
for plugin in [value for value in enabled_plugins if value not in loaded_plugins]:
print(f"{plugin: <33}|{' '*12}" f"{click.style(' (not found)', fg='red')}")

View File

@ -1,6 +1,11 @@
import os
from flask_migrate import Migrate, Config
from flask_sqlalchemy import SQLAlchemy from flask_sqlalchemy import SQLAlchemy
from importlib_metadata import EntryPoint
from sqlalchemy import MetaData from sqlalchemy import MetaData
from flaschengeist import logger
# https://alembic.sqlalchemy.org/en/latest/naming.html # https://alembic.sqlalchemy.org/en/latest/naming.html
metadata = MetaData( metadata = MetaData(
naming_convention={ naming_convention={
@ -14,6 +19,45 @@ metadata = MetaData(
db = SQLAlchemy(metadata=metadata) db = SQLAlchemy(metadata=metadata)
migrate = Migrate()
@migrate.configure
def configure_alembic(config: Config):
"""Alembic configuration hook
Inject all migrations paths into the ``version_locations`` config option.
This includes even disabled plugins, as simply disabling a plugin without
uninstall can break the alembic version management.
"""
from importlib_metadata import entry_points, distribution
# Set main script location
config.set_main_option(
"script_location", str(distribution("flaschengeist").locate_file("") / "flaschengeist" / "alembic")
)
# Set Flaschengeist's migrations
migrations = [config.get_main_option("script_location") + "/migrations"]
# Gather all migration paths
ep: EntryPoint
for ep in entry_points(group="flaschengeist.plugins"):
try:
directory = ep.dist.locate_file("")
for loc in ep.module.split(".") + ["migrations"]:
directory /= loc
if directory.exists():
logger.debug(f"Adding migration version path {directory}")
migrations.append(str(directory.resolve()))
except:
logger.warning(f"Could not load migrations of plugin {ep.name} for database migration.")
logger.debug("Plugin loading failed", exc_info=True)
# write back seperator (we changed it if neither seperator nor locations were specified)
config.set_main_option("version_path_separator", os.pathsep)
config.set_main_option("version_locations", os.pathsep.join(set(migrations)))
return config
def case_sensitive(s): def case_sensitive(s):

View File

@ -3,56 +3,82 @@
.. include:: docs/plugin_development.md .. include:: docs/plugin_development.md
""" """
from typing import Optional
from importlib_metadata import Distribution, EntryPoint from importlib_metadata import Distribution, EntryPoint
from werkzeug.datastructures import FileStorage
from werkzeug.exceptions import MethodNotAllowed, NotFound from werkzeug.exceptions import MethodNotAllowed, NotFound
from werkzeug.datastructures import FileStorage
from flaschengeist.models.user import _Avatar, User from flaschengeist.models.user import _Avatar, User
from flaschengeist.utils.hook import HookBefore, HookAfter from flaschengeist.utils.hook import HookBefore, HookAfter
__all__ = [
"plugins_installed",
"plugins_loaded",
"before_delete_user",
"before_role_updated",
"before_update_user",
"after_role_updated",
"Plugin",
"AuthPlugin",
]
# Documentation hacks, see https://github.com/mitmproxy/pdoc/issues/320
plugins_installed = HookAfter("plugins.installed") plugins_installed = HookAfter("plugins.installed")
"""Hook decorator for when all plugins are installed plugins_installed.__doc__ = """Hook decorator for when all plugins are installed
Possible use case would be to populate the database with some presets.
Args: Possible use case would be to populate the database with some presets.
hook_result: void (kwargs)
""" """
plugins_loaded = HookAfter("plugins.loaded") plugins_loaded = HookAfter("plugins.loaded")
"""Hook decorator for when all plugins are loaded plugins_loaded.__doc__ = """Hook decorator for when all plugins are loaded
Possible use case would be to check if a specific other plugin is loaded and change own behavior
Args: Possible use case would be to check if a specific other plugin is loaded and change own behavior
app: Current flask app instance (args)
hook_result: void (kwargs) Passed args:
- *app:* Current flask app instance (args)
""" """
before_role_updated = HookBefore("update_role") before_role_updated = HookBefore("update_role")
"""Hook decorator for when roles are modified before_role_updated.__doc__ = """Hook decorator for when roles are modified
Args:
role: Role object to modify Passed args:
new_name: New name if the name was changed (None if delete) - *role:* `flaschengeist.models.user.Role` to modify
- *new_name:* New name if the name was changed (*None* if delete)
""" """
after_role_updated = HookAfter("update_role") after_role_updated = HookAfter("update_role")
"""Hook decorator for when roles are modified after_role_updated.__doc__ = """Hook decorator for when roles are modified
Args:
role: Role object containing the modified role Passed args:
new_name: New name if the name was changed (None if deleted) - *role:* modified `flaschengeist.models.user.Role`
- *new_name:* New name if the name was changed (*None* if deleted)
""" """
before_update_user = HookBefore("update_user") before_update_user = HookBefore("update_user")
"""Hook decorator, when ever an user update is done, this is called before. before_update_user.__doc__ = """Hook decorator, when ever an user update is done, this is called before.
Args:
user: User object Passed args:
- *user:* `flaschengeist.models.user.User` object
""" """
before_delete_user = HookBefore("delete_user") before_delete_user = HookBefore("delete_user")
"""Hook decorator,this is called before an user gets deleted. before_delete_user.__doc__ = """Hook decorator,this is called before an user gets deleted.
Args:
user: User object Passed args:
- *user:* `flaschengeist.models.user.User` object
""" """
class Plugin: class Plugin:
"""Base class for all Plugins """Base class for all Plugins
If your class uses custom models add a static property called ``models``. All plugins must derived from this class.
Optional:
- *blueprint*: `flask.Blueprint` providing your routes
- *permissions*: List of your custom permissions
- *models*: Your models, used for API export
""" """
name: str name: str
@ -77,8 +103,16 @@ class Plugin:
models = None models = None
"""Optional module containing the SQLAlchemy models used by the plugin""" """Optional module containing the SQLAlchemy models used by the plugin"""
migrations_path = None migrations: Optional[tuple[str, str]] = None
"""Optional location of the path to migration files, required if custome db tables are used""" """Optional identifiers of the migration versions
If custom database tables are used migrations must be provided and the
head and removal versions had to be defined, e.g.
```
migrations = ("head_hash", "removal_hash")
```
"""
def __init__(self, entry_point: EntryPoint, config=None): def __init__(self, entry_point: EntryPoint, config=None):
"""Constructor called by create_app """Constructor called by create_app
@ -96,6 +130,17 @@ class Plugin:
""" """
pass pass
def uninstall(self):
"""Uninstall routine
If the plugin has custom database tables, make sure to remove them.
This can be either done by downgrading the plugin *head* to the *base*.
Or use custom migrations for the uninstall and *stamp* some version.
Is always called with Flask application context.
"""
pass
def get_setting(self, name: str, **kwargs): def get_setting(self, name: str, **kwargs):
"""Get plugin setting from database """Get plugin setting from database
@ -109,7 +154,7 @@ class Plugin:
""" """
from ..controller import pluginController from ..controller import pluginController
return pluginController.get_setting(self.id, name, **kwargs) return pluginController.get_setting(self.name, name, **kwargs)
def set_setting(self, name: str, value): def set_setting(self, name: str, value):
"""Save setting in database """Save setting in database
@ -148,6 +193,11 @@ class Plugin:
class AuthPlugin(Plugin): class AuthPlugin(Plugin):
"""Base class for all authentification plugins
See also `Plugin`
"""
def login(self, user, pw): def login(self, user, pw):
"""Login routine, MUST BE IMPLEMENTED! """Login routine, MUST BE IMPLEMENTED!
@ -210,7 +260,7 @@ class AuthPlugin(Plugin):
""" """
raise MethodNotAllowed raise MethodNotAllowed
def get_avatar(self, user: User) -> _Avatar: def get_avatar(self, user):
"""Retrieve avatar for given user (if supported by auth backend) """Retrieve avatar for given user (if supported by auth backend)
Default behavior is to use native Image objects, Default behavior is to use native Image objects,
@ -224,14 +274,14 @@ class AuthPlugin(Plugin):
""" """
raise NotFound raise NotFound
def set_avatar(self, user: User, file: FileStorage): def set_avatar(self, user, file: FileStorage):
"""Set the avatar for given user (if supported by auth backend) """Set the avatar for given user (if supported by auth backend)
Default behavior is to use native Image objects stored on the Flaschengeist server Default behavior is to use native Image objects stored on the Flaschengeist server
Args: Args:
user: User to set the avatar for user: User to set the avatar for
file: FileStorage object uploaded by the user file: `werkzeug.datastructures.FileStorage` uploaded by the user
Raises: Raises:
MethodNotAllowed: If not supported by Backend MethodNotAllowed: If not supported by Backend
Any valid HTTP exception Any valid HTTP exception
@ -242,7 +292,7 @@ class AuthPlugin(Plugin):
user.avatar_ = imageController.upload_image(file) user.avatar_ = imageController.upload_image(file)
def delete_avatar(self, user: User): def delete_avatar(self, user):
"""Delete the avatar for given user (if supported by auth backend) """Delete the avatar for given user (if supported by auth backend)
Default behavior is to use the imageController and native Image objects. Default behavior is to use the imageController and native Image objects.

View File

@ -13,8 +13,8 @@ from flaschengeist.controller import sessionController, userController
class AuthRoutePlugin(Plugin): class AuthRoutePlugin(Plugin):
name = "auth" id = "dev.flaschengeist.auth"
blueprint = Blueprint(name, __name__) blueprint = Blueprint("auth", __name__)
@AuthRoutePlugin.blueprint.route("/auth", methods=["POST"]) @AuthRoutePlugin.blueprint.route("/auth", methods=["POST"])

View File

@ -18,7 +18,7 @@ from flaschengeist.plugins import AuthPlugin, before_role_updated
class AuthLDAP(AuthPlugin): class AuthLDAP(AuthPlugin):
def __init__(self, entry_point, config): def __init__(self, entry_point, config):
super().__init__(entry_point) super().__init__(entry_point, config)
app.config.update( app.config.update(
LDAP_SERVER=config.get("host", "localhost"), LDAP_SERVER=config.get("host", "localhost"),
LDAP_PORT=config.get("port", 389), LDAP_PORT=config.get("port", 389),

View File

@ -14,6 +14,8 @@ from flaschengeist import logger
class AuthPlain(AuthPlugin): class AuthPlain(AuthPlugin):
id = "auth_plain"
def install(self): def install(self):
plugins_installed(self.post_install) plugins_installed(self.post_install)

View File

@ -3,7 +3,7 @@
Extends users plugin with balance functions Extends users plugin with balance functions
""" """
from flask import Blueprint, current_app from flask import current_app
from werkzeug.local import LocalProxy from werkzeug.local import LocalProxy
from werkzeug.exceptions import NotFound from werkzeug.exceptions import NotFound
@ -57,8 +57,10 @@ def service_debit():
class BalancePlugin(Plugin): class BalancePlugin(Plugin):
permissions = permissions.permissions permissions = permissions.permissions
plugin: "BalancePlugin" = LocalProxy(lambda: current_app.config["FG_PLUGINS"][BalancePlugin.name])
models = models models = models
migrations = True
plugin: "BalancePlugin" = LocalProxy(lambda: current_app.config["FG_PLUGINS"][BalancePlugin.name])
def __init__(self, entry_point, config): def __init__(self, entry_point, config):
super(BalancePlugin, self).__init__(entry_point, config) super(BalancePlugin, self).__init__(entry_point, config)

View File

@ -0,0 +1,47 @@
"""balance: initial
Revision ID: 98f2733bbe45
Revises:
Create Date: 2022-02-23 14:41:03.089145
"""
from alembic import op
import sqlalchemy as sa
import flaschengeist
# revision identifiers, used by Alembic.
revision = "98f2733bbe45"
down_revision = None
branch_labels = ("balance",)
depends_on = "flaschengeist"
def upgrade():
# ### commands auto generated by Alembic - please adjust! ###
op.create_table(
"balance_transaction",
sa.Column("receiver_id", flaschengeist.models.Serial(), nullable=True),
sa.Column("sender_id", flaschengeist.models.Serial(), nullable=True),
sa.Column("author_id", flaschengeist.models.Serial(), nullable=False),
sa.Column("id", flaschengeist.models.Serial(), nullable=False),
sa.Column("time", flaschengeist.models.UtcDateTime(), nullable=False),
sa.Column("amount", sa.Numeric(precision=5, scale=2, asdecimal=False), nullable=False),
sa.Column("reversal_id", flaschengeist.models.Serial(), nullable=True),
sa.ForeignKeyConstraint(["author_id"], ["user.id"], name=op.f("fk_balance_transaction_author_id_user")),
sa.ForeignKeyConstraint(["receiver_id"], ["user.id"], name=op.f("fk_balance_transaction_receiver_id_user")),
sa.ForeignKeyConstraint(
["reversal_id"],
["balance_transaction.id"],
name=op.f("fk_balance_transaction_reversal_id_balance_transaction"),
),
sa.ForeignKeyConstraint(["sender_id"], ["user.id"], name=op.f("fk_balance_transaction_sender_id_user")),
sa.PrimaryKeyConstraint("id", name=op.f("pk_balance_transaction")),
)
# ### end Alembic commands ###
def downgrade():
# ### commands auto generated by Alembic - please adjust! ###
op.drop_table("balance_transaction")
# ### end Alembic commands ###

View File

@ -134,7 +134,7 @@ def get_balance(userid, current_session: Session):
Route: ``/users/<userid>/balance`` | Method: ``GET`` Route: ``/users/<userid>/balance`` | Method: ``GET``
GET-parameters: ```{from?: string, to?: string}``` GET-parameters: ``{from?: string, to?: string}``
Args: Args:
userid: Userid of user to get balance from userid: Userid of user to get balance from
@ -173,7 +173,7 @@ def get_transactions(userid, current_session: Session):
Route: ``/users/<userid>/balance/transactions`` | Method: ``GET`` Route: ``/users/<userid>/balance/transactions`` | Method: ``GET``
GET-parameters: ```{from?: string, to?: string, limit?: int, offset?: int}``` GET-parameters: ``{from?: string, to?: string, limit?: int, offset?: int}``
Args: Args:
userid: Userid of user to get transactions from userid: Userid of user to get transactions from

View File

@ -13,7 +13,7 @@ from . import Plugin
class MailMessagePlugin(Plugin): class MailMessagePlugin(Plugin):
def __init__(self, entry_point, config): def __init__(self, entry_point, config):
super().__init__(entry_point) super().__init__(entry_point, config)
self.server = config["SERVER"] self.server = config["SERVER"]
self.port = config["PORT"] self.port = config["PORT"]
self.user = config["USER"] self.user = config["USER"]

View File

@ -1,5 +1,6 @@
"""Pricelist plugin""" """Pricelist plugin"""
import pathlib
from flask import Blueprint, jsonify, request, current_app from flask import Blueprint, jsonify, request, current_app
from werkzeug.local import LocalProxy from werkzeug.local import LocalProxy
from werkzeug.exceptions import BadRequest, Forbidden, NotFound, Unauthorized from werkzeug.exceptions import BadRequest, Forbidden, NotFound, Unauthorized
@ -26,6 +27,7 @@ class PriceListPlugin(Plugin):
def __init__(self, entry_point, config=None): def __init__(self, entry_point, config=None):
super().__init__(entry_point, config) super().__init__(entry_point, config)
self.blueprint = blueprint self.blueprint = blueprint
self.migrations_path = (pathlib.Path(__file__).parent / "migrations").resolve()
config = {"discount": 0} config = {"discount": 0}
config.update(config) config.update(config)

View File

@ -0,0 +1,141 @@
"""pricelist: initial
Revision ID: 58ab9b6a8839
Revises:
Create Date: 2022-02-23 14:45:30.563647
"""
from alembic import op
import sqlalchemy as sa
import flaschengeist
# revision identifiers, used by Alembic.
revision = "58ab9b6a8839"
down_revision = None
branch_labels = ("pricelist",)
depends_on = "flaschengeist"
def upgrade():
# ### commands auto generated by Alembic - please adjust! ###
op.create_table(
"drink_extra_ingredient",
sa.Column("id", flaschengeist.models.Serial(), nullable=False),
sa.Column("name", sa.String(length=30), nullable=False),
sa.Column("price", sa.Numeric(precision=5, scale=2, asdecimal=False), nullable=True),
sa.PrimaryKeyConstraint("id", name=op.f("pk_drink_extra_ingredient")),
sa.UniqueConstraint("name", name=op.f("uq_drink_extra_ingredient_name")),
)
op.create_table(
"drink_tag",
sa.Column("id", flaschengeist.models.Serial(), nullable=False),
sa.Column("name", sa.String(length=30), nullable=False),
sa.Column("color", sa.String(length=7), nullable=False),
sa.PrimaryKeyConstraint("id", name=op.f("pk_drink_tag")),
sa.UniqueConstraint("name", name=op.f("uq_drink_tag_name")),
)
op.create_table(
"drink_type",
sa.Column("id", flaschengeist.models.Serial(), nullable=False),
sa.Column("name", sa.String(length=30), nullable=False),
sa.PrimaryKeyConstraint("id", name=op.f("pk_drink_type")),
sa.UniqueConstraint("name", name=op.f("uq_drink_type_name")),
)
op.create_table(
"drink",
sa.Column("id", flaschengeist.models.Serial(), nullable=False),
sa.Column("article_id", sa.String(length=64), nullable=True),
sa.Column("package_size", sa.Integer(), nullable=True),
sa.Column("name", sa.String(length=60), nullable=False),
sa.Column("volume", sa.Numeric(precision=5, scale=2, asdecimal=False), nullable=True),
sa.Column("cost_per_volume", sa.Numeric(precision=5, scale=3, asdecimal=False), nullable=True),
sa.Column("cost_per_package", sa.Numeric(precision=5, scale=3, asdecimal=False), nullable=True),
sa.Column("receipt", sa.PickleType(), nullable=True),
sa.Column("type_id", flaschengeist.models.Serial(), nullable=True),
sa.Column("image_id", flaschengeist.models.Serial(), nullable=True),
sa.ForeignKeyConstraint(["image_id"], ["image.id"], name=op.f("fk_drink_image_id_image")),
sa.ForeignKeyConstraint(["type_id"], ["drink_type.id"], name=op.f("fk_drink_type_id_drink_type")),
sa.PrimaryKeyConstraint("id", name=op.f("pk_drink")),
)
op.create_table(
"drink_ingredient",
sa.Column("id", flaschengeist.models.Serial(), nullable=False),
sa.Column("volume", sa.Numeric(precision=5, scale=2, asdecimal=False), nullable=False),
sa.Column("ingredient_id", flaschengeist.models.Serial(), nullable=True),
sa.ForeignKeyConstraint(["ingredient_id"], ["drink.id"], name=op.f("fk_drink_ingredient_ingredient_id_drink")),
sa.PrimaryKeyConstraint("id", name=op.f("pk_drink_ingredient")),
)
op.create_table(
"drink_price_volume",
sa.Column("id", flaschengeist.models.Serial(), nullable=False),
sa.Column("drink_id", flaschengeist.models.Serial(), nullable=True),
sa.Column("volume", sa.Numeric(precision=5, scale=2, asdecimal=False), nullable=True),
sa.ForeignKeyConstraint(["drink_id"], ["drink.id"], name=op.f("fk_drink_price_volume_drink_id_drink")),
sa.PrimaryKeyConstraint("id", name=op.f("pk_drink_price_volume")),
)
op.create_table(
"drink_x_tag",
sa.Column("drink_id", flaschengeist.models.Serial(), nullable=True),
sa.Column("tag_id", flaschengeist.models.Serial(), nullable=True),
sa.ForeignKeyConstraint(["drink_id"], ["drink.id"], name=op.f("fk_drink_x_tag_drink_id_drink")),
sa.ForeignKeyConstraint(["tag_id"], ["drink_tag.id"], name=op.f("fk_drink_x_tag_tag_id_drink_tag")),
)
op.create_table(
"drink_x_type",
sa.Column("drink_id", flaschengeist.models.Serial(), nullable=True),
sa.Column("type_id", flaschengeist.models.Serial(), nullable=True),
sa.ForeignKeyConstraint(["drink_id"], ["drink.id"], name=op.f("fk_drink_x_type_drink_id_drink")),
sa.ForeignKeyConstraint(["type_id"], ["drink_type.id"], name=op.f("fk_drink_x_type_type_id_drink_type")),
)
op.create_table(
"drink_ingredient_association",
sa.Column("id", flaschengeist.models.Serial(), nullable=False),
sa.Column("volume_id", flaschengeist.models.Serial(), nullable=True),
sa.Column("_drink_ingredient_id", flaschengeist.models.Serial(), nullable=True),
sa.Column("_extra_ingredient_id", flaschengeist.models.Serial(), nullable=True),
sa.ForeignKeyConstraint(
["_drink_ingredient_id"],
["drink_ingredient.id"],
name=op.f("fk_drink_ingredient_association__drink_ingredient_id_drink_ingredient"),
),
sa.ForeignKeyConstraint(
["_extra_ingredient_id"],
["drink_extra_ingredient.id"],
name=op.f("fk_drink_ingredient_association__extra_ingredient_id_drink_extra_ingredient"),
),
sa.ForeignKeyConstraint(
["volume_id"],
["drink_price_volume.id"],
name=op.f("fk_drink_ingredient_association_volume_id_drink_price_volume"),
),
sa.PrimaryKeyConstraint("id", name=op.f("pk_drink_ingredient_association")),
)
op.create_table(
"drink_price",
sa.Column("id", flaschengeist.models.Serial(), nullable=False),
sa.Column("price", sa.Numeric(precision=5, scale=2, asdecimal=False), nullable=True),
sa.Column("volume_id", flaschengeist.models.Serial(), nullable=True),
sa.Column("public", sa.Boolean(), nullable=True),
sa.Column("description", sa.String(length=30), nullable=True),
sa.ForeignKeyConstraint(
["volume_id"], ["drink_price_volume.id"], name=op.f("fk_drink_price_volume_id_drink_price_volume")
),
sa.PrimaryKeyConstraint("id", name=op.f("pk_drink_price")),
)
# ### end Alembic commands ###
def downgrade():
# ### commands auto generated by Alembic - please adjust! ###
op.drop_table("drink_price")
op.drop_table("drink_ingredient_association")
op.drop_table("drink_x_type")
op.drop_table("drink_x_tag")
op.drop_table("drink_price_volume")
op.drop_table("drink_ingredient")
op.drop_table("drink")
op.drop_table("drink_type")
op.drop_table("drink_tag")
op.drop_table("drink_extra_ingredient")
# ### end Alembic commands ###

View File

@ -7,6 +7,7 @@ _hooks_after = {}
def Hook(function=None, id=None): def Hook(function=None, id=None):
"""Hook decorator """Hook decorator
Use to decorate functions as hooks, so plugins can hook up their custom functions. Use to decorate functions as hooks, so plugins can hook up their custom functions.
""" """
# `id` passed as `arg` not `kwarg` # `id` passed as `arg` not `kwarg`
@ -38,8 +39,10 @@ def Hook(function=None, id=None):
def HookBefore(id: str): def HookBefore(id: str):
"""Decorator for functions to be called before a Hook-Function is called """Decorator for functions to be called before a Hook-Function is called
The hooked up function must accept the same arguments as the function hooked onto, The hooked up function must accept the same arguments as the function hooked onto,
as the functions are called with the same arguments. as the functions are called with the same arguments.
Hint: This enables you to modify the arguments! Hint: This enables you to modify the arguments!
""" """
if not id or not isinstance(id, str): if not id or not isinstance(id, str):
@ -54,9 +57,18 @@ def HookBefore(id: str):
def HookAfter(id: str): def HookAfter(id: str):
"""Decorator for functions to be called after a Hook-Function is called """Decorator for functions to be called after a Hook-Function is called
As with the HookBefore, the hooked up function must accept the same As with the HookBefore, the hooked up function must accept the same
arguments as the function hooked onto, but also receives a arguments as the function hooked onto, but also receives a
`hook_result` kwarg containing the result of the function. `hook_result` kwarg containing the result of the function.
Example:
```py
@HookAfter("some.id")
def my_func(hook_result):
# This function is executed after the function registered with "some.id"
print(hook_result) # This is the result of the function
```
""" """
if not id or not isinstance(id, str): if not id or not isinstance(id, str):

View File

@ -22,16 +22,16 @@ include_package_data = True
python_requires = >=3.9 python_requires = >=3.9
packages = find: packages = find:
install_requires = install_requires =
Flask >= 2.0 Flask>=2.0
Pillow>=8.4.0 Pillow>=8.4.0
flask_cors flask_cors
flask_migrate>=3.1.0
flask_sqlalchemy>=2.5 flask_sqlalchemy>=2.5
# Importlib requirement can be dropped when python requirement is >= 3.10 # Importlib requirement can be dropped when python requirement is >= 3.10
importlib_metadata>=4.3 importlib_metadata>=4.3
sqlalchemy>=1.4.26 sqlalchemy>=1.4.26
toml toml
werkzeug werkzeug >= 2.0
[options.extras_require] [options.extras_require]
argon = argon2-cffi argon = argon2-cffi
@ -42,7 +42,7 @@ mysql =
mysqlclient;platform_system!='Windows' mysqlclient;platform_system!='Windows'
[options.package_data] [options.package_data]
* = *.toml * = *.toml, script.py.mako
[options.entry_points] [options.entry_points]
console_scripts = console_scripts =