feature/migrations, closes #19 #20

Merged
crimsen merged 28 commits from feature/migrations into develop 2023-03-02 05:37:11 +00:00
5 changed files with 133 additions and 51 deletions
Showing only changes of commit 4fbd20f78e - Show all commits

View File

@ -31,18 +31,35 @@ or if you want to also run the tests:
pip3 install --user ".[ldap,tests]" pip3 install --user ".[ldap,tests]"
You will also need a MySQL driver, recommended drivers are You will also need a MySQL driver, by default one of this is installed:
- `mysqlclient` - `mysqlclient` (non Windows)
- `PyMySQL` - `PyMySQL` (on Windows)
`setup.py` will try to install a matching driver. #### Hint on MySQL driver on Windows:
If you want to use `mysqlclient` instead of `PyMySQL` (performance?) you have to follow [this guide](https://www.radishlogic.com/coding/python-3/installing-mysqldb-for-python-3-in-windows/)
#### Windows ### Install database
Same as above, but if you want to use `mysqlclient` instead of `PyMySQL` (performance?) you have to follow this guide: The user needs to have full permissions to the database.
If not you need to create user and database manually do (or similar on Windows):
https://www.radishlogic.com/coding/python-3/installing-mysqldb-for-python-3-in-windows/ (
echo "CREATE DATABASE flaschengeist;"
echo "CREATE USER 'flaschengeist'@'localhost' IDENTIFIED BY 'flaschengeist';"
echo "GRANT ALL PRIVILEGES ON flaschengeist.* TO 'flaschengeist'@'localhost';"
echo "FLUSH PRIVILEGES;"
) | sudo mysql
### Configuration Then you can install the database tables, this will update all tables from core + all enabled plugins.
*Hint:* The same command can be later used to upgrade the database after plugins or core are updated.
$ flaschengeist db upgrade heads
## Plugins
To only upgrade one plugin (for example the `events` plugin):
$ flaschengeist db upgrade events@head
## Configuration
Configuration is done within the a `flaschengeist.toml`file, you can copy the one located inside the module path Configuration is done within the a `flaschengeist.toml`file, you can copy the one located inside the module path
(where flaschegeist is installed) or create an empty one and place it inside either: (where flaschegeist is installed) or create an empty one and place it inside either:
1. `~/.config/` 1. `~/.config/`
@ -63,21 +80,6 @@ So you have to configure one of the following options to call flaschengeists CRO
- Pros: Guaranteed execution interval, no impact on user experience (at least if you do not limit wsgi worker threads) - Pros: Guaranteed execution interval, no impact on user experience (at least if you do not limit wsgi worker threads)
- Cons: Uses one of the webserver threads while executing - Cons: Uses one of the webserver threads while executing
### Database installation
The user needs to have full permissions to the database.
If not you need to create user and database manually do (or similar on Windows):
(
echo "CREATE DATABASE flaschengeist;"
echo "CREATE USER 'flaschengeist'@'localhost' IDENTIFIED BY 'flaschengeist';"
echo "GRANT ALL PRIVILEGES ON flaschengeist.* TO 'flaschengeist'@'localhost';"
echo "FLUSH PRIVILEGES;"
) | sudo mysql
Then you can install the database tables and initial entries:
$ flaschengeist install
### Run ### Run
Flaschengeist provides a CLI, based on the flask CLI, respectivly called `flaschengeist`. Flaschengeist provides a CLI, based on the flask CLI, respectivly called `flaschengeist`.

View File

@ -5,9 +5,53 @@
- your_plugin/ - your_plugin/
- __init__.py - __init__.py
- ... - ...
- migrations/ (optional)
- ... - ...
- setup.cfg - setup.cfg
The basic layout of a plugin is quite simple, you will only need the `setup.cfg` or `setup.py` and The basic layout of a plugin is quite simple, you will only need the `setup.cfg` or `setup.py` and
the package containing your plugin code, at lease a `__init__.py` file with your `Plugin` class. the package containing your plugin code, at lease a `__init__.py` file with your `Plugin` class.
If you use custom database tables you need to provide a `migrations` directory within your package,
see next section.
## Database Tables / Migrations
To allow upgrades of installed plugins, the database is versioned and handled
through [Alembic](https://alembic.sqlalchemy.org/en/latest/index.html) migrations.
Each plugin, which uses custom database tables, is represented as an other base.
So you could simply follow the Alembic tutorial on [how to work with multiple bases](https://alembic.sqlalchemy.org/en/latest/branches.html#creating-a-labeled-base-revision).
A quick overview on how to work with migrations for your plugin:
$ flaschengeist db revision -m "Create my super plugin" \
--head=base --branch-label=myplugin_name --version-path=your/plugin/migrations
This would add a new base named `myplugin_name`, which should be the same as the pypi name of you plugin.
If your tables depend on an other plugin or a specific base version you could of cause add
--depends-on=VERSION
or
--depends-on=other_plugin
### Plugin Removal and Database Tables
As generic downgrades are most often hard to write, your plugin is not required to provide such functionallity.
For Flaschengeist only instable versions provide meaningful downgrade migrations down to the latest stable version.
So this means if you do not provide downgrades you must at lease provide a series of migrations toward removal of
the database tables in case the users wants to delete the plugin.
(base) ----> 1.0 <----> 1.1 <----> 1.2
|
--> removal
After the removal step the database is stamped to to "remove" your
## Useful Hooks
There are some predefined hooks, which might get handy for you.
For more information, please refer to
- `flaschengeist.utils.hook.HookBefore` and
- `flaschengeist.utils.hook.HookAfter`

View File

@ -12,49 +12,73 @@ from werkzeug.datastructures import FileStorage
from flaschengeist.models.user import _Avatar, User from flaschengeist.models.user import _Avatar, User
from flaschengeist.utils.hook import HookBefore, HookAfter from flaschengeist.utils.hook import HookBefore, HookAfter
plugins_installed = HookAfter("plugins.installed") __all__ = [
"""Hook decorator for when all plugins are installed "plugins_installed",
Possible use case would be to populate the database with some presets. "plugins_loaded",
"before_delete_user",
"before_role_updated",
"before_update_user",
"after_role_updated",
"Plugin",
"AuthPlugin",
]
Args: # Documentation hacks, see https://github.com/mitmproxy/pdoc/issues/320
hook_result: void (kwargs) plugins_installed = HookAfter("plugins.installed")
plugins_installed.__doc__ = """Hook decorator for when all plugins are installed
Possible use case would be to populate the database with some presets.
""" """
plugins_loaded = HookAfter("plugins.loaded") plugins_loaded = HookAfter("plugins.loaded")
"""Hook decorator for when all plugins are loaded plugins_loaded.__doc__ = """Hook decorator for when all plugins are loaded
Possible use case would be to check if a specific other plugin is loaded and change own behavior Possible use case would be to check if a specific other plugin is loaded and change own behavior
Args: Passed args:
app: Current flask app instance (args) - *app:* Current flask app instance (args)
hook_result: void (kwargs)
""" """
before_role_updated = HookBefore("update_role") before_role_updated = HookBefore("update_role")
"""Hook decorator for when roles are modified before_role_updated.__doc__ = """Hook decorator for when roles are modified
Args:
role: Role object to modify Passed args:
new_name: New name if the name was changed (None if delete) - *role:* `flaschengeist.models.user.Role` to modify
- *new_name:* New name if the name was changed (*None* if delete)
""" """
after_role_updated = HookAfter("update_role") after_role_updated = HookAfter("update_role")
"""Hook decorator for when roles are modified after_role_updated.__doc__ = """Hook decorator for when roles are modified
Args:
role: Role object containing the modified role Passed args:
new_name: New name if the name was changed (None if deleted) - *role:* modified `flaschengeist.models.user.Role`
- *new_name:* New name if the name was changed (*None* if deleted)
""" """
before_update_user = HookBefore("update_user") before_update_user = HookBefore("update_user")
"""Hook decorator, when ever an user update is done, this is called before. before_update_user.__doc__ = """Hook decorator, when ever an user update is done, this is called before.
Args:
user: User object Passed args:
- *user:* `flaschengeist.models.user.User` object
""" """
before_delete_user = HookBefore("delete_user") before_delete_user = HookBefore("delete_user")
"""Hook decorator,this is called before an user gets deleted. before_delete_user.__doc__ = """Hook decorator,this is called before an user gets deleted.
Args:
user: User object Passed args:
- *user:* `flaschengeist.models.user.User` object
""" """
class Plugin: class Plugin:
"""Base class for all Plugins """Base class for all Plugins
If your class uses custom models add a static property called ``models``. All plugins must derived from this class.
Optional:
- *blueprint*: `flask.Blueprint` providing your routes
- *permissions*: List of your custom permissions
- *models*: Your models, used for API export
""" """
name: str name: str
@ -250,14 +274,14 @@ class AuthPlugin(Plugin):
""" """
raise NotFound raise NotFound
def set_avatar(self, user: User, file: FileStorage): def set_avatar(self, user, file: FileStorage):
"""Set the avatar for given user (if supported by auth backend) """Set the avatar for given user (if supported by auth backend)
Default behavior is to use native Image objects stored on the Flaschengeist server Default behavior is to use native Image objects stored on the Flaschengeist server
Args: Args:
user: User to set the avatar for user: User to set the avatar for
file: FileStorage object uploaded by the user file: `werkzeug.datastructures.FileStorage` uploaded by the user
Raises: Raises:
MethodNotAllowed: If not supported by Backend MethodNotAllowed: If not supported by Backend
Any valid HTTP exception Any valid HTTP exception

View File

@ -134,7 +134,7 @@ def get_balance(userid, current_session: Session):
Route: ``/users/<userid>/balance`` | Method: ``GET`` Route: ``/users/<userid>/balance`` | Method: ``GET``
GET-parameters: ```{from?: string, to?: string}``` GET-parameters: ``{from?: string, to?: string}``
Args: Args:
userid: Userid of user to get balance from userid: Userid of user to get balance from
@ -173,7 +173,7 @@ def get_transactions(userid, current_session: Session):
Route: ``/users/<userid>/balance/transactions`` | Method: ``GET`` Route: ``/users/<userid>/balance/transactions`` | Method: ``GET``
GET-parameters: ```{from?: string, to?: string, limit?: int, offset?: int}``` GET-parameters: ``{from?: string, to?: string, limit?: int, offset?: int}``
Args: Args:
userid: Userid of user to get transactions from userid: Userid of user to get transactions from

View File

@ -7,6 +7,7 @@ _hooks_after = {}
def Hook(function=None, id=None): def Hook(function=None, id=None):
"""Hook decorator """Hook decorator
Use to decorate functions as hooks, so plugins can hook up their custom functions. Use to decorate functions as hooks, so plugins can hook up their custom functions.
""" """
# `id` passed as `arg` not `kwarg` # `id` passed as `arg` not `kwarg`
@ -38,8 +39,10 @@ def Hook(function=None, id=None):
def HookBefore(id: str): def HookBefore(id: str):
"""Decorator for functions to be called before a Hook-Function is called """Decorator for functions to be called before a Hook-Function is called
The hooked up function must accept the same arguments as the function hooked onto, The hooked up function must accept the same arguments as the function hooked onto,
as the functions are called with the same arguments. as the functions are called with the same arguments.
Hint: This enables you to modify the arguments! Hint: This enables you to modify the arguments!
""" """
if not id or not isinstance(id, str): if not id or not isinstance(id, str):
@ -54,9 +57,18 @@ def HookBefore(id: str):
def HookAfter(id: str): def HookAfter(id: str):
"""Decorator for functions to be called after a Hook-Function is called """Decorator for functions to be called after a Hook-Function is called
As with the HookBefore, the hooked up function must accept the same As with the HookBefore, the hooked up function must accept the same
arguments as the function hooked onto, but also receives a arguments as the function hooked onto, but also receives a
`hook_result` kwarg containing the result of the function. `hook_result` kwarg containing the result of the function.
Example:
```py
@HookAfter("some.id")
def my_func(hook_result):
# This function is executed after the function registered with "some.id"
print(hook_result) # This is the result of the function
```
""" """
if not id or not isinstance(id, str): if not id or not isinstance(id, str):