Compare commits

..

28 Commits

Author SHA1 Message Date
Jann Stute 200a285de6 fix: allow set_version to fail, but don't commit in that case 2026-07-23 22:24:19 +02:00
Jann Stute 75c586db79 fix: some syntax errors had slipped through 2026-07-23 22:09:38 +02:00
Jann Stute eef0da6901 refactor: package each migration in a class 2026-07-23 21:52:26 +02:00
Jann Stute 92809cc225 refactor: move migrations to different file 2026-07-23 21:10:59 +02:00
Jann Stute a22482df9e refactor: don't require setting library_dir to create a backup 2026-07-23 20:33:38 +02:00
Jann Stute c5402e6bda refactor: don't blindly create all tables in the beginning
The only table that has been added since DB version 6 (the earliest supported version), is the versions table in DB version 101.
This commit removes the "create all tables" statement, and instead creates the versions table in the 101 migration.
See 12e074b71d.
2026-07-23 20:23:21 +02:00
Jann Stute 24f9f27e63 refactor: remove unnecessary assurance
Bumping the auto increment value has been done since the original sql PR, so it doesn't need to be done on migrations.
See e5e7b8afc6.
2026-07-23 19:37:29 +02:00
Jann Stute 0d1597d520 refactor: inline make_tables + minor cleanup 2026-07-23 19:27:48 +02:00
Jann Stute 617ff710e3 fix: backup library before making any changes 2026-07-23 19:20:40 +02:00
Travis Abendshien f24653e030 docs: update styling 2026-07-22 18:31:18 -07:00
Travis Abendshien c30dee799a docs: update Nix verbage 2026-07-22 18:30:57 -07:00
Travis Abendshien 15230cc369 fix: assign new tag ID to aliases before adding them (#1455) 2026-07-21 20:46:08 -07:00
Travis Abendshien c3b6bec2ff chore: pass ripgrep label text to constructor 2026-07-19 19:49:37 -07:00
Travis Abendshien 58496a7d2d build: add nightly icons and strings 2026-07-19 19:48:27 -07:00
Travis Abendshien dc4251ff55 chore: bump version to v9.6.2 2026-07-19 19:19:29 -07:00
purpletennisball 174262b9b3 feat(thumbs): render .ai thumbnails (#1453) 2026-07-19 19:16:39 -07:00
Xarvex 27d761731c fix(ci): setup Ruff once, invoke manually
The parallel steps can have a race condition if they both have a lock on
the final Ruff installation path.

Initially, the action was reused for formatting and linting as the
ruff-action sets the necessary environment variables and matchers,
responsible for the annotations on the GitHub UI. Though, as it turns
out, this is already done whenever Ruff is installed, there is no
"special magic" whenever execution of the process happens.

So, only use the action for setup once, and in the parallel steps we can
invoke Ruff ourselves.
2026-07-19 16:11:40 -05:00
Jann Stute 51a9c16f50 refactor: remove dead folders table (#1444)
* refactor: remove dead folders table

* fix: remove folder param from all uses of Entry constructor

* fix: missing filename column after migration

* fix: bump db version to 300, since it is a breaking change

* fix: add empty library fixture for version 202
2026-07-17 17:35:08 -07:00
Travis Abendshien 6aa0cf74f9 chore: removed unused collage_icon.py file 2026-07-12 21:08:31 -07:00
Travis Abendshien 49b450c3a4 fix: don't prevent greater minor version DBs from opening 2026-07-12 20:58:53 -07:00
Ludvig Sandh a1dfa62e4a fix: correctly display duration statistic in preview panel (#1421)
* fix: correctly display duration statistic in preview panel

* style: format file_attributes.py with ruff

* refactor: consolidate preview thumb current file state

* refactor: remove another redundant current file state

* refactor: move generic format_duration function to string formatting util file

* refactor: tighten signal's declared types
2026-07-12 13:16:14 -07:00
Xarvex aa2d9d4815 fix(ci): misapplied rebase, cont 9d5200b
fixes: #1445 (for good)
2026-07-11 19:33:29 -05:00
Xarvex 16cfa8d2ff chore(ci): ignore translations pushes for reuse 2026-07-11 19:27:21 -05:00
Xarvex 9d5200b2f2 fix(ci): path matching fixes, corrected concurrency
Pull requests are now correctly handled and concurrency is per-job.
translations branch can safely be ignored for pushes.
2026-07-11 19:19:42 -05:00
Jann Stute f252a86fd5 refactor: split out sql migrations from library (#1432)
* refactor: minor simplification

* refactor: split open_library into new and not new case

Functionality is entirely unchanged, but entire method is duplicated.
Removing dead code from either copy is next.

* refactor: remove dead code

* doc: add todos and remove trivially true assert

* refactor: add assurance 1 version check

Assurance 1 was introduced with library version 101, specifically commmit 12e074b71d.
Thus all libraries created on version 101 and above already fullfill it and don't need it.
Furthermore, it only fails if the library didn't need the assurance, meaning the try-except and warning catching can be removed

* refactor: remove various unnecessary try-except statements in new_lib

All of these were introduced to account for the differences between new libraries and existing ones,
which makes them unnecessary after this split.

* refactor: remove unnecessary check in new_lib

* refactor: move folder assurance after migrations

The folder assurance has been present since the very first SQL commit e5e7b8afc6,
and thus only has an effect when the library is moved, meaning that is semantically not part of the migrations.

* refactor: massively simplify open_library

* refactor: move engine creation to static method

* refactor: add version check for assurance 3

Assurance 3 was introduced in commit 47c3d5338f
shortly (24h 20min) after the DB version had been bumped to 200.
Since it was also included in the first release with DB version 200,
all libraries created on DB version 200 and above
can be presumed to have already had this applied on creation.

* refactor: add assurance 3 to DB 200 migration

* refactor: move assurance 1 to a proper migration method

Moving the assurance after the migrations 7, 8, 9, and 100 is fine because it doesn't affect those.

* refactor: update version after every successfull migration

Since every migration migrates the library to a certain DB_VERSION, we can simply update the library version after such a successfull migration.
This way if a migration fails, the previous migrations won't be rerun the next time the library is opened.

* refactor: rewrite migration procedure as loop

* refactor: apply migration and update version in same transaction

None of the content of the migrations is changed, only the `with session:` statements are removed.

* refactor: replace all commits in the migrations with flushes

Also removes various try-except statements in the migrations.
These were introduced to catch the case where the migration is run twice due to a later migration failing,
this is not necessary anymore as every migration will only be executed at most once per library.

* refactor: make sure the migration log statements are consistent

* fix: pass library dir to migrations

* fix(db migration 8): only add colors that are actually new

DB Migration 8 was previously adding all colors except for the neon ones,
however, all but three of those have been around since DB version 4,
meaning that they don't need to be added at all (which caused the tests to fail).

* fix: json migration used outdated interface

* fix(open_library): create TS directory only if not opened in memory

* fix: enable sane transaction behaviour

By default in SQLAlchemy schema changes are automatically committed,
even if auto-commit is turned off.
This commit turns off auto-commit completely,
which caused some schema to not be applied correctly,
but those were fixed here as well.

* refactor: hide 'argument is not accessed' notices

* fix(db migration 9): filename property wasn't written correctly

* fix(db migration 104): include/exclude list was loaded incorrectly

* feat: log start and end of DB migrations

* fix: don't use double transaction in open_sqlite_library

* fix: only commit once when creating new library

* doc: add comment on removing Folder logic

* refactor: remove dunder naming
2026-07-11 12:36:55 -07:00
Xarvex a0fb679729 fix(ci): more path matching 2026-07-11 01:29:11 -05:00
Xarvex b182b2ff7e chore(ci): more paths for pytest to trigger on 2026-07-09 22:23:17 -05:00
purpletennisball 308b36b31e fix: thumbnail rendering for older pxd files (#1441) 2026-07-09 18:03:47 -07:00
40 changed files with 1053 additions and 850 deletions
+75 -16
View File
@@ -8,26 +8,29 @@ on:
paths: &on_paths
- .github/actions/setup-python/action.yml
- .github/workflows/checks_python.yml
- src/**/resources/**
- src/**/*.json
- src/**/*.qrc
- tests/**
- .editorconfig
- pyproject.toml
- uv.lock
- '**.py'
- '**.pyi'
- 'src/tagstudio/resources/**'
- '**.pyi?'
push:
branches-ignore:
- translations
paths: *on_paths
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
defaults:
run:
shell: sh
jobs:
run-conditions:
permissions:
pull-requests: read
name: Run Conditions
runs-on: ubuntu-latest
outputs:
@@ -36,14 +39,25 @@ jobs:
ruff: ${{ steps.run-conditions.outputs.ruff }}
steps:
- name: Checkout repository
if: github.event_name != 'pull_request'
uses: actions/checkout@v7
with:
fetch-depth: 0
# Largest positive number; infinite depth.
# Using 0 would grab all branches.
# See: https://github.com/actions/checkout/issues/520
# See: https://stackoverflow.com/questions/6802145/how-to-convert-a-git-shallow-clone-to-a-full-clone/6802238#6802238
# `git fetch --unshallow` as suggested in later answers would be an extra operation.
fetch-depth: '2147483647'
- name: Check changed files
id: changed-files
uses: tj-actions/changed-files@v47.0.6
with:
fail_on_initial_diff_error: 'true'
fail_on_submodule_diff_error: 'true'
skip_initial_fetch: 'true'
# WARNING: Does not support `?` glob operand!
files_yaml: |
generic:
- .github/workflows/checks_python.yml
@@ -55,20 +69,51 @@ jobs:
- .github/actions/setup-python/action.yml
pytest:
- .github/actions/setup-python/action.yml
- 'src/tagstudio/resources/**'
- src/**/resources/**
- src/**/*.json
- src/**/*.qrc
- tests/**
ruff:
- .editorconfig
- name: Set run conditions
id: run-conditions
env:
CHANGED_GENERIC: ${{ steps.changed-files.outputs.generic_any_changed }}
CHANGED_PYRIGHT: ${{ steps.changed-files.outputs.pyright_any_changed }}
CHANGED_PYTEST: ${{ steps.changed-files.outputs.pytest_any_changed }}
CHANGED_RUFF: ${{ steps.changed-files.outputs.ruff_any_changed }}
run: |
pyright=false
pytest=false
ruff=false
if [ "${CHANGED_GENERIC}" = true ]; then
pyright=true
pytest=true
ruff=true
else
if [ "${CHANGED_PYRIGHT}" = true ]; then
pyright=true
fi
if [ "${CHANGED_PYTEST}" = true ]; then
pytest=true
fi
if [ "${CHANGED_RUFF}" = true ]; then
ruff=true
fi
fi
cat <<EOF >>"${GITHUB_OUTPUT}"
pyright=${{ steps.changed-files.outputs.generic_any_changed || steps.changed-files.outputs.pyright_any_changed }}
pytest=${{ steps.changed-files.outputs.generic_any_changed || steps.changed-files.outputs.pytest_any_changed }}
ruff=${{ steps.changed-files.outputs.generic_any_changed || steps.changed-files.outputs.ruff_any_changed }}
pyright=${pyright}
pytest=${pytest}
ruff=${ruff}
EOF
check-pyright:
concurrency:
group: pyright-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
name: Pyright
needs: run-conditions
if: needs.run-conditions.outputs.pyright == 'true'
@@ -89,6 +134,9 @@ jobs:
run: pyright
check-pytest:
concurrency:
group: ${{ matrix.os }}-pytest-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
strategy:
fail-fast: false
matrix:
@@ -151,6 +199,10 @@ jobs:
run: pytest
check-ruff:
concurrency:
group: ruff-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
name: Ruff
needs: run-conditions
if: needs.run-conditions.outputs.ruff == 'true'
@@ -158,11 +210,18 @@ jobs:
steps:
- *checkout
- name: Setup Ruff
uses: astral-sh/ruff-action@v4.0.0
with:
# No-op operation, since executing Ruff cannot be disabled for the action.
# Note that `--version` has different behavior than `version`, as the
# latter will fail if passed any extra args, even if that arg is empty.
args: --version
src: ''
- parallel:
- name: Run Ruff linter
uses: astral-sh/ruff-action@v4.0.0
run: ruff check
- name: Run Ruff formatter
uses: astral-sh/ruff-action@v4.0.0
with:
args: format --check
run: ruff format --check
+6 -1
View File
@@ -3,7 +3,12 @@
---
name: REUSE Compliance Check
on: [pull_request, push]
on:
pull_request:
push:
branches-ignore:
- translations
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
+1 -1
View File
@@ -8,7 +8,7 @@ path = [
"docs/CNAME",
"docs/assets/**",
"src/tagstudio/qt/resources.json",
"src/tagstudio/resources/icon.*",
"src/tagstudio/resources/icon*.*",
"src/tagstudio/resources/tagstudio.desktop",
"src/tagstudio/resources/templates/ts_ignore_template.txt",
"src/tagstudio/resources/templates/ts_ignore_template_blank.txt",
+1 -1
View File
@@ -118,7 +118,7 @@ If you choose to manually set up a virtual environment and install dependencies
!!! Warning "Linux Library Dependencies"
If developing TagStudio on Linux, certain libraries are required that may not be included with your distribution. A full list of these can be found [here](install.md#linux).
## Nix(OS)
## Nix & NixOS
If using [Nix](https://nixos.org/), there is a development environment already provided in the [flake](https://wiki.nixos.org/wiki/Flakes) that is accessible with the following command:
+2 -2
View File
@@ -27,12 +27,12 @@ hide:
![TagStudio screenshot](./assets/tag_bubbles.png)
**TagStudio** is a photo & file organization application with an underlying tag-based system that focuses on giving freedom and flexibility to the user. No proprietary programs or formats, no sea of sidecar files, and no complete upheaval of your filesystem structure.
<span style="font-family: Bai Jamjuree, Roboto, sans-serif; font-size: 1.1rem; letter-spacing: -0.05rem;"><span style="font-weight: 900;">Tag</span><span style="font-weight: 500;"><i>Studio</i></span></span> is a photo & file organization application with an underlying tag-based system that focuses on giving freedom and flexibility to the user. No proprietary programs or formats, no sea of sidecar files, and no complete upheaval of your filesystem structure.
</div>
<figure markdown="span">
[:material-download: Download Latest Release](https://github.com/TagStudioDev/TagStudio/releases){ .md-button .md-button--primary }
[:material-github: Download Latest Release](https://github.com/TagStudioDev/TagStudio/releases){ .md-button .md-button--primary }
</figure>
## :material-star: Core Features
+3 -3
View File
@@ -95,9 +95,9 @@ Some external dependencies are required for TagStudio to execute. Below is a tab
Aborted (core dumped)
```
### :material-nix: Nix(OS)
### :material-nix: Nix & NixOS
For [Nix(OS)](https://nixos.org/), the TagStudio repository includes a [flake](https://wiki.nixos.org/wiki/Flakes) that provides some outputs such as a development shell and package.
For [Nix](https://nixos.org/), the TagStudio repository includes a [flake](https://wiki.nixos.org/wiki/Flakes) that provides some outputs such as a development shell and package.
Two packages are provided: `tagstudio` and `tagstudio-jxl`. The distinction was made because `tagstudio-jxl` has an extra compilation step for [JPEG-XL](https://jpeg.org/jpegxl) image support. To give either of them a test run, you can execute `nix run github:TagStudioDev/TagStudio#tagstudio`. If you are in a cloned repository and wish to run a package with the context of the repository, you can simply use `nix run` with no arguments.
@@ -256,4 +256,4 @@ To generate thumbnails for RAR-based files (like `.cbr`) you'll need an extracto
### ripgrep
A recommended tool to improve the performance of directory scanning is [`ripgrep`](https://github.com/BurntSushi/ripgrep), a Rust-based directory walker that natively integrates with our [`.ts_ignore`](ignore.md) (`.gitignore`-style) pattern matching system for excluding files and directories. Ripgrep is already pre-installed on some Linux distributions and also available from several package managers.
A recommended tool to improve the performance of directory scanning is [ripgrep](https://github.com/BurntSushi/ripgrep), a Rust-based directory walker that natively integrates with our [`.ts_ignore`](ignore.md) (`.gitignore`-style) pattern matching system for excluding files and directories. Ripgrep is already pre-installed on some Linux distributions and also available from several package managers.
+69 -2
View File
@@ -3,11 +3,14 @@
* SPDX-License-Identifier: GPL-3.0-only
*/
/* Dark Theme */
[data-md-color-scheme="slate"] {
--md-primary-fg-color: rgb(197, 110, 255);
--md-accent-fg-color: rgb(92, 222, 255);
--md-default-bg-color: #060617;
--md-default-fg-color: #eae1ff;
--md-default-fg-color--light: #b898ff;
--md-code-fg-color: #eae1ffcc;
--md-default-fg-color--light: #c2a5ff;
--md-code-fg-color: #d8c7ffcc;
--md-code-hl-string-color: rgb(92, 255, 228);
--md-code-hl-keyword-color: rgb(61, 155, 255);
--md-code-hl-constant-color: rgb(205, 78, 255);
@@ -17,6 +20,8 @@
/* Light Theme */
[data-md-color-scheme="default"] {
--md-primary-fg-color: #7758ff;
--md-accent-fg-color: rgb(22, 166, 255);
--md-default-fg-color--light: #090a26;
}
@@ -73,6 +78,11 @@ td {
padding: 0.5em 1em 0.5em 1em !important;
}
hr {
border-bottom-width: 2px !important;
border-color: #9988ff50 !important;
}
.md-typeset ul li ul {
margin-top: 0;
margin-bottom: 0.1rem;
@@ -119,6 +129,18 @@ h2,
margin-right: -0.8rem;
}
.md-code__nav,
.md-content__button,
.headerlink {
border-radius: 0.2rem;
background: none;
color: #9988ff50 !important;
}
.md-code__nav:hover {
background-color: #9988ff50;
}
figcaption {
margin-top: 0 !important;
}
@@ -149,6 +171,51 @@ td code {
padding: 0 !important;
}
.admonition {
border-radius: 0.4rem !important;
border-width: 2px !important;
}
.highlight > .filename {
border-width: 2px !important;
border: solid;
border-color: #9988ff10 !important;
border-radius: 0.4rem 0.4rem 0 0 !important;
}
code {
border-radius: 0.4rem !important;
border-width: 2px !important;
border: solid;
border-color: #9988ff10;
}
:is(span, ul, li, td, p, a) > code {
border-width: 1px !important;
border-radius: 0.1rem !important;
}
.filename + pre > code {
border-top-width: 0 !important;
border-radius: 0 0 0.4rem 0.4rem !important;
}
.tabbed-labels.tabbed-labels--linked {
padding-left: 0.4rem;
padding-right: 0.4rem;
box-shadow: none !important;
}
.admonition-title {
font-size: 0.7rem;
font-family: "Bai Jamjuree", Roboto, sans-serif;
font-weight: 600 !important;
}
.admonition-title span {
margin-top: 0.05rem !important;
}
/* Matches the palette used by mkdocs-material */
.priority-high {
color: #f1185a;
+27
View File
@@ -29,3 +29,30 @@ h2 {
margin-right: 0;
}
}
.grid.cards > ul > li {
border-radius: 0.4rem !important;
border-width: 2px !important;
border-color: #9988ff20 !important;
}
.md-button--primary {
margin: 1rem;
padding: 0.3rem 1.2rem !important;
border-radius: 0.4rem !important;
font-size: 1rem;
font-family: "Bai Jamjuree", Roboto, sans-serif;
background: linear-gradient(60deg, rgb(205, 78, 255) 0%, rgb(116, 123, 255) 100%);
border-style: solid;
border-width: 0 0 2px 0 !important;
border-color: #ffffff33 !important;
}
.md-button--primary:hover {
background: linear-gradient(60deg, rgb(205, 78, 255) 30%, rgb(116, 123, 255) 100%);
border-color: #ffffff55 !important;
}
.md-button--primary span {
margin-top: 0.1rem !important;
}
+4 -4
View File
@@ -70,16 +70,16 @@ theme:
# Palette toggle for light mode
- media: "(prefers-color-scheme: light)"
scheme: default
primary: purple
accent: purple
primary: custom
accent: custom
toggle:
icon: material/lightbulb
name: Switch to Dark Mode
# Palette toggle for dark mode
- media: "(prefers-color-scheme: dark)"
scheme: slate
primary: purple
accent: purple
primary: custom
accent: custom
toggle:
icon: material/lightbulb-night-outline
name: Switch to System Preference
+1 -1
View File
@@ -9,7 +9,7 @@ build-backend = "hatchling.build"
[project]
name = "TagStudio"
description = "A User-Focused Photo & File Management System."
version = "9.6.1"
version = "9.6.2"
license = "GPL-3.0-only"
readme = "README.md"
requires-python = ">=3.12,<3.14"
+2 -2
View File
@@ -3,8 +3,8 @@
from importlib.metadata import version
VERSION: str = version("tagstudio") # Major.Minor.Patch
VERSION_BRANCH: str = "" # Usually "" or "Pre-Release"
VERSION: str = version("tagstudio")
BUILD_TYPE: str = "" # Usually "", "app.nightly", or "app.pre_release"
COPYRIGHT_YEARS: str = "2021-2026"
COPYRIGHT: str = f"© {COPYRIGHT_YEARS} Travis Abendshien & TagStudio Contributors"
COPYRIGHT_COMPACT: str = f"© {COPYRIGHT_YEARS} Travis Abendshien\n& TagStudio Contributors"
@@ -4,12 +4,17 @@
from sqlalchemy import text
from tagstudio.core.library.alchemy.fields import (
DatetimeFieldTemplate,
TextFieldTemplate,
)
SQL_FILENAME: str = "ts_library.sqlite"
JSON_FILENAME: str = "ts_library.json"
DB_VERSION_CURRENT_KEY: str = "CURRENT"
DB_VERSION_INITIAL_KEY: str = "INITIAL"
DB_VERSION: int = 202
DB_VERSION: int = 300
TAG_CHILDREN_QUERY = text("""
WITH RECURSIVE ChildTags AS (
@@ -32,3 +37,15 @@ WITH RECURSIVE ChildTags AS (
)
SELECT tag_id FROM ChildTags;
""")
DEFAULT_FIELD_TEMPLATES = (
TextFieldTemplate(name="Title"),
TextFieldTemplate(name="Author"),
TextFieldTemplate(name="Artist"),
TextFieldTemplate(name="URL"),
TextFieldTemplate(name="Description", is_multiline=True),
TextFieldTemplate(name="Notes", is_multiline=True),
TextFieldTemplate(name="Comments", is_multiline=True),
DatetimeFieldTemplate(name="Date"),
)
+1 -35
View File
@@ -6,12 +6,9 @@ from pathlib import Path
from typing import override
import structlog
from sqlalchemy import Dialect, Engine, String, TypeDecorator, create_engine, text
from sqlalchemy.exc import OperationalError
from sqlalchemy import Dialect, String, TypeDecorator
from sqlalchemy.orm import DeclarativeBase
from tagstudio.core.constants import RESERVED_TAG_END
logger = structlog.getLogger(__name__)
@@ -34,34 +31,3 @@ class PathType(TypeDecorator):
class Base(DeclarativeBase):
type_annotation_map = {Path: PathType}
def make_engine(connection_string: str) -> Engine:
return create_engine(connection_string)
def make_tables(engine: Engine) -> None:
logger.info("[Library] Creating DB tables...")
Base.metadata.create_all(engine)
# tag IDs < 1000 are reserved
# create tag and delete it to bump the autoincrement sequence
# TODO - find a better way
# is this the better way?
with engine.connect() as conn:
result = conn.execute(text("SELECT SEQ FROM sqlite_sequence WHERE name='tags'"))
autoincrement_val = result.scalar()
if not autoincrement_val or autoincrement_val <= RESERVED_TAG_END:
try:
conn.execute(
text(
"INSERT INTO tags "
"(id, name, color_namespace, color_slug, is_category, is_hidden) VALUES "
f"({RESERVED_TAG_END}, 'temp', NULL, NULL, false, false)"
)
)
conn.execute(text(f"DELETE FROM tags WHERE id = {RESERVED_TAG_END}"))
conn.commit()
except OperationalError as e:
logger.error("Could not initialize built-in tags", error=e)
conn.rollback()
+159 -554
View File
@@ -2,11 +2,6 @@
# SPDX-License-Identifier: GPL-3.0-only
# NOTE: This file contains necessary use of deprecated first-party code until that
# code is removed in a future version (prefs).
# pyright: reportDeprecated=false
import re
import shutil
import sys
@@ -18,8 +13,6 @@ from datetime import UTC, datetime
from os import makedirs
from pathlib import Path
from typing import TYPE_CHECKING
from uuid import uuid4
from warnings import catch_warnings
import sqlalchemy
import structlog
@@ -45,7 +38,7 @@ from sqlalchemy import (
update,
)
from sqlalchemy.dialects import sqlite
from sqlalchemy.exc import IntegrityError
from sqlalchemy.exc import IntegrityError, OperationalError
from sqlalchemy.orm import (
InstanceState,
Session,
@@ -73,11 +66,12 @@ from tagstudio.core.library.alchemy.constants import (
DB_VERSION,
DB_VERSION_CURRENT_KEY,
DB_VERSION_INITIAL_KEY,
DEFAULT_FIELD_TEMPLATES,
JSON_FILENAME,
SQL_FILENAME,
TAG_CHILDREN_QUERY,
)
from tagstudio.core.library.alchemy.db import make_tables
from tagstudio.core.library.alchemy.db import Base as ModelBase
from tagstudio.core.library.alchemy.enums import (
MAX_SQL_VARIABLES,
BrowsingState,
@@ -93,9 +87,9 @@ from tagstudio.core.library.alchemy.fields import (
TextFieldTemplate,
)
from tagstudio.core.library.alchemy.joins import TagEntry, TagParent
from tagstudio.core.library.alchemy.migrations import DBMigrations, MigrationError
from tagstudio.core.library.alchemy.models import (
Entry,
Folder,
Namespace,
Tag,
TagAlias,
@@ -106,7 +100,6 @@ from tagstudio.core.library.alchemy.visitors import SQLBoolExpressionBuilder
from tagstudio.core.library.ignore import migrate_ext_list
from tagstudio.core.library.json.library import Library as JsonLibrary
from tagstudio.core.utils.types import unwrap
from tagstudio.qt.translations import Translations
if TYPE_CHECKING:
from sqlalchemy import Select
@@ -172,20 +165,6 @@ def get_default_tags() -> tuple[Tag, ...]:
return archive_tag, favorite_tag, meta_tag
def get_default_field_templates() -> tuple[BaseFieldTemplate, ...]:
"""Return the default field templates for a new TagStudio library."""
title = TextFieldTemplate(name="Title")
author = TextFieldTemplate(name="Author")
artist = TextFieldTemplate(name="Artist")
url = TextFieldTemplate(name="URL")
description = TextFieldTemplate(name="Description", is_multiline=True)
notes = TextFieldTemplate(name="Notes", is_multiline=True)
comments = TextFieldTemplate(name="Comments", is_multiline=True)
date = DatetimeFieldTemplate(name="Date")
return title, author, artist, url, description, notes, comments, date
# The difference in the number of default JSON tags vs default tags in the current version.
DEFAULT_TAG_DIFF: int = len(get_default_tags()) - len([TAG_ARCHIVED, TAG_FAVORITE])
@@ -234,7 +213,6 @@ class Library:
library_dir: Path | None = None
engine: Engine | None = None
folder: Folder | None = None
included_files: set[Path] = set()
def __init__(self) -> None:
@@ -259,7 +237,6 @@ class Library:
"""Migrate JSON library data to the SQLite database."""
logger.info("Starting Library Conversion...")
start_time = time.time()
folder: Folder = Folder(path=self.library_dir, uuid=str(uuid4()))
# Tags
for tag in json_lib.tags:
@@ -312,7 +289,6 @@ class Library:
[
Entry(
path=entry.path / entry.filename,
folder=folder,
fields=[],
id=entry.id + 1, # NOTE: JSON IDs start at 0 instead of 1
date_added=datetime.now(),
@@ -381,20 +357,19 @@ class Library:
return tag.name
def open_library(self, library_dir: Path, in_memory: bool = False) -> LibraryStatus:
"""Wrapper for open_sqlite_library.
"""Wrapper for open_sqlite_library and create_sqlite_library.
Handles in-memory storage and checks whether a JSON-migration is necessary.
"""
assert isinstance(library_dir, Path)
if in_memory:
return self.open_sqlite_library(library_dir, is_new=True, storage_path=":memory:")
is_new = True
sql_path = library_dir / TS_FOLDER_NAME / SQL_FILENAME
if self.verify_ts_folder(library_dir) and (is_new := not sql_path.exists()):
json_path = library_dir / TS_FOLDER_NAME / JSON_FILENAME
if json_path.exists():
json_path = library_dir / TS_FOLDER_NAME / JSON_FILENAME
is_new = not sql_path.exists()
if not in_memory:
self.verify_ts_folder(library_dir) # ensure .TagStudio directory exists
if is_new and json_path.exists():
return LibraryStatus(
success=False,
library_path=library_dir,
@@ -402,14 +377,18 @@ class Library:
json_migration_req=True,
)
return self.open_sqlite_library(library_dir, is_new, str(sql_path))
if is_new:
return self.create_sqlite_library(library_dir, in_memory)
def open_sqlite_library(
self, library_dir: Path, is_new: bool, storage_path: str
) -> LibraryStatus:
return self.open_sqlite_library(library_dir, in_memory)
@staticmethod
def __get_engine(library_dir: Path, in_memory: bool, sql_filename: str):
connection_string = URL.create(
drivername="sqlite",
database=storage_path,
database=(
":memory:" if in_memory else str(library_dir / TS_FOLDER_NAME / sql_filename)
),
)
# NOTE: File-based databases should use NullPool to create new DB connection in order to
# keep connections on separate threads, which prevents the DB files from being locked
@@ -418,169 +397,92 @@ class Library:
# More info can be found on the SQLAlchemy docs:
# https://docs.sqlalchemy.org/en/20/changelog/migration_07.html
# Under -> sqlite-the-sqlite-dialect-now-uses-nullpool-for-file-based-databases
poolclass = None if storage_path == ":memory:" else NullPool
loaded_db_version: int = 0
initial_db_version: int = DB_VERSION
poolclass = None if in_memory else NullPool
logger.info(
"[Library] Creating SQLAlchemy Engine",
connection_string=connection_string,
poolclass=poolclass,
)
return create_engine(
connection_string, poolclass=poolclass, connect_args={"autocommit": False}
)
def create_sqlite_library(
self, library_dir: Path, in_memory: bool, sql_filename: str = SQL_FILENAME
) -> LibraryStatus:
self.engine = self.__get_engine(library_dir, in_memory, sql_filename)
logger.info(
"[Library] Opening SQLite Library",
library_dir=library_dir,
connection_string=connection_string,
)
self.engine = create_engine(connection_string, poolclass=poolclass)
with Session(self.engine) as session:
# Don't check DB version when creating new library
if not is_new:
loaded_db_version = self.get_version(DB_VERSION_CURRENT_KEY)
initial_db_version = self.get_version(DB_VERSION_INITIAL_KEY)
# ======================== Library Database Version Checking =======================
# DB_VERSION 6 is the first supported SQLite DB version.
# If the DB_VERSION is >= 100, that means it's a compound major + minor version.
# - Dividing by 100 and flooring gives the major (breaking changes) version.
# - If a DB has major version higher than the current program, don't load it.
# - If only the minor version is higher, it's still allowed to load.
if loaded_db_version < 6 or (
loaded_db_version >= 100 and loaded_db_version // 100 > DB_VERSION // 100
):
mismatch_text = Translations["status.library_version_mismatch"]
found_text = Translations["status.library_version_found"]
expected_text = Translations["status.library_version_expected"]
return LibraryStatus(
success=False,
message=(
f"{mismatch_text}\n"
f"{found_text} v{loaded_db_version}, "
f"{expected_text} v{DB_VERSION}"
),
logger.info("[Library] Creating DB tables...")
with self.engine.connect() as conn:
ModelBase.metadata.create_all(conn)
conn.commit()
# TODO - find a better way
# is this the better way?
# Could we perhaps update the row we are reading from here?
result = conn.execute(text("SELECT SEQ FROM sqlite_sequence WHERE name='tags'"))
autoincrement_val = result.scalar()
if not autoincrement_val or autoincrement_val <= RESERVED_TAG_END:
try:
conn.execute(
text(
"INSERT INTO tags "
"(id, name, color_namespace, color_slug, is_category, is_hidden) "
f"VALUES ({RESERVED_TAG_END}, 'temp', NULL, NULL, false, false)"
)
)
conn.execute(text(f"DELETE FROM tags WHERE id = {RESERVED_TAG_END}"))
conn.commit()
except OperationalError as e:
logger.error("Could not initialize built-in tags", error=e)
conn.rollback()
logger.info(f"[Library] Library DB version: {loaded_db_version}")
make_tables(self.engine)
with Session(self.engine) as session:
# Add default tag color namespaces.
namespaces = default_color_groups.namespaces()
if is_new:
# Add default tag color namespaces.
namespaces = default_color_groups.namespaces()
try:
session.add_all(namespaces)
session.commit()
except IntegrityError as e:
logger.error("[Library] Couldn't add default tag color namespaces", error=e)
session.rollback()
session.add_all(namespaces)
session.flush()
# Add default tag colors.
tag_colors: list[TagColorGroup] = default_color_groups.standard()
tag_colors += default_color_groups.pastels()
tag_colors += default_color_groups.shades()
tag_colors += default_color_groups.grayscale()
tag_colors += default_color_groups.earth_tones()
tag_colors += default_color_groups.neon()
if is_new:
try:
session.add_all(tag_colors)
session.commit()
except IntegrityError as e:
logger.error("[Library] Couldn't add default tag colors", error=e)
session.rollback()
# Add default tag colors.
tag_colors: list[TagColorGroup] = default_color_groups.standard()
tag_colors += default_color_groups.pastels()
tag_colors += default_color_groups.shades()
tag_colors += default_color_groups.grayscale()
tag_colors += default_color_groups.earth_tones()
tag_colors += default_color_groups.neon()
# Add default tags.
tags = get_default_tags()
try:
session.add_all(tags)
session.commit()
except IntegrityError:
session.rollback()
session.add_all(tag_colors)
session.flush()
# Add default tags.
session.add_all(get_default_tags())
session.flush()
# Add default field templates
if is_new:
for template in get_default_field_templates():
try:
session.add(template)
session.commit()
except IntegrityError:
logger.info(
"[Library] FieldTemplate already exists", field_template=template
)
session.rollback()
for template in DEFAULT_FIELD_TEMPLATES:
session.add(template)
session.flush()
# Ensure version rows are present
with catch_warnings(record=True):
try:
initial = DB_VERSION if is_new else 100
session.add(Version(key=DB_VERSION_INITIAL_KEY, value=initial))
session.commit()
except IntegrityError:
session.rollback()
try:
session.add(Version(key=DB_VERSION_CURRENT_KEY, value=DB_VERSION))
session.commit()
except IntegrityError:
session.rollback()
# check if folder matching current path exists already
self.folder = session.scalar(select(Folder).where(Folder.path == library_dir))
if not self.folder:
folder = Folder(
path=library_dir,
uuid=str(uuid4()),
)
session.add(folder)
session.expunge(folder)
session.commit()
self.folder = folder
session.add(Version(key=DB_VERSION_INITIAL_KEY, value=DB_VERSION))
session.add(Version(key=DB_VERSION_CURRENT_KEY, value=DB_VERSION))
session.flush()
# Generate default .ts_ignore file
if is_new:
try:
ts_ignore_template = (
Path(__file__).parents[3] / "resources/templates/ts_ignore_template.txt"
)
shutil.copy2(ts_ignore_template, library_dir / TS_FOLDER_NAME / IGNORE_NAME)
except Exception as e:
logger.error("[ERROR][Library] Could not generate '.ts_ignore' file!", error=e)
# Apply any post-SQL migration patches.
if not is_new:
assert loaded_db_version >= 6
# save backup if patches will be applied
if loaded_db_version < DB_VERSION:
self.library_dir = library_dir
self.save_library_backup_to_disk()
self.library_dir = None
# migrate DB step by step from one version to the next
if loaded_db_version < 7:
# changes: value_type, tags
self.__apply_db7_migration(session)
if loaded_db_version < 8:
# changes: tag_colors
self.__apply_db8_migration(session)
if loaded_db_version < 9:
# changes: entries
self.__apply_db9_migration(session)
if loaded_db_version < 100:
# changes: tag_parents
self.__apply_db100_migration(session)
if loaded_db_version < 102:
# changes: tag_parents
self.__apply_db102_migration(session)
if loaded_db_version < 103:
# changes: tags
self.__apply_db103_migration(session)
if loaded_db_version < 104:
# changes: deletes preferences
self.__apply_db104_migration(session, library_dir)
if loaded_db_version < 200:
# changes: field tables
self.__apply_db200_migration(session)
if initial_db_version < 200 and loaded_db_version < 201:
# changes: field tables
self.__apply_db201_migration(session)
if loaded_db_version < 202:
# changes: tag_parents
self.__apply_db202_migration(session)
try:
ts_ignore_template = (
Path(__file__).parents[3] / "resources/templates/ts_ignore_template.txt"
)
shutil.copy2(ts_ignore_template, library_dir / TS_FOLDER_NAME / IGNORE_NAME)
except Exception as e:
logger.error("[ERROR][Library] Could not generate '.ts_ignore' file!", error=e)
session.execute(
text("CREATE INDEX IF NOT EXISTS idx_tags_name_shorthand ON tags (name, shorthand)")
@@ -596,337 +498,33 @@ class Library:
)
)
# Update DB_VERSION
if loaded_db_version < DB_VERSION:
logger.info(f"[Library] Library migrated to DB version {DB_VERSION}")
self.set_version(DB_VERSION_CURRENT_KEY, DB_VERSION)
session.commit()
# everything is fine, set the library path
self.library_dir = library_dir
return LibraryStatus(success=True, library_path=library_dir)
def __apply_db7_migration(self, session: Session):
"""Migrate DB from DB_VERSION 6 to 7."""
logger.info("[Library][Migration] Applying patches to DB_VERSION: 6 library...")
with session:
# Repair tags that may have a disambiguation_id pointing towards a deleted tag.
all_tag_ids = session.scalars(text("SELECT DISTINCT id FROM tags")).all()
disam_stmt = (
update(Tag)
.where(Tag.disambiguation_id.not_in(all_tag_ids))
.values(disambiguation_id=None)
)
session.execute(disam_stmt)
session.commit()
def open_sqlite_library(
self, library_dir: Path, in_memory: bool, sql_filename: str = SQL_FILENAME
) -> LibraryStatus:
logger.info("[Library] Opening SQLite Library", library_dir=library_dir)
self.engine = self.__get_engine(library_dir, in_memory, sql_filename)
def __apply_db8_migration(self, session: Session):
"""Migrate DB from DB_VERSION 7 to 8."""
# Add the missing color_border column to the TagColorGroups table.
color_border_stmt = text(
"ALTER TABLE tag_colors ADD COLUMN color_border BOOLEAN DEFAULT FALSE NOT NULL"
)
try:
session.execute(color_border_stmt)
session.commit()
logger.info("[Library][Migration] Added color_border column to tag_colors table")
except Exception as e:
logger.error(
"[Library][Migration] Could not create color_border column in tag_colors table!",
error=e,
)
session.rollback()
migrations = DBMigrations(library_dir, self.engine)
# collect new default tag colors
tag_colors: list[TagColorGroup] = default_color_groups.standard()
tag_colors += default_color_groups.pastels()
tag_colors += default_color_groups.shades()
tag_colors += default_color_groups.grayscale()
tag_colors += default_color_groups.earth_tones()
# tag_colors += default_color_groups.neon() # NOTE: Neon is handled separately
# save backup if patches will be applied
if migrations.required:
Library.save_library_backup_to_disk(library_dir)
# Add any new default colors introduced in DB_VERSION 8
for color in tag_colors:
try:
session.add(color)
logger.info(
"[Library][Migration] Migrated tag color to DB_VERSION 8+",
color_name=color.name,
)
session.commit()
except IntegrityError:
session.rollback()
migrations.run()
except MigrationError as e:
return LibraryStatus(success=False, message=e.args[0])
# Update Neon colors to use the the color_border property
for color in default_color_groups.neon():
try:
neon_stmt = (
update(TagColorGroup)
.where(
and_(
TagColorGroup.namespace == color.namespace,
TagColorGroup.slug == color.slug,
)
)
.values(
slug=color.slug,
namespace=color.namespace,
name=color.name,
primary=color.primary,
secondary=color.secondary,
color_border=color.color_border,
)
)
session.execute(neon_stmt)
session.commit()
except IntegrityError as e:
logger.error(
"[Library] Could not migrate Neon colors to DB_VERSION 8+!",
error=e,
)
session.rollback()
def __apply_db9_migration(self, session: Session):
"""Migrate DB from DB_VERSION 8 to 9."""
# Apply database schema changes
add_filename_column = text(
"ALTER TABLE entries ADD COLUMN filename TEXT NOT NULL DEFAULT ''"
)
try:
session.execute(add_filename_column)
session.commit()
logger.info("[Library][Migration] Added filename column to entries table")
except Exception as e:
logger.error(
"[Library][Migration] Could not create filename column in entries table!",
error=e,
)
session.rollback()
# Populate the new filename column.
for entry in self.all_entries():
session.merge(entry).filename = entry.path.name
session.commit()
logger.info("[Library][Migration] Populated filename column in entries table")
def __apply_db100_migration(self, session: Session):
"""Migrate DB to DB_VERSION 100."""
with session:
# Repair parent-child tag relationships that are the wrong way around.
stmt = update(TagParent).values(
parent_id=TagParent.child_id,
child_id=TagParent.parent_id,
)
session.execute(stmt)
session.commit()
logger.info("[Library][Migration] Refactored TagParent table")
def __apply_db102_migration(self, session: Session):
"""Migrate DB to DB_VERSION 102."""
with session:
stmt = delete(TagParent).where(TagParent.parent_id.not_in(select(Tag.id).distinct()))
session.execute(stmt)
session.commit()
logger.info("[Library][Migration] Verified TagParent table data")
def __apply_db103_migration(self, session: Session):
"""Migrate DB from DB_VERSION 102 to 103."""
# add the new hidden column for tags
add_is_hidden_column = text(
"ALTER TABLE tags ADD COLUMN is_hidden BOOLEAN NOT NULL DEFAULT 0"
)
try:
session.execute(add_is_hidden_column)
session.commit()
logger.info("[Library][Migration] Added is_hidden column to tags table")
except Exception as e:
logger.error(
"[Library][Migration] Could not create is_hidden column in tags table!",
error=e,
)
session.rollback()
# mark the "Archived" tag as hidden
try:
session.query(Tag).filter(Tag.id == TAG_ARCHIVED).update({"is_hidden": True})
session.commit()
logger.info("[Library][Migration] Updated archived tag to be hidden")
except Exception as e:
logger.error(
"[Library][Migration] Could not update archived tag to be hidden!",
error=e,
)
session.rollback()
def __apply_db104_migration(self, session: Session, library_dir: Path):
"""Migrate DB from DB_VERSION 103 to 104."""
# Convert file extension list to ts_ignore file, if a .ts_ignore file does not exist
self.__migrate_sql_to_ts_ignore(library_dir)
session.execute(text("DROP TABLE preferences"))
session.commit()
def __migrate_sql_to_ts_ignore(self, library_dir: Path):
# Do not continue if existing '.ts_ignore' file is found
ts_ignore = library_dir / TS_FOLDER_NAME / IGNORE_NAME
if Path(ts_ignore).exists():
return
# Load legacy extension data
with Session(self.engine) as session:
extensions: list[str] = unwrap(
session.scalar(text("SELECT value FROM preferences WHERE key = 'EXTENSION_LIST'"))
)
is_exclude_list: bool = unwrap(
session.scalar(text("SELECT value FROM preferences WHERE key = 'IS_EXCLUDE_LIST'"))
)
with open(ts_ignore, "w") as f:
f.write(migrate_ext_list(extensions, is_exclude_list))
def __apply_db200_migration(self, session: Session):
"""Migrate DB to DB_VERSION 200."""
with session:
# Drop unused 'boolean_fields' and 'value_type' tables
logger.info(
"[Library][Migration][200] Dropping boolean_fields and value_type tables..."
)
session.execute(text("DROP TABLE boolean_fields"))
session.execute(text("DROP TABLE value_type"))
# Add 'name' column to text_fields and datetime_fields tables
logger.info("[Library][Migration][200] Adding name columns to field tables...")
stmt = text('ALTER TABLE text_fields ADD COLUMN name VARCHAR DEFAULT ""')
session.execute(stmt)
stmt = text('ALTER TABLE datetime_fields ADD COLUMN name VARCHAR DEFAULT ""')
session.execute(stmt)
# Drop unnecessary 'position' columns
logger.info("[Library][Migration][200] Dropping position columns to field tables...")
session.execute(text("ALTER TABLE datetime_fields DROP COLUMN position"))
session.execute(text("ALTER TABLE text_fields DROP COLUMN position"))
# Add 'is_multiline' column to text_fields table
logger.info("[Library][Migration][200] Adding is_multiline column to text_fields...")
stmt = text(
"ALTER TABLE text_fields ADD COLUMN is_multiline BOOLEAN NOT NULL DEFAULT 0"
)
session.execute(stmt)
session.flush()
# Move values from old `type_key` columns into new `name` columns
logger.info("[Library][Migration][200] Moving values from type_key columns to name...")
session.execute(text("UPDATE text_fields SET name = type_key"))
session.execute(text("UPDATE datetime_fields SET name = type_key"))
session.flush()
# Change `name` values to title case
logger.info("[Library][Migration][200] Normalizing TextField names...")
for text_field in session.execute(select(TextField)).scalars():
# NOTE: The only exception to the "Title Case" conversion is the "URL" field.
text_field.name = text_field.name.title().replace("Url", "URL").replace("_", " ")
logger.info("[Library][Migration][200] Normalizing DatetimeField names...")
for datetime_field in session.execute(select(DatetimeField)).scalars():
datetime_field.name = datetime_field.name.title().replace("_", " ")
session.flush()
# Add correct `is_multiline` values to text_fields table
logger.info("[Library][Migration][200] Updating is_multiline for legacy TEXT_BOXes...")
text_boxes = [
x.get("name") for x in LEGACY_FIELD_MAP.values() if x.get("is_multiline") is True
]
update_stmt = (
update(TextField).where(TextField.name.in_(text_boxes)).values(is_multiline=True)
)
session.execute(update_stmt)
session.flush()
# Repair legacy "Description" fields to use is_multiline = True
logger.info("[Library][Migration][200] Repairing legacy Description fields...")
desc_stmt = (
update(TextField)
.where(TextField.name == "Description" and TextField.is_multiline == False) # noqa: E712
.values(is_multiline=True)
)
session.execute(desc_stmt)
# Repair legacy "Comments" fields to use is_multiline = True
logger.info("[Library][Migration][200] Repairing legacy Comment fields...")
comm_stmt = (
update(TextField)
.where(TextField.name == "Comments" and TextField.is_multiline == False) # noqa: E712
.values(is_multiline=True)
)
session.execute(comm_stmt)
# Add default field templates
logger.info("[Library][Migration][200] Adding default field templates...")
for template in get_default_field_templates():
try:
session.add(template)
session.flush()
except IntegrityError:
logger.error("[Library] FieldTemplate already exists", field_template=template)
session.rollback()
session.commit()
def __apply_db201_migration(self, session: Session):
"""Migrate DB to DB_VERSION 201."""
with session:
create_text_fields_table = text("""
CREATE TABLE text_fields_new (
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
name VARCHAR NOT NULL,
entry_id INTEGER NOT NULL,
value VARCHAR,
is_multiline BOOLEAN NOT NULL,
FOREIGN KEY(entry_id) REFERENCES entries (id)
)
""")
create_datetime_fields_table = text("""
CREATE TABLE datetime_fields_new (
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
name VARCHAR NOT NULL,
entry_id INTEGER NOT NULL,
value VARCHAR,
FOREIGN KEY(entry_id) REFERENCES entries (id)
)
""")
logger.info("[Library][Migration][201] Dropping type_key from text_fields table...")
session.execute(create_text_fields_table)
session.flush()
session.execute(
text("""
INSERT INTO text_fields_new (id, name, entry_id, value, is_multiline)
SELECT id, name, entry_id, value, is_multiline
FROM text_fields
""")
)
session.execute(text("DROP TABLE text_fields"))
session.execute(text("ALTER TABLE text_fields_new RENAME TO text_fields"))
logger.info("[Library][Migration][201] Dropping type_key from datetime_fields table...")
session.execute(create_datetime_fields_table)
session.flush()
session.execute(
text("""
INSERT INTO datetime_fields_new (id, name, entry_id, value)
SELECT id, name, entry_id, value
FROM datetime_fields
""")
)
session.execute(text("DROP TABLE datetime_fields"))
session.execute(text("ALTER TABLE datetime_fields_new RENAME TO datetime_fields"))
session.commit()
def __apply_db202_migration(self, session: Session):
"""Migrate DB to DB_VERSION 202."""
with session:
stmt = delete(TagParent).where(TagParent.child_id.not_in(select(Tag.id).distinct()))
session.execute(stmt)
session.commit()
logger.info("[Library][Migration] Verified TagParent table data")
# everything is fine, set the library path
self.library_dir = library_dir
return LibraryStatus(success=True, library_path=library_dir)
@property
def field_templates(self) -> Sequence[BaseFieldTemplate]:
@@ -1075,32 +673,37 @@ class Library:
with Session(self.engine) as session:
return unwrap(session.scalar(select(func.count(Entry.id))))
@staticmethod
def _all_entries(session: Session, with_joins: bool = False) -> Iterator[Entry]:
"""Load entries without joins."""
stmt = select(Entry)
if with_joins:
# load Entry with all joins and all tags
stmt = (
stmt.outerjoin(Entry.text_fields)
.outerjoin(Entry.datetime_fields)
.outerjoin(Entry.tags)
)
stmt = stmt.options(
contains_eager(Entry.text_fields),
contains_eager(Entry.datetime_fields),
contains_eager(Entry.tags),
)
stmt = stmt.distinct()
entries = session.execute(stmt).scalars()
if with_joins:
entries = entries.unique()
for entry in entries:
yield entry
session.expunge(entry)
def all_entries(self, with_joins: bool = False) -> Iterator[Entry]:
"""Load entries without joins."""
with Session(self.engine) as session:
stmt = select(Entry)
if with_joins:
# load Entry with all joins and all tags
stmt = (
stmt.outerjoin(Entry.text_fields)
.outerjoin(Entry.datetime_fields)
.outerjoin(Entry.tags)
)
stmt = stmt.options(
contains_eager(Entry.text_fields),
contains_eager(Entry.datetime_fields),
contains_eager(Entry.tags),
)
stmt = stmt.distinct()
entries = session.execute(stmt).scalars()
if with_joins:
entries = entries.unique()
for entry in entries:
yield entry
session.expunge(entry)
return Library._all_entries(session, with_joins)
@property
def tags(self) -> list[Tag]:
@@ -1128,11 +731,12 @@ class Library:
raise ValueError("Invalid library directory.")
full_ts_path = library_dir / TS_FOLDER_NAME
if not full_ts_path.exists():
logger.info("creating library directory", dir=full_ts_path)
full_ts_path.mkdir(parents=True, exist_ok=True)
return False
return True
if full_ts_path.exists():
return True
logger.info("creating library directory", dir=full_ts_path)
full_ts_path.mkdir(parents=True, exist_ok=True)
return False
def add_entries(self, items: list[Entry]) -> list[int]:
"""Add multiple Entry records to the Library."""
@@ -1737,6 +1341,8 @@ class Library:
session.flush()
if aliases is not None:
for a in aliases:
a.tag_id = tag.id
self.update_aliases(tag, aliases, session)
session.flush()
@@ -1845,16 +1451,17 @@ class Library:
session.rollback()
return None
def save_library_backup_to_disk(self) -> Path:
assert isinstance(self.library_dir, Path)
makedirs(str(self.library_dir / TS_FOLDER_NAME / BACKUP_FOLDER_NAME), exist_ok=True)
@staticmethod
def save_library_backup_to_disk(library_dir: Path) -> Path:
assert isinstance(library_dir, Path)
makedirs(str(library_dir / TS_FOLDER_NAME / BACKUP_FOLDER_NAME), exist_ok=True)
filename = f"ts_library_backup_{datetime.now(UTC).strftime('%Y_%m_%d_%H%M%S')}.sqlite"
target_path = self.library_dir / TS_FOLDER_NAME / BACKUP_FOLDER_NAME / filename
target_path = library_dir / TS_FOLDER_NAME / BACKUP_FOLDER_NAME / filename
shutil.copy2(
self.library_dir / TS_FOLDER_NAME / SQL_FILENAME,
library_dir / TS_FOLDER_NAME / SQL_FILENAME,
target_path,
)
@@ -2146,8 +1753,12 @@ class Library:
Args:
key(str): The key for the name of the version type to set.
"""
with Session(self.engine) as session:
engine = sqlalchemy.inspect(self.engine)
return Library._get_version(self.engine, key)
@staticmethod
def _get_version(engine, key: str) -> int:
with Session(engine) as session:
engine = sqlalchemy.inspect(engine)
try:
# "Version" table added in DB_VERSION 101
if engine and engine.has_table("versions"):
@@ -2167,23 +1778,17 @@ class Library:
except Exception:
return 0
def set_version(self, key: str, value: int) -> None:
@staticmethod
def _set_version(session: Session, key: str, value: int) -> None:
"""Set a version value to the DB.
Args:
session(Session): The SQLAlchemy DB Session to use.
key(str): The key for the name of the version type to set.
value(int): The version value to set.
"""
with Session(self.engine) as session:
try:
version = session.scalar(select(Version).where(Version.key == key))
assert version
version.value = value
session.add(version)
session.commit()
except (IntegrityError, AssertionError) as e:
logger.error("[Library][ERROR] Couldn't add default tag color namespaces", error=e)
session.rollback()
# Insert if key has no value yet, otherwise update the value
session.merge(Version(key=key, value=value))
def mirror_entry_fields(self, entries: list[Entry]) -> None:
"""Mirror fields among multiple Entry items."""
@@ -0,0 +1,549 @@
# SPDX-FileCopyrightText: (c) TagStudio Contributors
# SPDX-License-Identifier: MIT
from collections.abc import Callable
from pathlib import Path
import structlog
import ujson
from sqlalchemy import (
Engine,
and_,
delete,
select,
text,
update,
)
from sqlalchemy.orm import (
Session,
)
from tagstudio.core.constants import (
IGNORE_NAME,
TAG_ARCHIVED,
TS_FOLDER_NAME,
)
from tagstudio.core.library.alchemy import default_color_groups
from tagstudio.core.library.alchemy.constants import (
DB_VERSION,
DB_VERSION_CURRENT_KEY,
DB_VERSION_INITIAL_KEY,
DEFAULT_FIELD_TEMPLATES,
)
from tagstudio.core.library.alchemy.fields import (
LEGACY_FIELD_MAP,
DatetimeField,
TextField,
)
from tagstudio.core.library.alchemy.joins import TagParent
from tagstudio.core.library.alchemy.models import (
Tag,
TagColorGroup,
Version,
)
from tagstudio.core.library.ignore import migrate_ext_list
from tagstudio.core.utils.types import unwrap
from tagstudio.qt.translations import Translations
logger = structlog.get_logger(__name__)
class MigrationError(Exception):
pass
class DBMigration:
version: int = None # pyright: ignore[reportAssignmentType]
initial_version: int | None = None
@classmethod
def run(cls, session: Session, library_dir: Path, fmt_log: Callable[[str], str]):
raise NotImplementedError
class DBMigrations:
def __init__(self, library_dir: Path, engine: Engine) -> None:
from tagstudio.core.library.alchemy.library import Library
self.library_dir = library_dir
self.engine = engine
# Don't check DB version when creating new library
self.loaded_db_version = Library._get_version(engine, DB_VERSION_CURRENT_KEY)
self.initial_db_version = Library._get_version(engine, DB_VERSION_INITIAL_KEY)
# ======================== Library Database Version Checking =======================
# DB_VERSION 6 is the first supported SQLite DB version.
# If the DB_VERSION is >= 100, that means it's a compound major + minor version.
# - Dividing by 100 and flooring gives the major (breaking changes) version.
# - If a DB has major version higher than the current program, don't load it.
# - If only the minor version is higher, it's still allowed to load.
if self.loaded_db_version < 6 or (
self.loaded_db_version >= 100 and self.loaded_db_version // 100 > DB_VERSION // 100
):
mismatch_text = Translations["status.library_version_mismatch"]
found_text = Translations["status.library_version_found"]
expected_text = Translations["status.library_version_expected"]
raise MigrationError(
f"{mismatch_text}\n"
f"{found_text} v{self.loaded_db_version}, "
f"{expected_text} v{DB_VERSION}"
)
logger.info(
f"[Library][Migration] Starting with library DB version: {self.loaded_db_version}"
)
@property
def required(self) -> bool:
return self.loaded_db_version < DB_VERSION
def run(self):
from tagstudio.core.library.alchemy.library import Library
# migrate DB step by step from one version to the next
# (migration_method, db_version, initial_db_version)
migrations: list[type[DBMigration]] = [
MigrationTo7, # changes: value_type, tags
MigrationTo8, # changes: tag_colors
MigrationTo9, # changes: entries
MigrationTo100, # changes: tag_parents
MigrationTo101, # changes: versions
MigrationTo102, # changes: tag_parents
MigrationTo103, # changes: tags
MigrationTo104, # changes: deletes preferences
MigrationTo200, # changes: field tables
MigrationTo201, # changes: field tables
MigrationTo202, # changes: tag_parents
MigrationTo300, # changes: deletes folders
]
with Session(self.engine) as session:
for migration in migrations:
if self.loaded_db_version < migration.version and (
migration.initial_version is None
or self.initial_db_version < migration.initial_version
):
logger.info(f"[Library][Migration][{migration.version}] Starting DB Migration")
# any error causes transaction to rollback
migration.run(
session,
self.library_dir,
lambda msg, v=migration.version: f"[Library][Migration][{v}] {msg}",
)
self.loaded_db_version = migration.version
try:
Library._set_version(session, DB_VERSION_CURRENT_KEY, migration.version)
except Exception as e:
logger.info(
f"[Library][Migration][{migration.version}] "
"Couldn't update version, continuing without commit",
error=e,
)
session.flush()
else:
session.commit()
logger.info(f"[Library][Migration][{migration.version}] Completed DB Migration")
assert self.loaded_db_version >= DB_VERSION, (
"Ran all migrations, but the DB is still not on the newest version"
)
logger.info(f"[Library][Migration] Library migrated to DB version {DB_VERSION}")
class MigrationTo7(DBMigration):
version = 7
@classmethod
def run(cls, session: Session, library_dir: Path, fmt_log):
"""Migrate DB from DB_VERSION 6 to 7."""
logger.info(fmt_log("Applying patches to DB_VERSION: 6 library..."))
# Repair tags that may have a disambiguation_id pointing towards a deleted tag.
# TODO: combine into single sql statement
all_tag_ids = session.scalars(text("SELECT DISTINCT id FROM tags")).all()
disam_stmt = (
update(Tag)
.where(Tag.disambiguation_id.not_in(all_tag_ids))
.values(disambiguation_id=None)
)
session.execute(disam_stmt)
session.flush()
class MigrationTo8(DBMigration):
version = 8
@classmethod
def run(cls, session: Session, library_dir: Path, fmt_log):
"""Migrate DB from DB_VERSION 7 to 8."""
# Add the missing color_border column to the TagColorGroups table.
session.execute(
text("ALTER TABLE tag_colors ADD COLUMN color_border BOOLEAN DEFAULT FALSE NOT NULL")
)
session.flush()
logger.info(fmt_log("Added color_border column to tag_colors table"))
# collect new default tag colors
tag_colors: list[TagColorGroup] = [
color
for color in default_color_groups.shades()
if color.slug in ["burgundy", "dark-teal", "dark_lavender"]
]
# Add any new default colors introduced in DB_VERSION 8
for color in tag_colors:
session.add(color)
session.flush()
logger.info(
fmt_log("Migrated tag colors to DB_VERSION 8+"),
color_name=tag_colors,
)
# Update Neon colors to use the the color_border property
for color in default_color_groups.neon():
neon_stmt = (
update(TagColorGroup)
.where(
and_(
TagColorGroup.namespace == color.namespace,
TagColorGroup.slug == color.slug,
)
)
.values(
slug=color.slug,
namespace=color.namespace,
name=color.name,
primary=color.primary,
secondary=color.secondary,
color_border=color.color_border,
)
)
session.execute(neon_stmt)
session.flush()
class MigrationTo9(DBMigration):
version = 9
@classmethod
def run(cls, session: Session, library_dir: Path, fmt_log):
"""Migrate DB from DB_VERSION 8 to 9."""
# Apply database schema changes
add_filename_column = text(
"ALTER TABLE entries ADD COLUMN filename TEXT NOT NULL DEFAULT ''"
)
session.execute(add_filename_column)
session.flush()
logger.info(fmt_log("Added filename column to entries table"))
# Populate the new filename column.
from tagstudio.core.library.alchemy.library import Library
for entry in Library._all_entries(session):
entry.filename = entry.path.name
session.merge(entry)
session.flush()
logger.info(fmt_log("Populated filename column in entries table"))
class MigrationTo100(DBMigration):
version = 100
@classmethod
def run(cls, session: Session, library_dir: Path, fmt_log):
"""Migrate DB to DB_VERSION 100."""
# Repair parent-child tag relationships that are the wrong way around.
stmt = update(TagParent).values(
parent_id=TagParent.child_id,
child_id=TagParent.parent_id,
)
session.execute(stmt)
session.flush()
logger.info(fmt_log("Refactored TagParent table"))
class MigrationTo101(DBMigration):
version = 101
@classmethod
def run(cls, session: Session, library_dir: Path, fmt_log):
"""Migrate DB to DB_VERSION 101."""
# Create versions table
session.execute(
text("""
CREATE TABLE versions (
"key" VARCHAR NOT NULL PRIMARY KEY,
value INTEGER NOT NULL
)
""")
)
session.flush()
# Ensure version rows are present
session.add(Version(key=DB_VERSION_INITIAL_KEY, value=100))
session.flush()
logger.info(fmt_log("Created versions table"))
class MigrationTo102(DBMigration):
version = 102
@classmethod
def run(cls, session: Session, library_dir: Path, fmt_log):
"""Migrate DB to DB_VERSION 102."""
# delete TagParents with a dangling parent reference
stmt = delete(TagParent).where(TagParent.parent_id.not_in(select(Tag.id).distinct()))
session.execute(stmt)
session.flush()
logger.info(fmt_log("Verified TagParent table data"))
class MigrationTo103(DBMigration):
version = 103
@classmethod
def run(cls, session: Session, library_dir: Path, fmt_log):
"""Migrate DB from DB_VERSION 102 to 103."""
# add the new hidden column for tags
session.execute(text("ALTER TABLE tags ADD COLUMN is_hidden BOOLEAN NOT NULL DEFAULT 0"))
session.flush()
logger.info(fmt_log("Added is_hidden column to tags table"))
# mark the "Archived" tag as hidden
session.query(Tag).filter(Tag.id == TAG_ARCHIVED).update({"is_hidden": True})
session.flush()
logger.info(fmt_log("Updated archived tag to be hidden"))
class MigrationTo104(DBMigration):
version = 104
@classmethod
def run(cls, session: Session, library_dir: Path, fmt_log):
"""Migrate DB from DB_VERSION 103 to 104."""
# Convert file extension list to ts_ignore file, if a .ts_ignore file does not exist
cls.__migrate_sql_to_ts_ignore(session, library_dir)
session.execute(text("DROP TABLE preferences"))
session.flush()
@classmethod
def __migrate_sql_to_ts_ignore(cls, session: Session, library_dir: Path):
# Do not continue if existing '.ts_ignore' file is found
ts_ignore = library_dir / TS_FOLDER_NAME / IGNORE_NAME
if Path(ts_ignore).exists():
return
# Load legacy extension data
extensions: list[str] = ujson.loads(
unwrap(
session.scalar(text("SELECT value FROM preferences WHERE key = 'EXTENSION_LIST'"))
)
)
is_exclude_list: bool = unwrap(
session.scalar(text("SELECT value FROM preferences WHERE key = 'IS_EXCLUDE_LIST'"))
)
with open(ts_ignore, "w") as f:
f.write(migrate_ext_list(extensions, is_exclude_list))
class MigrationTo200(DBMigration):
version = 200
@classmethod
def run(cls, session: Session, library_dir: Path, fmt_log):
"""Migrate DB to DB_VERSION 200."""
# Drop unused 'boolean_fields' and 'value_type' tables
logger.info(fmt_log("Dropping boolean_fields and value_type tables..."))
session.execute(text("DROP TABLE boolean_fields"))
session.execute(text("DROP TABLE value_type"))
# Add 'name' column to text_fields and datetime_fields tables
logger.info(fmt_log("Adding name columns to field tables..."))
stmt = text('ALTER TABLE text_fields ADD COLUMN name VARCHAR DEFAULT ""')
session.execute(stmt)
stmt = text('ALTER TABLE datetime_fields ADD COLUMN name VARCHAR DEFAULT ""')
session.execute(stmt)
# Drop unnecessary 'position' columns
logger.info(fmt_log("Dropping position columns to field tables..."))
session.execute(text("ALTER TABLE datetime_fields DROP COLUMN position"))
session.execute(text("ALTER TABLE text_fields DROP COLUMN position"))
# Add 'is_multiline' column to text_fields table
logger.info(fmt_log("Adding is_multiline column to text_fields..."))
stmt = text("ALTER TABLE text_fields ADD COLUMN is_multiline BOOLEAN NOT NULL DEFAULT 0")
session.execute(stmt)
session.flush()
# Move values from old `type_key` columns into new `name` columns
logger.info(fmt_log("Moving values from type_key columns to name..."))
session.execute(text("UPDATE text_fields SET name = type_key"))
session.execute(text("UPDATE datetime_fields SET name = type_key"))
session.flush()
# Change `name` values to title case
logger.info(fmt_log("Normalizing TextField names..."))
for text_field in session.execute(select(TextField)).scalars():
# NOTE: The only exception to the "Title Case" conversion is the "URL" field.
text_field.name = text_field.name.title().replace("Url", "URL").replace("_", " ")
logger.info(fmt_log("Normalizing DatetimeField names..."))
for datetime_field in session.execute(select(DatetimeField)).scalars():
datetime_field.name = datetime_field.name.title().replace("_", " ")
session.flush()
# Add correct `is_multiline` values to text_fields table
logger.info(fmt_log("Updating is_multiline for legacy TEXT_BOXes..."))
text_boxes = [
x.get("name") for x in LEGACY_FIELD_MAP.values() if x.get("is_multiline") is True
]
update_stmt = (
update(TextField).where(TextField.name.in_(text_boxes)).values(is_multiline=True)
)
session.execute(update_stmt)
session.flush()
# Repair legacy "Description" fields to use is_multiline = True
logger.info(fmt_log("Repairing legacy Description fields..."))
desc_stmt = (
update(TextField)
.where(TextField.name == "Description" and TextField.is_multiline == False) # noqa: E712
.values(is_multiline=True)
)
session.execute(desc_stmt)
# Repair legacy "Comments" fields to use is_multiline = True
logger.info(fmt_log("Repairing legacy Comment fields..."))
comm_stmt = (
update(TextField)
.where(TextField.name == "Comments" and TextField.is_multiline == False) # noqa: E712
.values(is_multiline=True)
)
session.execute(comm_stmt)
# Add default field templates
logger.info(fmt_log("Adding default field templates..."))
for template in DEFAULT_FIELD_TEMPLATES:
session.add(template)
session.flush()
# DB indices for improved performance
session.execute(
text("CREATE INDEX IF NOT EXISTS idx_tags_name_shorthand ON tags (name, shorthand)")
)
session.execute(
text("CREATE INDEX IF NOT EXISTS idx_tag_parents_child_id ON tag_parents (child_id)")
)
session.execute(
text("CREATE INDEX IF NOT EXISTS idx_tag_entries_entry_id ON tag_entries (entry_id)")
)
class MigrationTo201(DBMigration):
version = 201
initial_version = 200
@classmethod
def run(cls, session: Session, library_dir: Path, fmt_log):
"""Migrate DB to DB_VERSION 201."""
create_text_fields_table = text("""
CREATE TABLE text_fields_new (
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
name VARCHAR NOT NULL,
entry_id INTEGER NOT NULL,
value VARCHAR,
is_multiline BOOLEAN NOT NULL,
FOREIGN KEY(entry_id) REFERENCES entries (id)
)
""")
create_datetime_fields_table = text("""
CREATE TABLE datetime_fields_new (
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
name VARCHAR NOT NULL,
entry_id INTEGER NOT NULL,
value VARCHAR,
FOREIGN KEY(entry_id) REFERENCES entries (id)
)
""")
logger.info(fmt_log("Dropping type_key from text_fields table..."))
session.execute(create_text_fields_table)
session.flush()
session.execute(
text("""
INSERT INTO text_fields_new (id, name, entry_id, value, is_multiline)
SELECT id, name, entry_id, value, is_multiline
FROM text_fields
""")
)
session.execute(text("DROP TABLE text_fields"))
session.execute(text("ALTER TABLE text_fields_new RENAME TO text_fields"))
logger.info(fmt_log("Dropping type_key from datetime_fields table..."))
session.execute(create_datetime_fields_table)
session.flush()
session.execute(
text("""
INSERT INTO datetime_fields_new (id, name, entry_id, value)
SELECT id, name, entry_id, value
FROM datetime_fields
""")
)
session.execute(text("DROP TABLE datetime_fields"))
session.execute(text("ALTER TABLE datetime_fields_new RENAME TO datetime_fields"))
session.flush()
class MigrationTo202(DBMigration):
version = 202
@classmethod
def run(cls, session: Session, library_dir: Path, fmt_log):
"""Migrate DB to DB_VERSION 202."""
stmt = delete(TagParent).where(TagParent.child_id.not_in(select(Tag.id).distinct()))
session.execute(stmt)
session.flush()
logger.info(fmt_log("Verified TagParent table data"))
class MigrationTo300(DBMigration):
version = 300
@classmethod
def run(cls, session: Session, library_dir: Path, fmt_log):
## remove folder_id column from entries table
# create new table in the desired scheme (without folder_id column)
session.execute(
text("""
CREATE TABLE entries_new (
id INTEGER NOT NULL,
path VARCHAR NOT NULL,
suffix VARCHAR NOT NULL,
date_created DATETIME,
date_modified DATETIME,
date_added DATETIME,
filename TEXT NOT NULL DEFAULT '',
PRIMARY KEY (id),
UNIQUE (path)
)
""")
)
session.flush()
# transfer data to new table
session.execute(
text("""
INSERT INTO entries_new (id, path, suffix, date_created, date_modified, date_added,
filename)
SELECT id, path, suffix, date_created, date_modified, date_added, filename
FROM entries
""")
)
# delete old table
session.execute(text("DROP TABLE entries"))
# rename new table to old table
session.execute(text("ALTER TABLE entries_new RENAME TO entries"))
session.flush()
## drop table "folders"
session.execute(text("DROP TABLE folders"))
session.flush()
@@ -182,23 +182,11 @@ class Tag(Base):
return self.name >= other.name
class Folder(Base):
__tablename__ = "folders"
# TODO - implement this
id: Mapped[int] = mapped_column(primary_key=True)
path: Mapped[Path] = mapped_column(PathType, unique=True)
uuid: Mapped[str] = mapped_column(unique=True)
class Entry(Base):
__tablename__ = "entries"
id: Mapped[int] = mapped_column(primary_key=True)
folder_id: Mapped[int] = mapped_column(ForeignKey("folders.id"))
folder: Mapped[Folder] = relationship("Folder")
path: Mapped[Path] = mapped_column(PathType, unique=True)
filename: Mapped[str] = mapped_column()
suffix: Mapped[str] = mapped_column()
@@ -235,7 +223,6 @@ class Entry(Base):
def __init__(
self,
path: Path,
folder: Folder,
fields: list[BaseField],
id: int | None = None,
date_created: dt | None = None,
@@ -244,7 +231,6 @@ class Entry(Base):
) -> None:
super().__init__()
self.path = path
self.folder = folder
self.id = id # pyright: ignore[reportAttributeAccessIssue]
self.filename = path.name
self.suffix = path.suffix.lstrip(".").lower()
-2
View File
@@ -16,7 +16,6 @@ from tagstudio.core.library.alchemy.library import Library
from tagstudio.core.library.alchemy.models import Entry
from tagstudio.core.library.ignore import PATH_GLOB_FLAGS, Ignore, ignore_to_glob
from tagstudio.core.utils.silent_subprocess import silent_run # pyright: ignore
from tagstudio.core.utils.types import unwrap
logger = structlog.get_logger(__name__)
@@ -41,7 +40,6 @@ class RefreshTracker:
entries = [
Entry(
path=entry_path,
folder=unwrap(self.library.folder),
fields=[],
date_added=dt.now(),
)
+2 -1
View File
@@ -105,6 +105,7 @@ class MediaCategories:
# These sets are used either individually or together to form the final sets
# for the MediaCategory(s).
# These sets may be combined and are NOT 1:1 with the final categories.
_ADOBE_ILLUSTRATOR_SET: set[str] = {".ai"}
_ADOBE_PHOTOSHOP_SET: set[str] = {
".pdd",
".psb",
@@ -580,7 +581,7 @@ class MediaCategories:
)
PDF_TYPES = MediaCategory(
media_type=MediaType.PDF,
extensions=_PDF_SET,
extensions=_PDF_SET | _ADOBE_ILLUSTRATOR_SET,
is_iana=False,
name="pdf",
)
@@ -49,3 +49,14 @@ def is_version_outdated(current: str, latest: str) -> bool:
return vcur.patch < vlat.patch
else:
return vcur.prerelease is not None or vcur.build is not None
def format_duration(duration: int | float) -> str:
"""Format a duration in seconds as M:SS or H:MM:SS."""
try:
seconds = int(float(duration))
hours, seconds = divmod(seconds, 3600)
minutes, seconds = divmod(seconds, 60)
return f"{hours}:{minutes:02}:{seconds:02}" if hours else f"{minutes}:{seconds:02}"
except (OverflowError, ValueError):
return "-:--"
+3 -2
View File
@@ -10,7 +10,8 @@ import traceback
import structlog
from tagstudio.core.constants import VERSION, VERSION_BRANCH
from tagstudio.core.constants import BUILD_TYPE, VERSION
from tagstudio.qt.translations import Translations
from tagstudio.qt.ts_qt import QtDriver
logger = structlog.get_logger(__name__)
@@ -59,7 +60,7 @@ def main():
"--version",
action="version",
help="Displays TagStudio version information.",
version=f"TagStudio v{VERSION} {VERSION_BRANCH}",
version=f"TagStudio v{VERSION} {Translations[BUILD_TYPE] if BUILD_TYPE else ''}",
)
args = parser.parse_args()
@@ -32,8 +32,6 @@ Image.MAX_IMAGE_PIXELS = None
class PreviewThumb(PreviewThumbView):
__current_file: Path
def __init__(self, library: Library, driver: "QtDriver"):
super().__init__(library, driver)
@@ -114,7 +112,7 @@ class PreviewThumb(PreviewThumbView):
def display_file(self, filepath: Path) -> FileAttributeData:
"""Render a single file preview."""
self.__current_file = filepath
self._current_file = filepath
ext = filepath.suffix.lower()
@@ -150,21 +148,26 @@ class PreviewThumb(PreviewThumbView):
@override
def _open_file_action_callback(self):
open_file(
self.__current_file, windows_start_command=self.__driver.settings.windows_start_command
)
if self._current_file:
open_file(
self._current_file,
windows_start_command=self.__driver.settings.windows_start_command,
)
@override
def _open_explorer_action_callback(self):
open_file(self.__current_file, file_manager=True)
if self._current_file:
open_file(self._current_file, file_manager=True)
@override
def _delete_action_callback(self):
if bool(self.__current_file):
self.__driver.delete_files_callback(self.__current_file)
if self._current_file:
self.__driver.delete_files_callback(self._current_file)
@override
def _button_wrapper_callback(self):
open_file(
self.__current_file, windows_start_command=self.__driver.settings.windows_start_command
)
if self._current_file:
open_file(
self._current_file,
windows_start_command=self.__driver.settings.windows_start_command,
)
+8 -4
View File
@@ -19,12 +19,12 @@ from PySide6.QtWidgets import (
)
from tagstudio.core.constants import (
BUILD_TYPE,
COPYRIGHT,
DISCORD_URL,
DOCS_URL,
GITHUB_REPO_URL,
VERSION,
VERSION_BRANCH,
)
from tagstudio.core.ts_core import TagStudioCore
from tagstudio.core.utils.ffmpeg_status import FfmpegStatus, FfprobeStatus
@@ -42,7 +42,12 @@ from tagstudio.qt.views.stylesheets.stylesheets import form_content_style, heade
class AboutModal(QWidget):
"""Modal window showing information about the TagStudio application."""
VERSION_STR: str = f"{Translations['about.version']} {VERSION} {(' (' + VERSION_BRANCH + ')') if VERSION_BRANCH else ''}" # noqa: E501
VERSION_STR: str = " ".join(
[
f"{Translations['about.version']}",
f"{VERSION} {(' (' + Translations[BUILD_TYPE] + ')') if BUILD_TYPE else ''}",
]
)
def __init__(self, config_path: Path | str):
super().__init__()
@@ -196,8 +201,7 @@ class AboutModal(QWidget):
# ripgrep Status
ripgrep_path_title = QLabel("ripgrep") # NOTE: Don't localize
ripgrep_path_content = ClickableLabel()
ripgrep_path_content.setText(f"{ripgrep_status}") # TODO: Pass in constructor after #1386
ripgrep_path_content = ClickableLabel(f"{ripgrep_status}")
ripgrep_location = RipgrepStatus.which()
if ripgrep_location:
ripgrep_path_content.clicked.connect(
-133
View File
@@ -1,133 +0,0 @@
# SPDX-FileCopyrightText: (c) TagStudio Contributors
# SPDX-License-Identifier: GPL-3.0-only
import math
from pathlib import Path
import cv2
import structlog
from PIL import Image, ImageChops, UnidentifiedImageError
from PIL.Image import DecompressionBombError
from PySide6.QtCore import QObject, Signal
from tagstudio.core.library.alchemy.library import Library
from tagstudio.core.media_types import MediaCategories
from tagstudio.core.utils.types import unwrap
from tagstudio.qt.helpers.file_tester import is_readable_video
logger = structlog.get_logger(__name__)
class CollageIconRenderer(QObject):
rendered = Signal(Image.Image)
done = Signal()
def __init__(self, library: Library):
QObject.__init__(self)
self.lib = library
def render(
self,
entry_id: int,
size: tuple[int, int],
data_tint_mode: bool,
data_only_mode: bool,
keep_aspect: bool,
):
entry = unwrap(self.lib.get_entry(entry_id))
filepath = unwrap(self.lib.library_dir) / entry.path
color: str = ""
try:
if data_tint_mode or data_only_mode:
color = "#28bb48" if entry.tags else "#e22c3c"
if data_only_mode:
pic = Image.new("RGB", size, color)
# collage.paste(pic, (y*thumb_size, x*thumb_size))
self.rendered.emit(pic)
if not data_only_mode:
logger.info(
"Combining icons",
entry=entry,
color=self.get_file_color(filepath.suffix.lower()),
)
ext: str = filepath.suffix.lower()
if MediaCategories.is_ext_in_category(ext, MediaCategories.IMAGE_TYPES):
try:
with Image.open(filepath) as pic:
if keep_aspect:
pic.thumbnail(size)
else:
pic = pic.resize(size)
if data_tint_mode and color:
pic = pic.convert(mode="RGB")
pic = ImageChops.hard_light(pic, Image.new("RGB", size, color))
self.rendered.emit(pic)
except DecompressionBombError as e:
logger.info(f"[ERROR] One of the images was too big ({e})")
elif MediaCategories.is_ext_in_category(
ext, MediaCategories.VIDEO_TYPES
) and is_readable_video(filepath):
video = cv2.VideoCapture(str(filepath), cv2.CAP_FFMPEG)
video.set(
cv2.CAP_PROP_POS_FRAMES,
(video.get(cv2.CAP_PROP_FRAME_COUNT) // 2),
)
success, frame = video.read()
# NOTE: Depending on the video format, compression, and
# frame count, seeking halfway does not work and the thumb
# must be pulled from the earliest available frame.
max_frame_seek: int = 10
for i in range(
0,
min(
max_frame_seek,
math.floor(video.get(cv2.CAP_PROP_FRAME_COUNT)),
),
):
success, frame = video.read()
if not success:
video.set(cv2.CAP_PROP_POS_FRAMES, i)
else:
break
frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
with Image.fromarray(frame, mode="RGB") as pic:
if keep_aspect:
pic.thumbnail(size)
else:
pic = pic.resize(size)
if data_tint_mode and color:
pic = ImageChops.hard_light(pic, Image.new("RGB", size, color))
self.rendered.emit(pic)
except (UnidentifiedImageError, FileNotFoundError):
logger.error("Couldn't read entry", entry=entry.path)
with Image.open(
str(Path(__file__).parents[1] / "resources/qt/images/thumb_broken_512.png")
) as pic:
pic.thumbnail(size)
if data_tint_mode and color:
pic = pic.convert(mode="RGB")
pic = ImageChops.hard_light(pic, Image.new("RGB", size, color))
# collage.paste(pic, (y*thumb_size, x*thumb_size))
self.rendered.emit(pic)
except KeyboardInterrupt:
logger.info("Collage operation cancelled.")
except Exception:
logger.exception("render failed", entry=entry.path)
self.done.emit()
def get_file_color(self, ext: str):
if ext.lower() == "gif":
return "\033[93m"
if MediaCategories.is_ext_in_category(ext, MediaCategories.IMAGE_TYPES):
return "\033[37m"
elif MediaCategories.is_ext_in_category(ext, MediaCategories.VIDEO_TYPES):
return "\033[96m"
elif MediaCategories.is_ext_in_category(ext, MediaCategories.PLAINTEXT_TYPES):
return "\033[92m"
else:
return "\033[97m"
+2 -10
View File
@@ -7,7 +7,6 @@ import platform
import typing
from dataclasses import dataclass
from datetime import datetime as dt
from datetime import timedelta
from pathlib import Path
import structlog
@@ -20,6 +19,7 @@ from tagstudio.core.enums import ShowFilepathOption
from tagstudio.core.library.alchemy.library import Library
from tagstudio.core.library.ignore import Ignore
from tagstudio.core.media_types import MediaCategories
from tagstudio.core.utils.str_formatting import format_duration
from tagstudio.core.utils.types import unwrap
from tagstudio.qt.models.palette import ColorType, UiColor, get_ui_color
from tagstudio.qt.translations import Translations
@@ -224,15 +224,7 @@ class FileAttributes(QWidget):
if stats.duration is not None:
stats_label_text = add_newline(stats_label_text)
try:
dur_str = str(timedelta(seconds=float(stats.duration)))[:-7]
if dur_str.startswith("0:"):
dur_str = dur_str[2:]
if dur_str.startswith("0"):
dur_str = dur_str[1:]
except OverflowError:
dur_str = "-:--"
stats_label_text += f"{dur_str}"
stats_label_text += format_duration(stats.duration)
if font_family:
stats_label_text = add_newline(stats_label_text)
+6 -5
View File
@@ -395,14 +395,15 @@ class JsonMigrationModal(QObject):
# Convert JSON Library to SQLite
yield Translations["json_migration.creating_database_tables"]
self.sql_lib = SqliteLibrary()
self.temp_path: Path = (
self.json_lib.library_dir / TS_FOLDER_NAME / "migration_ts_library.sqlite"
)
temp_filename = "migration_ts_library.sqlite"
self.temp_path: Path = self.json_lib.library_dir / TS_FOLDER_NAME / temp_filename
if self.temp_path.exists():
logger.info('Temporary migration file "temp_path" already exists. Removing...')
self.temp_path.unlink()
self.sql_lib.open_sqlite_library(
self.json_lib.library_dir, is_new=True, storage_path=str(self.temp_path)
self.sql_lib.create_sqlite_library(
self.json_lib.library_dir,
in_memory=False,
sql_filename=temp_filename,
)
yield Translations.format(
"json_migration.migrating_files_entries", entries=len(self.json_lib.entries)
+7 -4
View File
@@ -1229,6 +1229,8 @@ class ThumbRenderer(QObject):
"QuickLook/Preview.heic",
"QuickLook/Thumbnail.jpg",
"QuickLook/Thumbnail.heic",
"QuickLook/Thumbnail.webp",
"QuickLook/Icon.webp",
]
im: Image.Image | None = None
@@ -1293,11 +1295,12 @@ class ThumbRenderer(QObject):
return im
@staticmethod
def _pdf_thumb(filepath: Path, size: int) -> Image.Image | None:
"""Render a thumbnail for a PDF file.
def _pdf_thumb(filepath: Path, size: int, ext: str) -> Image.Image | None:
"""Render a thumbnail for a PDF or Adobe Illustator file.
filepath (Path): The path of the file.
size (int): The size of the icon.
ext (str): The file extension.
"""
im: Image.Image | None = None
@@ -1319,7 +1322,7 @@ class ThumbRenderer(QObject):
else:
page_size *= size / page_size.width()
# Enlarge image for anti-aliasing
scale_factor = 2.5
scale_factor = 2.5 if ext in {".pdf"} else 1
page_size *= scale_factor
# Render image with no anti-aliasing for speed
render_options: QPdfDocumentRenderOptions = QPdfDocumentRenderOptions()
@@ -1908,7 +1911,7 @@ class ThumbRenderer(QObject):
elif MediaCategories.is_ext_in_category(
ext, MediaCategories.PDF_TYPES, mime_fallback=True
):
image = self._pdf_thumb(_filepath, adj_size)
image = self._pdf_thumb(_filepath, adj_size, ext)
# Archives =====================================================
elif MediaCategories.is_ext_in_category(ext, MediaCategories.ARCHIVE_TYPES):
image = self._archive_thumb(_filepath, ext)
+3 -3
View File
@@ -40,7 +40,7 @@ from PySide6.QtGui import (
from PySide6.QtWidgets import QApplication, QFileDialog, QMessageBox, QPushButton, QScrollArea
import tagstudio.qt.resources_rc # noqa: F401 # pyright: ignore[reportUnusedImport]
from tagstudio.core.constants import TAG_ARCHIVED, TAG_FAVORITE, VERSION, VERSION_BRANCH
from tagstudio.core.constants import BUILD_TYPE, TAG_ARCHIVED, TAG_FAVORITE, VERSION
from tagstudio.core.driver import DriverMixin
from tagstudio.core.enums import AppCacheItems, MacroID, ShowFilepathOption
from tagstudio.core.library.alchemy.enums import BrowsingState, SortingModeEnum
@@ -203,7 +203,7 @@ class QtDriver(DriverMixin, QObject):
self.scrollbar_pos = 0
self.spacing = None
self.branch: str = (" (" + VERSION_BRANCH + ")") if VERSION_BRANCH else ""
self.branch: str = (" (" + Translations[BUILD_TYPE] + ")") if BUILD_TYPE else ""
self.base_title: str = f"TagStudio Alpha {VERSION}{self.branch}"
# self.title_text: str = self.base_title
# self.buffer = {}
@@ -837,7 +837,7 @@ class QtDriver(DriverMixin, QObject):
logger.info("Backing Up Library...")
self.main_window.status_bar.showMessage(Translations["status.library_backup_in_progress"])
start_time = time.time()
target_path = self.lib.save_library_backup_to_disk()
target_path = Library.save_library_backup_to_disk(unwrap(self.lib.library_dir))
end_time = time.time()
self.main_window.status_bar.showMessage(
Translations.format(
@@ -51,6 +51,7 @@ class PreviewPanelView(QWidget):
self._containers = FieldContainers(
self.lib, driver
) # TODO: this should be name mangled, but is still needed on the controller side atm
self.__current_stats: FileAttributeData | None = None
preview_section = QWidget()
preview_layout = QVBoxLayout(preview_section)
@@ -132,6 +133,7 @@ class PreviewPanelView(QWidget):
def __connect_callbacks(self) -> None:
self.__add_field_button.clicked.connect(self._add_field_button_callback)
self.__add_tag_button.clicked.connect(self._add_tag_button_callback)
self._thumb.stats_updated.connect(self.__thumb_stats_updated_callback)
def _add_field_button_callback(self) -> None:
raise NotImplementedError()
@@ -139,6 +141,25 @@ class PreviewPanelView(QWidget):
def _add_tag_button_callback(self) -> None:
raise NotImplementedError()
def __thumb_stats_updated_callback(self, filepath: Path, stats: FileAttributeData) -> None:
if len(self._selected) != 1:
return
if filepath != self._thumb.current_file:
return
if self.__current_stats is None:
self.__current_stats = FileAttributeData()
if stats.width is not None:
self.__current_stats.width = stats.width
if stats.height is not None:
self.__current_stats.height = stats.height
if stats.duration is not None:
self.__current_stats.duration = stats.duration
self._file_attrs.update_stats(filepath, self.__current_stats)
def _set_selection_callback(self) -> None:
raise NotImplementedError()
@@ -155,6 +176,7 @@ class PreviewPanelView(QWidget):
# No Items Selected
if len(selected) == 0:
self._thumb.hide_preview()
self.__current_stats = None
self._file_attrs.update_stats()
self._file_attrs.update_date_label()
self._containers.hide_containers()
@@ -167,9 +189,12 @@ class PreviewPanelView(QWidget):
entry: Entry = unwrap(self.lib.get_entry(entry_id))
filepath: Path = unwrap(self.lib.library_dir) / entry.path
if filepath != self._thumb.current_file:
self.__current_stats = None
if update_preview:
stats: FileAttributeData = self._thumb.display_file(filepath)
self.__current_stats = stats
self._file_attrs.update_stats(filepath, stats)
self._file_attrs.update_date_label(filepath)
self._containers.update_from_entry(entry_id)
@@ -182,6 +207,7 @@ class PreviewPanelView(QWidget):
elif len(selected) > 1:
# items: list[Entry] = [self.lib.get_entry_full(x) for x in self.driver.selected]
self._thumb.hide_preview() # TODO: Render mixed selection
self.__current_stats = None
self._file_attrs.update_multi_selection(len(selected))
self._file_attrs.update_date_label()
self._containers.hide_containers() # TODO: Allow for mixed editing
+43 -13
View File
@@ -34,11 +34,13 @@ class PreviewThumbView(QWidget):
"""The Preview Panel Widget."""
check_ffmpeg = Signal(bool)
stats_updated = Signal(Path, FileAttributeData)
__img_button_size: tuple[int, int]
__image_ratio: float
__filepath: Path | None
_current_file: Path | None
__should_render_on_resize: bool
__rendered_res: tuple[int, int]
def __init__(self, library: Library, driver: "QtDriver") -> None:
@@ -47,6 +49,8 @@ class PreviewThumbView(QWidget):
self.__img_button_size = (266, 266)
self.__image_ratio = 1.0
self.__should_render_on_resize = False
self.__image_layout = QStackedLayout(self)
self.__image_layout.setAlignment(Qt.AlignmentFlag.AlignCenter)
self.__image_layout.setStackingMode(QStackedLayout.StackingMode.StackAll)
@@ -92,6 +96,10 @@ class PreviewThumbView(QWidget):
self.__media_player.addAction(open_file_action)
self.__media_player.addAction(open_explorer_action)
self.__media_player.addAction(delete_action)
# QMediaPlayer loads duration asynchronously after setSource().
self.__media_player.player.durationChanged.connect(
self.__media_player_duration_changed_callback
)
# Need to watch for this to resize the player appropriately.
self.__media_player.player.hasVideoChanged.connect(
@@ -128,6 +136,16 @@ class PreviewThumbView(QWidget):
def __media_player_video_changed_callback(self, video: bool) -> None:
self.__update_image_size((self.size().width(), self.size().height()))
def __media_player_duration_changed_callback(self, duration_ms: int) -> None:
filepath = self.__media_player.filepath
if filepath is None or duration_ms <= 0:
return
self.stats_updated.emit(
filepath,
FileAttributeData(duration=duration_ms // 1000),
)
def __thumb_renderer_updated_callback(
self, _timestamp: float, img: QPixmap, _size: QSize, _path: Path
) -> None:
@@ -207,7 +225,8 @@ class PreviewThumbView(QWidget):
self.__preview_gif.hide()
def __render_thumb(self, filepath: Path) -> None:
self.__filepath = filepath
self.__should_render_on_resize = True
self.__rendered_res = (
math.ceil(self.__img_button_size[0] * THUMB_SIZE_FACTOR),
math.ceil(self.__img_button_size[1] * THUMB_SIZE_FACTOR),
@@ -221,17 +240,16 @@ class PreviewThumbView(QWidget):
update_on_ratio_change=True,
)
def __update_media_player(self, filepath: Path) -> int:
"""Display either audio or video.
Returns the duration of the audio / video.
"""
def __update_media_player(self, filepath: Path) -> None:
"""Display either audio or video."""
self.__media_player.play(filepath)
return self.__media_player.player.duration() * 1000
def _display_video(self, filepath: Path, size: QSize | None) -> FileAttributeData:
self.__should_render_on_resize = False
self.__switch_preview(MediaType.VIDEO)
stats = FileAttributeData(duration=self.__update_media_player(filepath))
self.__update_media_player(filepath)
stats = FileAttributeData()
if size is not None:
stats.width = size.width()
@@ -250,10 +268,13 @@ class PreviewThumbView(QWidget):
def _display_audio(self, filepath: Path) -> FileAttributeData:
self.__switch_preview(MediaType.AUDIO)
self.__render_thumb(filepath)
return FileAttributeData(duration=self.__update_media_player(filepath))
self.__update_media_player(filepath)
return FileAttributeData()
def _display_gif(self, gif_data: bytes, size: tuple[int, int]) -> FileAttributeData | None:
"""Update the animated image preview from a filepath."""
self.__should_render_on_resize = False
stats = FileAttributeData()
# Ensure that any movie and buffer from previous animations are cleared.
@@ -296,17 +317,26 @@ class PreviewThumbView(QWidget):
def hide_preview(self) -> None:
"""Completely hide the file preview."""
self.__switch_preview(None)
self.__filepath = None
self._current_file = None
self.__should_render_on_resize = False
@override
def resizeEvent(self, event: QResizeEvent) -> None:
self.__update_image_size((self.size().width(), self.size().height()))
if self.__filepath is not None and self.__rendered_res < self.__img_button_size:
self.__render_thumb(self.__filepath)
if (
self._current_file is not None
and self.__should_render_on_resize
and self.__rendered_res < self.__img_button_size
):
self.__render_thumb(self._current_file)
return super().resizeEvent(event)
@property
def media_player(self) -> MediaPlayer:
return self.__media_player
@property
def current_file(self) -> Path | None:
return self._current_file
+7 -2
View File
@@ -10,7 +10,7 @@ from PySide6.QtCore import QRect, Qt
from PySide6.QtGui import QColor, QFont, QPainter, QPen, QPixmap
from PySide6.QtWidgets import QSplashScreen, QWidget
from tagstudio.core.constants import COPYRIGHT, COPYRIGHT_COMPACT, VERSION, VERSION_BRANCH
from tagstudio.core.constants import BUILD_TYPE, COPYRIGHT, COPYRIGHT_COMPACT, VERSION
from tagstudio.qt.global_settings import Splash
from tagstudio.qt.resource_manager import ResourceManager
from tagstudio.qt.translations import Translations
@@ -21,7 +21,12 @@ logger = structlog.get_logger(__name__)
class SplashScreen:
"""The custom splash screen widget for TagStudio."""
VERSION_STR: str = f"{Translations['about.version']} {VERSION} {(' (' + VERSION_BRANCH + ')') if VERSION_BRANCH else ''}" # noqa: E501
VERSION_STR: str = " ".join(
[
f"{Translations['about.version']}",
f"{VERSION} {(' (' + Translations[BUILD_TYPE] + ')') if BUILD_TYPE else ''}",
]
)
DEFAULT_SPLASH = Splash.AURORA
def __init__(
Binary file not shown.

Before

Width:  |  Height:  |  Size: 43 KiB

After

Width:  |  Height:  |  Size: 112 KiB

Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 124 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 126 KiB

@@ -10,6 +10,7 @@
"about.version.latest": "{built_version} (Latest Release: {latest_version})",
"about.website": "Website",
"app.git": "Git Commit",
"app.nightly": "Nightly",
"app.pre_release": "Pre-Release",
"app.title": "{base_title} - Library '{library_dir}'",
"color_manager.title": "Manage Tag Colors",
-8
View File
@@ -20,7 +20,6 @@ sys.path.insert(0, str(CWD.parent))
from tagstudio.core.constants import THUMB_CACHE_NAME, TS_FOLDER_NAME
from tagstudio.core.library.alchemy.library import Library
from tagstudio.core.library.alchemy.models import Entry, Tag
from tagstudio.core.utils.types import unwrap
from tagstudio.qt.thumb_grid_layout import ThumbGridLayout
from tagstudio.qt.ts_qt import QtDriver
@@ -36,22 +35,18 @@ def file_mediatypes_library():
status = lib.open_library(Path(""), in_memory=True)
assert status.success
folder = unwrap(lib.folder)
entry1 = Entry(
folder=folder,
path=Path("foo.png"),
fields=[TextField(name="Title", value="I'm a Test Title")],
)
entry2 = Entry(
folder=folder,
path=Path("bar.png"),
fields=[TextField(name="Title", value="I'm a Test Title")],
)
entry3 = Entry(
folder=folder,
path=Path("baz.apng"),
fields=[TextField(name="Title", value="I'm a Test Title")],
)
@@ -87,7 +82,6 @@ def library(request, library_dir: Path): # pyright: ignore
lib = Library()
status = lib.open_library(library_path, in_memory=True)
assert status.success
folder = unwrap(lib.folder)
tag = Tag(
name="foo",
@@ -116,7 +110,6 @@ def library(request, library_dir: Path): # pyright: ignore
# default item with deterministic name
entry = Entry(
id=1,
folder=folder,
path=Path("foo.txt"),
fields=[TextField(name="Title", value="I'm a Test Title")],
)
@@ -124,7 +117,6 @@ def library(request, library_dir: Path): # pyright: ignore
entry2 = Entry(
id=2,
folder=folder,
path=Path("one/two/bar.md"),
fields=[TextField(name="Title", value="I'm a Test Title")],
)
Binary file not shown.
-4
View File
@@ -8,25 +8,21 @@ from tagstudio.core.library.alchemy.fields import BaseField, TextField
from tagstudio.core.library.alchemy.library import Library
from tagstudio.core.library.alchemy.models import Entry
from tagstudio.core.library.alchemy.registries.dupe_files_registry import DupeFilesRegistry
from tagstudio.core.utils.types import unwrap
CWD = Path(__file__).parent
def test_refresh_dupe_files(library: Library):
library.library_dir = Path("/tmp/")
folder = unwrap(library.folder)
fields: list[BaseField] = [TextField(name="Title", value="I'm a Test Title")]
entry = Entry(
folder=folder,
path=Path("bar/foo.txt"),
fields=fields,
)
entry2 = Entry(
folder=folder,
path=Path("foo/foo.txt"),
fields=fields,
)
+1 -10
View File
@@ -77,7 +77,6 @@ def test_library_add_file(library: Library):
"""Check Entry.path handling for insert vs lookup"""
entry = Entry(
path=Path("bar.txt"),
folder=unwrap(library.folder),
fields=[TextField(name="Title", value="I'm a Test Title")],
)
@@ -139,8 +138,7 @@ def test_get_entry(library: Library, entry_min: Entry):
def test_entries_count(library: Library):
folder = unwrap(library.folder)
entries = [Entry(path=Path(f"{x}.txt"), folder=folder, fields=[]) for x in range(10)]
entries = [Entry(path=Path(f"{x}.txt"), fields=[]) for x in range(10)]
new_ids = library.add_entries(entries)
assert len(new_ids) == 10
@@ -254,7 +252,6 @@ def test_update_entry_with_multiple_identical_text_fields(library: Library, entr
def test_mirror_entry_fields(library: Library):
# Create and add entries with fields
entry_a = Entry(
folder=unwrap(library.folder),
path=Path("title_and_date.txt"),
fields=[
TextField(name="Title", value="I'm a Test Title"),
@@ -262,7 +259,6 @@ def test_mirror_entry_fields(library: Library):
],
)
entry_b = Entry(
folder=unwrap(library.folder),
path=Path("notes.txt"),
fields=[
TextField(name="Notes", value="These are my notes.\nNo peeking!", is_multiline=True),
@@ -270,7 +266,6 @@ def test_mirror_entry_fields(library: Library):
],
)
entry_c = Entry(
folder=unwrap(library.folder),
path=Path("date_published.txt"),
fields=[
DatetimeField(name="Date Published", value="2000-01-01 12:00:00"),
@@ -319,14 +314,11 @@ def test_mirror_entry_fields(library: Library):
def test_merge_entries(library: Library):
folder = unwrap(library.folder)
tag_0: Tag = unwrap(library.add_tag(Tag(id=1010, name="tag_0")))
tag_1: Tag = unwrap(library.add_tag(Tag(id=1011, name="tag_1")))
tag_2: Tag = unwrap(library.add_tag(Tag(id=1012, name="tag_2")))
entry_a = Entry(
folder=folder,
path=Path("a"),
fields=[
TextField(name="Author", value="Author McAuthorson"),
@@ -334,7 +326,6 @@ def test_merge_entries(library: Library):
],
)
entry_b = Entry(
folder=folder,
path=Path("b"),
fields=[TextField(name="Notes", value="test note", is_multiline=True)],
)