calvin.erfmann/TagStudio
1
0
Fork 0
mirror of https://github.com/TagStudioDev/TagStudio.git synced 2026-06-10 01:23:04 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor!: uncouple fields from hardcoded values

Browse Source
This commit is contained in:
Travis Abendshien
2026-05-08 12:49:17 -07:00
parent 47d4de5825
commit 39c5c0a90f
23 changed files with 640 additions and 751 deletions
Download Patch File Download Diff File Expand all files Collapse all files

81
docs/library-changes.md
View File

@@ -64,8 +64,8 @@ Migration from the legacy JSON format is provided via a walkthrough when opening
| ------------------------------------------------------------------------------- | ------ | ----------------------------------------------- |
| [v9.5.0-pr2](https://github.com/TagStudioDev/TagStudio/releases/tag/v9.5.0-pr2) | SQLite | `<Library Folder>`/.TagStudio/ts_library.sqlite |
- Repairs "Description" fields to use a TEXT_LINE key instead of a TEXT_BOX key.
- Repairs tags that may have a disambiguation_id pointing towards a deleted tag.
- ~~Repairs "Description" fields to use a TEXT_LINE key instead of a TEXT_BOX key.~~ _See [Version 200](#version-200)_
- Repairs tags that may have a disambiguation_id pointing towards a deleted tag.
---
@@ -75,9 +75,9 @@ Migration from the legacy JSON format is provided via a walkthrough when opening
| ------------------------------------------------------------------------------- | ------ | ----------------------------------------------- |
| [v9.5.0-pr4](https://github.com/TagStudioDev/TagStudio/releases/tag/v9.5.0-pr4) | SQLite | `<Library Folder>`/.TagStudio/ts_library.sqlite |
- Adds the `color_border` column to the `tag_colors` table. Used for instructing the [secondary color](colors.md#secondary-color) to apply to a tag's border as a new optional behavior.
- Adds three new default colors: "Burgundy (TagStudio Shades)", "Dark Teal (TagStudio Shades)", and "Dark Lavender (TagStudio Shades)".
- Updates Neon colors to use the new `color_border` property.
- Adds the `color_border` column to the `tag_colors` table. Used for instructing the [secondary color](colors.md#secondary-color) to apply to a tag's border as a new optional behavior.
- Adds three new default colors: "Burgundy (TagStudio Shades)", "Dark Teal (TagStudio Shades)", and "Dark Lavender (TagStudio Shades)".
- Updates Neon colors to use the new `color_border` property.
---
@@ -87,48 +87,69 @@ Migration from the legacy JSON format is provided via a walkthrough when opening
| ----------------------------------------------------------------------- | ------ | ----------------------------------------------- |
| [v9.5.2](https://github.com/TagStudioDev/TagStudio/releases/tag/v9.5.2) | SQLite | `<Library Folder>`/.TagStudio/ts_library.sqlite |
- Adds the `filename` column to the `entries` table. Used for sorting entries by filename in search results.
- Adds the `filename` column to the `entries` table. Used for sorting entries by filename in search results.
---
### Version 100
### Versions 100 - 1xx
#### Version 100
| Used From | Format | Location |
| ---------------------------------------------------------------------------------------------------- | ------ | ----------------------------------------------- |
| [74383e3](https://github.com/TagStudioDev/TagStudio/commit/74383e3c3c12f72be1481ab0b86c7360b95c2d85) | SQLite | `<Library Folder>`/.TagStudio/ts_library.sqlite |
- Introduces built-in minor versioning
- The version number divided by 100 (and floored) constitutes the **major** version. Major version indicate breaking changes that prevent libraries from being opened in TagStudio versions older than the ones they were created in.
- Values more precise than this ("ones" through "tens" columns) constitute the **minor** version. These indicate minor changes that don't prevent a newer library from being opened in an older version of TagStudio, as long as the major version is not also increased.
- Swaps `parent_id` and `child_id` values in the `tag_parents` table
- Introduces built-in minor versioning
- The version number divided by 100 (and floored) constitutes the **major** version. Major version indicate breaking changes that prevent libraries from being opened in TagStudio versions older than the ones they were created in.
- Values more precise than this ("ones" through "tens" columns) constitute the **minor** version. These indicate minor changes that don't prevent a newer library from being opened in an older version of TagStudio, as long as the major version is not also increased.
- Swaps `parent_id` and `child_id` values in the `tag_parents` table
#### Version 101
| Used From | Format | Location |
| ----------------------------------------------------------------------- | ------ | ----------------------------------------------- |
| [v9.5.4](https://github.com/TagStudioDev/TagStudio/releases/tag/v9.5.4) | SQLite | `<Library Folder>`/.TagStudio/ts_library.sqlite |
| Used From | Format | Location |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ | ----------------------------------------------- |
| [12e074b](https://github.com/TagStudioDev/TagStudio/commit/12e074b71d8860282b44e49e0e1a41b7a2e4bae8)/[v9.5.4](https://github.com/TagStudioDev/TagStudio/releases/tag/v9.5.4) | SQLite | `<Library Folder>`/.TagStudio/ts_library.sqlite |
- Deprecates the `preferences` table, set to be removed in a future TagStudio version.
- Introduces the `versions` table
- Has a string `key` column and an int `value` column
- The `key` column stores one of two values: `'INITIAL'` and `'CURRENT'`
- `'INITIAL'` stores the database version number in which in was created
- Pre-existing databases set this number to `100`
- `'CURRENT'` stores the current database version number
- Deprecates the `preferences` table, set to be removed in a future TagStudio version.
- Introduces the `versions` table
- Has a string `key` column and an int `value` column
- The `key` column stores one of two values: `'INITIAL'` and `'CURRENT'`
- `'INITIAL'` stores the database version number in which in was created
- Pre-existing databases set this number to `100`
- `'CURRENT'` stores the current database version number
#### Version 102
| Used From | Format | Location |
| ----------------------------------------------------------------------- | ------ | ----------------------------------------------- |
| [v9.5.4](https://github.com/TagStudioDev/TagStudio/releases/tag/v9.5.4) | SQLite | `<Library Folder>`/.TagStudio/ts_library.sqlite |
| Used From | Format | Location |
| ---------------------------------------------------------------------------------------------------- | ------ | ----------------------------------------------- |
| [71d0425](https://github.com/TagStudioDev/TagStudio/commit/71d04254cf87f4200bb7ffc81656e50dfb122e4d) | SQLite | `<Library Folder>`/.TagStudio/ts_library.sqlite |
- Applies repairs to the `tag_parents` table created in [version 100](#version-100), removing rows that reference tags that have been deleted.
- Applies repairs to the `tag_parents` table created in [version 100](#version-100), removing rows that reference tags that have been deleted.
#### Version 103
| Used From | Format | Location |
| ----------------------------------------------------------------------- | ------ | ----------------------------------------------- |
| [#1139](https://github.com/TagStudioDev/TagStudio/pull/1139) | SQLite | `<Library Folder>`/.TagStudio/ts_library.sqlite |
| Used From | Format | Location |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ | ----------------------------------------------- |
| [88d0b47](https://github.com/TagStudioDev/TagStudio/commit/88d0b47a86821ccfadba653f30a515abce5b24b0)/[v9.5.7](https://github.com/TagStudioDev/TagStudio/releases/tag/v9.5.7) | SQLite | `<Library Folder>`/.TagStudio/ts_library.sqlite |
- Adds the `is_hidden` column to the `tags` table (default `0`). Used for excluding entries tagged with hidden tags from library searches.
- Sets the `is_hidden` field on the built-in Archived tag to `1`, to match the Archived tag now being hidden by default.
- Adds the `is_hidden` column to the `tags` table (default `0`). Used for excluding entries tagged with hidden tags from library searches.
- Sets the `is_hidden` field on the built-in Archived tag to `1`, to match the Archived tag now being hidden by default.
### Versions 200 - 2xx
#### Version 200
| Used From | Format | Location |
| --------- | ------ | ----------------------------------------------- |
| TBD | SQLite | `<Library Folder>`/.TagStudio/ts_library.sqlite |
- Adds `text_field_templates` and `date_field_templates` tables.
- Drops `boolean_fields` and `value_type` tables.
- Adds `name` columns to `text_fields` and `datetime_fields` tables.
- Values in the `name` columns are taken from the `type_key` columns and are changed to "Title Case".
- **Example:** "DATE_CREATED" -> "Date Created"
- Drops `position` columns from `text_fields` and `datetime_fields` tables.
- Adds `is_multiline` column to `text_fields` table.
- Values are set to `TRUE` if the field row was previously a "TEXT_BOX" type.
- Repairs existing "Description" fields inside the `text_fields` table to have their `is_multiline` column set to `TRUE` _(Previously done in [Version 7](#version-7))_.
- Repairs existing "Comments" fields inside the `text_fields` table to have their `is_multiline` column set to `TRUE`.

2
src/tagstudio/core/library/alchemy/constants.py
View File

@@ -11,7 +11,7 @@ JSON_FILENAME: str = "ts_library.json"
DB_VERSION_LEGACY_KEY: str = "DB_VERSION"
DB_VERSION_CURRENT_KEY: str = "CURRENT"
DB_VERSION_INITIAL_KEY: str = "INITIAL"
DB_VERSION: int = 103
DB_VERSION: int = 200
TAG_CHILDREN_QUERY = text("""
WITH RECURSIVE ChildTags AS (

5
src/tagstudio/core/library/alchemy/db.py
View File

@@ -66,8 +66,3 @@ def make_tables(engine: Engine) -> None:
except OperationalError as e:
logger.error("Could not initialize built-in tags", error=e)
conn.rollback()
def drop_tables(engine: Engine) -> None:
logger.info("dropping db tables")
Base.metadata.drop_all(engine)

8
src/tagstudio/core/library/alchemy/enums.py
View File

@@ -152,11 +152,3 @@ class BrowsingState:
def with_show_hidden_entries(self, show_hidden_entries: bool) -> "BrowsingState":
return replace(self, show_hidden_entries=show_hidden_entries)
class FieldTypeEnum(enum.Enum):
TEXT_LINE = "Text Line"
TEXT_BOX = "Text Box"
TAGS = "Tags"
DATETIME = "Datetime"
BOOLEAN = "Checkbox"

143
src/tagstudio/core/library/alchemy/fields.py
View File

@@ -5,18 +5,15 @@
from __future__ import annotations
from dataclasses import dataclass, field
from enum import Enum
from typing import TYPE_CHECKING, Any, override
from typing import TYPE_CHECKING, Any
from sqlalchemy import ForeignKey
from sqlalchemy.orm import Mapped, declared_attr, mapped_column, relationship
from tagstudio.core.library.alchemy.db import Base
from tagstudio.core.library.alchemy.enums import FieldTypeEnum
if TYPE_CHECKING:
from tagstudio.core.library.alchemy.models import Entry, ValueType
from tagstudio.core.library.alchemy.models import Entry
class BaseField(Base):
@@ -27,12 +24,8 @@ class BaseField(Base):
return mapped_column(primary_key=True, autoincrement=True)
@declared_attr
def type_key(self) -> Mapped[str]:
return mapped_column(ForeignKey("value_type.key"))
@declared_attr
def type(self) -> Mapped[ValueType]:
return relationship(foreign_keys=[self.type_key], lazy=False) # type: ignore # pyright: ignore[reportArgumentType]
def name(self) -> Mapped[str]:
return mapped_column(nullable=False, default="")
@declared_attr
def entry_id(self) -> Mapped[int]:
@@ -42,50 +35,14 @@ class BaseField(Base):
def entry(self) -> Mapped[Entry]:
return relationship(foreign_keys=[self.entry_id]) # type: ignore # pyright: ignore[reportArgumentType]
@declared_attr
def position(self) -> Mapped[int]:
return mapped_column(default=0)
@override
def __hash__(self):
return hash(self.__key())
def __key(self): # pyright: ignore[reportUnknownParameterType]
raise NotImplementedError
value: Any # pyright: ignore
class BooleanField(BaseField):
__tablename__ = "boolean_fields"
value: Mapped[bool]
def __key(self):
return (self.type, self.value)
@override
def __eq__(self, value: object) -> bool:
if isinstance(value, BooleanField):
return self.__key() == value.__key()
raise NotImplementedError
class TextField(BaseField):
__tablename__ = "text_fields"
value: Mapped[str | None]
def __key(self) -> tuple[ValueType, str | None]:
return self.type, self.value
@override
def __eq__(self, value: object) -> bool:
if isinstance(value, TextField):
return self.__key() == value.__key()
elif isinstance(value, DatetimeField):
return False
raise NotImplementedError
is_multiline: Mapped[bool] = mapped_column(nullable=False, default=False)
class DatetimeField(BaseField):
@@ -93,52 +50,56 @@ class DatetimeField(BaseField):
value: Mapped[str | None]
def __key(self):
return (self.type, self.value)
@override
def __eq__(self, value: object) -> bool:
if isinstance(value, DatetimeField):
return self.__key() == value.__key()
raise NotImplementedError
class BaseFieldTemplate(Base):
__abstract__ = True
@declared_attr
def id(self) -> Mapped[int]:
return mapped_column(primary_key=True, autoincrement=True)
@declared_attr
def name(self) -> Mapped[str]:
return mapped_column(nullable=False, default="")
@dataclass
class DefaultField:
id: int
name: str
type: FieldTypeEnum
is_default: bool = field(default=False)
class TextFieldTemplate(BaseFieldTemplate):
__tablename__ = "text_field_templates"
is_multiline: Mapped[bool] = mapped_column(nullable=False, default=False)
class FieldID(Enum):
"""Only for bootstrapping content of DB table."""
class DatetimeFieldTemplate(BaseFieldTemplate):
__tablename__ = "datetime_field_templates"
TITLE = DefaultField(id=0, name="Title", type=FieldTypeEnum.TEXT_LINE, is_default=True)
AUTHOR = DefaultField(id=1, name="Author", type=FieldTypeEnum.TEXT_LINE)
ARTIST = DefaultField(id=2, name="Artist", type=FieldTypeEnum.TEXT_LINE)
URL = DefaultField(id=3, name="URL", type=FieldTypeEnum.TEXT_LINE)
DESCRIPTION = DefaultField(id=4, name="Description", type=FieldTypeEnum.TEXT_BOX)
NOTES = DefaultField(id=5, name="Notes", type=FieldTypeEnum.TEXT_BOX)
COLLATION = DefaultField(id=9, name="Collation", type=FieldTypeEnum.TEXT_LINE)
DATE = DefaultField(id=10, name="Date", type=FieldTypeEnum.DATETIME)
DATE_CREATED = DefaultField(id=11, name="Date Created", type=FieldTypeEnum.DATETIME)
DATE_MODIFIED = DefaultField(id=12, name="Date Modified", type=FieldTypeEnum.DATETIME)
DATE_TAKEN = DefaultField(id=13, name="Date Taken", type=FieldTypeEnum.DATETIME)
DATE_PUBLISHED = DefaultField(id=14, name="Date Published", type=FieldTypeEnum.DATETIME)
# ARCHIVED = DefaultField(id=15, name="Archived", type=CheckboxField.checkbox)
# FAVORITE = DefaultField(id=16, name="Favorite", type=CheckboxField.checkbox)
BOOK = DefaultField(id=17, name="Book", type=FieldTypeEnum.TEXT_LINE)
COMIC = DefaultField(id=18, name="Comic", type=FieldTypeEnum.TEXT_LINE)
SERIES = DefaultField(id=19, name="Series", type=FieldTypeEnum.TEXT_LINE)
MANGA = DefaultField(id=20, name="Manga", type=FieldTypeEnum.TEXT_LINE)
SOURCE = DefaultField(id=21, name="Source", type=FieldTypeEnum.TEXT_LINE)
DATE_UPLOADED = DefaultField(id=22, name="Date Uploaded", type=FieldTypeEnum.DATETIME)
DATE_RELEASED = DefaultField(id=23, name="Date Released", type=FieldTypeEnum.DATETIME)
VOLUME = DefaultField(id=24, name="Volume", type=FieldTypeEnum.TEXT_LINE)
ANTHOLOGY = DefaultField(id=25, name="Anthology", type=FieldTypeEnum.TEXT_LINE)
MAGAZINE = DefaultField(id=26, name="Magazine", type=FieldTypeEnum.TEXT_LINE)
PUBLISHER = DefaultField(id=27, name="Publisher", type=FieldTypeEnum.TEXT_LINE)
GUEST_ARTIST = DefaultField(id=28, name="Guest Artist", type=FieldTypeEnum.TEXT_LINE)
COMPOSER = DefaultField(id=29, name="Composer", type=FieldTypeEnum.TEXT_LINE)
COMMENTS = DefaultField(id=30, name="Comments", type=FieldTypeEnum.TEXT_LINE)
# Used for migrating legacy libraries.
# Legacy JSON libraries (<v9.4) use an integer ID.
# SQLite libraries 6 until 200 use a slugfield name (e.g. "DATE_CREATED").
LEGACY_FIELD_MAP = {
0: {"type": TextField, "name": "Title", "is_multiline": False},
1: {"type": TextField, "name": "Author", "is_multiline": False},
2: {"type": TextField, "name": "Artist", "is_multiline": False},
3: {"type": TextField, "name": "URL", "is_multiline": False},
4: {"type": TextField, "name": "Description", "is_multiline": True},
5: {"type": TextField, "name": "Notes", "is_multiline": True},
9: {"type": TextField, "name": "Collation", "is_multiline": False},
10: {"type": DatetimeField, "name": "Date", "is_multiline": False},
11: {"type": DatetimeField, "name": "Date Created"},
12: {"type": DatetimeField, "name": "Date Modified"},
13: {"type": DatetimeField, "name": "Date Taken"},
14: {"type": DatetimeField, "name": "Date Published"},
17: {"type": TextField, "name": "Book", "is_multiline": False},
18: {"type": TextField, "name": "Comic", "is_multiline": False},
19: {"type": TextField, "name": "Series", "is_multiline": False},
20: {"type": TextField, "name": "Manga", "is_multiline": False},
21: {"type": TextField, "name": "Source", "is_multiline": False},
22: {"type": DatetimeField, "name": "Date Uploaded"},
23: {"type": DatetimeField, "name": "Date Released"},
24: {"type": TextField, "name": "Volume", "is_multiline": False},
25: {"type": TextField, "name": "Anthology", "is_multiline": False},
26: {"type": TextField, "name": "Magazine", "is_multiline": False},
27: {"type": TextField, "name": "Publisher", "is_multiline": False},
28: {"type": TextField, "name": "Guest Artist", "is_multiline": False},
29: {"type": TextField, "name": "Composer", "is_multiline": False},
30: {"type": TextField, "name": "Comments", "is_multiline": True},
}

526
src/tagstudio/core/library/alchemy/library.py
View File

@@ -11,7 +11,7 @@ import re
import shutil
import time
import unicodedata
from collections.abc import Iterable, Iterator
from collections.abc import Iterable, Iterator, Sequence
from dataclasses import dataclass
from datetime import UTC, datetime
from os import makedirs
@@ -82,14 +82,16 @@ from tagstudio.core.library.alchemy.db import make_tables
from tagstudio.core.library.alchemy.enums import (
MAX_SQL_VARIABLES,
BrowsingState,
FieldTypeEnum,
SortingModeEnum,
)
from tagstudio.core.library.alchemy.fields import (
LEGACY_FIELD_MAP,
BaseField,
BaseFieldTemplate,
DatetimeField,
FieldID,
DatetimeFieldTemplate,
TextField,
TextFieldTemplate,
)
from tagstudio.core.library.alchemy.joins import TagEntry, TagParent
from tagstudio.core.library.alchemy.models import (
@@ -100,7 +102,6 @@ from tagstudio.core.library.alchemy.models import (
Tag,
TagAlias,
TagColorGroup,
ValueType,
Version,
)
from tagstudio.core.library.alchemy.visitors import SQLBoolExpressionBuilder
@@ -141,6 +142,7 @@ def slugify(input_string: str, allow_reserved: bool = False) -> str:
def get_default_tags() -> tuple[Tag, ...]:
"""Return the built-in tags for a new TagStudio library."""
meta_tag = Tag(
id=TAG_META,
name="Meta Tags",
@@ -171,6 +173,20 @@ def get_default_tags() -> tuple[Tag, ...]:
return archive_tag, favorite_tag, meta_tag
def get_default_field_templates() -> tuple[BaseFieldTemplate, ...]:
"""Return the default field templates for a new TagStudio library."""
title = TextFieldTemplate(name="Title")
author = TextFieldTemplate(name="Author")
artist = TextFieldTemplate(name="Artist")
url = TextFieldTemplate(name="URL")
description = TextFieldTemplate(name="Description", is_multiline=True)
notes = TextFieldTemplate(name="Notes", is_multiline=True)
comments = TextFieldTemplate(name="Comments", is_multiline=True)
date = DatetimeFieldTemplate(name="Date")
return title, author, artist, url, description, notes, comments, date
# The difference in the number of default JSON tags vs default tags in the current version.
DEFAULT_TAG_DIFF: int = len(get_default_tags()) - len([TAG_ARCHIVED, TAG_FAVORITE])
@@ -301,24 +317,48 @@ class Library:
path=entry.path / entry.filename,
folder=folder,
fields=[],
id=entry.id + 1, # JSON IDs start at 0 instead of 1
id=entry.id + 1, # NOTE: JSON IDs start at 0 instead of 1
date_added=datetime.now(),
)
for entry in json_lib.entries
]
)
for entry in json_lib.entries:
for field in entry.fields: # pyright: ignore[reportUnknownVariableType]
for k, v in field.items(): # pyright: ignore[reportUnknownVariableType]
for legacy_field_id, value in field.items(): # pyright: ignore[reportUnknownVariableType]
# Old tag fields get added as tags
if k in LEGACY_TAG_FIELD_IDS:
self.add_tags_to_entries(entry_ids=entry.id + 1, tag_ids=v)
if legacy_field_id in LEGACY_TAG_FIELD_IDS:
self.add_tags_to_entries(entry_ids=entry.id + 1, tag_ids=value)
else:
self.add_field_to_entry(
entry_id=(entry.id + 1), # JSON IDs start at 0 instead of 1
field_id=self.get_field_name_from_id(k),
value=v,
)
try:
if LEGACY_FIELD_MAP[legacy_field_id]["type"] == TextField:
self.add_text_field_to_entry(
entry_id=(
entry.id + 1
), # NOTE: JSON IDs start at 0 instead of 1
name=str(LEGACY_FIELD_MAP[legacy_field_id]["name"]),
value=value,
is_multiline=bool(
LEGACY_FIELD_MAP[legacy_field_id]["is_multiline"]
),
)
elif LEGACY_FIELD_MAP[legacy_field_id]["type"] == DatetimeField:
self.add_text_field_to_entry(
entry_id=(
entry.id + 1
), # NOTE: JSON IDs start at 0 instead of 1
name=str(LEGACY_FIELD_MAP[legacy_field_id]["name"]),
value=value,
)
except Exception as e:
logger.error(
"[Library][JSON Migration] Error reading field",
error=e,
entry_id=entry.id + 1,
legacy_field_id=legacy_field_id,
value=value,
)
# Preferences
self.set_prefs(LibraryPrefs.EXTENSION_LIST, [x.strip(".") for x in json_lib.ext_list])
@@ -327,12 +367,6 @@ class Library:
end_time = time.time()
logger.info(f"Library Converted! ({format_timespan(end_time - start_time)})")
def get_field_name_from_id(self, field_id: int) -> FieldID | None:
for f in FieldID:
if field_id == f.value.id:
return f
return None
def tag_display_name(self, tag: Tag | None) -> str:
if not tag:
return "<NO TAG>"
@@ -457,6 +491,22 @@ class Library:
except IntegrityError:
session.rollback()
# Add default field templates
if is_new:
for ft in get_default_field_templates():
try:
if type(ft) is TextFieldTemplate:
session.add(
TextFieldTemplate(name=ft.name, is_multiline=ft.is_multiline)
)
elif type(ft) is DatetimeFieldTemplate:
session.add(DatetimeFieldTemplate(name=ft.name))
session.commit()
except IntegrityError:
logger.info("[Library] FieldTemplate already exists", field_template=ft)
session.rollback()
# Ensure version rows are present
with catch_warnings(record=True):
# NOTE: The "Preferences" table is depreciated and will be removed in the future.
@@ -490,22 +540,6 @@ class Library:
except IntegrityError:
session.rollback()
for field in FieldID:
try:
session.add(
ValueType(
key=field.name,
name=field.value.name,
type=field.value.type,
position=field.value.id,
is_default=field.value.is_default,
)
)
session.commit()
except IntegrityError:
logger.debug("ValueType already exists", field=field)
session.rollback()
# check if folder matching current path exists already
self.folder = session.scalar(select(Folder).where(Folder.path == library_dir))
if not self.folder:
@@ -545,18 +579,21 @@ class Library:
if loaded_db_version < 103:
self.__apply_db103_schema_changes(session)
if loaded_db_version == 6:
self.__apply_repairs_for_db6(session)
self.__apply_db6_data_repairs(session)
if loaded_db_version >= 6 and loaded_db_version < 8:
self.__apply_db8_default_data(session)
self.__apply_db8_data_additions(session)
if loaded_db_version < 9:
self.__apply_db9_filename_population(session)
self.__apply_db9_data_additions(session)
if loaded_db_version < 100:
self.__apply_db100_parent_repairs(session)
self.__apply_db100_data_repairs(session)
if loaded_db_version < 102:
self.__apply_db102_repairs(session)
self.__apply_db102_data_repairs(session)
if loaded_db_version < 103:
self.__apply_db103_default_data(session)
self.__apply_db103_data_additions(session)
if loaded_db_version < 200:
self.__apply_db200_schema_changes(session)
self.__apply_db200_data_repairs(session)
# Convert file extension list to ts_ignore file, if a .ts_ignore file does not exist
self.migrate_sql_to_ts_ignore(library_dir)
@@ -569,19 +606,10 @@ class Library:
self.library_dir = library_dir
return LibraryStatus(success=True, library_path=library_dir)
def __apply_repairs_for_db6(self, session: Session):
def __apply_db6_data_repairs(self, session: Session):
"""Apply database repairs introduced in DB_VERSION 7."""
logger.info("[Library][Migration] Applying patches to DB_VERSION: 6 library...")
with session:
# Repair "Description" fields with a TEXT_LINE key instead of a TEXT_BOX key.
desc_stmt = (
update(ValueType)
.where(ValueType.key == FieldID.DESCRIPTION.name)
.values(type=FieldTypeEnum.TEXT_BOX.name)
)
session.execute(desc_stmt)
session.flush()
# Repair tags that may have a disambiguation_id pointing towards a deleted tag.
all_tag_ids: set[int] = {tag.id for tag in self.tags}
disam_stmt = (
@@ -610,7 +638,7 @@ class Library:
)
session.rollback()
def __apply_db8_default_data(self, session: Session):
def __apply_db8_data_additions(self, session: Session):
"""Apply default data changes introduced in DB_VERSION 8."""
tag_colors: list[TagColorGroup] = default_color_groups.standard()
tag_colors += default_color_groups.pastels()
@@ -676,14 +704,14 @@ class Library:
)
session.rollback()
def __apply_db9_filename_population(self, session: Session):
def __apply_db9_data_additions(self, session: Session):
"""Populate the filename column introduced in DB_VERSION 9."""
for entry in self.all_entries():
session.merge(entry).filename = entry.path.name
session.commit()
logger.info("[Library][Migration] Populated filename column in entries table")
def __apply_db100_parent_repairs(self, session: Session):
def __apply_db100_data_repairs(self, session: Session):
"""Swap the child_id and parent_id values in the TagParent table."""
with session:
# Repair parent-child tag relationships that are the wrong way around.
@@ -695,7 +723,7 @@ class Library:
session.commit()
logger.info("[Library][Migration] Refactored TagParent table")
def __apply_db102_repairs(self, session: Session):
def __apply_db102_data_repairs(self, session: Session):
"""Repair tag_parents rows with references to deleted tags."""
with session:
all_tag_ids: list[int] = [t.id for t in self.tags]
@@ -720,13 +748,12 @@ class Library:
)
session.rollback()
def __apply_db103_default_data(self, session: Session):
def __apply_db103_data_additions(self, session: Session):
"""Apply default data changes introduced in DB_VERSION 103."""
try:
session.query(Tag).filter(Tag.id == TAG_ARCHIVED).update({"is_hidden": True})
session.commit()
logger.info("[Library][Migration] Updated archived tag to be hidden")
session.commit()
except Exception as e:
logger.error(
"[Library][Migration] Could not update archived tag to be hidden!",
@@ -734,6 +761,106 @@ class Library:
)
session.rollback()
def __apply_db200_schema_changes(self, session: Session):
logger.info("[Library][Migration][200] Migrating schema to library version 200...")
with session:
# Drop unused 'boolean_fields' and 'value_type' tables
session.execute(text("DROP TABLE boolean_fields"))
session.execute(text("DROP TABLE value_type"))
session.commit()
logger.info("[Library][Migration][200] Dropped boolean_fields and value_type tables")
# Add 'name' column to text_fields and datetime_fields tables
stmt = text('ALTER TABLE text_fields ADD COLUMN name VARCHAR NOT NULL DEFAULT ""')
session.execute(stmt)
stmt = text('ALTER TABLE datetime_fields ADD COLUMN name VARCHAR NOT NULL DEFAULT ""')
session.execute(stmt)
session.commit()
logger.info("[Library][Migration][200] Added name columns to field tables")
# Drop unnecessary 'position' columns
session.execute(text("ALTER TABLE datetime_fields DROP COLUMN position"))
session.execute(text("ALTER TABLE text_fields DROP COLUMN position"))
session.commit()
logger.info("[Library][Migration][200] Dropped position columns to field tables")
# Add 'is_multiline' column to text_fields table
stmt = text(
"ALTER TABLE text_fields ADD COLUMN is_multiline BOOLEAN NOT NULL DEFAULT 0"
)
session.execute(stmt)
session.commit()
logger.info("[Library][Migration][200] Added is_multiline column to text_fields table")
# Move values from old `type_key` columns into new `name` columns
session.execute(text("UPDATE text_fields SET name = type_key"))
session.execute(text("UPDATE datetime_fields SET name = type_key"))
session.commit()
logger.info("[Library][Migration][200] Moved values from type_key columns to name")
# TODO: Remove `type_key` columns from text_fields and datetime_fields tables.
# See issue with dropping columns foreign keys in SQLite:
# https://www.sqlite.org/lang_altertable.html#making_other_kinds_of_table_schema_changes
# Change `name` values to title case
for text_field in session.execute(select(TextField)).scalars():
# NOTE: The only exception to the "Title Case" conversion is the "URL" field.
text_field.name = text_field.name.title().replace("Url", "URL").replace("_", " ")
logger.info("[Library][Migration][200] Normalized TextField names")
session.commit()
for datetime_field in session.execute(select(DatetimeField)).scalars():
datetime_field.name = datetime_field.name.title().replace("_", " ")
logger.info("[Library][Migration][200] Normalized DatetimeField names")
session.commit()
# Add correct `is_multiline` values to text_fields table
text_boxes = [
x.get("name") for x in LEGACY_FIELD_MAP.values() if x.get("is_multiline") is True
]
update_stmt = (
update(TextField).where(TextField.name.in_(text_boxes)).values(is_multiline=True)
)
session.execute(update_stmt)
logger.info(
"[Library][Migration][200] Updated is_multiline columns for legacy TEXT_BOX fields"
)
session.commit()
pass
def __apply_db200_data_repairs(self, session: Session):
logger.info("[Library][Migration] Repairing data for library below version 200...")
with session:
# Repair legacy "Description" fields to use is_multiline = True
desc_stmt = (
update(TextField)
.where(TextField.name == "Description" and TextField.is_multiline == False) # noqa: E712
.values(is_multiline=True)
)
session.execute(desc_stmt)
# Repair legacy "Comments" fields to use is_multiline = True
comm_stmt = (
update(TextField)
.where(TextField.name == "Comments" and TextField.is_multiline == False) # noqa: E712
.values(is_multiline=True)
)
session.execute(comm_stmt)
session.commit()
# Add default field templates
for ft in get_default_field_templates():
try:
if type(ft) is TextFieldTemplate:
session.add(TextFieldTemplate(name=ft.name, is_multiline=ft.is_multiline))
elif type(ft) is DatetimeFieldTemplate:
session.add(DatetimeFieldTemplate(name=ft.name))
session.commit()
except IntegrityError:
logger.info("[Library] FieldTemplate already exists", field_template=ft)
session.rollback()
def migrate_sql_to_ts_ignore(self, library_dir: Path):
# Do not continue if existing '.ts_ignore' file is found
if Path(library_dir / TS_FOLDER_NAME / IGNORE_NAME).exists():
@@ -763,15 +890,11 @@ class Library:
f.writelines([f"{prefix}*.{x.lstrip('.')}\n" for x in extensions])
@property
def default_fields(self) -> list[BaseField]:
def field_templates(self) -> Sequence[BaseFieldTemplate]:
with Session(self.engine) as session:
types = session.scalars(
select(ValueType).where(
# check if field is default
ValueType.is_default.is_(True)
)
)
return [x.as_field for x in types]
text_templates = list(session.scalars(select(TextFieldTemplate)))
datetime_templates = list(session.scalars(select(DatetimeFieldTemplate)))
return text_templates + datetime_templates
def get_entry(self, entry_id: int) -> Entry | None:
"""Load entry without joins."""
@@ -1002,8 +1125,8 @@ class Library:
session.query(Entry).where(Entry.id.in_(sub_list)).delete()
session.commit()
def has_path_entry(self, path: Path) -> bool:
"""Check if item with given path is in library already."""
def has_entry_with_path(self, path: Path) -> bool:
"""Check if an entry with this path is in the library."""
with Session(self.engine) as session:
return session.query(exists().where(Entry.path == path)).scalar()
@@ -1151,7 +1274,7 @@ class Library:
Returns True if the action succeeded and False if the path already exists.
"""
if self.has_path_entry(path):
if self.has_entry_with_path(path):
return False
if isinstance(entry_id, Entry):
entry_id = entry_id.id
@@ -1195,165 +1318,136 @@ class Library:
return False
return True
def update_field_position(
self,
field_class: type[BaseField],
field_type: str,
entry_ids: list[int] | int,
):
if isinstance(entry_ids, int):
entry_ids = [entry_ids]
with Session(self.engine) as session:
for entry_id in entry_ids:
rows = list(
session.scalars(
select(field_class)
.where(
and_(
field_class.entry_id == entry_id,
field_class.type_key == field_type,
)
)
.order_by(field_class.id)
)
)
# Reassign `order` starting from 0
for index, row in enumerate(rows):
row.position = index
session.add(row)
session.flush()
if rows:
session.commit()
def remove_entry_field(
self,
field: BaseField,
entry_ids: list[int],
) -> None:
FieldClass = type(field) # noqa: N806
field_ = type(field)
logger.info(
"remove_entry_field",
field=field,
type=field_,
entry_ids=entry_ids,
field_type=field.type,
cls=FieldClass,
pos=field.position,
)
with Session(self.engine) as session:
# remove all fields matching entry and field_type
delete_stmt = delete(FieldClass).where(
delete_stmt = delete(field_).where(
and_(
FieldClass.position == field.position,
FieldClass.type_key == field.type_key,
FieldClass.entry_id.in_(entry_ids),
field_.id == field.id,
)
)
session.execute(delete_stmt)
session.commit()
# recalculate the remaining positions
# self.update_field_position(type(field), field.type, entry_ids)
def update_entry_field(
self,
entry_ids: list[int] | int,
field: BaseField,
content: str | datetime,
def update_text_field(
self, entry_ids: list[int] | int, field: TextField, value: str, is_multiline: bool
):
"""Update a TextField field on one or more Entries."""
if isinstance(entry_ids, int):
entry_ids = [entry_ids]
FieldClass = type(field) # noqa: N806
field_ = type(field)
with Session(self.engine) as session:
update_stmt = (
update(FieldClass)
update(field_)
.where(
and_(
FieldClass.position == field.position,
FieldClass.type == field.type,
FieldClass.entry_id.in_(entry_ids),
field_.id == field.id,
)
)
.values(value=content)
.values(value=value, is_multiline=is_multiline)
)
session.execute(update_stmt)
session.commit()
@property
def field_types(self) -> dict[str, ValueType]:
with Session(self.engine) as session:
return {x.key: x for x in session.scalars(select(ValueType)).all()}
def get_value_type(self, field_key: str) -> ValueType:
with Session(self.engine) as session:
field = unwrap(session.scalar(select(ValueType).where(ValueType.key == field_key)))
session.expunge(field)
return field
def add_field_to_entry(
def update_datetime_field(
self,
entry_id: int,
*,
field: ValueType | None = None,
field_id: FieldID | str | None = None,
value: str | datetime | None = None,
entry_ids: list[int] | int,
field: DatetimeField,
value: datetime,
):
"""Update a DatetimeField field on one or more Entries."""
if isinstance(entry_ids, int):
entry_ids = [entry_ids]
field_ = type(field)
with Session(self.engine) as session:
update_stmt = (
update(field_)
.where(
and_(
field_.id == field.id,
)
)
.values(value=value)
)
session.execute(update_stmt)
session.commit()
def add_text_field_to_entry(
self, entry_id: int, name: str, value: str | None = None, is_multiline: bool = False
) -> bool:
"""Add a TextField field to an Entry."""
logger.info(
"[Library][add_field_to_entry]",
"[Library] Adding text field to entry",
entry_id=entry_id,
field_type=field,
field_id=field_id,
name=name,
value=value,
is_multiline=is_multiline,
)
# supply only instance or ID, not both
assert bool(field) != (field_id is not None)
if not field:
if isinstance(field_id, FieldID):
field_id = field_id.name
field = self.get_value_type(unwrap(field_id))
field_model: TextField | DatetimeField
if field.type in (FieldTypeEnum.TEXT_LINE, FieldTypeEnum.TEXT_BOX):
field_model = TextField(
type_key=field.key,
value=value or "",
)
elif field.type == FieldTypeEnum.DATETIME:
field_model = DatetimeField(
type_key=field.key,
value=value,
)
else:
raise NotImplementedError(f"field type not implemented: {field.type}")
field = TextField(entry_id=entry_id, name=name, value=value, is_multiline=is_multiline)
with Session(self.engine) as session:
try:
field_model.entry_id = entry_id
session.add(field_model)
session.add(field)
session.flush()
session.commit()
except IntegrityError as e:
logger.error(e)
session.rollback()
return False
# TODO - trigger error signal
# recalculate the positions of fields
self.update_field_position(
field_class=type(field_model),
field_type=field.key,
entry_ids=entry_id,
return True
def add_datetime_field_to_entry(
self,
entry_id: int,
name: str,
value: str | None = None,
) -> bool:
"""Add a DatetimeField field to an Entry."""
logger.info(
"[Library] Adding datetime field to entry",
entry_id=entry_id,
name=name,
value=value,
)
field = DatetimeField(
entry_id=entry_id,
name=name,
value=value,
)
with Session(self.engine) as session:
try:
session.add(field)
session.flush()
session.commit()
except IntegrityError as e:
logger.error(e)
session.rollback()
return False
return True
def tag_from_strings(self, strings: list[str] | str) -> list[int]:
@@ -1942,39 +2036,75 @@ class Library:
session.commit()
# TODO - try/except
def mirror_entry_fields(self, *entries: Entry) -> None:
def mirror_entry_fields(self, entries: list[Entry]) -> None:
"""Mirror fields among multiple Entry items."""
fields = {}
# load all fields
existing_fields = {field.type_key for field in entries[0].fields}
for entry in entries:
for entry_field in entry.fields:
fields[entry_field.type_key] = entry_field
all_tuples_to_fields_map = {}
# assign the field to all entries
# Track all fields across all entries
for entry in entries:
for field_key, field in fields.items(): # pyright: ignore[reportUnknownVariableType]
if field_key not in existing_fields:
self.add_field_to_entry(
entry_id=entry.id,
field_id=field.type_key,
value=field.value,
)
for field in entry.fields:
field_tuple: tuple | None = None
if type(field) is TextField:
field_tuple = (type(field), field.name, field.value, field.is_multiline)
elif type(field) is DatetimeField:
field_tuple = (type(field), field.name, field.value)
all_tuples_to_fields_map[field_tuple] = field
logger.info(
"[Library][mirror_fields]", entry_id=entry.id, field_count_before=len(entry.fields)
)
# Apply all (remaining) fields to all entries, avoiding duplicates
for entry in entries:
for field_tuple, field in all_tuples_to_fields_map.items(): # pyright: ignore[reportUnknownVariableType]
entry_field_tuples: set[tuple[Any, ...]] = set() # pyright: ignore[reportExplicitAny]
# Locally process the entry's fields into parsable tuples
for entry_field in entry.fields:
entry_field_tuple: tuple | None = None
if type(entry_field) is TextField:
entry_field_tuple = (
type(entry_field),
entry_field.name,
entry_field.value,
entry_field.is_multiline,
)
entry_field_tuples.add(entry_field_tuple)
elif type(entry_field) is DatetimeField:
entry_field_tuple = (type(entry_field), entry_field.name, entry_field.value)
entry_field_tuples.add(entry_field_tuple)
if field_tuple not in entry_field_tuples:
if type(field) is TextField:
self.add_text_field_to_entry(
entry_id=entry.id,
name=field.name,
value=field.value,
is_multiline=field.is_multiline,
)
elif type(field) is DatetimeField:
self.add_datetime_field_to_entry(
entry_id=entry.id, name=field.name, value=field.value
)
logger.info(
"[Library][mirror_fields]", entry_id=entry.id, field_count_after=len(entry.fields)
)
def merge_entries(self, from_entry: Entry, into_entry: Entry) -> bool:
"""Add fields and tags from the first entry to the second, and then delete the first."""
success = True
for field in from_entry.fields:
result = self.add_field_to_entry(
entry_id=into_entry.id,
field_id=field.type_key,
value=field.value,
success = False
try:
self.mirror_entry_fields([from_entry, into_entry])
tag_ids = [tag.id for tag in from_entry.tags]
self.add_tags_to_entries(into_entry.id, tag_ids)
self.remove_entries([from_entry.id])
success = True
except Exception as e:
logger.error(
"[Library][merge_entries] Could not merge entires",
error=e,
from_entry_id=from_entry.id,
into_entry_id=into_entry.id,
)
if not result:
success = False
tag_ids = [tag.id for tag in from_entry.tags]
self.add_tags_to_entries(into_entry.id, tag_ids)
self.remove_entries([from_entry.id])
return success

56
src/tagstudio/core/library/alchemy/models.py
View File

@@ -6,16 +6,14 @@ from datetime import datetime as dt
from pathlib import Path
from typing import override
from sqlalchemy import JSON, ForeignKey, ForeignKeyConstraint, Integer, event
from sqlalchemy import JSON, ForeignKey, ForeignKeyConstraint, Integer
from sqlalchemy.orm import Mapped, mapped_column, relationship
from typing_extensions import deprecated
from tagstudio.core.constants import TAG_ARCHIVED, TAG_FAVORITE
from tagstudio.core.library.alchemy.db import Base, PathType
from tagstudio.core.library.alchemy.enums import FieldTypeEnum
from tagstudio.core.library.alchemy.fields import (
BaseField,
BooleanField,
DatetimeField,
TextField,
)
@@ -224,7 +222,6 @@ class Entry(Base):
fields: list[BaseField] = []
fields.extend(self.text_fields)
fields.extend(self.datetime_fields)
fields = sorted(fields, key=lambda field: field.type.position)
return fields
@property
@@ -276,57 +273,6 @@ class Entry(Base):
self.tags.remove(tag)
class ValueType(Base):
"""Define Field Types in the Library.
Example:
key: content_tags (this field is slugified `name`)
name: Content Tags (this field is human readable name)
kind: type of content (Text Line, Text Box, Tags, Datetime, Checkbox)
is_default: Should the field be present in new Entry?
order: position of the field widget in the Entry form
"""
__tablename__ = "value_type"
key: Mapped[str] = mapped_column(primary_key=True)
name: Mapped[str] = mapped_column(nullable=False)
type: Mapped[FieldTypeEnum] = mapped_column(default=FieldTypeEnum.TEXT_LINE)
is_default: Mapped[bool] # pyright: ignore[reportUninitializedInstanceVariable]
position: Mapped[int] # pyright: ignore[reportUninitializedInstanceVariable]
# add relations to other tables
text_fields: Mapped[list[TextField]] = relationship("TextField", back_populates="type")
datetime_fields: Mapped[list[DatetimeField]] = relationship(
"DatetimeField", back_populates="type"
)
boolean_fields: Mapped[list[BooleanField]] = relationship("BooleanField", back_populates="type")
@property
def as_field(self) -> BaseField:
FieldClass = { # noqa: N806
FieldTypeEnum.TEXT_LINE: TextField,
FieldTypeEnum.TEXT_BOX: TextField,
FieldTypeEnum.DATETIME: DatetimeField,
FieldTypeEnum.BOOLEAN: BooleanField,
}
return FieldClass[self.type](
type_key=self.key,
position=self.position,
)
@event.listens_for(ValueType, "before_insert")
def slugify_field_key(mapper, connection, target): # pyright: ignore
"""Slugify the field key before inserting into the database."""
if not target.key:
from tagstudio.core.library.alchemy.library import slugify
target.key = slugify(target.tag)
# NOTE: The "Preferences" table has been depreciated as of TagStudio 9.5.4
# and is set to be removed in a future release.
@deprecated("Use `Version` for storing version, and `ts_ignore` system for file exclusion.")

4
src/tagstudio/core/library/refresh.py
View File

@@ -145,7 +145,7 @@ class RefreshTracker:
dir_file_count += 1
self.library.included_files.add(f)
if not self.library.has_path_entry(f):
if not self.library.has_entry_with_path(f):
self.files_not_in_library.append(f)
end_time_total = time()
@@ -190,7 +190,7 @@ class RefreshTracker:
relative_path = f.relative_to(library_dir)
if not self.library.has_path_entry(relative_path):
if not self.library.has_entry_with_path(relative_path):
self.files_not_in_library.append(relative_path)
except ValueError:
logger.info("[Refresh]: ValueError when refreshing directory with wcmatch!")

170
src/tagstudio/core/ts_core.py
View File

@@ -4,19 +4,13 @@
"""The core classes and methods of TagStudio."""
import json
import re
from functools import lru_cache
from pathlib import Path
import requests
import structlog
from tagstudio.core.constants import TS_FOLDER_NAME
from tagstudio.core.library.alchemy.fields import FieldID
from tagstudio.core.library.alchemy.library import Library
from tagstudio.core.library.alchemy.models import Entry
from tagstudio.core.utils.types import unwrap
logger = structlog.get_logger(__name__)
@@ -27,170 +21,6 @@ class TagStudioCore:
def __init__(self):
self.lib: Library = Library()
@classmethod
def get_gdl_sidecar(cls, filepath: Path, source: str = "") -> dict:
"""Attempt to open and dump a Gallery-DL Sidecar file for the filepath.
Return a formatted object with notable values or an empty object if none is found.
"""
raise NotImplementedError("This method is currently broken and needs to be fixed.")
info = {}
_filepath = filepath.parent / (filepath.name + ".json")
# NOTE: This fixes an unknown (recent?) bug in Gallery-DL where Instagram sidecar
# files may be downloaded with indices starting at 1 rather than 0, unlike the posts.
# This may only occur with sidecar files that are downloaded separate from posts.
if source == "instagram" and not _filepath.is_file():
newstem = _filepath.stem[:-16] + "1" + _filepath.stem[-15:]
_filepath = _filepath.parent / (newstem + ".json")
logger.info("get_gdl_sidecar", filepath=filepath, source=source, sidecar=_filepath)
try:
with open(_filepath, encoding="utf8") as f:
json_dump = json.load(f)
if not json_dump:
return {}
if source == "twitter":
info[FieldID.DESCRIPTION] = json_dump["content"].strip()
info[FieldID.DATE_PUBLISHED] = json_dump["date"]
elif source == "instagram":
info[FieldID.DESCRIPTION] = json_dump["description"].strip()
info[FieldID.DATE_PUBLISHED] = json_dump["date"]
elif source == "artstation":
info[FieldID.TITLE] = json_dump["title"].strip()
info[FieldID.ARTIST] = json_dump["user"]["full_name"].strip()
info[FieldID.DESCRIPTION] = json_dump["description"].strip()
info[FieldID.TAGS] = json_dump["tags"]
# info["tags"] = [x for x in json_dump["mediums"]["name"]]
info[FieldID.DATE_PUBLISHED] = json_dump["date"]
elif source == "newgrounds":
# info["title"] = json_dump["title"]
# info["artist"] = json_dump["artist"]
# info["description"] = json_dump["description"]
info[FieldID.TAGS] = json_dump["tags"]
info[FieldID.DATE_PUBLISHED] = json_dump["date"]
info[FieldID.ARTIST] = json_dump["user"].strip()
info[FieldID.DESCRIPTION] = json_dump["description"].strip()
info[FieldID.SOURCE] = json_dump["post_url"].strip()
except Exception:
logger.exception("Error handling sidecar file.", path=_filepath)
return info
# def scrape(self, entry_id):
# entry = self.lib.get_entry(entry_id)
# if entry.fields:
# urls: list[str] = []
# if self.lib.get_field_index_in_entry(entry, 21):
# urls.extend([self.lib.get_field_attr(entry.fields[x], 'content')
# for x in self.lib.get_field_index_in_entry(entry, 21)])
# if self.lib.get_field_index_in_entry(entry, 3):
# urls.extend([self.lib.get_field_attr(entry.fields[x], 'content')
# for x in self.lib.get_field_index_in_entry(entry, 3)])
# # try:
# if urls:
# for url in urls:
# url = "https://" + url if 'https://' not in url else url
# html_doc = requests.get(url).text
# soup = bs(html_doc, "html.parser")
# print(soup)
# input()
# # except:
# # # print("Could not resolve URL.")
# # pass
@classmethod
def match_conditions(cls, lib: Library, entry_id: int) -> bool:
"""Match defined conditions against a file to add Entry data."""
# TODO - what even is this file format?
# TODO: Make this stored somewhere better instead of temporarily in this JSON file.
cond_file = unwrap(lib.library_dir) / TS_FOLDER_NAME / "conditions.json"
if not cond_file.is_file():
return False
entry: Entry = unwrap(lib.get_entry(entry_id))
try:
with open(cond_file, encoding="utf8") as f:
json_dump = json.load(f)
for c in json_dump["conditions"]:
match: bool = False
for path_c in c["path_conditions"]:
if Path(path_c).is_relative_to(entry.path):
match = True
break
if not match:
return False
if not c.get("fields"):
return False
fields = c["fields"]
entry_field_types = {field.type_key: field for field in entry.fields}
for field in fields:
is_new = field["id"] not in entry_field_types
field_key = field["id"]
if is_new:
lib.add_field_to_entry(
entry.id, field_id=field_key, value=field["value"]
)
else:
lib.update_entry_field(entry.id, field_key, field["value"])
except Exception:
logger.exception("Error matching conditions.", entry=entry)
return False
@classmethod
def build_url(cls, entry: Entry, source: str):
"""Try to rebuild a source URL given a specific filename structure."""
source = source.lower().replace("-", " ").replace("_", " ")
if "twitter" in source:
return cls._build_twitter_url(entry)
elif "instagram" in source:
return cls._build_instagram_url(entry)
@classmethod
def _build_twitter_url(cls, entry: Entry):
"""Build a Twitter URL given a specific filename structure.
Method expects filename to be formatted as 'USERNAME_TWEET-ID_INDEX_YEAR-MM-DD'
"""
try:
stubs = str(entry.path.name).rsplit("_", 3)
url = f"www.twitter.com/{stubs[0]}/status/{stubs[-3]}/photo/{stubs[-2]}"
return url
except Exception:
logger.exception("Error building Twitter URL.", entry=entry)
return ""
@classmethod
def _build_instagram_url(cls, entry: Entry):
"""Build an Instagram URL given a specific filename structure.
Method expects filename to be formatted as 'USERNAME_POST-ID_INDEX_YEAR-MM-DD'
"""
try:
stubs = str(entry.path.name).rsplit("_", 2)
# stubs[0] = stubs[0].replace(f"{author}_", '', 1)
# print(stubs)
# NOTE: Both Instagram usernames AND their ID can have underscores in them,
# so unless you have the exact username (which can change) on hand to remove,
# your other best bet is to hope that the ID is only 11 characters long, which
# seems to more or less be the case... for now...
url = f"www.instagram.com/p/{stubs[-3][-11:]}"
return url
except Exception:
logger.exception("Error building Instagram URL.", entry=entry)
return ""
@staticmethod
@lru_cache(maxsize=1)
def get_most_recent_release_version() -> str:

2
src/tagstudio/qt/controllers/library_info_window_controller.py
View File

@@ -65,7 +65,7 @@ class LibraryInfoWindow(LibraryInfoWindowView):
def update_stats(self):
self.entry_count_label.setText(f"<b>{self.lib.entries_count}</b>")
self.tag_count_label.setText(f"<b>{len(self.lib.tags)}</b>")
self.field_count_label.setText(f"<b>{len(self.lib.field_types)}</b>")
self.field_count_label.setText(f"<b>{len(self.lib.field_templates)}</b>")
self.namespaces_count_label.setText(f"<b>{len(self.lib.namespaces)}</b>")
colors_total = 0
for c in self.lib.tag_color_groups.values():

3
src/tagstudio/qt/controllers/preview_panel_controller.py
View File

@@ -22,12 +22,15 @@ class PreviewPanel(PreviewPanelView):
self.__add_field_modal = AddFieldModal(self.lib)
self.__add_tag_modal = TagSearchModal(self.lib, is_tag_chooser=True)
@typing.override
def _add_field_button_callback(self):
self.__add_field_modal.show()
@typing.override
def _add_tag_button_callback(self):
self.__add_tag_modal.show()
@typing.override
def _set_selection_callback(self):
with catch_warnings(record=True):
self.__add_field_modal.done.disconnect()

13
src/tagstudio/qt/mixed/add_field.py
View File

@@ -19,7 +19,7 @@ from PySide6.QtWidgets import (
)
from tagstudio.core.library.alchemy.library import Library
from tagstudio.qt.translations import Translations
from tagstudio.qt.translations import FIELD_TYPE_KEYS, Translations
logger = structlog.get_logger(__name__)
@@ -73,13 +73,18 @@ class AddFieldModal(QWidget):
self.root_layout.addStretch(1)
self.root_layout.addWidget(self.button_container)
@override
def show(self):
self.list_widget.clear()
for df in self.lib.field_types.values():
item = QListWidgetItem(f"{df.name} ({df.type.value})")
item.setData(Qt.ItemDataRole.UserRole, df.key)
for field_template in self.lib.field_templates:
field_name_key: str = FIELD_TYPE_KEYS.get(
field_template.__class__.__name__, "field_type.unknown"
)
item = QListWidgetItem(f"{field_template.name} ({Translations[field_name_key]})")
item.setData(Qt.ItemDataRole.UserRole, field_template)
self.list_widget.addItem(item)
self.list_widget.setFocus()
self.list_widget.setCurrentRow(0)
super().show()

133
src/tagstudio/qt/mixed/field_containers.py
View File

@@ -15,6 +15,7 @@ from PySide6.QtGui import QGuiApplication
from PySide6.QtWidgets import (
QFrame,
QHBoxLayout,
QListWidgetItem,
QMessageBox,
QScrollArea,
QSizePolicy,
@@ -23,11 +24,13 @@ from PySide6.QtWidgets import (
)
from tagstudio.core.enums import Theme
from tagstudio.core.library.alchemy.enums import FieldTypeEnum
from tagstudio.core.library.alchemy.fields import (
BaseField,
BaseFieldTemplate,
DatetimeField,
DatetimeFieldTemplate,
TextField,
TextFieldTemplate,
)
from tagstudio.core.library.alchemy.library import Library
from tagstudio.core.library.alchemy.models import Entry, Tag
@@ -36,7 +39,7 @@ from tagstudio.qt.controllers.tag_box_controller import TagBoxWidget
from tagstudio.qt.mixed.datetime_picker import DatetimePicker
from tagstudio.qt.mixed.field_widget import FieldContainer
from tagstudio.qt.mixed.text_field import TextWidget
from tagstudio.qt.translations import Translations
from tagstudio.qt.translations import FIELD_TYPE_KEYS, Translations
from tagstudio.qt.views.edit_text_box_modal import EditTextBox
from tagstudio.qt.views.edit_text_line_modal import EditTextLine
from tagstudio.qt.views.panel_modal import PanelModal
@@ -205,7 +208,7 @@ class FieldContainers(QWidget):
def remove_field_prompt(self, name: str) -> str:
return Translations.format("library.field.confirm_remove", name=name)
def add_field_to_selected(self, field_list: list):
def add_field_to_selected(self, field_list: list[QListWidgetItem]):
"""Add list of entry fields to one or more selected items.
Uses the current driver selection, NOT the field containers cache.
@@ -216,11 +219,24 @@ class FieldContainers(QWidget):
fields=field_list,
)
for entry_id in self.driver.selected:
for field_item in field_list:
self.lib.add_field_to_entry(
entry_id,
field_id=field_item.data(Qt.ItemDataRole.UserRole),
for field in field_list:
field_: BaseFieldTemplate = field.data(Qt.ItemDataRole.UserRole)
logger.info(
"[FieldContainers][add_field_to_selected] Adding field",
name=field_.name,
type=field_.__class__.__name__,
)
if type(field_) is TextFieldTemplate:
self.lib.add_text_field_to_entry(
entry_id=entry_id,
name=field_.name,
is_multiline=field_.is_multiline,
)
elif type(field_) is DatetimeFieldTemplate:
self.lib.add_datetime_field_to_entry(
entry_id=entry_id,
name=field_.name,
)
def add_tags_to_selected(self, tags: int | list[int]):
"""Add list of tags to one or more selected items.
@@ -250,7 +266,12 @@ class FieldContainers(QWidget):
If True, field is not present in all selected items.
"""
logger.info("[FieldContainers][write_field_container]", index=index)
logger.info(
"[FieldContainers][write_container]",
index=index,
name=field.name,
type=field.__class__.__name__,
)
if len(self.containers) < (index + 1):
container = FieldContainer()
self.containers.append(container)
@@ -258,8 +279,13 @@ class FieldContainers(QWidget):
else:
container = self.containers[index]
if field.type.type == FieldTypeEnum.TEXT_LINE:
container.set_title(field.type.name)
# Set field title
field_name_key: str = FIELD_TYPE_KEYS.get(field.__class__.__name__, "field_type.unknown")
title = f"{field.name} ({Translations[field_name_key]})"
# Single-line Text
if type(field) is TextField and not field.is_multiline:
container.set_title(field.name)
container.set_inline(False)
# Normalize line endings in any text content.
@@ -267,19 +293,18 @@ class FieldContainers(QWidget):
assert isinstance(field.value, str | type(None))
text = field.value or ""
else:
text = "<i>Mixed Data</i>"
text = "<i>Mixed Data</i>" # TODO: Localize this
title = f"{field.type.name} ({field.type.type.value})"
inner_widget = TextWidget(title, text)
container.set_inner_widget(inner_widget)
if not is_mixed:
modal = PanelModal(
EditTextLine(field.value),
title=title,
window_title=f"Edit {field.type.type.value}",
save_callback=(
window_title=f"Edit {field.name}", # TODO: Localize this
save_callback=( # pyright: ignore[reportArgumentType]
lambda content: (
self.update_field(field, content), # type: ignore
self.update_text_field(field, content, is_multiline=False),
self.update_from_entry(self.cached_entries[0].id),
)
),
@@ -291,7 +316,7 @@ class FieldContainers(QWidget):
container.set_edit_callback(modal.show)
container.set_remove_callback(
lambda: self.remove_message_box(
prompt=self.remove_field_prompt(field.type.type.value),
prompt=self.remove_field_prompt(title),
callback=lambda: (
self.remove_field(field),
self.update_from_entry(self.cached_entries[0].id),
@@ -299,26 +324,26 @@ class FieldContainers(QWidget):
)
)
elif field.type.type == FieldTypeEnum.TEXT_BOX:
container.set_title(field.type.name)
# Multiline Text
elif type(field) is TextField and field.is_multiline:
container.set_title(field.name)
container.set_inline(False)
# Normalize line endings in any text content.
if not is_mixed:
assert isinstance(field.value, str | type(None))
text = (field.value or "").replace("\r", "\n")
else:
text = "<i>Mixed Data</i>"
title = f"{field.type.name} (Text Box)"
text = "<i>Mixed Data</i>" # TODO: Localize this
inner_widget = TextWidget(title, text)
container.set_inner_widget(inner_widget)
if not is_mixed:
modal = PanelModal(
EditTextBox(field.value),
title=title,
window_title=f"Edit {field.type.name}",
save_callback=(
window_title=f"Edit {field.name}", # TODO: Localize this
save_callback=( # pyright: ignore[reportArgumentType]
lambda content: (
self.update_field(field, content), # type: ignore
self.update_text_field(field, content, is_multiline=True),
self.update_from_entry(self.cached_entries[0].id),
)
),
@@ -326,7 +351,7 @@ class FieldContainers(QWidget):
container.set_edit_callback(modal.show)
container.set_remove_callback(
lambda: self.remove_message_box(
prompt=self.remove_field_prompt(field.type.name),
prompt=self.remove_field_prompt(field.name),
callback=lambda: (
self.remove_field(field),
self.update_from_entry(self.cached_entries[0].id),
@@ -334,20 +359,18 @@ class FieldContainers(QWidget):
)
)
elif field.type.type == FieldTypeEnum.DATETIME:
elif type(field) is DatetimeField:
logger.info("[FieldContainers][write_container] Datetime Field", field=field)
if not is_mixed:
container.set_title(field.type.name)
container.set_title(field.name)
container.set_inline(False)
title = f"{field.type.name} (Date)"
try:
assert field.value is not None
text = self.driver.settings.format_datetime(
DatetimePicker.string2dt(field.value)
)
except (ValueError, AssertionError):
title += " (Unknown Format)"
text = str(field.value)
inner_widget = TextWidget(title, text)
@@ -355,10 +378,10 @@ class FieldContainers(QWidget):
modal = PanelModal(
DatetimePicker(self.driver, field.value or dt.now()),
title=f"Edit {field.type.name}",
save_callback=(
title=f"Edit {field.name}",
save_callback=( # pyright: ignore[reportArgumentType]
lambda content: (
self.update_field(field, content), # type: ignore
self.update_datetime_field(field, content),
self.update_from_entry(self.cached_entries[0].id),
)
),
@@ -367,7 +390,7 @@ class FieldContainers(QWidget):
container.set_edit_callback(modal.show)
container.set_remove_callback(
lambda: self.remove_message_box(
prompt=self.remove_field_prompt(field.type.name),
prompt=self.remove_field_prompt(field.name),
callback=lambda: (
self.remove_field(field),
self.update_from_entry(self.cached_entries[0].id),
@@ -375,20 +398,20 @@ class FieldContainers(QWidget):
)
)
else:
text = "<i>Mixed Data</i>"
title = f"{field.type.name} (Wacky Date)"
text = "<i>Mixed Data</i>" # TODO: Localize this
inner_widget = TextWidget(title, text)
container.set_inner_widget(inner_widget)
else:
logger.warning("[FieldContainers][write_container] Unknown Field", field=field)
container.set_title(field.type.name)
logger.warning(
"[FieldContainers][write_container] Unknown Field", field=field
) # TODO: Localize this
container.set_title(field.name)
container.set_inline(False)
title = f"{field.type.name} (Unknown Field Type)"
inner_widget = TextWidget(title, field.type.name)
inner_widget = TextWidget(title, field.name)
container.set_inner_widget(inner_widget)
container.set_remove_callback(
lambda: self.remove_message_box(
prompt=self.remove_field_prompt(field.type.name),
prompt=self.remove_field_prompt(field.name),
callback=lambda: (
self.remove_field(field),
self.update_from_entry(self.cached_entries[0].id),
@@ -419,7 +442,9 @@ class FieldContainers(QWidget):
else:
container = self.containers[index]
container.set_title("Tags" if not category_tag else category_tag.name)
container.set_title(
"Tags" if not category_tag else category_tag.name
) # TODO: Localize this
container.set_inline(False)
if not is_mixed:
@@ -431,7 +456,7 @@ class FieldContainers(QWidget):
else:
inner_widget = TagBoxWidget(
"Tags",
"Tags", # TODO: Localize this
self.driver,
)
container.set_inner_widget(inner_widget)
@@ -460,26 +485,24 @@ class FieldContainers(QWidget):
entry_ids = [e.id for e in self.cached_entries]
self.lib.remove_entry_field(field, entry_ids)
def update_field(self, field: BaseField, content: str) -> None:
"""Update a field in all selected Entries, given a field object."""
assert isinstance(
field,
TextField | DatetimeField,
), f"instance: {type(field)}"
def update_text_field(self, field: TextField, value: str, is_multiline: bool):
"""Update a text field across selected entries."""
entry_ids = [e.id for e in self.cached_entries]
assert entry_ids, "No entries selected"
self.lib.update_entry_field(
entry_ids,
field,
content,
)
self.lib.update_text_field(entry_ids, field, value, is_multiline)
def update_datetime_field(self, field: DatetimeField, value: str):
"""Update a datetime field across selected entries."""
entry_ids = [e.id for e in self.cached_entries]
assert entry_ids, "No entries selected"
self.lib.update_datetime_field(entry_ids, field, dt.fromisoformat(value))
def remove_message_box(self, prompt: str, callback: Callable) -> None:
remove_mb = QMessageBox()
remove_mb.setText(prompt)
remove_mb.setWindowTitle("Remove Field")
remove_mb.setWindowTitle("Remove Field") # TODO: Localize
remove_mb.setIcon(QMessageBox.Icon.Warning)
cancel_button = remove_mb.addButton(
Translations["generic.cancel_alt"], QMessageBox.ButtonRole.DestructiveRole

21
src/tagstudio/qt/mixed/migration_modal.py
View File

@@ -33,6 +33,7 @@ from tagstudio.core.constants import (
from tagstudio.core.enums import LibraryPrefs
from tagstudio.core.library.alchemy import default_color_groups
from tagstudio.core.library.alchemy.constants import SQL_FILENAME
from tagstudio.core.library.alchemy.fields import LEGACY_FIELD_MAP
from tagstudio.core.library.alchemy.joins import TagParent
from tagstudio.core.library.alchemy.library import Library as SqliteLibrary
from tagstudio.core.library.alchemy.models import Entry, TagAlias
@@ -550,9 +551,6 @@ class JsonMigrationModal(QObject):
def check_field_parity(self) -> bool:
"""Check if all JSON field and tag data matches the new SQL data."""
def sanitize_field(entry: Entry, value, type, type_key):
return value if value else None
def sanitize_json_field(value):
if isinstance(value, list):
return set(value) if value else None
@@ -563,7 +561,7 @@ class JsonMigrationModal(QObject):
sql_fields: list[tuple] = []
json_fields: list[tuple] = []
sql_entry: Entry = unwrap(self.sql_lib.get_entry_full(json_entry.id + 1))
sql_entry: Entry | None = self.sql_lib.get_entry_full(json_entry.id + 1)
if not sql_entry:
logger.info(
"[Field Comparison]",
@@ -576,14 +574,13 @@ class JsonMigrationModal(QObject):
return self.field_parity
for sf in sql_entry.fields:
if sf.type.type.value not in LEGACY_TAG_FIELD_IDS:
sql_fields.append(
(
sql_entry.id,
sf.type.key,
sanitize_field(sql_entry, sf.value, sf.type.type, sf.type_key),
)
sql_fields.append(
(
sql_entry.id,
sf.name.upper().replace(" ", "_"),
sf.value if sf.value else None,
)
)
sql_fields.sort()
# NOTE: The JSON database stored tags inside of special "tag field" types which
@@ -597,7 +594,7 @@ class JsonMigrationModal(QObject):
tags_count += 1
json_tags = json_tags.union(value or [])
else:
key: str = unwrap(self.sql_lib.get_field_name_from_id(int_key)).name
key: str = str(LEGACY_FIELD_MAP[int_key]["name"]).upper().replace(" ", "_")
json_fields.append((json_entry.id + 1, key, value))
json_fields.sort()

2
src/tagstudio/qt/mixed/mirror_entries_modal.py
View File

@@ -95,7 +95,7 @@ class MirrorEntriesModal(QWidget):
mirrored: list = []
lib = self.driver.lib
for i, entries in enumerate(self.tracker.groups):
lib.mirror_entry_fields(*entries)
lib.mirror_entry_fields(entries)
sleep(0.005)
yield i

8
src/tagstudio/qt/translations.py
View File

@@ -39,6 +39,14 @@ LANGUAGES = {
"Viossa": "qpv",
}
# A map of field class names to their respective translation keys.
FIELD_TYPE_KEYS = {
"DatetimeField": "field_type.datetime",
"DatetimeFieldTemplate": "field_type.datetime",
"TextField": "field_type.text",
"TextFieldTemplate": "field_type.text",
}
class Translator:
_default_strings: dict[str, str]

31
src/tagstudio/qt/ts_qt.py
View File

@@ -52,10 +52,8 @@ from tagstudio.core.driver import DriverMixin
from tagstudio.core.enums import MacroID, SettingItems, ShowFilepathOption
from tagstudio.core.library.alchemy.enums import (
BrowsingState,
FieldTypeEnum,
SortingModeEnum,
)
from tagstudio.core.library.alchemy.fields import FieldID
from tagstudio.core.library.alchemy.library import Library, LibraryStatus
from tagstudio.core.library.alchemy.models import Entry
from tagstudio.core.library.ignore import Ignore
@@ -63,7 +61,7 @@ from tagstudio.core.library.refresh import RefreshTracker
from tagstudio.core.media_types import MediaCategories
from tagstudio.core.query_lang.util import ParsingError
from tagstudio.core.ts_core import TagStudioCore
from tagstudio.core.utils.str_formatting import is_version_outdated, strip_web_protocol
from tagstudio.core.utils.str_formatting import is_version_outdated
from tagstudio.core.utils.types import unwrap
from tagstudio.qt.cache_manager import CacheManager
from tagstudio.qt.controllers.ffmpeg_missing_message_box import FfmpegMissingMessageBox
@@ -1123,7 +1121,6 @@ class QtDriver(DriverMixin, QObject):
def run_macro(self, name: MacroID, entry_id: int):
"""Run a specific Macro on an Entry given a Macro name."""
entry: Entry = unwrap(self.lib.get_entry(entry_id))
full_path = unwrap(self.lib.library_dir) / entry.path
source = "" if entry.path.parent == Path(".") else entry.path.parts[0].lower()
logger.info(
@@ -1140,32 +1137,6 @@ class QtDriver(DriverMixin, QObject):
continue
self.run_macro(macro_id, entry_id)
elif name == MacroID.SIDECAR:
parsed_items = TagStudioCore.get_gdl_sidecar(full_path, source)
for field_id, value in parsed_items.items():
if isinstance(value, list) and len(value) > 0 and isinstance(value[0], str):
value = self.lib.tag_from_strings(value)
self.lib.add_field_to_entry(
entry.id,
field_id=field_id,
value=value,
)
elif name == MacroID.BUILD_URL:
url = TagStudioCore.build_url(entry, source)
if url is not None:
self.lib.add_field_to_entry(entry.id, field_id=FieldID.SOURCE, value=url)
elif name == MacroID.MATCH:
TagStudioCore.match_conditions(self.lib, entry.id)
elif name == MacroID.CLEAN_URL:
for field in entry.text_fields:
if field.type.type == FieldTypeEnum.TEXT_LINE and field.value:
self.lib.update_entry_field(
entry_ids=entry.id,
field=field,
content=strip_web_protocol(field.value),
)
def sorting_direction_callback(self):
logger.info("Sorting Direction Changed", ascending=self.main_window.sorting_direction)
self.update_browsing_state(

3
src/tagstudio/resources/translations/en.json
View File

@@ -71,6 +71,9 @@
"entries.unlinked.unlinked_count": "Unlinked Entries: {count}",
"ffmpeg.missing.description": "FFmpeg and/or FFprobe were not found. FFmpeg is required for multimedia playback and thumbnails.",
"ffmpeg.missing.status": "{ffmpeg}: {ffmpeg_status}<br>{ffprobe}: {ffprobe_status}",
"field_type.datetime": "Datetime",
"field_type.text": "Text",
"field_type.unknown": "Unknown Type",
"field.copy": "Copy Field",
"field.edit": "Edit Field",
"field.paste": "Paste Field",

12
tests/conftest.py
View File

@@ -12,6 +12,8 @@ from unittest.mock import Mock, patch
import pytest
from PySide6.QtWidgets import QScrollArea
from tagstudio.core.library.alchemy.fields import TextField
CWD = Path(__file__).parent
# this needs to be above `src` imports
sys.path.insert(0, str(CWD.parent))
@@ -40,19 +42,19 @@ def file_mediatypes_library():
entry1 = Entry(
folder=folder,
path=Path("foo.png"),
fields=lib.default_fields,
fields=[TextField(name="Title", value="I'm a Test Title")],
)
entry2 = Entry(
folder=folder,
path=Path("bar.png"),
fields=lib.default_fields,
fields=[TextField(name="Title", value="I'm a Test Title")],
)
entry3 = Entry(
folder=folder,
path=Path("baz.apng"),
fields=lib.default_fields,
fields=[TextField(name="Title", value="I'm a Test Title")],
)
assert lib.add_entries([entry1, entry2, entry3])
@@ -117,7 +119,7 @@ def library(request, library_dir: Path): # pyright: ignore
id=1,
folder=folder,
path=Path("foo.txt"),
fields=lib.default_fields,
fields=[TextField(name="Title", value="I'm a Test Title")],
)
assert lib.add_tags_to_entries(entry.id, tag.id)
@@ -125,7 +127,7 @@ def library(request, library_dir: Path): # pyright: ignore
id=2,
folder=folder,
path=Path("one/two/bar.md"),
fields=lib.default_fields,
fields=[TextField(name="Title", value="I'm a Test Title")],
)
assert lib.add_tags_to_entries(entry2.id, tag2.id)

BIN
tests/fixtures/search_library/.TagStudio/ts_library.sqlite vendored
View File

Binary file not shown.

5
tests/macros/test_dupe_files.py
View File

@@ -4,6 +4,7 @@
from pathlib import Path
from tagstudio.core.library.alchemy.fields import TextField
from tagstudio.core.library.alchemy.library import Library
from tagstudio.core.library.alchemy.models import Entry
from tagstudio.core.library.alchemy.registries.dupe_files_registry import DupeFilesRegistry
@@ -19,13 +20,13 @@ def test_refresh_dupe_files(library: Library):
entry = Entry(
folder=folder,
path=Path("bar/foo.txt"),
fields=library.default_fields,
fields=[TextField(name="Title", value="I'm a Test Title")],
)
entry2 = Entry(
folder=folder,
path=Path("foo/foo.txt"),
fields=library.default_fields,
fields=[TextField(name="Title", value="I'm a Test Title")],
)
library.add_entries([entry, entry2])

4
tests/test_db_migrations.py
View File

@@ -27,6 +27,10 @@ EMPTY_LIBRARIES = "empty_libraries"
str(Path(CWD.parent / FIXTURES / EMPTY_LIBRARIES / "DB_VERSION_8")),
str(Path(CWD.parent / FIXTURES / EMPTY_LIBRARIES / "DB_VERSION_9")),
str(Path(CWD.parent / FIXTURES / EMPTY_LIBRARIES / "DB_VERSION_100")),
str(Path(CWD.parent / FIXTURES / EMPTY_LIBRARIES / "DB_VERSION_101")),
# str(Path(CWD.parent / FIXTURES / EMPTY_LIBRARIES / "DB_VERSION_102")),
str(Path(CWD.parent / FIXTURES / EMPTY_LIBRARIES / "DB_VERSION_103")),
str(Path(CWD.parent / FIXTURES / EMPTY_LIBRARIES / "DB_VERSION_200")),
],
)
def test_library_migrations(path: str):

159
tests/test_library.py
View File

@@ -13,7 +13,7 @@ import structlog
from tagstudio.core.enums import DefaultEnum, LibraryPrefs
from tagstudio.core.library.alchemy.enums import BrowsingState
from tagstudio.core.library.alchemy.fields import (
FieldID, # pyright: ignore[reportPrivateUsage]
DatetimeField,
TextField,
)
from tagstudio.core.library.alchemy.library import Library
@@ -82,12 +82,12 @@ def test_library_add_file(library: Library):
entry = Entry(
path=Path("bar.txt"),
folder=unwrap(library.folder),
fields=library.default_fields,
fields=[TextField(name="Title", value="I'm a Test Title")],
)
assert not library.has_path_entry(entry.path)
assert not library.has_entry_with_path(entry.path)
assert library.add_entries([entry])
assert library.has_path_entry(entry.path)
assert library.has_entry_with_path(entry.path)
def test_create_tag(library: Library, generate_tag: Callable[..., Tag]):
@@ -213,13 +213,18 @@ def test_remove_entry_field(library: Library, entry_full: Entry):
assert not entry.text_fields
def test_remove_field_entry_with_multiple_field(library: Library, entry_full: Entry):
def test_remove_text_field_entry_with_multiple_field(library: Library, entry_full: Entry):
# Given
title_field = entry_full.text_fields[0]
# When
# add identical field
assert library.add_field_to_entry(entry_full.id, field_id=title_field.type_key)
assert library.add_text_field_to_entry(
entry_full.id,
name=title_field.name,
value=title_field.value,
is_multiline=title_field.is_multiline,
)
# remove entry field
library.remove_entry_field(title_field, [entry_full.id])
@@ -232,30 +237,22 @@ def test_remove_field_entry_with_multiple_field(library: Library, entry_full: En
def test_update_entry_field(library: Library, entry_full: Entry):
title_field = entry_full.text_fields[0]
library.update_entry_field(
entry_full.id,
title_field,
"new value",
)
library.update_text_field(entry_full.id, title_field, "new value", title_field.is_multiline)
entry = next(library.all_entries(with_joins=True))
assert entry.text_fields[0].value == "new value"
def test_update_entry_with_multiple_identical_fields(library: Library, entry_full: Entry):
def test_update_entry_with_multiple_identical_text_fields(library: Library, entry_full: Entry):
# Given
title_field = entry_full.text_fields[0]
# When
# add identical field
library.add_field_to_entry(entry_full.id, field_id=title_field.type_key)
library.add_text_field_to_entry(entry_full.id, name="Title", value="")
# update one of the fields
library.update_entry_field(
entry_full.id,
title_field,
"new value",
)
library.update_text_field(entry_full.id, title_field, "new value", title_field.is_multiline)
# Then only one should be updated
entry = next(library.all_entries(with_joins=True))
@@ -263,37 +260,64 @@ def test_update_entry_with_multiple_identical_fields(library: Library, entry_ful
assert entry.text_fields[1].value == "new value"
def test_mirror_entry_fields(library: Library, entry_full: Entry):
# new entry
target_entry = Entry(
def test_mirror_entry_fields(library: Library):
# Create and add entries with fields
entry_a = Entry(
folder=unwrap(library.folder),
path=Path("xxx"),
path=Path("title_and_date.txt"),
fields=[
TextField(
type_key=FieldID.NOTES.name,
value="notes",
position=0,
)
TextField(name="Title", value="I'm a Test Title"),
DatetimeField(name="Date", value="2026-05-07 12:59:24"),
],
)
entry_b = Entry(
folder=unwrap(library.folder),
path=Path("notes.txt"),
fields=[
TextField(name="Notes", value="These are my notes.\nNo peeking!", is_multiline=True)
],
)
entry_c = Entry(
folder=unwrap(library.folder),
path=Path("date_published.txt"),
fields=[
DatetimeField(name="Date Published", value="2000-01-01 12:00:00"),
],
)
entry_a_id, entry_b_id, entry_c_id = library.add_entries([entry_a, entry_b, entry_c])
# insert new entry and get id
entry_id = library.add_entries([target_entry])[0]
# Retrieve from library
entry_a_ = unwrap(library.get_entry_full(entry_a_id))
entry_b_ = unwrap(library.get_entry_full(entry_b_id))
entry_c_ = unwrap(library.get_entry_full(entry_c_id))
# get new entry from library
new_entry = unwrap(library.get_entry_full(entry_id))
# Sanity check for initial fields
assert entry_a_.fields[0].name == "Title"
assert entry_a_.fields[1].name == "Date"
assert entry_b_.fields[0].name == "Notes"
assert entry_c_.fields[0].name == "Date Published"
assert len(entry_a_.fields) == 2
assert len(entry_b_.fields) == 1
assert len(entry_c_.fields) == 1
# mirror fields onto new entry
library.mirror_entry_fields(new_entry, entry_full)
# Mirror fields between entries
library.mirror_entry_fields([entry_b_, entry_a_, entry_c_])
# get new entry from library again
entry = unwrap(library.get_entry_full(entry_id))
# Retrieve from library, again
entry_a_mirrored = unwrap(library.get_entry_full(entry_a_id))
entry_b_mirrored = unwrap(library.get_entry_full(entry_b_id))
entry_c_mirrored = unwrap(library.get_entry_full(entry_c_id))
# make sure fields are there after getting it from the library again
assert len(entry.fields) == 2
assert {x.type_key for x in entry.fields} == {
FieldID.TITLE.name,
FieldID.NOTES.name,
# Assert presence of all fields on all entries
assert len(entry_a_mirrored.fields) == 4
assert len(entry_b_mirrored.fields) == 4
assert len(entry_c_mirrored.fields) == 4
assert {(type(x), x.name) for x in entry_a_mirrored.fields} == {
(TextField, "Title"),
(TextField, "Notes"),
(DatetimeField, "Date"),
(DatetimeField, "Date Published"),
}
@@ -304,32 +328,32 @@ def test_merge_entries(library: Library):
tag_1: Tag = unwrap(library.add_tag(Tag(id=1011, name="tag_1")))
tag_2: Tag = unwrap(library.add_tag(Tag(id=1012, name="tag_2")))
a = Entry(
entry_a = Entry(
folder=folder,
path=Path("a"),
fields=[
TextField(type_key=FieldID.AUTHOR.name, value="Author McAuthorson", position=0),
TextField(type_key=FieldID.DESCRIPTION.name, value="test description", position=2),
TextField(name="Author", value="Author McAuthorson"),
TextField(name="Description", value="test description", is_multiline=True),
],
)
b = Entry(
entry_b = Entry(
folder=folder,
path=Path("b"),
fields=[TextField(type_key=FieldID.NOTES.name, value="test note", position=1)],
fields=[TextField(name="Notes", value="test note", is_multiline=True)],
)
ids = library.add_entries([a, b])
entry_a_id, entry_b_id = library.add_entries([entry_a, entry_b])
library.add_tags_to_entries(ids[0], [tag_0.id, tag_2.id])
library.add_tags_to_entries(ids[1], [tag_1.id])
library.add_tags_to_entries(entry_a_id, [tag_0.id, tag_2.id])
library.add_tags_to_entries(entry_b_id, [tag_1.id])
entry_a: Entry = unwrap(library.get_entry_full(ids[0]))
entry_b: Entry = unwrap(library.get_entry_full(ids[1]))
entry_a_: Entry = unwrap(library.get_entry_full(entry_a_id))
entry_b_: Entry = unwrap(library.get_entry_full(entry_b_id))
assert library.merge_entries(entry_a, entry_b)
assert not library.has_path_entry(Path("a"))
assert library.has_path_entry(Path("b"))
assert library.merge_entries(entry_a_, entry_b_)
assert not library.has_entry_with_path(Path("a"))
assert library.has_entry_with_path(Path("b"))
entry_b_merged = unwrap(library.get_entry_full(ids[1]))
entry_b_merged = unwrap(library.get_entry_full(entry_b_id))
fields = [field.value for field in entry_b_merged.fields]
assert "Author McAuthorson" in fields
@@ -366,33 +390,6 @@ def test_search_entry_id(library: Library, query_name: int, has_result: bool):
assert (result is not None) == has_result
def test_update_field_order(library: Library, entry_full: Entry):
# Given
title_field = entry_full.text_fields[0]
# When add two more fields
library.add_field_to_entry(entry_full.id, field_id=title_field.type_key, value="first")
library.add_field_to_entry(entry_full.id, field_id=title_field.type_key, value="second")
# remove the one on first position
assert title_field.position == 0
library.remove_entry_field(title_field, [entry_full.id])
# recalculate the positions
library.update_field_position(
type(title_field),
title_field.type_key,
entry_full.id,
)
# Then
entry = next(library.all_entries(with_joins=True))
assert entry.text_fields[0].position == 0
assert entry.text_fields[0].value == "first"
assert entry.text_fields[1].position == 1
assert entry.text_fields[1].value == "second"
def test_library_prefs_multiple_identical_vals():
# check the preferences are inherited from DefaultEnum
assert issubclass(LibraryPrefs, DefaultEnum)