diff --git a/docs/macros.md b/docs/macros.md index 0573cd28..5e045f74 100644 --- a/docs/macros.md +++ b/docs/macros.md @@ -28,13 +28,11 @@ triggers = ["on_new_entry"] ## Actions -Actions are the general task that a macro will perform. They are represented by TOML tables and must have a unique name in your macro file, but the name itself has no importance to the macro. The root table will contain general configuration info about your action, and nested tables (e.g. `[action.task]`) will further define the specifics of your action. +Actions are broad categories of operations that your macro will perform. They are represented by TOML tables and must have a unique name in your macro file, but the name itself has no importance to the macro. A single macro file can contain multiple actions and each action can contain multiple tasks. -```toml -[newgrounds] -``` +An action table with a name of your choosing (e.g. `[action]`) will contain the general configuration for your action, and nested task tables (e.g. `[action.task]`) will define the specifics of your action's tasks. -Action tables must have an `action` key with one of the following values: +Action tables must have an `action` key with one of the following valid action values: - [`import_data`](#import-data): Import data from a supported external source. - [`add_data`](#add-data): Add data declared inside the macro file. @@ -44,15 +42,33 @@ Action tables must have an `action` key with one of the following values: action = "import_data" ``` +Most of the configuration of actions comes at the [task configuration](#task-configuration) level. This is where you will build out exactly how your action will translate data and instructions into results for your TagStudio library. + +--- + ### Add Data The `add_data` action lets you add data to a [file entry](entries.md) given one or more conditional statements. Unlike the [`import_data`](#import-data) action, the `add_data` action adds data declared in the macro itself rather than importing it form a source external to the macro. +Compatible Keys: + +- [`source_filters`](#source_filters) +- [`value`](#value) + +--- + ### Import Data -The `import_data` action allows you to import external data into your TagStudio library in the form of [tags](tags.md) and [fields](fields.md). While some sources need explicit support (e.g. ID3, EXIF) generic sources such as JSON sidecar files can leverage wide array of data shaping options that allow the underlying data structure to be abstracted from TagStudio's internal data structures. This macro pairs very well with tools such as [gallery-dl](https://github.com/mikf/gallery-dl). +The `import_data` action allows you to import external data into your TagStudio library in the form of [tags](tags.md) and [fields](fields.md). While some sources need explicit support (e.g. ID3, EXIF) generic sources such as JSON sidecar files can leverage a wide array of data shaping options that allow the underlying data structure to be abstracted from TagStudio's internal data structures. This macro pairs very well with tools that download sidecar files for data such as [gallery-dl](https://github.com/mikf/gallery-dl). -If you're importing from an object or table-like source (i.e. JSON), you'll need to create a nested table with the name format of "`[action.task]`" and provide a [`key`](#key) field filled with the name of the targeted source key. In this case the "task" name does not matter as long as it doesn't conflict with one of the built-in names (i.e. "`map`", "`template`, "`inverse_map`"). +Compatible Keys: + +- [`key`](#key) +- [`source_location`](#source_location) +- [`source_format`](#source_format) +- [`is_embedded`](#is_embedded) + +If you're importing from an object-like source (e.g. JSON), you'll need to create a nested task table with the format `[action.task]` and provide a [`key`](#key) field filled with the name of the targeted source key. In this case the task name does not matter as long as it doesn't conflict with one of the built-in task names (i.e. "`map`", "`inverse_map`, "`template`"). === "Importable JSON Data" @@ -73,11 +89,129 @@ If you're importing from an object or table-like source (i.e. JSON), you'll need Inside the new table we can now declare additional information about the native data formats and how they should be imported into TagStudio. +--- + ### Action Configuration -#### Key +#### `source_format` -The `key` key is used in conjunction with the [`import_data`](#import-data) action to specify the object/table key to target in your import data. If you're targeting a nested object, separate the names of the keys with a dot. + +!!! note "" + Compatible Actions: [`import_data`](#import-data) + +The `source_format` key is used to declare what type of source data will be imported from. + +```toml +[newgrounds] +action = "import_data" +source_format = "json" +``` + +- `exif`: Embedded EXIF metadata +- `id3`: Embedded ID3 metadata +- `json`: A JSON formatted file +- `text`: A plaintext file +- `xml`: An XML formatted file +- `xmp`: Embedded XMP metadata or an XMP sidecar file + +--- + +#### `source_location` + + +!!! note "" + Compatible Actions: [`import_data`](#import-data) + +The `source_location` key is used to declare where the metadata should be imported from. This can be a relative or absolute path, and can reference the targeted filename with the `{filename}` placeholder. + +```toml +[newgrounds] +action = "import_data" +source_format = "json" +source_location = "{filename}.json" # Relative sidecar file +``` + + + +--- + +#### `is_embedded` + + +!!! note "" + Compatible Actions: [`import_data`](#import-data) + +If targeting embedded data, add the `is_embedded` key and set it to `true`. If no `source_location` is used then the file this macro is targeting will be used as a source. + +```toml +[newgrounds] +action = "import_data" +source_format = "id3" +is_embedded = true +``` + +--- + +#### `source_filters` + + +!!! note "" + Compatible Actions: [`add_data`](#add-data), [`import_data`](#import-data) + +`source_filters` are used to declare a glob list of files that are able to be targeted by this action. An entry filepath only needs to fall under one of the given source filters in order for the macro to continue. If not, then the macro will be skipped for this file entry. + + +=== "import_data" + ```toml + [newgrounds] + action = "import_data" + source_format = "json" + source_location = "{filename}.json" + source_filters = ["**/Newgrounds/**"] + ``` +=== "add_data" + ```toml + [animated] + action = "add_data" + source_filters = ["**/*.gif", "**/*.apng"] + ``` + + + +--- + +## Task Configuration + +An [action's](#actions) tasks need to be configured using the built-in keys available to each action. These keys may be specific to certain actions, required or optional, or expect other specific formatting. The actions section will list each action's available keys, and the following list of keys will likewise list which actions they are compatible with along with any other rules. + +Along with generally defining your own custom tasks, there are a few built-in tasks that have reserved names and offer extra functionality on top of your own tasks. These currently include: + +- [`.inverse-map`](#many-to-1-inverse-map) (Inverse Tag Maps) +- [`.map`](#manual-tag-mapping) (Tag Maps) +- [`.template`](#templates) (Templates) + +--- + +### `key` + + +!!! note "" + Compatible Actions: [`import_data`](#import-data) + +The `key` key is used to specify the object key to target in your data source. If you're targeting a nested object, separate the names of the keys with a dot. ```toml [artstation] @@ -91,7 +225,7 @@ key="mediums.name" # Nested key ts_type = "tags" ``` -When importing from the same key multiple times, you have the option to either choose different names for your "task" tables or use the same name with these tables wrapped in an extra pair of brackets. +When importing from the same key multiple times, you have the option to either choose different names for your task tables or use the same name with these tables wrapped in an extra pair of brackets. === "Single Import" @@ -133,108 +267,13 @@ When importing from the same key multiple times, you have the option to either c name = "Artist" ``` -#### Source Format +--- -The `source_format` key is used in conjunction with the [`import_data`](#import-data) action to declare what type of source data will be imported from. - -```toml -[newgrounds] -action = "import_data" -source_format = "json" -``` - -- `exif`: Embedded EXIF metadata -- `id3`: Embedded ID3 metadata -- `json`: A JSON formatted file -- `text`: A plain text file -- `xml`: An XML formatted file -- `xmp`: Embedded XMP metadata or an XMP sidecar file - -#### Source Location - -The `source_location` key is used in conjunction with the `import_data` key to declare where the metadata should be imported from. This can be a relative or absolute path, and can reference the targeted filename with the `{filename}` placeholder. - -```toml -[newgrounds] -action = "import_data" -source_format = "json" -source_location = "{filename}.json" # Relative sidecar file -``` - - - -#### Embedded Metadata - -If targeting embedded data, add the `is_embedded` key and set it to `true`. If no `source_location` is used then the file this macro is targeting will be used as a source. - -```toml -[newgrounds] -action = "import_data" -source_format = "id3" -is_embedded = true -``` - -#### Source Filters - -`source_filters` are used to declare a glob list of files that are able to be targeted by this action. An entry filepath only needs to fall under one of the given source filters in order for the macro to continue. If not, then the macro will be skipped for this file entry. +### `ts_type` -=== "import_data" - ```toml - [newgrounds] - action = "import_data" - source_format = "json" - source_location = "{filename}.json" - source_filters = ["**/Newgrounds/**"] - ``` -=== "add_data" - ```toml - [animated] - action = "add_data" - source_filters = ["**/*.gif", "**/*.apng"] - ``` - -#### Value - -The `value` key is specifically used with the [`add_data`](#add-data) action to define what value should be added to the file entry. - - -=== "Title Field" - ```toml - [animated] - action = "add_data" - source_filters = ["**/*.gif", "**/*.apng"] - [animated.title] - ts_type = "text_line" - name = "Title" - value = "Animated Image" - ``` -=== "Tags" - ```toml - [animated] - action = "add_data" - source_filters = ["**/*.gif", "**/*.apng"] - [animated.tags] - ts_type = "tags" - value = ["Animated"] - ``` - - - -#### TagStudio Types +!!! note "" + Compatible Actions: [`add_data`](#add-data), [`import_data`](#import-data) The required `ts_type` key defines the destination data format inside TagStudio itself. This can be [tags](tags.md) or any [field](fields.md) type. @@ -260,9 +299,7 @@ The required `ts_type` key defines the destination data format inside TagStudio ts_type = "tags" ``` -### Field Specific Keys - -#### Name +#### Field Specific Keys `name`: The name of the field to import into. @@ -286,12 +323,10 @@ The required `ts_type` key defines the destination data format inside TagStudio !!! note As of writing (v9.5.3) TagStudio fields still do not allow for custom names. The macro system is designed to be forward-thinking with this feature in mind, however only existing TagStudio field names are currently considered valid. Any invalid field names will default to the "Notes" field. -### Tag Specific Keys +#### Tag Specific Keys Since TagStudio tags are more complex than other traditional tag formats, there are several options for fine-tuning how tags should be imported. -#### Delimiter - `delimiter`: The delimiter between string tags to use. @@ -308,36 +343,6 @@ Since TagStudio tags are more complex than other traditional tag formats, there delimiter = "\n" ``` -#### Prefix - -`prefix`: An optional prefix to remove. - - -!!! example - Given a list of tags such as `["#tag1", "#tag2", "#tag3"]`, you may wish to remove the "`#`" prefix. - -```toml -[instagram.tags] -ts_type = "tags" -prefix = "#" -``` - -#### Strict - -`strict`: Determines what [names](tags.md#naming-tags) of the TagStudio tags should be used to compare against the source data when matching. - -- `true`: Only match against the TagStudio tag [name](tags.md#name) field. -- `false` (Default): Match against any TagStudio tag name field including [shorthands](tags.md#shorthand), [aliases](tags.md#aliases), and the [disambiguation name](tags.md#disambiguation). - -#### Use Context - -`use_context`: Determines if TagStudio should use context clues from other source tags to provide more accurate tag matches. - -- `true` (Default): Use context clue matching (slower, less ambiguous). -- `false`: Ignore surrounding source tags (faster, more ambiguous). - -#### On Missing - `on_missing`: Determines the behavior of how to react to source tags with no match in the library. - `"prompt"`: Ask the user if they wish to create, skip, or manually choose an existing tag. @@ -352,9 +357,65 @@ use_context = true on_missing = "create" ``` +`prefix`: An optional prefix to remove. + + +!!! example + Given a list of tags such as `["#tag1", "#tag2", "#tag3"]`, you may wish to remove the "`#`" prefix. + +```toml +[instagram.tags] +ts_type = "tags" +prefix = "#" +``` + +`strict`: A flag that determines what [names](tags.md#naming-tags) of the TagStudio tags should be used to compare against the source data when matching. + +- `true`: Only match against the TagStudio tag [name](tags.md#name) field. +- `false` (Default): Match against any TagStudio tag name field including [shorthands](tags.md#shorthand), [aliases](tags.md#aliases), and the [disambiguation name](tags.md#disambiguation). + +`use_context`: A flag that determines if TagStudio should use context clues from other source tags to provide more accurate tag matches. + +- `true` (Default): Use context clue matching (slower, less ambiguous). +- `false`: Ignore surrounding source tags (faster, more ambiguous). + \*\* + +--- + +### `value` + + +!!! note "" + Compatible Actions: [`add_data`](#add-data) + +The `value` key is use specifically with the [`add_data`](#add-data) action to define what value should be added to the file entry. + + +=== "Title Field" + ```toml + [animated] + action = "add_data" + source_filters = ["**/*.gif", "**/*.apng"] + [animated.title] + ts_type = "text_line" + name = "Title" + value = "Animated Image" + ``` +=== "Tags" + ```toml + [animated] + action = "add_data" + source_filters = ["**/*.gif", "**/*.apng"] + [animated.tags] + ts_type = "tags" + value = ["Animated"] + ``` + +--- + ### Manual Tag Mapping -If the results from the standard tag matching system aren't good enough to properly import specific source data into your TagStudio library, you have the option to manually specify mappings between source and destination tags. A table with the `.map` or `.inverse_map` suffixes will be used to map tags in the nearest scope. +If the automatic tag matching system isn't enough to import tags the way you'd like, you can manually specify mappings between source and destination tags. Tables with the `.map` or `.inverse_map` task suffixes will be used to map tags in the nearest scope. === "Global Scope" @@ -364,21 +425,23 @@ If the results from the standard tag matching system aren't good enough to prope ``` === "Action Scope" ```toml - # Applies to all tag keys in the "newgrounds" action + # Applies to all tasks in the "newgrounds" action [newgrounds.map] ``` === "Key Scope" ```toml - # Only applies to tags within the "ratings" key inside the "newgrounds" action + # Only applies to the "ratings" task inside the "newgrounds" action [newgrounds.ratings.map] ``` -- `map`: Used for [1 to 0](#1-to-0-ignore-matches), [1 to 1](#1-to-1), and [1 to Many](#1-to-many) mappings. -- `inverse_map`: Used for [Many to 1](#many-to-1-inverse-map) mappings. +- `map`: Used for "[1 to 0](#1-to-0-ignore-matches)", "[1 to 1](#1-to-1)", and "[1 to many](#1-to-many)" mappings +- `inverse_map`: Used for "[many to 1](#many-to-1-inverse-map)" mappings + +--- #### 1 to 0 (Ignore Matches) -By mapping the key of the source tag name to an empty string, you can ignore that tag when matching with your own library. This is useful if you're importing from a source that uses tags you don't wish to use or create inside your own libraries. +By mapping the key of the source tag name to an empty string, you can ignore that tag when matching with your own tags. This is useful if you're importing from a source that uses tags you don't wish to use or create inside your own libraries. ```toml [newgrounds.tags.map] @@ -386,9 +449,11 @@ By mapping the key of the source tag name to an empty string, you can ignore tha favorite = "" ``` +--- + #### 1 to 1 -By mapping the key of the source tag name to the name of one of your TagStudio tags, you can directly specify a destination tag while bypassing the matching algorithm. +By mapping the key or quoted string of a source tag to one of your TagStudio tags, you can directly specify a destination tag while bypassing the matching algorithm. !!! tip @@ -398,11 +463,14 @@ By mapping the key of the source tag name to the name of one of your TagStudio t [newgrounds.tags.map] # Source Tag Name = TagStudio Tag Name colored_pencil = "Drawing" +"Colored Pencil" = "Drawing" ``` +--- + #### 1 to Many -By mapping the key of the source tag name to a list of your TagStudio tag names, you can cause one source tag to import as more than one of your TagStudio tags. +By mapping the key or quoted string of a source tag to a **list of your TagStudio tags**, you can cause one source tag to import as more than one of your TagStudio tags. ```toml [newgrounds.tags.map] @@ -411,9 +479,11 @@ drawing = ["Drawing (2D)", "Image (Meta Tags)"] video = ["Animation (2D)", "Animated (Meta Tags)"] ``` +--- + #### Many to 1 (Inverse Map) -By mapping a key with the name of one of your TagStudio tags to a list of source tags, you can declare a combination of required source tags that result in a wholly new matched TagStudio tag. This is useful if you use a single tag in your TagStudio library that is represented by multiple separate tags from your source. +By mapping the key or quoted string of one of your TagStudio tags to a **list of source tags**, you can declare a combination of required source tags that result in a wholly new matched TagStudio tag. This is useful if you use a single tag in your TagStudio library that is represented by multiple separate tags from your source. ```toml [newgrounds.tags.inverse_map] @@ -422,6 +492,8 @@ By mapping a key with the name of one of your TagStudio tags to a list of source "Animation (3D)" = ["3D", "video"] ``` +--- + ### Templates Templates are part of the `input_data` action and allow you to take data from one or more keys of a source and combine them into a single value. Template sub-action tables must begin with the action name and end with `.template` (e.g. `[action.template]`). Source object keys can be embedded in a string value if surrounded by curly braces (`{}`). Nested keys are accessed by separating the keys with a dot (e.g. `{key.nested_key}`). diff --git a/src/tagstudio/core/macro_parser.py b/src/tagstudio/core/macro_parser.py index e8824eef..fc85c343 100644 --- a/src/tagstudio/core/macro_parser.py +++ b/src/tagstudio/core/macro_parser.py @@ -464,6 +464,8 @@ def _fill_template( if isinstance(value, dict): for v in value: + # NOTE: This f-string is the only thing defining how the nested key syntax works. + # If instead you wanted to use key[nested] syntax for example, use: f"{key}[{str(v)}]" normalized_key: str = f"{key}.{str(v)}" template = _fill_template(template, value, str(v), normalized_key)