Compare commits

..

37 Commits

Author SHA1 Message Date
tyrrrz b89dd4c924 Refactor 2026-07-25 21:18:39 +03:00
tyrrrz 427c6021ac Fix grammar 2026-07-25 20:28:29 +03:00
Kushida 91a8daf3a7 Fix reverse exports with an after boundary (#1569) 2026-07-25 20:28:06 +03:00
Cap LaBuff 0358045c94 Only display http(s) URLs as links in HTML export (#1568) 2026-07-14 12:09:42 +03:00
tyrrrz 31de8a5b06 Apply updated formatting rules 2026-07-04 18:51:27 +03:00
dependabot[bot] efd430ed29 Bump the actions group with 3 updates (#1564)
Signed-off-by: dependabot[bot] <support@github.com>
2026-07-04 18:49:28 +03:00
dependabot[bot] 623c688e1b Bump the nuget group with 14 updates (#1565)
Signed-off-by: dependabot[bot] <support@github.com>
2026-07-04 18:48:21 +03:00
tyrrrz 05f8df51e9 Refactor 2026-06-26 13:31:15 +03:00
Jacob Pfundstein e2c633b004 Include starter message in thread exports and skip placeholder (#1557)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 12:59:20 +03:00
tyrrrz b11a57a825 Fix formatting 2026-06-20 14:33:04 +03:00
Nick f502e577c2 Check for media.discordapp.net in addition to cdn.discordapp.com when stripping signature parameters from media URLs (#1554)
Co-authored-by: Oleksii Holub <1935960+Tyrrrz@users.noreply.github.com>
2026-06-20 14:30:34 +03:00
Nick 8bc9fe7c72 Catch both old uppercase 5-char hash and newer lowercase 5-char hash when migrating to new hash (#1552)
Co-authored-by: Oleksii Holub <1935960+Tyrrrz@users.noreply.github.com>
2026-06-20 14:16:48 +03:00
Jacob Pfundstein 97485c280b Fix output path detection to correctly identify directories and files (#1556) 2026-06-20 13:55:49 +03:00
Jacob Pfundstein 5a57c4202d Validate output path early to prevent errors when exporting multiple channels (#1555)
Co-authored-by: Oleksii Holub <1935960+Tyrrrz@users.noreply.github.com>
2026-06-18 15:37:58 +03:00
tyrrrz acac8f7bbb Update packages 2026-06-08 15:52:10 +03:00
Oleksii Holub 5632d6c386 HTML-encode markdown content when formatting is disabled (#1545) 2026-06-04 13:09:03 +03:00
Oleksii Holub f4d1e630f7 Implement missing HTML encoding for several attribute injections (#1544) 2026-06-04 12:38:09 +03:00
Colton 799dd1ad81 Stream asset downloads to avoid OutOfMemoryException on large files (#1540)
* Stream asset downloads to avoid OutOfMemoryException on large files

ExportAssetDownloader used HttpClient.GetAsync(url, ct), which defaults to
HttpCompletionOption.ResponseContentRead and buffers the entire response body
into memory before returning. Downloading a large attachment therefore tried to
grow a single in-memory buffer to the full file size and threw
OutOfMemoryException (HttpContent.LoadIntoBufferAsync -> GrowAndWrite).

Pass HttpCompletionOption.ResponseHeadersRead so the body is streamed straight
to disk via CopyToAsync in bounded chunks, matching how DiscordClient already
issues its requests.

* Update ExportAssetDownloader.cs

---------

Co-authored-by: Oleksii Holub <1935960+Tyrrrz@users.noreply.github.com>
2026-06-03 15:47:43 +03:00
dependabot[bot] 64a4ba296e Bump the actions group with 4 updates (#1541)
Signed-off-by: dependabot[bot] <support@github.com>
2026-06-02 19:41:24 +03:00
dependabot[bot] f0e99765f7 Bump the nuget group with 13 updates (#1542)
Signed-off-by: dependabot[bot] <support@github.com>
2026-06-02 19:40:41 +03:00
tyrrrz 45629c9d7f Refactor message boxes 2026-05-26 15:00:42 +03:00
Copilot 2aec84884b Extract app shutdown fallback into App.Shutdown (#1539)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
2026-05-25 19:10:19 +03:00
tyrrrz 6258394fc0 Remove lang version from build props 2026-05-03 16:01:32 +03:00
tyrrrz e6472543e0 Refactor fetching Avalonia TopLevel 2026-05-02 11:30:14 +03:00
dependabot[bot] ee163d6de7 Bump actions/upload-artifact from 7.0.0 to 7.0.1 in the actions group (#1528)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-01 16:31:01 +03:00
tyrrrz 5838c0546e Update for Avalonia 12 2026-05-01 16:30:11 +03:00
dependabot[bot] 07a02773be Bump the nuget group with 15 updates (#1529)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-01 16:05:31 +03:00
tyrrrz 0a623b24d1 Refactor GUI spacing 2026-04-22 17:18:28 +03:00
tyrrrz da4f5ea74a Use PowerKit's Uri.TryGetFilaName() 2026-04-20 22:22:11 +03:00
tyrrrz 34b5ccc75b Remove BOM 2026-04-19 23:13:12 +03:00
Copilot 7456f0fe3a Add PowerKit and replace custom utility extensions (#1525)
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-04-19 23:10:45 +03:00
tyrrrz 757daa7a60 Forward slashes 2026-04-19 18:13:26 +03:00
tyrrrz 9590d4461a Standardize repo structure 2026-04-19 16:24:23 +03:00
tyrrrz a27fc3e72e Update Avalonia 2026-04-17 15:02:28 +03:00
Oleksii Holub f10bba63e6 Add warning against user account automation 2026-04-14 22:52:10 +03:00
tyrrrz b76b42549b Fix security issues 2026-04-14 13:23:38 +03:00
tyrrrz ae70fe9741 Move theme initialization to the correct place 2026-04-07 17:54:05 +03:00
229 changed files with 2282 additions and 2514 deletions
+1 -1
View File
@@ -16,7 +16,7 @@ $ docker pull tyrrrz/discordchatexporter:stable
Note the `:stable` tag. DiscordChatExporter images are tagged according to the following patterns:
- `stable` — latest stable version release. This tag is updated with each release of a new project version. Recommended for personal use.
- `x.y.z` (e.g. `2.30.1`) — specific stable version release. This tag is pushed when the corresponding version is released and never updated thereafter. Recommended for use in automation scenarios.
- `x.y.z` (e.g., `2.30.1`) — specific stable version release. This tag is pushed when the corresponding version is released and never updated thereafter. Recommended for use in automation scenarios.
- `latest` — latest (potentially unstable) build. This tag is updated with each new commit to the `prime` branch. Not recommended, unless you want to test a new feature that has not been released in a stable version yet.
You can see all available tags [here](https://hub.docker.com/r/tyrrrz/discordchatexporter/tags?ordering=name).
+2 -2
View File
@@ -64,9 +64,9 @@ exit 0
- `tokenhere` with your [Token](Token-and-IDs.md).
- `channelhere` with a [Channel ID](Token-and-IDs.md).
- `dceFOLDERpathhere` with DCE's **directory path** (e.g. `/path/to/folder`, NOT `/path/to/folder/DiscordChatExporter.dll`).
- `dceFOLDERpathhere` with DCE's **directory path** (e.g., `/path/to/folder`, NOT `/path/to/folder/DiscordChatExporter.dll`).
- `filenamehere` with the exported channel's filename, without spaces.
- `dirhere` with the export directory (e.g. /home/user/Documents/Discord\ Exports).
- `dirhere` with the export directory (e.g., /home/user/Documents/Discord\ Exports).
- `formathere` with one of the available export formats.
> **Note**:
+2 -2
View File
@@ -68,9 +68,9 @@ exit 0
- `tokenhere` with your [Token](Token-and-IDs.md)
- `channelhere` with a [Channel ID](Token-and-IDs.md)
- `dceFOLDERpathhere` with DCE's **directory's path** (e.g. `/Users/user/Desktop/DiscordChatExporterFolder`, NOT `/Users/user/Desktop/DiscordChatExporterFolder/DiscordChatExporter.DLL`)
- `dceFOLDERpathhere` with DCE's **directory's path** (e.g., `/Users/user/Desktop/DiscordChatExporterFolder`, NOT `/Users/user/Desktop/DiscordChatExporterFolder/DiscordChatExporter.DLL`)
- `filenamehere` with the exported channel's filename, without spaces
- `dirhere` with the directory you want the files to be saved at (e.g. `/Users/user/Documents/Discord\ Exports`)
- `dirhere` with the directory you want the files to be saved at (e.g., `/Users/user/Documents/Discord\ Exports`)
- `formathere` with one of the available export formats
To quickly get file or folder paths, select the file/folder, then hit Command+I (⌘I) and copy what's after `Where:`.
+2 -2
View File
@@ -33,9 +33,9 @@ exit
- `tokenhere` with your [Token](Token-and-IDs.md)
- `channelhere` with a [Channel ID](Token-and-IDs.md)
- `exefolderhere` with the .exe **directory's path** (e.g. C:\Users\User\Desktop\DiscordChatExporter)
- `exefolderhere` with the .exe **directory's path** (e.g., C:\Users\User\Desktop\DiscordChatExporter)
- `filenamehere` with a filename without spaces
- `dirhere` with the export directory (e.g. C:\Users\User\Documents\Exports)
- `dirhere` with the export directory (e.g., C:\Users\User\Documents\Exports)
- `formathere` with one of the available export formats
Make sure not to delete the quotes (")
+62 -118
View File
@@ -23,104 +23,84 @@ Now we're ready to run the commands.
Type the following command in your terminal of choice, then press ENTER to run it. This will list all available subcommands and options.
```console
./dce
./DiscordChatExporter.Cli
```
> **Note**:
> On Windows, use `dce` instead of `./dce`.
> On Windows, if you're using the default Command Prompt (`cmd`), omit the leading `./` at the start of the command.
> **Docker** users, please refer to the [Docker usage instructions](Docker.md).
## CLI commands
| Command | Description |
| ----------------- | -------------------------------------------------------------------- |
| export | Exports one or more channels |
| list channels | Outputs the list of channels in the given server(s) |
| list channels dm | Outputs the list of direct message channels |
| list servers | Outputs the list of accessible servers |
| list unwrap | Resolves categories and forums in a channel list to their child channels and threads |
| guide | Explains how to obtain token, server, and channel ID |
| Command | Description |
| ----------- | ---------------------------------------------------- |
| export | Exports a channel |
| exportdm | Exports all direct message channels |
| exportguild | Exports all channels within the specified server |
| exportall | Exports all accessible channels |
| channels | Outputs the list of channels in the given server |
| dm | Outputs the list of direct message channels |
| guilds | Outputs the list of accessible servers |
| guide | Explains how to obtain token, server, and channel ID |
To use the commands, you'll need a token. For the instructions on how to get a token, please refer to [this page](Token-and-IDs.md), or run `./dce guide`.
To pass the token, use the `-t|--token` option:
```console
./dce export 53555 -t "mfa.Ifrn"
```
Alternatively, you can set the `DISCORD_TOKEN` environment variable and omit `-t`:
**Linux/macOS:**
```console
export DISCORD_TOKEN="mfa.Ifrn"
```
**Windows:**
```console
set DISCORD_TOKEN=mfa.Ifrn
```
The pipeline examples in this guide assume `DISCORD_TOKEN` is already set.
To use the commands, you'll need a token. For the instructions on how to get a token, please refer to [this page](Token-and-IDs.md), or run `./DiscordChatExporter.Cli guide`.
To get help with a specific command, run:
```console
./dce command --help
./DiscordChatExporter.Cli command --help
```
For example, to figure out how to use the `export` command, run:
```console
./dce export --help
./DiscordChatExporter.Cli export --help
```
## Export a specific channel
You can quickly export with DCE's default settings by providing the channel ID as a positional argument and `-t|--token`.
You can quickly export with DCE's default settings by using just `-t token` and `-c channelid`.
```console
./dce export 53555 -t "mfa.Ifrn"
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555
```
#### Changing the format
You can change the export format to `HtmlDark`, `HtmlLight`, `PlainText` `Json` or `Csv` with `-f|--format`. The default
You can change the export format to `HtmlDark`, `HtmlLight`, `PlainText` `Json` or `Csv` with `-f format`. The default
format is `HtmlDark`.
```console
./dce export 53555 -t "mfa.Ifrn" -f Json
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555 -f Json
```
#### Changing the output filename
You can change the filename by using `-o|--output`. e.g. for the `HTML` format:
You can change the filename by using `-o name.ext`. e.g., for the `HTML` format:
```console
./dce export 53555 -t "mfa.Ifrn" -o myserver.html
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555 -o myserver.html
```
#### Changing the output directory
You can change the export directory by using `-o|--output` and providing a path that ends with a slash or does not have a file
You can change the export directory by using `-o` and providing a path that ends with a slash or does not have a file
extension.
If any of the folders in the path have a space in its name, escape them with quotes (").
```console
./dce export 53555 -t "mfa.Ifrn" -o "C:\Discord Exports"
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555 -o "C:\Discord Exports"
```
#### Changing the filename and output directory
You can change both the filename and export directory by using `-o|--output`.
You can change both the filename and export directory by using `-o directory\name.ext`.
Note that the filename must have an extension, otherwise it will be considered a directory name.
If any of the folders in the path have a space in its name, escape them with quotes (").
```console
./dce export 53555 -t "mfa.Ifrn" -o "C:\Discord Exports\myserver.html"
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555 -o "C:\Discord Exports\myserver.html"
```
#### Generating the filename and output directory dynamically
@@ -128,7 +108,7 @@ If any of the folders in the path have a space in its name, escape them with quo
You can use template tokens to generate the output file path based on the server and channel metadata.
```console
./dce export 53555 -t "mfa.Ifrn" -o "C:\Discord Exports\%G\%T\%C.html"
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555 -o "C:\Discord Exports\%G\%T\%C.html"
```
Assuming you are exporting a channel named `"my-channel"` in the `"Text channels"` category from a server
@@ -156,13 +136,13 @@ You can use partitioning to split files after a given number of messages or file
For example, a channel with 36 messages set to be partitioned every 10 messages will output 4 files.
```console
./dce export 53555 -t "mfa.Ifrn" -p 10
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555 -p 10
```
A 45 MB channel set to be partitioned every 20 MB will output 3 files.
```console
./dce export 53555 -t "mfa.Ifrn" -p 20mb
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555 -p 20mb
```
#### Downloading assets
@@ -173,7 +153,7 @@ downloaded when using the plain text (TXT) export format.
A folder containing the assets will be created along with the exported chat. They must be kept together.
```console
./dce export 53555 -t "mfa.Ifrn" --media
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555 --media
```
#### Reusing assets
@@ -182,7 +162,7 @@ Previously downloaded assets can be reused to skip redundant downloads as long a
same folder. Using this option can speed up future exports. This option requires the `--media` option.
```console
./dce export 53555 -t "mfa.Ifrn" --media --reuse-media
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555 --media --reuse-media
```
#### Changing the media directory
@@ -191,7 +171,7 @@ By default, the media directory is created alongside the exported chat. You can
providing a path that ends with a slash. All of the exported media will be stored in this directory.
```console
./dce export 53555 -t "mfa.Ifrn" --media --media-dir "C:\Discord Media"
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555 --media --media-dir "C:\Discord Media"
```
#### Changing the date format
@@ -200,31 +180,31 @@ You can customize how dates are formatted in the exported files by using `--loca
locales. The default locale is `en-US`.
```console
./dce export 53555 -t "mfa.Ifrn" --locale "de-DE"
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555 --locale "de-DE"
```
#### Date ranges
**Messages sent before a date**
Use `--before` to export messages sent before the provided date. E.g. messages sent before September 18th, 2019:
Use `--before` to export messages sent before the provided date. e.g., messages sent before September 18th, 2019:
```console
./dce export 53555 -t "mfa.Ifrn" --before 2019-09-18
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555 --before 2019-09-18
```
**Messages sent after a date**
Use `--after` to export messages sent after the provided date. E.g. messages sent after September 17th, 2019 11:34 PM:
Use `--after` to export messages sent after the provided date. e.g., messages sent after September 17th, 2019 11:34 PM:
```console
./dce export 53555 -t "mfa.Ifrn" --after "2019-09-17 23:34"
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555 --after "2019-09-17 23:34"
```
**Messages sent in a date range**
Use `--before` and `--after` to export messages sent during the provided date range. E.g. messages sent between
Use `--before` and `--after` to export messages sent during the provided date range. e.g., messages sent between
September 17th, 2019 11:34 PM and September 18th:
```console
./dce export 53555 -t "mfa.Ifrn" --after "2019-09-17 23:34" --before "2019-09-18"
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555 --after "2019-09-17 23:34" --before "2019-09-18"
```
You can try different formats like `17-SEP-2019 11:34 PM` or even refine your ranges down to
@@ -238,111 +218,75 @@ formats [here](https://docs.microsoft.com/en-us/dotnet/standard/base-types/custo
Use `--filter` to filter what messages are included in the export.
```console
./dce export 53555 -t "mfa.Ifrn" --filter "from:Tyrrrz has:image"
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555 --filter "from:Tyrrrz has:image"
```
Documentation on message filter syntax can be found [here](https://github.com/Tyrrrz/DiscordChatExporter/blob/prime/.docs/Message-filters.md).
### Export channels from a specific server
> [!IMPORTANT]
> The following examples assume `DISCORD_TOKEN` is already set. See [CLI commands](#cli-commands) for instructions.
To export all channels in a specific server, use `list channels` to list channels and pipe the result to `export`.
**Linux/macOS:**
To export all channels in a specific server, use the `exportguild` command and provide the server ID through the `-g|--guild` option:
```console
./dce list channels 21814 | ./dce export
```
**Windows:**
```console
dce list channels 21814 | dce export
```
You can also list channels for multiple servers at once:
```console
./dce list channels 21814 35930 | ./dce export
./DiscordChatExporter.Cli exportguild -t "mfa.Ifrn" -g 21814
```
#### Including threads
By default, threads are not included. You can change this behavior by passing `--include-threads` to the `list channels` command. It has possible values of `none`, `active`, or `all`, indicating which threads should be included. To include both active and archived threads, use `--include-threads all`.
By default, threads are not included in the export. You can change this behavior by using `--include-threads` and
specifying which threads should be included. It has possible values of `none`, `active`, or `all`, indicating which
threads should be included. To include both active and archived threads, use `--include-threads all`.
```console
./dce list channels 21814 --include-threads all | ./dce export
./DiscordChatExporter.Cli exportguild -t "mfa.Ifrn" -g 21814 --include-threads all
```
#### Including voice channels
By default, voice channels are included. You can change this behavior by passing `--include-vc false` to the `list channels` command.
By default, voice channels are included in the export. You can change this behavior by using `--include-vc` and
specifying whether to include voice channels in the export. It has possible values of `true` or `false`, to exclude
voice channels, use `--include-vc false`.
```console
./dce list channels 21814 --include-vc false | ./dce export
./DiscordChatExporter.Cli exportguild -t "mfa.Ifrn" -g 21814 --include-vc false
```
### Export all DMs
### Export all channels
> [!IMPORTANT]
> The following examples assume `DISCORD_TOKEN` is already set. See [CLI commands](#cli-commands) for instructions.
To export all DMs:
**Linux/macOS:**
To export all accessible channels, use the `exportall` command:
```console
./dce list channels dm | ./dce export
./DiscordChatExporter.Cli exportall -t "mfa.Ifrn"
```
**Windows:**
#### Excluding DMs
To exclude DMs, add the `--include-dm false` option.
```console
dce list channels dm | dce export
./DiscordChatExporter.Cli exportall -t "mfa.Ifrn" --include-dm false
```
### List channels in a server
To list the channels available in a specific server, use the `list channels` command and provide the server ID as an argument:
To list the channels available in a specific server, use the `channels` command and provide the server ID through the `-g|--guild` option:
```console
./dce list channels 21814 -t "mfa.Ifrn"
```
The `list channels` command outputs a JSON array of channel objects. You can pipe this directly to the `export` command:
```console
./dce list channels 21814 | ./dce export
./DiscordChatExporter.Cli channels -t "mfa.Ifrn" -g 21814
```
### List direct message channels
To list all DM channels accessible to the current account, use the `list channels dm` command:
To list all DM channels accessible to the current account, use the `dm` command:
```console
./dce list channels dm -t "mfa.Ifrn"
```
The `list channels dm` command outputs a JSON array of channel objects. You can pipe this directly to the `export` command:
```console
./dce list channels dm | ./dce export
```
### Unwrap categories and forums
To resolve category and forum channels in a list to their child channels and thread posts, use the `list unwrap` command:
```console
./dce list channels 21814 | ./dce list unwrap | ./dce export
./DiscordChatExporter.Cli dm -t "mfa.Ifrn"
```
### List servers
To list all servers accessible by the current account, use the `list servers` command:
To list all servers accessible by the current account, use the `guilds` command:
```console
./dce list servers -t "mfa.Ifrn" > C:\path\to\output.txt
./DiscordChatExporter.Cli guilds -t "mfa.Ifrn" > C:\path\to\output.txt
```
+8 -8
View File
@@ -12,7 +12,7 @@
After extracting the `.zip`, run `DiscordChatExporter.exe` **(Windows)**, or `DiscordChatExporter` **(Linux)**.
If you're using **macOS**, you'll need to manually grant permission for the app to run.
If you're using **macOS**, you'll need to manually grant permission for the app to run.
If you skip these steps, the "DiscordChatExporter is damaged and cant be opened" error will be shown.
1. Open Terminal.app. You can search for it in Spotlight (press <kbd>⌘</kbd> + <kbd>Space</kbd> and type "Terminal").
@@ -56,10 +56,11 @@ In this screen you can customize the following:
- **Export format** - HTML (Dark), HTML (Light), TXT, CSV and JSON
- **Date range (after/before)** (Optional) - If set, only messages sent in the provided date range will be exported. Only one value (either after or before) is required if you want to use this option.
> **Note**:
> Please note that the time defaults to **12:00 AM** (midnight/00:00). This means that if you choose to export between Sep 17th and Sep 18th, messages from Sep 18th won't be exported.
- **Partition limit** (Optional) - Split output into partitions, each limited to this number of messages (e.g. 100) or file size (e.g. 10mb). For example, a channel with 36 messages set to be partitioned every 10 messages will output 4 files.
- **Partition limit** (Optional) - Split output into partitions, each limited to this number of messages (e.g., 100) or file size (e.g., 10mb). For example, a channel with 36 messages set to be partitioned every 10 messages will output 4 files.
- **Message Filter** (Optional) - Special notation for filtering the messages that get included in the export. See [Message filters](Message-filters.md) for more info.
@@ -77,29 +78,28 @@ In this screen you can customize the following:
## Settings
- **Auto-update** - Perform automatic updates on every launch.
Default: Enabled
Default: Enabled
> **Note**:
> Keep this option enabled to receive the latest features and bug fixes!
- **Dark mode** - Use darker colors in the UI (User Interface).
Default: Disabled
Default: Disabled
- **Persist token** - Persist last used token between sessions.
Default: Enabled
Default: Enabled
- **Show threads** - Controls whether threads are shown in the channel list.
Default: none
Default: none
- **Locale** - Customize how dates are formatted in the exported files.
- **Date format** - Customize how dates are formatted in the exported files in the settings menu ().
- **Parallel limit** - The number of channels that will be exported at the same time.
Default: 1
Default: 1
> **Note**:
> Try to keep this number low so that your account doesn't get flagged.
- **Normalize to UTC** - Convert all dates to UTC before exporting.
+1 -1
View File
@@ -2,7 +2,7 @@ blank_issues_enabled: false
contact_links:
- name: ⚠ Feature request
url: https://github.com/Tyrrrz/.github/blob/prime/docs/project-status.md
about: Sorry, but this project is in maintenance mode and no longer accepts new feature requests.
about: Sorry, but this project is in maintenance mode and currently doesn't accept new feature requests.
- name: 📖 Documentation
url: https://github.com/Tyrrrz/DiscordChatExporter/blob/prime/.docs
about: Find usage guides and frequently asked questions.
+18
View File
@@ -0,0 +1,18 @@
changelog:
exclude:
authors:
- dependabot
- dependabot[bot]
categories:
- title: Enhancements
labels:
- enhancement
- title: Bugs
labels:
- bug
- title: Other
labels:
- "*"
+5 -5
View File
@@ -23,10 +23,10 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Install Docker Buildx
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
- name: Build image
run: >
@@ -37,7 +37,7 @@ jobs:
--output type=tar,dest=DiscordChatExporter.Cli.Docker.tar
- name: Upload image
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: DiscordChatExporter.Cli.Docker
path: DiscordChatExporter.Cli.Docker.tar
@@ -55,10 +55,10 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Install Docker Buildx
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
- name: Login to DockerHub
run: >
+9 -9
View File
@@ -26,10 +26,10 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Install .NET
uses: actions/setup-dotnet@c2fa09f4bde5ebb9d1777cf28262a3eb3db3ced7 # v5.2.0
uses: actions/setup-dotnet@26b0ec14cb23fa6904739307f278c14f94c95bf1 # v5.4.0
# Build the project separately to discern between build and format errors
- name: Build
@@ -62,10 +62,10 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Install .NET
uses: actions/setup-dotnet@c2fa09f4bde5ebb9d1777cf28262a3eb3db3ced7 # v5.2.0
uses: actions/setup-dotnet@26b0ec14cb23fa6904739307f278c14f94c95bf1 # v5.4.0
- name: Run tests
env:
@@ -81,7 +81,7 @@ jobs:
DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=opencover
- name: Upload coverage
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v6.0.0
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
token: ${{ secrets.CODECOV_TOKEN }}
@@ -117,10 +117,10 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Install .NET
uses: actions/setup-dotnet@c2fa09f4bde5ebb9d1777cf28262a3eb3db3ced7 # v5.2.0
uses: actions/setup-dotnet@26b0ec14cb23fa6904739307f278c14f94c95bf1 # v5.4.0
- name: Publish app
run: >
@@ -135,7 +135,7 @@ jobs:
--self-contained
- name: Upload app binaries
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ${{ matrix.asset }}.${{ matrix.rid }}
path: ${{ matrix.app }}/bin/publish/
@@ -236,7 +236,7 @@ jobs:
steps:
- name: Notify Discord
uses: tyrrrz/action-http-request@25f132e48dea89c0f6b7955398270b506e1d51cb # 1.1.4
uses: tyrrrz/action-http-request@110f1a0f0f7e91c3b2de349539249f4573c243fa # 1.1.5
with:
url: ${{ secrets.DISCORD_WEBHOOK }}
method: POST
-1
View File
@@ -4,7 +4,6 @@
<Version>999.9.9-dev</Version>
<Company>Tyrrrz</Company>
<Copyright>Copyright (c) Oleksii Holub</Copyright>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
+26 -26
View File
@@ -4,43 +4,43 @@
</PropertyGroup>
<ItemGroup>
<PackageVersion Include="AngleSharp" Version="1.4.0" />
<PackageVersion Include="AsyncImageLoader.Avalonia" Version="3.7.0" />
<PackageVersion Include="AngleSharp" Version="1.5.1" />
<PackageVersion Include="AsyncImageLoader.Avalonia" Version="3.8.0" />
<PackageVersion Include="AsyncKeyedLock" Version="8.0.2" />
<PackageVersion Include="Avalonia" Version="11.3.13" />
<PackageVersion Include="Avalonia.Desktop" Version="11.3.13" />
<PackageVersion Include="Avalonia.Diagnostics" Version="11.3.13" />
<PackageVersion Include="Avalonia" Version="12.0.5" />
<PackageVersion Include="Avalonia.Desktop" Version="12.0.5" />
<PackageVersion Include="CliFx" Version="3.0.0" />
<PackageVersion Include="Cogwheel" Version="2.1.1" />
<PackageVersion Include="CommunityToolkit.Mvvm" Version="8.4.2" />
<PackageVersion Include="coverlet.collector" Version="8.0.1" />
<PackageVersion Include="CSharpier.MsBuild" Version="1.2.6" />
<PackageVersion Include="Deorcify" Version="1.1.0" />
<PackageVersion Include="ThisAssembly.Project" Version="2.1.2" />
<PackageVersion Include="DialogHost.Avalonia" Version="0.11.0" />
<PackageVersion Include="FluentAssertions" Version="8.9.0" />
<PackageVersion Include="GitHubActionsTestLogger" Version="3.0.3" />
<PackageVersion Include="Gress" Version="2.1.1" />
<PackageVersion Include="JsonExtensions" Version="1.2.0" />
<PackageVersion Include="Markdig" Version="1.1.2" />
<PackageVersion Include="Material.Avalonia" Version="3.9.2" />
<PackageVersion Include="Material.Icons.Avalonia" Version="2.2.0" />
<PackageVersion Include="Microsoft.Extensions.Configuration" Version="10.0.5" />
<PackageVersion Include="coverlet.collector" Version="10.0.1" />
<PackageVersion Include="CSharpier.MsBuild" Version="1.3.0" />
<PackageVersion Include="Deorcify" Version="2.0.1" />
<PackageVersion Include="DialogHost.Avalonia" Version="0.12.2" />
<PackageVersion Include="FluentAssertions" Version="8.10.0" />
<PackageVersion Include="GitHubActionsTestLogger" Version="3.0.4" />
<PackageVersion Include="Gress" Version="2.2.0" />
<PackageVersion Include="JsonExtensions" Version="1.2.3" />
<PackageVersion Include="Markdig" Version="1.3.2" />
<PackageVersion Include="Material.Avalonia" Version="3.16.1" />
<PackageVersion Include="Material.Icons.Avalonia" Version="3.0.2" />
<PackageVersion Include="Microsoft.Extensions.Configuration" Version="10.0.9" />
<PackageVersion
Include="Microsoft.Extensions.Configuration.EnvironmentVariables"
Version="10.0.5"
Version="10.0.9"
/>
<PackageVersion Include="Microsoft.Extensions.Configuration.UserSecrets" Version="10.0.5" />
<PackageVersion Include="Microsoft.Extensions.DependencyInjection" Version="10.0.5" />
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="18.3.0" />
<PackageVersion Include="Microsoft.Extensions.Configuration.UserSecrets" Version="10.0.9" />
<PackageVersion Include="Microsoft.Extensions.DependencyInjection" Version="10.0.9" />
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="18.7.0" />
<PackageVersion Include="Onova" Version="2.6.13" />
<PackageVersion Include="Polly" Version="8.6.6" />
<PackageVersion Include="RazorBlade" Version="0.11.0" />
<PackageVersion Include="Spectre.Console" Version="0.54.0" />
<PackageVersion Include="Polly" Version="8.7.0" />
<PackageVersion Include="PowerKit" Version="2.0.2" />
<PackageVersion Include="RazorBlade" Version="1.0.0" />
<PackageVersion Include="Spectre.Console" Version="0.57.1" />
<PackageVersion Include="Superpower" Version="3.1.0" />
<PackageVersion Include="ThisAssembly.Project" Version="2.1.2" />
<PackageVersion Include="WebMarkupMin.Core" Version="2.21.0" />
<PackageVersion Include="xunit" Version="2.9.3" />
<PackageVersion Include="xunit.runner.visualstudio" Version="3.1.5" />
<PackageVersion Include="YoutubeExplode" Version="6.5.7" />
<PackageVersion Include="YoutubeExplode" Version="6.6.0" />
</ItemGroup>
</Project>
@@ -14,17 +14,18 @@
<PackageReference Include="coverlet.collector" PrivateAssets="all" />
<PackageReference Include="CSharpier.MsBuild" PrivateAssets="all" />
<PackageReference Include="FluentAssertions" />
<PackageReference Include="GitHubActionsTestLogger" PrivateAssets="all" />
<PackageReference Include="GitHubActionsTestLogger" />
<PackageReference Include="JsonExtensions" />
<PackageReference Include="Microsoft.Extensions.Configuration" />
<PackageReference Include="Microsoft.Extensions.Configuration.EnvironmentVariables" />
<PackageReference Include="Microsoft.Extensions.Configuration.UserSecrets" />
<PackageReference Include="Microsoft.NET.Test.Sdk" />
<PackageReference Include="PowerKit" />
<PackageReference Include="xunit" />
<PackageReference Include="xunit.runner.visualstudio" PrivateAssets="all" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\DiscordChatExporter.Cli\DiscordChatExporter.Cli.csproj" />
<ProjectReference Include="../DiscordChatExporter.Cli/DiscordChatExporter.Cli.csproj" />
</ItemGroup>
</Project>
@@ -1,4 +1,4 @@
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Discord;
namespace DiscordChatExporter.Cli.Tests.Infra;
@@ -1,4 +1,4 @@
using System;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
@@ -1,4 +1,4 @@
using System;
using System;
using System.Reflection;
using Microsoft.Extensions.Configuration;
+1 -1
View File
@@ -1,4 +1,4 @@
# DiscordChatExporter Tests
# DiscordChatExporter Tests
This test suite runs against a real Discord server, specifically created to exercise different behaviors required by the test scenarios.
In order to run these tests locally, you need to join the test server and configure your authentication token.
@@ -1,4 +1,4 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using FluentAssertions;
using Xunit;
@@ -1,15 +1,15 @@
using System;
using System;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Exporting;
using FluentAssertions;
using JsonExtensions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -60,6 +60,52 @@ public class DateRangeSpecs
);
}
[Fact]
public async Task I_can_filter_the_reverse_export_to_only_include_messages_sent_after_the_specified_date()
{
// Arrange
var after = new DateTimeOffset(2021, 07, 24, 0, 0, 0, TimeSpan.Zero);
using var file = TempFile.Create();
// Act
await new ExportChannelsCommand
{
Token = Secrets.DiscordToken,
ChannelIds = [ChannelIds.DateRangeTestCases],
ExportFormat = ExportFormat.Json,
OutputPath = file.Path,
After = Snowflake.FromDate(after),
IsReverseMessageOrder = true,
}.ExecuteAsync(new FakeConsole());
// Assert
var timestamps = Json.Parse(await File.ReadAllTextAsync(file.Path))
.GetProperty("messages")
.EnumerateArray()
.Select(j => j.GetProperty("timestamp").GetDateTimeOffset())
.ToArray();
timestamps.All(t => t > after).Should().BeTrue();
timestamps.Should().BeInDescendingOrder();
timestamps
.Should()
.BeEquivalentTo(
[
new DateTimeOffset(2021, 09, 08, 14, 26, 35, TimeSpan.Zero),
new DateTimeOffset(2021, 07, 24, 14, 52, 40, TimeSpan.Zero),
new DateTimeOffset(2021, 07, 24, 14, 52, 39, TimeSpan.Zero),
new DateTimeOffset(2021, 07, 24, 14, 52, 38, TimeSpan.Zero),
new DateTimeOffset(2021, 07, 24, 13, 49, 13, TimeSpan.Zero),
],
o =>
o.Using<DateTimeOffset>(ctx =>
ctx.Subject.Should().BeCloseTo(ctx.Expectation, TimeSpan.FromSeconds(1))
)
.WhenTypeIs<DateTimeOffset>()
);
}
[Fact]
public async Task I_can_filter_the_export_to_only_include_messages_sent_before_the_specified_date()
{
@@ -1,15 +1,15 @@
using System;
using System;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Exporting;
using DiscordChatExporter.Core.Exporting.Filtering;
using FluentAssertions;
using JsonExtensions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -1,4 +1,4 @@
using System;
using System;
using System.Linq;
using System.Threading.Tasks;
using AngleSharp.Dom;
@@ -1,4 +1,4 @@
using System.IO;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using AngleSharp.Dom;
@@ -8,6 +8,7 @@ using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Exporting;
using FluentAssertions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -1,11 +1,11 @@
using System;
using System;
using System.Linq;
using System.Threading.Tasks;
using AngleSharp.Dom;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Utils.Extensions;
using FluentAssertions;
using PowerKit.Extensions;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -1,9 +1,9 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using AngleSharp.Dom;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils.Extensions;
using DiscordChatExporter.Core.Discord;
using FluentAssertions;
using PowerKit.Extensions;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -22,7 +22,7 @@ public class HtmlForwardSpecs
// Assert
message
.Text()
.ReplaceWhiteSpace()
.ReplaceWhiteSpace(' ')
.Should()
.ContainAll("Forwarded", @"¯\_(ツ)_/¯", "12/29/2025 2:14 PM");
}
@@ -1,4 +1,4 @@
using System.IO;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using AngleSharp.Dom;
@@ -8,6 +8,7 @@ using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Exporting;
using FluentAssertions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -1,9 +1,9 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using AngleSharp.Dom;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils.Extensions;
using DiscordChatExporter.Core.Discord;
using FluentAssertions;
using PowerKit.Extensions;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -22,11 +22,14 @@ public class HtmlMarkdownSpecs
// Assert
message
.Text()
.ReplaceWhiteSpace()
.ReplaceWhiteSpace(' ')
.Should()
.Contain("Default timestamp: 2/12/2023 1:36 PM");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -39,8 +42,11 @@ public class HtmlMarkdownSpecs
);
// Assert
message.Text().ReplaceWhiteSpace().Should().Contain("Short time timestamp: 1:36 PM");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
message.Text().ReplaceWhiteSpace(' ').Should().Contain("Short time timestamp: 1:36 PM");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -53,8 +59,11 @@ public class HtmlMarkdownSpecs
);
// Assert
message.Text().ReplaceWhiteSpace().Should().Contain("Long time timestamp: 1:36:12 PM");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
message.Text().ReplaceWhiteSpace(' ').Should().Contain("Long time timestamp: 1:36:12 PM");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -67,8 +76,11 @@ public class HtmlMarkdownSpecs
);
// Assert
message.Text().ReplaceWhiteSpace().Should().Contain("Short date timestamp: 2/12/2023");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
message.Text().ReplaceWhiteSpace(' ').Should().Contain("Short date timestamp: 2/12/2023");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -83,11 +95,14 @@ public class HtmlMarkdownSpecs
// Assert
message
.Text()
.ReplaceWhiteSpace()
.ReplaceWhiteSpace(' ')
.Should()
.Contain("Long date timestamp: Sunday, February 12, 2023");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -102,11 +117,14 @@ public class HtmlMarkdownSpecs
// Assert
message
.Text()
.ReplaceWhiteSpace()
.ReplaceWhiteSpace(' ')
.Should()
.Contain("Full timestamp: Sunday, February 12, 2023 1:36 PM");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -121,11 +139,14 @@ public class HtmlMarkdownSpecs
// Assert
message
.Text()
.ReplaceWhiteSpace()
.ReplaceWhiteSpace(' ')
.Should()
.Contain("Full long timestamp: Sunday, February 12, 2023 1:36:12 PM");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -140,11 +161,14 @@ public class HtmlMarkdownSpecs
// Assert
message
.Text()
.ReplaceWhiteSpace()
.ReplaceWhiteSpace(' ')
.Should()
.Contain("Relative timestamp: 2/12/2023 1:36 PM");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -1,4 +1,4 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using AngleSharp.Dom;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -1,4 +1,4 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using AngleSharp.Dom;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -1,4 +1,4 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
using FluentAssertions;
@@ -1,4 +1,4 @@
using System.Linq;
using System.Linq;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -1,13 +1,13 @@
using System.IO;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Exporting;
using FluentAssertions;
using JsonExtensions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -1,4 +1,4 @@
using System.Linq;
using System.Linq;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -1,4 +1,4 @@
using System.Linq;
using System.Linq;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -1,4 +1,4 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
using FluentAssertions;
@@ -1,4 +1,4 @@
using System.Linq;
using System.Linq;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -1,4 +1,4 @@
using System.Linq;
using System.Linq;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -1,12 +1,12 @@
using System.IO;
using System.IO;
using System.Threading.Tasks;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Exporting;
using DiscordChatExporter.Core.Exporting.Partitioning;
using FluentAssertions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -17,7 +17,7 @@ public class PartitioningSpecs
public async Task I_can_export_a_channel_with_partitioning_based_on_message_count()
{
// Arrange
using var dir = TempDir.Create();
using var dir = TempDirectory.Create();
var filePath = Path.Combine(dir.Path, "output.html");
// Act
@@ -38,7 +38,7 @@ public class PartitioningSpecs
public async Task I_can_export_a_channel_with_partitioning_based_on_file_size()
{
// Arrange
using var dir = TempDir.Create();
using var dir = TempDirectory.Create();
var filePath = Path.Combine(dir.Path, "output.html");
// Act
@@ -1,4 +1,4 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using FluentAssertions;
using Xunit;
@@ -1,7 +1,7 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils.Extensions;
using FluentAssertions;
using PowerKit.Extensions;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -16,7 +16,7 @@ public class PlainTextForwardSpecs
// Assert
document
.ReplaceWhiteSpace()
.ReplaceWhiteSpace(' ')
.Should()
.ContainAll("{Forwarded Message}", @"¯\_(ツ)_/¯", "12/28/2025 10:52 PM");
}
@@ -1,4 +1,4 @@
using System.IO;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Infrastructure;
@@ -7,6 +7,7 @@ using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Exporting;
using FluentAssertions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -17,7 +18,7 @@ public class SelfContainedSpecs
public async Task I_can_export_a_channel_and_download_all_referenced_assets()
{
// Arrange
using var dir = TempDir.Create();
using var dir = TempDirectory.Create();
var filePath = Path.Combine(dir.Path, "output.html");
// Act
@@ -1,19 +0,0 @@
using System.Text;
namespace DiscordChatExporter.Cli.Tests.Utils.Extensions;
internal static class StringExtensions
{
extension(string str)
{
public string ReplaceWhiteSpace(string replacement = " ")
{
var buffer = new StringBuilder(str.Length);
foreach (var ch in str)
buffer.Append(char.IsWhiteSpace(ch) ? replacement : ch);
return buffer.ToString();
}
}
}
+1 -1
View File
@@ -1,4 +1,4 @@
using AngleSharp.Html.Dom;
using AngleSharp.Html.Dom;
using AngleSharp.Html.Parser;
namespace DiscordChatExporter.Cli.Tests.Utils;
@@ -1,36 +0,0 @@
using System;
using System.IO;
using System.Reflection;
namespace DiscordChatExporter.Cli.Tests.Utils;
internal partial class TempDir(string path) : IDisposable
{
public string Path { get; } = path;
public void Dispose()
{
try
{
Directory.Delete(Path, true);
}
catch (DirectoryNotFoundException) { }
}
}
internal partial class TempDir
{
public static TempDir Create()
{
var dirPath = System.IO.Path.Combine(
System.IO.Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location)
?? Directory.GetCurrentDirectory(),
"Temp",
Guid.NewGuid().ToString()
);
Directory.CreateDirectory(dirPath);
return new TempDir(dirPath);
}
}
@@ -1,37 +0,0 @@
using System;
using System.IO;
using System.Reflection;
namespace DiscordChatExporter.Cli.Tests.Utils;
internal partial class TempFile(string path) : IDisposable
{
public string Path { get; } = path;
public void Dispose()
{
try
{
File.Delete(Path);
}
catch (FileNotFoundException) { }
}
}
internal partial class TempFile
{
public static TempFile Create()
{
var dirPath = System.IO.Path.Combine(
System.IO.Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location)
?? Directory.GetCurrentDirectory(),
"Temp"
);
Directory.CreateDirectory(dirPath);
var filePath = System.IO.Path.Combine(dirPath, Guid.NewGuid() + ".tmp");
return new TempFile(filePath);
}
}
@@ -1,4 +1,4 @@
{
{
"$schema": "https://xunit.net/schema/current/xunit.runner.schema.json",
"methodDisplayOptions": "all",
"methodDisplay": "method"
@@ -1,4 +1,4 @@
using System;
using System;
using System.Diagnostics.CodeAnalysis;
using System.Threading.Tasks;
using CliFx;
@@ -30,7 +30,7 @@ public abstract class DiscordCommandBase : ICommand
[CommandOption(
"respect-rate-limits",
Description = "Whether to respect advisory rate limits. "
+ "If disabled, only hard rate limits (i.e. 429 responses) will be respected."
+ "If disabled, only hard rate limits (i.e., 429 responses) will be respected."
)]
public bool ShouldRespectRateLimits { get; set; } = true;
@@ -1,4 +1,4 @@
using System;
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
@@ -59,7 +59,7 @@ public abstract class ExportCommandBase : DiscordCommandBase
"partition",
'p',
Description = "Split the output into partitions, each limited to the specified "
+ "number of messages (e.g. '100') or file size (e.g. '10mb')."
+ "number of messages (e.g., '100') or file size (e.g., '10mb')."
)]
public PartitionLimit PartitionLimit { get; set; } = PartitionLimit.Null;
@@ -165,6 +165,37 @@ public abstract class ExportCommandBase : DiscordCommandBase
throw new CommandException("Option --media-dir cannot be used without --media.");
}
// Make sure the user does not try to export multiple channels into one file.
// Output path must either be a directory or contain template tokens for this to work.
// Validate this up-front, before fetching threads, because thread fetching can take a
// long time and it's frustrating to fail only after it completes.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/799
// https://github.com/Tyrrrz/DiscordChatExporter/issues/917
// https://github.com/Tyrrrz/DiscordChatExporter/issues/1549
var mayExportMultipleChannels =
// Multiple channels were provided explicitly
channels.Count > 1
// Thread inclusion can add more channels to the export
|| ThreadInclusionMode != ThreadInclusionMode.None;
var isValidOutputPath =
// Anything is valid when exporting a single channel
!mayExportMultipleChannels
// When using template tokens, assume the user knows what they're doing
|| OutputPath.Contains('%')
// Otherwise, require an existing directory or an unambiguous directory path
|| Directory.Exists(OutputPath)
|| Path.EndsInDirectorySeparator(OutputPath);
if (!isValidOutputPath)
{
throw new CommandException(
"Attempted to export multiple channels, but the output path is neither a directory nor a template. "
+ "If the provided output path is meant to be treated as a directory, make sure it ends with a slash. "
+ $"Provided output path: '{OutputPath}'."
);
}
var unwrappedChannels = new List<Channel>(channels);
// Unwrap threads
@@ -205,28 +236,6 @@ public abstract class ExportCommandBase : DiscordCommandBase
await console.Output.WriteLineAsync($"Fetched {fetchedThreadsCount} thread(s).");
}
// Make sure the user does not try to export multiple channels into one file.
// Output path must either be a directory or contain template tokens for this to work.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/799
// https://github.com/Tyrrrz/DiscordChatExporter/issues/917
var isValidOutputPath =
// Anything is valid when exporting a single channel
unwrappedChannels.Count <= 1
// When using template tokens, assume the user knows what they're doing
|| OutputPath.Contains('%')
// Otherwise, require an existing directory or an unambiguous directory path
|| Directory.Exists(OutputPath)
|| Path.EndsInDirectorySeparator(OutputPath);
if (!isValidOutputPath)
{
throw new CommandException(
"Attempted to export multiple channels, but the output path is neither a directory nor a template. "
+ "If the provided output path is meant to be treated as a directory, make sure it ends with a slash. "
+ $"Provided output path: '{OutputPath}'."
);
}
// Export
var errorsByChannel = new ConcurrentDictionary<Channel, string>();
var warningsByChannel = new ConcurrentDictionary<Channel, string>();
@@ -1,4 +1,4 @@
using System;
using System;
using CliFx.Activation;
using DiscordChatExporter.Cli.Commands.Shared;
@@ -1,4 +1,4 @@
using System.Globalization;
using System.Globalization;
using CliFx.Activation;
namespace DiscordChatExporter.Cli.Commands.Converters;
@@ -0,0 +1,156 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Binding;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Cli.Utils.Extensions;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Discord.Dump;
using DiscordChatExporter.Core.Exceptions;
using Spectre.Console;
namespace DiscordChatExporter.Cli.Commands;
[Command("exportall", Description = "Exports all accessible channels.")]
public partial class ExportAllCommand : ExportCommandBase
{
[CommandOption("include-dm", Description = "Include direct message channels.")]
public bool IncludeDirectChannels { get; set; } = true;
[CommandOption("include-guilds", Description = "Include server channels.")]
public bool IncludeGuildChannels { get; set; } = true;
[CommandOption("include-vc", Description = "Include voice channels.")]
public bool IncludeVoiceChannels { get; set; } = true;
[CommandOption(
"data-package",
Description = "Path to the personal data package (ZIP file) requested from Discord. "
+ "If provided, only channels referenced in the dump will be exported."
)]
public string? DataPackageFilePath { get; set; }
public override async ValueTask ExecuteAsync(IConsole console)
{
await base.ExecuteAsync(console);
var cancellationToken = console.RegisterCancellationHandler();
var channels = new List<Channel>();
// Pull from the API
if (string.IsNullOrWhiteSpace(DataPackageFilePath))
{
await foreach (var guild in Discord.GetUserGuildsAsync(cancellationToken))
{
// Regular channels
await console.Output.WriteLineAsync(
$"Fetching channels for server '{guild.Name}'..."
);
var fetchedChannelsCount = 0;
await console
.CreateStatusTicker()
.StartAsync(
"...",
async ctx =>
{
await foreach (
var channel in Discord.GetGuildChannelsAsync(
guild.Id,
cancellationToken
)
)
{
if (channel.IsCategory)
continue;
if (!IncludeVoiceChannels && channel.IsVoice)
continue;
channels.Add(channel);
ctx.Status(
Markup.Escape($"Fetched '{channel.GetHierarchicalName()}'.")
);
fetchedChannelsCount++;
}
}
);
await console.Output.WriteLineAsync($"Fetched {fetchedChannelsCount} channel(s).");
}
}
// Pull from the data package
else
{
await console.Output.WriteLineAsync("Extracting channels...");
var dump = await DataDump.LoadAsync(DataPackageFilePath, cancellationToken);
var inaccessibleChannels = new List<DataDumpChannel>();
await console
.CreateStatusTicker()
.StartAsync(
"...",
async ctx =>
{
foreach (var dumpChannel in dump.Channels)
{
ctx.Status(
Markup.Escape(
$"Fetching '{dumpChannel.Name}' ({dumpChannel.Id})..."
)
);
try
{
var channel = await Discord.GetChannelAsync(
dumpChannel.Id,
cancellationToken
);
channels.Add(channel);
}
catch (DiscordChatExporterException)
{
inaccessibleChannels.Add(dumpChannel);
}
}
}
);
await console.Output.WriteLineAsync($"Fetched {channels} channel(s).");
// Print inaccessible channels
if (inaccessibleChannels.Any())
{
await console.Output.WriteLineAsync();
using (console.WithForegroundColor(ConsoleColor.Red))
{
await console.Error.WriteLineAsync(
"Failed to access the following channel(s):"
);
}
foreach (var dumpChannel in inaccessibleChannels)
await console.Error.WriteLineAsync($"{dumpChannel.Name} ({dumpChannel.Id})");
await console.Error.WriteLineAsync();
}
}
// Filter out unwanted channels
if (!IncludeDirectChannels)
channels.RemoveAll(c => c.IsDirect);
if (!IncludeGuildChannels)
channels.RemoveAll(c => c.IsGuild);
if (!IncludeVoiceChannels)
channels.RemoveAll(c => c.IsVoice);
await ExportAsync(console, channels);
}
}
@@ -1,27 +1,25 @@
using System.Collections.Generic;
using System.Text.Json;
using System.Collections.Generic;
using System.Threading.Tasks;
using CliFx;
using CliFx.Binding;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Cli.Utils.Extensions;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Discord.Data;
using PowerKit.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("export", Description = "Exports one or multiple channels.")]
public partial class ExportChannelsCommand : ExportCommandBase
{
[CommandParameter(
0,
Name = "channel-ids",
// TODO: change this to plural (breaking change)
[CommandOption(
"channel",
'c',
Description = "Channel ID(s). "
+ "If not provided, channel IDs are read from standard input (one per line or as a JSON array), "
+ "enabling piping from the 'list channels' or 'list channels dm' commands."
+ "If provided with category ID(s), all channels inside those categories will be exported."
)]
public IReadOnlyList<Snowflake> ChannelIds { get; set; } = [];
public required IReadOnlyList<Snowflake> ChannelIds { get; set; }
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -29,46 +27,35 @@ public partial class ExportChannelsCommand : ExportCommandBase
var cancellationToken = console.RegisterCancellationHandler();
// If no channel IDs were specified, read them from stdin
var channelIds = new List<Snowflake>(ChannelIds);
if (channelIds.Count == 0 && console.IsInputRedirected)
{
await foreach (var line in console.Input.ReadLinesAsync(cancellationToken))
{
var trimmed = line.Trim();
if (string.IsNullOrEmpty(trimmed))
continue;
// Snowflake IDs are numeric; non-numeric input is treated as a JSON array
if (!char.IsAsciiDigit(trimmed[0]))
{
using var doc = JsonDocument.Parse(trimmed);
foreach (var element in doc.RootElement.EnumerateArray())
channelIds.Add(Snowflake.Parse(element.GetProperty("id").GetString()!));
}
else
{
channelIds.Add(Snowflake.Parse(trimmed));
}
}
}
if (channelIds.Count == 0)
{
throw new CommandException(
"No channel IDs provided. "
+ "Specify channel IDs as arguments or pipe them from the 'list channels' or 'list channels dm' commands."
);
}
await console.Output.WriteLineAsync("Resolving channel(s)...");
var channels = new List<Channel>();
var channelsByGuild = new Dictionary<Snowflake, IReadOnlyList<Channel>>();
foreach (var channelId in channelIds)
foreach (var channelId in ChannelIds)
{
var channel = await Discord.GetChannelAsync(channelId, cancellationToken);
channels.Add(channel);
// Unwrap categories
if (channel.IsCategory)
{
var guildChannels =
channelsByGuild.GetValueOrDefault(channel.GuildId)
?? await Discord.GetGuildChannelsAsync(channel.GuildId, cancellationToken);
foreach (var guildChannel in guildChannels)
{
if (guildChannel.Parent?.Id == channel.Id)
channels.Add(guildChannel);
}
// Cache the guild channels to avoid redundant work
channelsByGuild[channel.GuildId] = guildChannels;
}
else
{
channels.Add(channel);
}
}
await ExportAsync(console, channels);
@@ -0,0 +1,27 @@
using System.Threading.Tasks;
using CliFx.Binding;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Core.Discord.Data;
using PowerKit.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("exportdm", Description = "Exports all direct message channels.")]
public partial class ExportDirectMessagesCommand : ExportCommandBase
{
public override async ValueTask ExecuteAsync(IConsole console)
{
await base.ExecuteAsync(console);
var cancellationToken = console.RegisterCancellationHandler();
await console.Output.WriteLineAsync("Fetching channels...");
var channels = await Discord.GetGuildChannelsAsync(
Guild.DirectMessages.Id,
cancellationToken
);
await ExportAsync(console, channels);
}
}
@@ -0,0 +1,61 @@
using System.Collections.Generic;
using System.Threading.Tasks;
using CliFx.Binding;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Cli.Utils.Extensions;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Discord.Data;
using Spectre.Console;
namespace DiscordChatExporter.Cli.Commands;
[Command("exportguild", Description = "Exports all channels within the specified server.")]
public partial class ExportGuildCommand : ExportCommandBase
{
[CommandOption("guild", 'g', Description = "Server ID.")]
public required Snowflake GuildId { get; set; }
[CommandOption("include-vc", Description = "Include voice channels.")]
public bool IncludeVoiceChannels { get; set; } = true;
public override async ValueTask ExecuteAsync(IConsole console)
{
await base.ExecuteAsync(console);
var cancellationToken = console.RegisterCancellationHandler();
var channels = new List<Channel>();
await console.Output.WriteLineAsync("Fetching channels...");
var fetchedChannelsCount = 0;
await console
.CreateStatusTicker()
.StartAsync(
"...",
async ctx =>
{
await foreach (
var channel in Discord.GetGuildChannelsAsync(GuildId, cancellationToken)
)
{
if (channel.IsCategory)
continue;
if (!IncludeVoiceChannels && channel.IsVoice)
continue;
channels.Add(channel);
ctx.Status(Markup.Escape($"Fetched '{channel.GetHierarchicalName()}'."));
fetchedChannelsCount++;
}
}
);
await console.Output.WriteLineAsync($"Fetched {fetchedChannelsCount} channel(s).");
await ExportAsync(console, channels);
}
}
@@ -1,24 +1,21 @@
using System.Collections.Generic;
using System;
using System.Linq;
using System.Text.Json;
using System.Threading.Tasks;
using CliFx.Binding;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Cli.Commands.Converters;
using DiscordChatExporter.Cli.Commands.Shared;
using DiscordChatExporter.Cli.Utils.Json;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Utils.Extensions;
using PowerKit.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("list channels", Description = "Gets the list of channels in one or more servers.")]
[Command("channels", Description = "Get the list of channels in a server.")]
public partial class GetChannelsCommand : DiscordCommandBase
{
[CommandParameter(0, Name = "server-ids", Description = "Server ID(s).")]
public required IReadOnlyList<Snowflake> ServerIds { get; set; }
[CommandOption("guild", 'g', Description = "Server ID.")]
public required Snowflake GuildId { get; set; }
[CommandOption("include-vc", Description = "Include voice channels.")]
public bool IncludeVoiceChannels { get; set; } = true;
@@ -36,44 +33,82 @@ public partial class GetChannelsCommand : DiscordCommandBase
var cancellationToken = console.RegisterCancellationHandler();
var allChannels = new List<Channel>();
var channels = (await Discord.GetGuildChannelsAsync(GuildId, cancellationToken))
.Where(c => !c.IsCategory)
.Where(c => IncludeVoiceChannels || !c.IsVoice)
.OrderBy(c => c.Parent?.Position)
.ThenBy(c => c.Name)
.ToArray();
foreach (var serverId in ServerIds)
{
var channels = (await Discord.GetGuildChannelsAsync(serverId, cancellationToken))
.Where(c => !c.IsCategory)
.Where(c => IncludeVoiceChannels || !c.IsVoice)
.OrderBy(c => c.Parent?.Position)
.ThenBy(c => c.Name)
.ToArray();
var channelIdMaxLength = channels
.Select(c => c.Id.ToString().Length)
.OrderDescending()
.FirstOrDefault();
var threads =
ThreadInclusionMode != ThreadInclusionMode.None
? (
await Discord.GetGuildThreadsAsync(
serverId,
ThreadInclusionMode == ThreadInclusionMode.All,
null,
null,
cancellationToken
)
var threads =
ThreadInclusionMode != ThreadInclusionMode.None
? (
await Discord.GetGuildThreadsAsync(
GuildId,
ThreadInclusionMode == ThreadInclusionMode.All,
null,
null,
cancellationToken
)
.OrderBy(c => c.Name)
.ToArray()
: [];
)
.OrderBy(c => c.Name)
.ToArray()
: [];
foreach (var channel in channels)
foreach (var channel in channels)
{
// Channel ID
await console.Output.WriteAsync(
channel.Id.ToString().PadRight(channelIdMaxLength, ' ')
);
// Separator
using (console.WithForegroundColor(ConsoleColor.DarkGray))
await console.Output.WriteAsync(" | ");
// Channel name
using (console.WithForegroundColor(ConsoleColor.White))
await console.Output.WriteLineAsync(channel.GetHierarchicalName());
var channelThreads = threads.Where(t => t.Parent?.Id == channel.Id).ToArray();
var channelThreadIdMaxLength = channelThreads
.Select(t => t.Id.ToString().Length)
.OrderDescending()
.FirstOrDefault();
foreach (var channelThread in channelThreads)
{
allChannels.Add(channel);
allChannels.AddRange(threads.Where(t => t.Parent?.Id == channel.Id));
// Indent
await console.Output.WriteAsync(" * ");
// Thread ID
await console.Output.WriteAsync(
channelThread.Id.ToString().PadRight(channelThreadIdMaxLength, ' ')
);
// Separator
using (console.WithForegroundColor(ConsoleColor.DarkGray))
await console.Output.WriteAsync(" | ");
// Thread name
using (console.WithForegroundColor(ConsoleColor.White))
await console.Output.WriteAsync($"Thread / {channelThread.Name}");
// Separator
using (console.WithForegroundColor(ConsoleColor.DarkGray))
await console.Output.WriteAsync(" | ");
// Thread status
using (console.WithForegroundColor(ConsoleColor.White))
await console.Output.WriteLineAsync(
channelThread.IsArchived ? "Archived" : "Active"
);
}
}
await console.Output.WriteLineAsync(
JsonSerializer.Serialize(
allChannels.ToArray(),
CliJsonSerializerContext.Instance.ChannelArray
)
);
}
}
@@ -1,16 +1,15 @@
using System.Linq;
using System.Text.Json;
using System;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Binding;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Cli.Utils.Json;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Utils.Extensions;
using PowerKit.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("list channels dm", Description = "Gets the list of direct message channels.")]
[Command("dm", Description = "Gets the list of all direct message channels.")]
public partial class GetDirectChannelsCommand : DiscordCommandBase
{
public override async ValueTask ExecuteAsync(IConsole console)
@@ -26,8 +25,25 @@ public partial class GetDirectChannelsCommand : DiscordCommandBase
.ThenBy(c => c.Name)
.ToArray();
await console.Output.WriteLineAsync(
JsonSerializer.Serialize(channels, CliJsonSerializerContext.Instance.ChannelArray)
);
var channelIdMaxLength = channels
.Select(c => c.Id.ToString().Length)
.OrderDescending()
.FirstOrDefault();
foreach (var channel in channels)
{
// Channel ID
await console.Output.WriteAsync(
channel.Id.ToString().PadRight(channelIdMaxLength, ' ')
);
// Separator
using (console.WithForegroundColor(ConsoleColor.DarkGray))
await console.Output.WriteAsync(" | ");
// Channel name
using (console.WithForegroundColor(ConsoleColor.White))
await console.Output.WriteLineAsync(channel.GetHierarchicalName());
}
}
}
@@ -1,16 +1,15 @@
using System.Linq;
using System.Text.Json;
using System;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Binding;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Cli.Utils.Json;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Utils.Extensions;
using PowerKit.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("list servers", Description = "Gets the list of accessible servers.")]
[Command("guilds", Description = "Gets the list of accessible servers.")]
public partial class GetGuildsCommand : DiscordCommandBase
{
public override async ValueTask ExecuteAsync(IConsole console)
@@ -25,8 +24,23 @@ public partial class GetGuildsCommand : DiscordCommandBase
.ThenBy(g => g.Name)
.ToArray();
await console.Output.WriteLineAsync(
JsonSerializer.Serialize(guilds, CliJsonSerializerContext.Instance.GuildArray)
);
var guildIdMaxLength = guilds
.Select(g => g.Id.ToString().Length)
.OrderDescending()
.FirstOrDefault();
foreach (var guild in guilds)
{
// Guild ID
await console.Output.WriteAsync(guild.Id.ToString().PadRight(guildIdMaxLength, ' '));
// Separator
using (console.WithForegroundColor(ConsoleColor.DarkGray))
await console.Output.WriteAsync(" | ");
// Guild name
using (console.WithForegroundColor(ConsoleColor.White))
await console.Output.WriteLineAsync(guild.Name);
}
}
}
@@ -1,13 +0,0 @@
using System.Threading.Tasks;
using CliFx;
using CliFx.Binding;
using CliFx.Infrastructure;
namespace DiscordChatExporter.Cli.Commands;
[Command("list", Description = "Lists channels, DMs, or servers.")]
public partial class ListCommand : ICommand
{
public ValueTask ExecuteAsync(IConsole console) =>
throw new CommandException("Use one of the named commands listed below.", showHelp: true);
}
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Cli.Commands.Shared;
namespace DiscordChatExporter.Cli.Commands.Shared;
public enum ThreadInclusionMode
{
@@ -1,95 +0,0 @@
using System.Collections.Generic;
using System.Text;
using System.Text.Json;
using System.Threading.Tasks;
using CliFx;
using CliFx.Binding;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Cli.Utils.Extensions;
using DiscordChatExporter.Cli.Utils.Json;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command(
"list unwrap",
Description = "Resolves categories and forums in a channel list to their child channels and threads."
)]
public partial class UnwrapChannelsCommand : DiscordCommandBase
{
public override async ValueTask ExecuteAsync(IConsole console)
{
await base.ExecuteAsync(console);
var cancellationToken = console.RegisterCancellationHandler();
// Read all JSON from stdin (produced by 'list channels' or 'list channels dm')
var sb = new StringBuilder();
await foreach (var line in console.Input.ReadLinesAsync(cancellationToken))
sb.Append(line);
Channel[] channels;
try
{
channels =
JsonSerializer.Deserialize(
sb.ToString().Trim(),
CliJsonSerializerContext.Instance.ChannelArray
) ?? [];
}
catch (JsonException)
{
throw new CommandException(
"Failed to parse input as a JSON channel array. "
+ "Pipe the output of 'list channels' or 'list channels dm' to this command."
);
}
var result = new List<Channel>();
var channelsByGuild = new Dictionary<Snowflake, IReadOnlyList<Channel>>();
foreach (var channel in channels)
{
if (channel.IsCategory)
{
// Expand category to its child channels
var guildChannels =
channelsByGuild.GetValueOrDefault(channel.GuildId)
?? await Discord.GetGuildChannelsAsync(channel.GuildId, cancellationToken);
foreach (var guildChannel in guildChannels)
{
if (guildChannel.Parent?.Id == channel.Id)
result.Add(guildChannel);
}
channelsByGuild[channel.GuildId] = guildChannels;
}
else if (channel.Kind == ChannelKind.GuildForum)
{
// Expand forum to its thread posts
await foreach (
var thread in Discord.GetChannelThreadsAsync(
[channel],
cancellationToken: cancellationToken
)
)
result.Add(thread);
}
else
{
result.Add(channel);
}
}
await console.Output.WriteLineAsync(
JsonSerializer.Serialize(
result.ToArray(),
CliJsonSerializerContext.Instance.ChannelArray
)
);
}
}
@@ -1,7 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<ApplicationIcon>..\favicon.ico</ApplicationIcon>
<ApplicationIcon>../favicon.ico</ApplicationIcon>
<PublishTrimmed>true</PublishTrimmed>
<CopyOutputSymbolsToPublishDirectory>false</CopyOutputSymbolsToPublishDirectory>
</PropertyGroup>
@@ -9,25 +9,13 @@
<ItemGroup>
<PackageReference Include="CliFx" />
<PackageReference Include="CSharpier.MsBuild" PrivateAssets="all" />
<PackageReference Include="Deorcify" PrivateAssets="all" />
<PackageReference Include="Deorcify" />
<PackageReference Include="Gress" />
<PackageReference Include="PowerKit" />
<PackageReference Include="Spectre.Console" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\DiscordChatExporter.Core\DiscordChatExporter.Core.csproj" />
</ItemGroup>
<ItemGroup>
<None
Include="dce"
CopyToOutputDirectory="PreserveNewest"
CopyToPublishDirectory="PreserveNewest"
/>
<None
Include="dce.bat"
CopyToOutputDirectory="PreserveNewest"
CopyToPublishDirectory="PreserveNewest"
/>
<ProjectReference Include="../DiscordChatExporter.Core/DiscordChatExporter.Core.csproj" />
</ItemGroup>
</Project>
+1 -1
View File
@@ -1,4 +1,4 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using CliFx;
namespace DiscordChatExporter.Cli;
@@ -1,8 +1,4 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Runtime.CompilerServices;
using System.Threading;
using System;
using System.Threading.Tasks;
using CliFx.Infrastructure;
using Spectre.Console;
@@ -65,15 +61,4 @@ internal static class ConsoleExtensions
progressTask.StopTask();
}
}
public static async IAsyncEnumerable<string> ReadLinesAsync(
this TextReader reader,
[EnumeratorCancellation] CancellationToken cancellationToken = default
)
{
while (await reader.ReadLineAsync(cancellationToken).ConfigureAwait(false) is { } line)
{
yield return line;
}
}
}
@@ -1,26 +0,0 @@
using System.Text.Json;
using System.Text.Json.Serialization;
using DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Cli.Utils.Json;
[JsonSourceGenerationOptions(
PropertyNamingPolicy = JsonKnownNamingPolicy.CamelCase,
GenerationMode = JsonSourceGenerationMode.Metadata
)]
[JsonSerializable(typeof(Channel[]))]
[JsonSerializable(typeof(Guild[]))]
internal partial class CliJsonSerializerContext : JsonSerializerContext
{
// Instance pre-configured with converters for Snowflake (serialised as a string)
// and all enum types (serialised as their name). Defined here so the Core types
// are never touched.
public static CliJsonSerializerContext Instance { get; } =
new(
new JsonSerializerOptions
{
PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
Converters = { new SnowflakeJsonConverter(), new JsonStringEnumConverter() },
}
);
}
@@ -1,21 +0,0 @@
using System;
using System.Text.Json;
using System.Text.Json.Serialization;
using DiscordChatExporter.Core.Discord;
namespace DiscordChatExporter.Cli.Utils.Json;
internal class SnowflakeJsonConverter : JsonConverter<Snowflake>
{
public override Snowflake Read(
ref Utf8JsonReader reader,
Type typeToConvert,
JsonSerializerOptions options
) => Snowflake.Parse(reader.GetString()!);
public override void Write(
Utf8JsonWriter writer,
Snowflake value,
JsonSerializerOptions options
) => writer.WriteStringValue(value.ToString());
}
-2
View File
@@ -1,2 +0,0 @@
#!/usr/bin/env sh
exec "$(dirname "$0")/DiscordChatExporter.Cli" "$@"
-2
View File
@@ -1,2 +0,0 @@
@echo off
"%~dp0DiscordChatExporter.Cli.exe" %*
@@ -1,6 +1,6 @@
using System.Text.Json;
using DiscordChatExporter.Core.Utils.Extensions;
using System.Text.Json;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,4 +1,4 @@
using System;
using System;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,9 +1,9 @@
using System;
using System;
using System.IO;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,9 +1,9 @@
using System.Collections.Generic;
using System.Collections.Generic;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,4 +1,4 @@
using System.Collections.Generic;
using System.Collections.Generic;
using System.Linq;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Discord.Data;
// https://discord.com/developers/docs/resources/channel#channel-object-channel-types
public enum ChannelKind
@@ -1,4 +1,4 @@
using System;
using System;
using System.Diagnostics.CodeAnalysis;
using System.Globalization;
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Discord.Data.Common;
namespace DiscordChatExporter.Core.Discord.Data.Common;
public interface IHasId
{
@@ -1,4 +1,4 @@
using System;
using System;
using System.Globalization;
using System.Linq;
@@ -3,8 +3,8 @@ using System.Collections.Generic;
using System.Drawing;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data.Embeds;
@@ -47,9 +47,7 @@ public partial record Embed
var kind =
json.GetPropertyOrNull("type")
?.GetStringOrNull()
?.Pipe(s =>
Enum.TryParse<EmbedKind>(s, true, out var result) ? result : (EmbedKind?)null
)
.Pipe(s => Enum.ParseOrNull<EmbedKind>(s, true))
?? EmbedKind.Rich;
var url = json.GetPropertyOrNull("url")?.GetNonWhiteSpaceStringOrNull();
@@ -58,7 +56,7 @@ public partial record Embed
var color = json.GetPropertyOrNull("color")
?.GetInt32OrNull()
?.Pipe(System.Drawing.Color.FromArgb)
.ResetAlpha();
.WithFullAlpha();
var author = json.GetPropertyOrNull("author")?.Pipe(EmbedAuthor.Parse);
var description = json.GetPropertyOrNull("description")?.GetStringOrNull();
@@ -74,7 +72,7 @@ public partial record Embed
// Under the Discord API model, embeds can only have at most one image.
// Because of that, embeds that are rendered with multiple images on the client
// (e.g. tweet embeds), are exposed from the API as multiple separate embeds.
// (e.g., tweet embeds), are exposed from the API as multiple separate embeds.
// Our embed model is consistent with the user-facing side of Discord, so images
// are stored as an array. The API will only ever return one image, but we deal
// with this by merging related embeds at the end of the message parsing process.
@@ -1,4 +1,4 @@
using System.Text.RegularExpressions;
using System.Text.RegularExpressions;
namespace DiscordChatExporter.Core.Discord.Data.Embeds;
@@ -1,4 +1,4 @@
using System.Text.RegularExpressions;
using System.Text.RegularExpressions;
namespace DiscordChatExporter.Core.Discord.Data.Embeds;
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Discord.Data.Embeds;
namespace DiscordChatExporter.Core.Discord.Data.Embeds;
public partial record YouTubeVideoEmbedProjection(string VideoId)
{
@@ -1,8 +1,8 @@
using System.Text.Json;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -10,14 +10,14 @@ namespace DiscordChatExporter.Core.Discord.Data;
public partial record Emoji(
// Only present on custom emoji
Snowflake? Id,
// Name of a custom emoji (e.g. LUL) or actual representation of a standard emoji (e.g. 🙂)
// Name of a custom emoji (e.g., LUL) or actual representation of a standard emoji (e.g., 🙂)
string Name,
bool IsAnimated
)
{
public bool IsCustomEmoji { get; } = Id is not null;
// Name of a custom emoji (e.g. LUL) or name of a standard emoji (e.g. slight_smile)
// Name of a custom emoji (e.g., LUL) or name of a standard emoji (e.g., slight_smile)
public string Code { get; } = Id is not null ? Name : EmojiIndex.TryGetCode(Name) ?? Name;
public string ImageUrl { get; } =
@@ -1,4 +1,4 @@
using System;
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
@@ -1,7 +1,7 @@
using System.Text.Json;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,6 +1,6 @@
using System.Text.Json;
using DiscordChatExporter.Core.Utils.Extensions;
using System.Text.Json;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,7 +1,7 @@
using System.Text.Json;
using System.Text.Json;
using System.Text.RegularExpressions;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,9 +1,9 @@
using System.Collections.Generic;
using System.Collections.Generic;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,11 +1,11 @@
using System;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Discord.Data.Embeds;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,4 +1,4 @@
using System;
using System;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Discord.Data;
// https://discord.com/developers/docs/resources/channel#message-object-message-types
public enum MessageKind
@@ -13,4 +13,5 @@ public enum MessageKind
GuildMemberJoin = 7,
ThreadCreated = 18,
Reply = 19,
ThreadStarterMessage = 21,
}
@@ -1,6 +1,6 @@
using System.Text.Json;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,5 +1,5 @@
using System.Text.Json;
using DiscordChatExporter.Core.Utils.Extensions;
using System.Text.Json;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,8 +1,8 @@
using System.Drawing;
using System.Drawing;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -18,7 +18,7 @@ public record Role(Snowflake Id, string Name, int Position, Color? Color) : IHas
var color = json.GetPropertyOrNull("color")
?.GetInt32OrNull()
?.Pipe(System.Drawing.Color.FromArgb)
.ResetAlpha()
.WithFullAlpha()
.NullIf(c => c.ToRgb() <= 0);
return new Role(id, name, position, color);
@@ -1,8 +1,8 @@
using System;
using System;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Discord.Data;
public enum StickerFormat
{
@@ -1,7 +1,7 @@
using System.Text.Json;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
+161 -105
View File
@@ -1,4 +1,4 @@
using System;
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
@@ -11,10 +11,10 @@ using System.Threading.Tasks;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Exceptions;
using DiscordChatExporter.Core.Utils;
using DiscordChatExporter.Core.Utils.Extensions;
using Gress;
using JsonExtensions.Http;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord;
@@ -30,9 +30,8 @@ public class DiscordClient(
string url,
TokenKind tokenKind,
CancellationToken cancellationToken = default
)
{
return await Http.ResponseResiliencePipeline.ExecuteAsync(
) =>
await Http.ResponseResiliencePipeline.ExecuteAsync(
async innerCancellationToken =>
{
using var request = new HttpRequestMessage(HttpMethod.Get, new Uri(_baseUri, url));
@@ -59,12 +58,12 @@ public class DiscordClient(
{
var remainingRequestCount = response
.Headers.TryGetValue("X-RateLimit-Remaining")
?.Pipe(s => int.Parse(s, CultureInfo.InvariantCulture));
?.Pipe(s => int.ParseOrNull(s, CultureInfo.InvariantCulture));
var resetAfterDelay = response
.Headers.TryGetValue("X-RateLimit-Reset-After")
?.Pipe(s => double.Parse(s, CultureInfo.InvariantCulture))
.Pipe(TimeSpan.FromSeconds);
?.Pipe(s => double.ParseOrNull(s, CultureInfo.InvariantCulture))
?.Pipe(TimeSpan.FromSeconds);
// If this was the last request available before hitting the rate limit,
// wait out the reset time so that future requests can succeed.
@@ -91,7 +90,6 @@ public class DiscordClient(
},
cancellationToken
);
}
private async ValueTask<TokenKind> ResolveTokenKindAsync(
CancellationToken cancellationToken = default
@@ -161,7 +159,7 @@ public class DiscordClient(
$"""
Request to '{url}' failed: {response
.StatusCode.ToString()
.ToSpaceSeparatedWords()
.SeparateWords(' ')
.ToLowerInvariant()}.
Response content: {await response.Content.ReadAsStringAsync(
cancellationToken
@@ -364,6 +362,7 @@ public class DiscordClient(
$"guilds/{guildId}/members/{memberId}",
cancellationToken
);
return response?.Pipe(j => Member.Parse(j, guildId));
}
@@ -412,14 +411,12 @@ public class DiscordClient(
?.GetNonWhiteSpaceStringOrNull()
?.Pipe(Snowflake.Parse);
Channel? parent = null;
if (parentId is not null)
{
// It's possible for the parent channel to be inaccessible, despite the
// child channel being accessible.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/1108
parent = await TryGetChannelAsync(parentId.Value, cancellationToken);
}
// It's possible for the parent channel to be inaccessible, despite the
// child channel being accessible.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/1108
var parent = parentId is not null
? await TryGetChannelAsync(parentId.Value, cancellationToken)
: null;
return Channel.Parse(response.Value, parent);
}
@@ -595,6 +592,33 @@ public class DiscordClient(
}
}
public async ValueTask<Message?> TryGetMessageAsync(
Snowflake channelId,
Snowflake messageId,
CancellationToken cancellationToken = default
)
{
// Use the regular message listing endpoint with the 'around' parameter instead of the
// dedicated single-message endpoint, because the latter is not accessible to user tokens.
var url = new UrlBuilder()
.SetPath($"channels/{channelId}/messages")
.SetQueryParameter("around", messageId.ToString())
.SetQueryParameter("limit", "1")
.Build();
// Can be null on channels that the user cannot access
var response = await TryGetJsonResponseAsync(url, cancellationToken);
if (response is null)
return null;
// The endpoint returns messages around the requested ID, so make sure to only return
// the message that exactly matches it (it may be absent if it has been deleted).
return response
.Value.EnumerateArray()
.Select(Message.Parse)
.FirstOrDefault(m => m.Id == messageId);
}
private async ValueTask<Message?> TryGetFirstMessageAsync(
Snowflake channelId,
Snowflake? after = null,
@@ -607,8 +631,12 @@ public class DiscordClient(
.SetQueryParameter("after", (after ?? Snowflake.Zero).ToString())
.Build();
var response = await GetJsonResponseAsync(url, cancellationToken);
var message = response.EnumerateArray().Select(Message.Parse).FirstOrDefault();
// Can be null on channels that the user cannot access
var response = await TryGetJsonResponseAsync(url, cancellationToken);
if (response is null)
return null;
var message = response.Value.EnumerateArray().Select(Message.Parse).FirstOrDefault();
return message;
}
@@ -625,36 +653,64 @@ public class DiscordClient(
.SetQueryParameter("before", before?.ToString())
.Build();
var response = await GetJsonResponseAsync(url, cancellationToken);
return response.EnumerateArray().Select(Message.Parse).LastOrDefault();
// Can be null on channels that the user cannot access
var response = await TryGetJsonResponseAsync(url, cancellationToken);
if (response is null)
return null;
return response.Value.EnumerateArray().Select(Message.Parse).LastOrDefault();
}
public async IAsyncEnumerable<Message> GetMessagesAsync(
private async IAsyncEnumerable<Message> GetMessagesAsync(
Snowflake channelId,
Snowflake? after = null,
Snowflake? before = null,
IProgress<Percentage>? progress = null,
[EnumeratorCancellation] CancellationToken cancellationToken = default
Snowflake? after,
Snowflake? before,
IProgress<Percentage>? progress,
bool isReverse,
[EnumeratorCancellation] CancellationToken cancellationToken
)
{
// Get the last message in the specified range, so we can later calculate the
// progress based on the difference between message timestamps.
// This also snapshots the boundaries, which means that messages posted after
// the export started will not appear in the output.
var lastMessage = await TryGetLastMessageAsync(channelId, before, cancellationToken);
if (lastMessage is null || lastMessage.Timestamp < after?.ToDate())
// To keep the understanding of message history independent of the fetching direction,
// we'll refer to the two ends of the range as Alpha and Omega.
// Depending on the direction, these are either 'before' and 'after', or 'after' and 'before'.
//
// Chronological order:
// <after> Alpha [----->----->----] Omega <before>
// Reverse chronological order:
// <after> Omega [-----<-----<----] Alpha <before>
// Because Discord API doesn't allow us to provide both 'after' and 'before' parameters
// at the same time, we have to establish at least one end of the boundary manually.
// To do that, we'll fetch the Omega message, which will be the terminal message in the range:
// last message in chronological order, or first message in reverse chronological order.
// This snapshotting also has the side benefit of allowing us to calculate progress by comparing
// the timestamps of the Alpha message, Omega message, and the message being currently processed.
var omegaMessage = !isReverse
? await TryGetLastMessageAsync(channelId, before, cancellationToken)
: await TryGetFirstMessageAsync(channelId, after, cancellationToken);
// If the Omega doesn't exist or falls outside of the range, then there are simply no messages
// satisfying the specified range.
if (
omegaMessage is null
|| (!isReverse && omegaMessage.Timestamp < after?.ToDate())
|| (isReverse && omegaMessage.Timestamp > before?.ToDate())
)
{
yield break;
}
// Keep track of the first message in range in order to calculate the progress
var firstMessage = default(Message);
// Persist the Alpha message as soon as we fetch the initial batch of messages.
// This is only used for calculating progress.
var alphaMessage = default(Message);
var currentAfter = after ?? Snowflake.Zero;
var currentBoundary = !isReverse ? after ?? Snowflake.Zero : before;
while (true)
{
var url = new UrlBuilder()
.SetPath($"channels/{channelId}/messages")
.SetQueryParameter("limit", "100")
.SetQueryParameter("after", currentAfter.ToString())
.SetQueryParameter(!isReverse ? "after" : "before", currentBoundary?.ToString())
.Build();
var response = await GetJsonResponseAsync(url, cancellationToken);
@@ -662,8 +718,8 @@ public class DiscordClient(
var messages = response
.EnumerateArray()
.Select(Message.Parse)
// Messages are returned from newest to oldest, so we need to reverse them
.Reverse()
// Messages in batches are always returned from newest to oldest, so reverse if needed
.Pipe(messages => isReverse ? messages : messages.Reverse())
.ToArray();
// Break if there are no messages (can happen if messages are deleted during execution)
@@ -678,32 +734,78 @@ public class DiscordClient(
foreach (var message in messages)
{
firstMessage ??= message;
// Ensure that the messages are in range
if (message.Timestamp > lastMessage.Timestamp)
// Ensure that we're still in range by checking against the Omega
if (!isReverse ? message.Id > omegaMessage.Id : message.Id < omegaMessage.Id)
{
yield break;
}
alphaMessage ??= message;
// Report progress based on timestamps
if (progress is not null)
{
var exportedDuration = (message.Timestamp - firstMessage.Timestamp).Duration();
var totalDuration = (lastMessage.Timestamp - firstMessage.Timestamp).Duration();
var fetchedDuration = isReverse
? alphaMessage.Timestamp - message.Timestamp
: message.Timestamp - alphaMessage.Timestamp;
var totalDuration = isReverse
? alphaMessage.Timestamp - omegaMessage.Timestamp
: omegaMessage.Timestamp - alphaMessage.Timestamp;
progress.Report(
Percentage.FromFraction(
// Avoid division by zero if all messages have the exact same timestamp
// (which happens when there's only one message in the channel)
totalDuration > TimeSpan.Zero
? exportedDuration / totalDuration
? fetchedDuration / totalDuration
: 1
)
);
}
yield return message;
currentAfter = message.Id;
// Some messages, for example thread starter messages, are returned by the API as content-less references.
// Try to resolve them to the actual message so that they appear as they do in the Discord client.
var actualMessage =
message.Kind == MessageKind.ThreadStarterMessage
&& message.Reference?.ChannelId is { } referencedChannelId
&& message.Reference?.MessageId is { } referencedMessageId
? await TryGetMessageAsync(
referencedChannelId,
referencedMessageId,
cancellationToken
)
: null;
yield return actualMessage ?? message;
}
// The new boundary is always determined by the last message in the batch,
// because we order the messages based on fetching direction.
currentBoundary = messages.Last().Id;
}
}
public async IAsyncEnumerable<Message> GetMessagesAsync(
Snowflake channelId,
Snowflake? after = null,
Snowflake? before = null,
IProgress<Percentage>? progress = null,
[EnumeratorCancellation] CancellationToken cancellationToken = default
)
{
await foreach (
var message in GetMessagesAsync(
channelId,
after,
before,
progress,
false,
cancellationToken
)
)
{
yield return message;
}
}
@@ -715,64 +817,18 @@ public class DiscordClient(
[EnumeratorCancellation] CancellationToken cancellationToken = default
)
{
// Get the first message in the specified range, so we can later calculate the
// progress based on the difference between message timestamps.
// Snapshotting is not necessary here because new messages can't appear in the past.
var firstMessage = await TryGetFirstMessageAsync(channelId, after, cancellationToken);
if (firstMessage is null || firstMessage.Timestamp > before?.ToDate())
yield break;
// Keep track of the last message in range in order to calculate the progress
var lastMessage = default(Message);
var currentBefore = before;
while (true)
await foreach (
var message in GetMessagesAsync(
channelId,
after,
before,
progress,
true,
cancellationToken
)
)
{
var url = new UrlBuilder()
.SetPath($"channels/{channelId}/messages")
.SetQueryParameter("limit", "100")
.SetQueryParameter("before", currentBefore?.ToString())
.Build();
var response = await GetJsonResponseAsync(url, cancellationToken);
var messages = response.EnumerateArray().Select(Message.Parse).ToArray();
// Break if there are no messages (can happen if messages are deleted during execution)
if (!messages.Any())
yield break;
// If all messages are empty, make sure that it's not because the bot account doesn't
// have the MESSAGE_CONTENT intent enabled.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/1106#issuecomment-1741548959
if (messages.All(m => m.IsEmpty))
await EnsureMessageContentIntentAsync(cancellationToken);
foreach (var message in messages)
{
lastMessage ??= message;
// Report progress based on timestamps
if (progress is not null)
{
var exportedDuration = (lastMessage.Timestamp - message.Timestamp).Duration();
var totalDuration = (lastMessage.Timestamp - firstMessage.Timestamp).Duration();
progress.Report(
Percentage.FromFraction(
// Avoid division by zero if all messages have the exact same timestamp
// (which happens when there's only one message in the channel)
totalDuration > TimeSpan.Zero
? exportedDuration / totalDuration
: 1
)
);
}
yield return message;
}
currentBefore = messages.Last().Id;
yield return message;
}
}
@@ -1,4 +1,4 @@
using System;
using System;
using System.Collections.Generic;
using System.IO.Compression;
using System.Linq;
@@ -1,3 +1,3 @@
namespace DiscordChatExporter.Core.Discord.Dump;
namespace DiscordChatExporter.Core.Discord.Dump;
public record DataDumpChannel(Snowflake Id, string Name);
@@ -1,4 +1,4 @@
using System;
using System;
namespace DiscordChatExporter.Core.Discord;
@@ -1,4 +1,4 @@
using System;
using System;
using System.Diagnostics.CodeAnalysis;
using System.Globalization;
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Discord;
namespace DiscordChatExporter.Core.Discord;
public enum TokenKind
{

Some files were not shown because too many files have changed in this diff Show More