Compare commits

..

7 Commits

Author SHA1 Message Date
copilot-swe-agent[bot] b7f55ff633 Remove TokenPersonalStep1Before/After split, fold step 1 into markdown instructions
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
2026-02-24 10:03:48 +00:00
copilot-swe-agent[bot] 5bbd3fa65e Adopt MarkdownToInlinesConverter for instruction text formatting
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
2026-02-24 09:55:08 +00:00
copilot-swe-agent[bot] c6871af438 Localize token field watermark and instruction text
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
2026-02-24 09:35:40 +00:00
copilot-swe-agent[bot] 2e29d03648 Fix Ukrainian translations per review feedback
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
2026-02-24 08:32:00 +00:00
copilot-swe-agent[bot] 7257332914 Fix Ukrainian translation: use 'Гілки' for threads
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
2026-02-24 08:26:26 +00:00
copilot-swe-agent[bot] b1b0dc1625 Implement localization in DiscordChatExporter following YoutubeDownloader pattern
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
2026-02-24 07:40:35 +00:00
copilot-swe-agent[bot] 10a5d1b2bf Initial plan 2026-02-24 07:24:02 +00:00
237 changed files with 2653 additions and 3385 deletions
+3 -3
View File
@@ -16,8 +16,8 @@ $ docker pull tyrrrz/discordchatexporter:stable
Note the `:stable` tag. DiscordChatExporter images are tagged according to the following patterns:
- `stable` — latest stable version release. This tag is updated with each release of a new project version. Recommended for personal use.
- `x.y.z` (e.g., `2.30.1`) — specific stable version release. This tag is pushed when the corresponding version is released and never updated thereafter. Recommended for use in automation scenarios.
- `latest` — latest (potentially unstable) build. This tag is updated with each new commit to the `prime` branch. Not recommended, unless you want to test a new feature that has not been released in a stable version yet.
- `x.y.z` (e.g. `2.30.1`) — specific stable version release. This tag is pushed when the corresponding version is released and never updated thereafter. Recommended for use in automation scenarios.
- `latest` — latest (potentially unstable) build. This tag is updated with each new commit to the `master` branch. Not recommended, unless you want to test a new feature that has not been released in a stable version yet.
You can see all available tags [here](https://hub.docker.com/r/tyrrrz/discordchatexporter/tags?ordering=name).
@@ -57,7 +57,7 @@ You can also use the current working directory as the output directory by specif
- `-v $PWD:/out` in Bash
- `-v $pwd.Path:/out` in PowerShell
For more information, please refer to the [Dockerfile](https://github.com/Tyrrrz/DiscordChatExporter/blob/prime/DiscordChatExporter.Cli.dockerfile) and [Docker documentation](https://docs.docker.com/engine/reference/run).
For more information, please refer to the [Dockerfile](https://github.com/Tyrrrz/DiscordChatExporter/blob/master/DiscordChatExporter.Cli.dockerfile) and [Docker documentation](https://docs.docker.com/engine/reference/run).
To get your Token and Channel IDs, please refer to [this page](Token-and-IDs.md).
+3 -3
View File
@@ -11,7 +11,7 @@
```bash
#!/bin/bash
# Info: https://github.com/Tyrrrz/DiscordChatExporter/blob/prime/.docs
# Info: https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs
TOKEN=tokenhere
CHANNELID=channelhere
@@ -64,9 +64,9 @@ exit 0
- `tokenhere` with your [Token](Token-and-IDs.md).
- `channelhere` with a [Channel ID](Token-and-IDs.md).
- `dceFOLDERpathhere` with DCE's **directory path** (e.g., `/path/to/folder`, NOT `/path/to/folder/DiscordChatExporter.dll`).
- `dceFOLDERpathhere` with DCE's **directory path** (e.g. `/path/to/folder`, NOT `/path/to/folder/DiscordChatExporter.dll`).
- `filenamehere` with the exported channel's filename, without spaces.
- `dirhere` with the export directory (e.g., /home/user/Documents/Discord\ Exports).
- `dirhere` with the export directory (e.g. /home/user/Documents/Discord\ Exports).
- `formathere` with one of the available export formats.
> **Note**:
+3 -3
View File
@@ -12,7 +12,7 @@
```bash
#!/bin/bash
# Info: https://github.com/Tyrrrz/DiscordChatExporter/blob/prime/.docs
# Info: https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs
TOKEN=tokenhere
CHANNELID=channelhere
@@ -68,9 +68,9 @@ exit 0
- `tokenhere` with your [Token](Token-and-IDs.md)
- `channelhere` with a [Channel ID](Token-and-IDs.md)
- `dceFOLDERpathhere` with DCE's **directory's path** (e.g., `/Users/user/Desktop/DiscordChatExporterFolder`, NOT `/Users/user/Desktop/DiscordChatExporterFolder/DiscordChatExporter.DLL`)
- `dceFOLDERpathhere` with DCE's **directory's path** (e.g. `/Users/user/Desktop/DiscordChatExporterFolder`, NOT `/Users/user/Desktop/DiscordChatExporterFolder/DiscordChatExporter.DLL`)
- `filenamehere` with the exported channel's filename, without spaces
- `dirhere` with the directory you want the files to be saved at (e.g., `/Users/user/Documents/Discord\ Exports`)
- `dirhere` with the directory you want the files to be saved at (e.g. `/Users/user/Documents/Discord\ Exports`)
- `formathere` with one of the available export formats
To quickly get file or folder paths, select the file/folder, then hit Command+I (⌘I) and copy what's after `Where:`.
+3 -3
View File
@@ -5,7 +5,7 @@
1. Open a text editor such as Notepad and paste:
```console
# Info: https://github.com/Tyrrrz/DiscordChatExporter/blob/prime/.docs
# Info: https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs
$TOKEN = "tokenhere"
$CHANNEL = "channelhere"
@@ -33,9 +33,9 @@ exit
- `tokenhere` with your [Token](Token-and-IDs.md)
- `channelhere` with a [Channel ID](Token-and-IDs.md)
- `exefolderhere` with the .exe **directory's path** (e.g., C:\Users\User\Desktop\DiscordChatExporter)
- `exefolderhere` with the .exe **directory's path** (e.g. C:\Users\User\Desktop\DiscordChatExporter)
- `filenamehere` with a filename without spaces
- `dirhere` with the export directory (e.g., C:\Users\User\Documents\Exports)
- `dirhere` with the export directory (e.g. C:\Users\User\Documents\Exports)
- `formathere` with one of the available export formats
Make sure not to delete the quotes (")
+2 -2
View File
@@ -27,7 +27,7 @@ Prerequisite step: Navigate to [discord.com](https://discord.com) and login.
3. Type
```js
let m;webpackChunkdiscord_app.push([[Math.random()],{},e=>{for(let i in e.c){let x=e.c[i];if(x?.exports?.getToken){m=x;break}}}]);m&&console.log("Token:",m.exports.getToken());
let m;webpackChunkdiscord_app.push([[Math.random()],{},e=>{for(let i in e.c){let x=e.c[i];if(x?.exports?.$8&&x.exports.LP&&x.exports.gK){m=x;break}}}]);m&&console.log("Token:",m.exports.LP());
```
into the console and press <kbd>Enter</kbd>. The console will display your user token.
@@ -120,7 +120,7 @@ Prerequisite step: Navigate to [discord.com](https://discord.com) and login.
1. Type
```js
let m;webpackChunkdiscord_app.push([[Math.random()],{},e=>{for(let i in e.c){let x=e.c[i];if(x?.exports?.getToken){m=x;break}}}]);m&&console.log("Token:",m.exports.getToken());
let m;webpackChunkdiscord_app.push([[Math.random()],{},e=>{for(let i in e.c){let x=e.c[i];if(x?.exports?.$8&&x.exports.LP&&x.exports.gK){m=x;break}}}]);m&&console.log("Token:",m.exports.LP());
```
into the console and press <kbd>Enter</kbd>. The console will display your user token.
+16 -15
View File
@@ -29,20 +29,21 @@ Type the following command in your terminal of choice, then press ENTER to run i
> **Note**:
> On Windows, if you're using the default Command Prompt (`cmd`), omit the leading `./` at the start of the command.
> **Docker** users, please refer to the [Docker usage instructions](Docker.md).
## CLI commands
| Command | Description |
| ----------- | ---------------------------------------------------- |
| export | Exports a channel |
| exportdm | Exports all direct message channels |
| exportguild | Exports all channels within the specified server |
| exportall | Exports all accessible channels |
| channels | Outputs the list of channels in the given server |
| dm | Outputs the list of direct message channels |
| guilds | Outputs the list of accessible servers |
| guide | Explains how to obtain token, server, and channel ID |
| Command | Description |
|-------------------------|------------------------------------------------------|
| export | Exports a channel |
| exportdm | Exports all direct message channels |
| exportguild | Exports all channels within the specified server |
| exportall | Exports all accessible channels |
| channels | Outputs the list of channels in the given server |
| dm | Outputs the list of direct message channels |
| guilds | Outputs the list of accessible servers |
| guide | Explains how to obtain token, server, and channel ID |
To use the commands, you'll need a token. For the instructions on how to get a token, please refer to [this page](Token-and-IDs.md), or run `./DiscordChatExporter.Cli guide`.
@@ -77,7 +78,7 @@ format is `HtmlDark`.
#### Changing the output filename
You can change the filename by using `-o name.ext`. e.g., for the `HTML` format:
You can change the filename by using `-o name.ext`. e.g. for the `HTML` format:
```console
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555 -o myserver.html
@@ -186,21 +187,21 @@ locales. The default locale is `en-US`.
#### Date ranges
**Messages sent before a date**
Use `--before` to export messages sent before the provided date. e.g., messages sent before September 18th, 2019:
Use `--before` to export messages sent before the provided date. E.g. messages sent before September 18th, 2019:
```console
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555 --before 2019-09-18
```
**Messages sent after a date**
Use `--after` to export messages sent after the provided date. e.g., messages sent after September 17th, 2019 11:34 PM:
Use `--after` to export messages sent after the provided date. E.g. messages sent after September 17th, 2019 11:34 PM:
```console
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555 --after "2019-09-17 23:34"
```
**Messages sent in a date range**
Use `--before` and `--after` to export messages sent during the provided date range. e.g., messages sent between
Use `--before` and `--after` to export messages sent during the provided date range. E.g. messages sent between
September 17th, 2019 11:34 PM and September 18th:
```console
@@ -221,7 +222,7 @@ Use `--filter` to filter what messages are included in the export.
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555 --filter "from:Tyrrrz has:image"
```
Documentation on message filter syntax can be found [here](https://github.com/Tyrrrz/DiscordChatExporter/blob/prime/.docs/Message-filters.md).
Documentation on message filter syntax can be found [here](https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs/Message-filters.md).
### Export channels from a specific server
+8 -8
View File
@@ -12,7 +12,7 @@
After extracting the `.zip`, run `DiscordChatExporter.exe` **(Windows)**, or `DiscordChatExporter` **(Linux)**.
If you're using **macOS**, you'll need to manually grant permission for the app to run.
If you're using **macOS**, you'll need to manually grant permission for the app to run.
If you skip these steps, the "DiscordChatExporter is damaged and cant be opened" error will be shown.
1. Open Terminal.app. You can search for it in Spotlight (press <kbd>⌘</kbd> + <kbd>Space</kbd> and type "Terminal").
@@ -56,11 +56,10 @@ In this screen you can customize the following:
- **Export format** - HTML (Dark), HTML (Light), TXT, CSV and JSON
- **Date range (after/before)** (Optional) - If set, only messages sent in the provided date range will be exported. Only one value (either after or before) is required if you want to use this option.
> **Note**:
> Please note that the time defaults to **12:00 AM** (midnight/00:00). This means that if you choose to export between Sep 17th and Sep 18th, messages from Sep 18th won't be exported.
- **Partition limit** (Optional) - Split output into partitions, each limited to this number of messages (e.g., 100) or file size (e.g., 10mb). For example, a channel with 36 messages set to be partitioned every 10 messages will output 4 files.
- **Partition limit** (Optional) - Split output into partitions, each limited to this number of messages (e.g. 100) or file size (e.g. 10mb). For example, a channel with 36 messages set to be partitioned every 10 messages will output 4 files.
- **Message Filter** (Optional) - Special notation for filtering the messages that get included in the export. See [Message filters](Message-filters.md) for more info.
@@ -78,28 +77,29 @@ In this screen you can customize the following:
## Settings
- **Auto-update** - Perform automatic updates on every launch.
Default: Enabled
Default: Enabled
> **Note**:
> Keep this option enabled to receive the latest features and bug fixes!
- **Dark mode** - Use darker colors in the UI (User Interface).
Default: Disabled
Default: Disabled
- **Persist token** - Persist last used token between sessions.
Default: Enabled
Default: Enabled
- **Show threads** - Controls whether threads are shown in the channel list.
Default: none
Default: none
- **Locale** - Customize how dates are formatted in the exported files.
- **Date format** - Customize how dates are formatted in the exported files in the settings menu ().
- **Parallel limit** - The number of channels that will be exported at the same time.
Default: 1
Default: 1
> **Note**:
> Try to keep this number low so that your account doesn't get flagged.
- **Normalize to UTC** - Convert all dates to UTC before exporting.
+3 -3
View File
@@ -1,10 +1,10 @@
blank_issues_enabled: false
contact_links:
- name: ⚠ Feature request
url: https://github.com/Tyrrrz/.github/blob/prime/docs/project-status.md
about: Sorry, but this project is in maintenance mode and currently doesn't accept new feature requests.
url: https://github.com/Tyrrrz/.github/blob/master/docs/project-status.md
about: Sorry, but this project is in maintenance mode and no longer accepts new feature requests.
- name: 📖 Documentation
url: https://github.com/Tyrrrz/DiscordChatExporter/blob/prime/.docs
url: https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs
about: Find usage guides and frequently asked questions.
- name: 🗨 Discussions
url: https://github.com/Tyrrrz/DiscordChatExporter/discussions/new
-18
View File
@@ -1,18 +0,0 @@
changelog:
exclude:
authors:
- dependabot
- dependabot[bot]
categories:
- title: Enhancements
labels:
- enhancement
- title: Bugs
labels:
- bug
- title: Other
labels:
- "*"
+9 -9
View File
@@ -4,12 +4,12 @@ on:
workflow_dispatch:
push:
branches:
- prime
- master
tags:
- "*"
pull_request:
branches:
- prime
- master
jobs:
# Outputs from this job aren't really used, but it's here to verify that the Dockerfile builds correctly
@@ -23,10 +23,10 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install Docker Buildx
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
- name: Build image
run: >
@@ -37,15 +37,15 @@ jobs:
--output type=tar,dest=DiscordChatExporter.Cli.Docker.tar
- name: Upload image
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
with:
name: DiscordChatExporter.Cli.Docker
path: DiscordChatExporter.Cli.Docker.tar
if-no-files-found: error
deploy:
# Deploy to DockerHub only on tag push or prime branch push
if: ${{ github.ref_type == 'tag' || github.ref_type == 'branch' && github.ref_name == 'prime' }}
# Deploy to DockerHub only on tag push or master branch push
if: ${{ github.ref_type == 'tag' || github.ref_type == 'branch' && github.ref_name == 'master' }}
runs-on: ubuntu-latest
timeout-minutes: 10
@@ -55,10 +55,10 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install Docker Buildx
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
- name: Login to DockerHub
run: >
+12 -13
View File
@@ -4,12 +4,12 @@ on:
workflow_dispatch:
push:
branches:
- prime
- master
tags:
- "*"
pull_request:
branches:
- prime
- master
env:
DOTNET_SKIP_FIRST_TIME_EXPERIENCE: true
@@ -26,10 +26,10 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install .NET
uses: actions/setup-dotnet@26b0ec14cb23fa6904739307f278c14f94c95bf1 # v5.4.0
uses: actions/setup-dotnet@baa11fbfe1d6520db94683bd5c7a3818018e4309 # v5.1.0
# Build the project separately to discern between build and format errors
- name: Build
@@ -62,10 +62,10 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install .NET
uses: actions/setup-dotnet@26b0ec14cb23fa6904739307f278c14f94c95bf1 # v5.4.0
uses: actions/setup-dotnet@baa11fbfe1d6520db94683bd5c7a3818018e4309 # v5.1.0
- name: Run tests
env:
@@ -81,7 +81,7 @@ jobs:
DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=opencover
- name: Upload coverage
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
with:
token: ${{ secrets.CODECOV_TOKEN }}
@@ -117,17 +117,16 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install .NET
uses: actions/setup-dotnet@26b0ec14cb23fa6904739307f278c14f94c95bf1 # v5.4.0
uses: actions/setup-dotnet@baa11fbfe1d6520db94683bd5c7a3818018e4309 # v5.1.0
- name: Publish app
run: >
dotnet publish ${{ matrix.app }}
-p:Version=${{ github.ref_type == 'tag' && github.ref_name || format('999.9.9-ci-{0}', github.sha) }}
-p:CSharpier_Bypass=true
-p:EncryptionSalt=${{ secrets.ENCRYPTION_SALT || 'HimalayanPinkSalt' }}
-p:PublishMacOSBundle=${{ startsWith(matrix.rid, 'osx-') }}
--output ${{ matrix.app }}/bin/publish/
--configuration Release
@@ -135,7 +134,7 @@ jobs:
--self-contained
- name: Upload app binaries
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
with:
name: ${{ matrix.asset }}.${{ matrix.rid }}
path: ${{ matrix.app }}/bin/publish/
@@ -200,7 +199,7 @@ jobs:
steps:
- name: Download app binaries
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0
with:
name: ${{ matrix.asset }}.${{ matrix.rid }}
path: ${{ matrix.app }}/
@@ -236,7 +235,7 @@ jobs:
steps:
- name: Notify Discord
uses: tyrrrz/action-http-request@110f1a0f0f7e91c3b2de349539249f4573c243fa # 1.1.5
uses: tyrrrz/action-http-request@1dd7ad841a34b9299f3741f7c7399f9feefdfb08 # 1.1.3
with:
url: ${{ secrets.DISCORD_WEBHOOK }}
method: POST
+4 -2
View File
@@ -1,12 +1,14 @@
<Project>
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Version>999.9.9-dev</Version>
<Company>Tyrrrz</Company>
<Copyright>Copyright (c) Oleksii Holub</Copyright>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<ILLinkTreatWarningsAsErrors>false</ILLinkTreatWarningsAsErrors>
</PropertyGroup>
</Project>
</Project>
-46
View File
@@ -1,46 +0,0 @@
<Project>
<PropertyGroup>
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
</PropertyGroup>
<ItemGroup>
<PackageVersion Include="AngleSharp" Version="1.5.1" />
<PackageVersion Include="AsyncImageLoader.Avalonia" Version="3.8.0" />
<PackageVersion Include="AsyncKeyedLock" Version="8.0.2" />
<PackageVersion Include="Avalonia" Version="12.0.5" />
<PackageVersion Include="Avalonia.Desktop" Version="12.0.5" />
<PackageVersion Include="CliFx" Version="3.0.0" />
<PackageVersion Include="Cogwheel" Version="2.1.1" />
<PackageVersion Include="CommunityToolkit.Mvvm" Version="8.4.2" />
<PackageVersion Include="coverlet.collector" Version="10.0.1" />
<PackageVersion Include="CSharpier.MsBuild" Version="1.3.0" />
<PackageVersion Include="Deorcify" Version="2.0.1" />
<PackageVersion Include="DialogHost.Avalonia" Version="0.12.2" />
<PackageVersion Include="FluentAssertions" Version="8.10.0" />
<PackageVersion Include="GitHubActionsTestLogger" Version="3.0.4" />
<PackageVersion Include="Gress" Version="2.2.0" />
<PackageVersion Include="JsonExtensions" Version="1.2.3" />
<PackageVersion Include="Markdig" Version="1.3.2" />
<PackageVersion Include="Material.Avalonia" Version="3.16.1" />
<PackageVersion Include="Material.Icons.Avalonia" Version="3.0.2" />
<PackageVersion Include="Microsoft.Extensions.Configuration" Version="10.0.9" />
<PackageVersion
Include="Microsoft.Extensions.Configuration.EnvironmentVariables"
Version="10.0.9"
/>
<PackageVersion Include="Microsoft.Extensions.Configuration.UserSecrets" Version="10.0.9" />
<PackageVersion Include="Microsoft.Extensions.DependencyInjection" Version="10.0.9" />
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="18.7.0" />
<PackageVersion Include="Onova" Version="2.6.13" />
<PackageVersion Include="Polly" Version="8.7.0" />
<PackageVersion Include="PowerKit" Version="2.0.2" />
<PackageVersion Include="RazorBlade" Version="1.0.0" />
<PackageVersion Include="Spectre.Console" Version="0.57.1" />
<PackageVersion Include="Superpower" Version="3.1.0" />
<PackageVersion Include="ThisAssembly.Project" Version="2.1.2" />
<PackageVersion Include="WebMarkupMin.Core" Version="2.21.0" />
<PackageVersion Include="xunit" Version="2.9.3" />
<PackageVersion Include="xunit.runner.visualstudio" Version="3.1.5" />
<PackageVersion Include="YoutubeExplode" Version="6.6.0" />
</ItemGroup>
</Project>
@@ -10,22 +10,24 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="AngleSharp" />
<PackageReference Include="coverlet.collector" PrivateAssets="all" />
<PackageReference Include="CSharpier.MsBuild" PrivateAssets="all" />
<PackageReference Include="FluentAssertions" />
<PackageReference Include="GitHubActionsTestLogger" />
<PackageReference Include="JsonExtensions" />
<PackageReference Include="Microsoft.Extensions.Configuration" />
<PackageReference Include="Microsoft.Extensions.Configuration.EnvironmentVariables" />
<PackageReference Include="Microsoft.Extensions.Configuration.UserSecrets" />
<PackageReference Include="Microsoft.NET.Test.Sdk" />
<PackageReference Include="PowerKit" />
<PackageReference Include="xunit" />
<PackageReference Include="xunit.runner.visualstudio" PrivateAssets="all" />
<PackageReference Include="AngleSharp" Version="1.4.0" />
<PackageReference Include="coverlet.collector" Version="6.0.4" PrivateAssets="all" />
<PackageReference Include="CSharpier.MsBuild" Version="1.2.5" PrivateAssets="all" />
<PackageReference Include="FluentAssertions" Version="8.8.0" />
<PackageReference Include="GitHubActionsTestLogger" Version="3.0.1" PrivateAssets="all" />
<PackageReference Include="JsonExtensions" Version="1.2.0" />
<PackageReference Include="Microsoft.Extensions.Configuration" Version="10.0.2" />
<PackageReference
Include="Microsoft.Extensions.Configuration.EnvironmentVariables"
Version="10.0.2"
/>
<PackageReference Include="Microsoft.Extensions.Configuration.UserSecrets" Version="10.0.2" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.0.1" />
<PackageReference Include="xunit" Version="2.9.3" />
<PackageReference Include="xunit.runner.visualstudio" Version="3.1.5" PrivateAssets="all" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="../DiscordChatExporter.Cli/DiscordChatExporter.Cli.csproj" />
<ProjectReference Include="..\DiscordChatExporter.Cli\DiscordChatExporter.Cli.csproj" />
</ItemGroup>
</Project>
@@ -1,4 +1,4 @@
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Discord;
namespace DiscordChatExporter.Cli.Tests.Infra;
@@ -16,8 +16,6 @@ public static class ChannelIds
public static Snowflake FilterTestCases { get; } = Snowflake.Parse("866744075033641020");
public static Snowflake ForwardTestCases { get; } = Snowflake.Parse("1455202357204877477");
public static Snowflake MarkdownTestCases { get; } = Snowflake.Parse("866459526819348521");
public static Snowflake MentionTestCases { get; } = Snowflake.Parse("866458801389174794");
@@ -1,4 +1,4 @@
using System;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
@@ -1,4 +1,4 @@
using System;
using System;
using System.Reflection;
using Microsoft.Extensions.Configuration;
+1 -1
View File
@@ -1,4 +1,4 @@
# DiscordChatExporter Tests
# DiscordChatExporter Tests
This test suite runs against a real Discord server, specifically created to exercise different behaviors required by the test scenarios.
In order to run these tests locally, you need to join the test server and configure your authentication token.
@@ -1,4 +1,4 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using FluentAssertions;
using Xunit;
@@ -1,15 +1,15 @@
using System;
using System;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Exporting;
using FluentAssertions;
using JsonExtensions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -60,52 +60,6 @@ public class DateRangeSpecs
);
}
[Fact]
public async Task I_can_filter_the_reverse_export_to_only_include_messages_sent_after_the_specified_date()
{
// Arrange
var after = new DateTimeOffset(2021, 07, 24, 0, 0, 0, TimeSpan.Zero);
using var file = TempFile.Create();
// Act
await new ExportChannelsCommand
{
Token = Secrets.DiscordToken,
ChannelIds = [ChannelIds.DateRangeTestCases],
ExportFormat = ExportFormat.Json,
OutputPath = file.Path,
After = Snowflake.FromDate(after),
IsReverseMessageOrder = true,
}.ExecuteAsync(new FakeConsole());
// Assert
var timestamps = Json.Parse(await File.ReadAllTextAsync(file.Path))
.GetProperty("messages")
.EnumerateArray()
.Select(j => j.GetProperty("timestamp").GetDateTimeOffset())
.ToArray();
timestamps.All(t => t > after).Should().BeTrue();
timestamps.Should().BeInDescendingOrder();
timestamps
.Should()
.BeEquivalentTo(
[
new DateTimeOffset(2021, 09, 08, 14, 26, 35, TimeSpan.Zero),
new DateTimeOffset(2021, 07, 24, 14, 52, 40, TimeSpan.Zero),
new DateTimeOffset(2021, 07, 24, 14, 52, 39, TimeSpan.Zero),
new DateTimeOffset(2021, 07, 24, 14, 52, 38, TimeSpan.Zero),
new DateTimeOffset(2021, 07, 24, 13, 49, 13, TimeSpan.Zero),
],
o =>
o.Using<DateTimeOffset>(ctx =>
ctx.Subject.Should().BeCloseTo(ctx.Expectation, TimeSpan.FromSeconds(1))
)
.WhenTypeIs<DateTimeOffset>()
);
}
[Fact]
public async Task I_can_filter_the_export_to_only_include_messages_sent_before_the_specified_date()
{
@@ -1,15 +1,15 @@
using System;
using System;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Exporting;
using DiscordChatExporter.Core.Exporting.Filtering;
using FluentAssertions;
using JsonExtensions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -1,4 +1,4 @@
using System;
using System;
using System.Linq;
using System.Threading.Tasks;
using AngleSharp.Dom;
@@ -1,14 +1,8 @@
using System.IO;
using System.Linq;
using System.Linq;
using System.Threading.Tasks;
using AngleSharp.Dom;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Exporting;
using FluentAssertions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -50,41 +44,4 @@ public class HtmlContentSpecs
"Yeet"
);
}
[Fact]
public async Task I_can_export_a_channel_in_the_HTML_format_in_the_reverse_order()
{
// Arrange
using var file = TempFile.Create();
// Act
await new ExportChannelsCommand
{
Token = Secrets.DiscordToken,
ChannelIds = [ChannelIds.DateRangeTestCases],
ExportFormat = ExportFormat.HtmlDark,
OutputPath = file.Path,
Locale = "en-US",
IsUtcNormalizationEnabled = true,
IsReverseMessageOrder = true,
}.ExecuteAsync(new FakeConsole());
var document = Html.Parse(await File.ReadAllTextAsync(file.Path));
var messages = document.QuerySelectorAll("[data-message-id]").ToArray();
// Assert
messages
.Select(e => e.GetAttribute("data-message-id"))
.Should()
.Equal(
"885169254029213696",
"868505973294268457",
"868505969821364245",
"868505966528835604",
"868490009366396958",
"866732113319428096",
"866710679758045195",
"866674314627121232"
);
}
}
@@ -1,11 +1,11 @@
using System;
using System;
using System.Linq;
using System.Threading.Tasks;
using AngleSharp.Dom;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Utils.Extensions;
using FluentAssertions;
using PowerKit.Extensions;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -1,29 +0,0 @@
using System.Threading.Tasks;
using AngleSharp.Dom;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
using FluentAssertions;
using PowerKit.Extensions;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
public class HtmlForwardSpecs
{
[Fact]
public async Task I_can_export_a_channel_that_contains_a_forwarded_message()
{
// Act
var message = await ExportWrapper.GetMessageAsHtmlAsync(
ChannelIds.ForwardTestCases,
Snowflake.Parse("1455202427115536514")
);
// Assert
message
.Text()
.ReplaceWhiteSpace(' ')
.Should()
.ContainAll("Forwarded", @"¯\_(ツ)_/¯", "12/29/2025 2:14 PM");
}
}
@@ -1,4 +1,4 @@
using System.IO;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using AngleSharp.Dom;
@@ -8,7 +8,6 @@ using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Exporting;
using FluentAssertions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -1,9 +1,9 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using AngleSharp.Dom;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils.Extensions;
using DiscordChatExporter.Core.Discord;
using FluentAssertions;
using PowerKit.Extensions;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -22,14 +22,11 @@ public class HtmlMarkdownSpecs
// Assert
message
.Text()
.ReplaceWhiteSpace(' ')
.ReplaceWhiteSpace()
.Should()
.Contain("Default timestamp: 2/12/2023 1:36 PM");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -42,11 +39,8 @@ public class HtmlMarkdownSpecs
);
// Assert
message.Text().ReplaceWhiteSpace(' ').Should().Contain("Short time timestamp: 1:36 PM");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
message.Text().ReplaceWhiteSpace().Should().Contain("Short time timestamp: 1:36 PM");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -59,11 +53,8 @@ public class HtmlMarkdownSpecs
);
// Assert
message.Text().ReplaceWhiteSpace(' ').Should().Contain("Long time timestamp: 1:36:12 PM");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
message.Text().ReplaceWhiteSpace().Should().Contain("Long time timestamp: 1:36:12 PM");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -76,11 +67,8 @@ public class HtmlMarkdownSpecs
);
// Assert
message.Text().ReplaceWhiteSpace(' ').Should().Contain("Short date timestamp: 2/12/2023");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
message.Text().ReplaceWhiteSpace().Should().Contain("Short date timestamp: 2/12/2023");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -95,14 +83,11 @@ public class HtmlMarkdownSpecs
// Assert
message
.Text()
.ReplaceWhiteSpace(' ')
.ReplaceWhiteSpace()
.Should()
.Contain("Long date timestamp: Sunday, February 12, 2023");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -117,14 +102,11 @@ public class HtmlMarkdownSpecs
// Assert
message
.Text()
.ReplaceWhiteSpace(' ')
.ReplaceWhiteSpace()
.Should()
.Contain("Full timestamp: Sunday, February 12, 2023 1:36 PM");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -139,14 +121,11 @@ public class HtmlMarkdownSpecs
// Assert
message
.Text()
.ReplaceWhiteSpace(' ')
.ReplaceWhiteSpace()
.Should()
.Contain("Full long timestamp: Sunday, February 12, 2023 1:36:12 PM");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -161,14 +140,11 @@ public class HtmlMarkdownSpecs
// Assert
message
.Text()
.ReplaceWhiteSpace(' ')
.ReplaceWhiteSpace()
.Should()
.Contain("Relative timestamp: 2/12/2023 1:36 PM");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -1,4 +1,4 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using AngleSharp.Dom;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -1,4 +1,4 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using AngleSharp.Dom;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -1,4 +1,4 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
using FluentAssertions;
@@ -1,4 +1,4 @@
using System.Linq;
using System.Linq;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -1,13 +1,7 @@
using System.IO;
using System.Linq;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Exporting;
using FluentAssertions;
using JsonExtensions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -49,43 +43,4 @@ public class JsonContentSpecs
"Yeet"
);
}
[Fact]
public async Task I_can_export_a_channel_in_the_JSON_format_in_the_reverse_order()
{
// Arrange
using var file = TempFile.Create();
// Act
await new ExportChannelsCommand
{
Token = Secrets.DiscordToken,
ChannelIds = [ChannelIds.DateRangeTestCases],
ExportFormat = ExportFormat.Json,
OutputPath = file.Path,
Locale = "en-US",
IsUtcNormalizationEnabled = true,
IsReverseMessageOrder = true,
}.ExecuteAsync(new FakeConsole());
var messages = Json.Parse(await File.ReadAllTextAsync(file.Path))
.GetProperty("messages")
.EnumerateArray()
.ToArray();
// Assert
messages
.Select(j => j.GetProperty("id").GetString())
.Should()
.Equal(
"885169254029213696",
"868505973294268457",
"868505969821364245",
"868505966528835604",
"868490009366396958",
"866732113319428096",
"866710679758045195",
"866674314627121232"
);
}
}
@@ -1,4 +1,4 @@
using System.Linq;
using System.Linq;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -1,4 +1,4 @@
using System.Linq;
using System.Linq;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -1,33 +0,0 @@
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
using FluentAssertions;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
public class JsonForwardSpecs
{
[Fact]
public async Task I_can_export_a_channel_that_contains_a_forwarded_message()
{
// Act
var message = await ExportWrapper.GetMessageAsJsonAsync(
ChannelIds.ForwardTestCases,
Snowflake.Parse("1455202427115536514")
);
// Assert
var reference = message.GetProperty("reference");
reference.GetProperty("type").GetString().Should().Be("Forward");
reference.GetProperty("guildId").GetString().Should().Be("869237470565392384");
var forwardedMessage = message.GetProperty("forwardedMessage");
forwardedMessage.GetProperty("content").GetString().Should().Contain(@"¯\_(ツ)_/¯");
forwardedMessage
.GetProperty("timestamp")
.GetString()
.Should()
.StartWith("2025-12-28T22:52:42.175+00:00");
}
}
@@ -1,4 +1,4 @@
using System.Linq;
using System.Linq;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -1,4 +1,4 @@
using System.Linq;
using System.Linq;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -1,12 +1,12 @@
using System.IO;
using System.IO;
using System.Threading.Tasks;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Exporting;
using DiscordChatExporter.Core.Exporting.Partitioning;
using FluentAssertions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -17,7 +17,7 @@ public class PartitioningSpecs
public async Task I_can_export_a_channel_with_partitioning_based_on_message_count()
{
// Arrange
using var dir = TempDirectory.Create();
using var dir = TempDir.Create();
var filePath = Path.Combine(dir.Path, "output.html");
// Act
@@ -38,7 +38,7 @@ public class PartitioningSpecs
public async Task I_can_export_a_channel_with_partitioning_based_on_file_size()
{
// Arrange
using var dir = TempDirectory.Create();
using var dir = TempDir.Create();
var filePath = Path.Combine(dir.Path, "output.html");
// Act
@@ -1,4 +1,4 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using FluentAssertions;
using Xunit;
@@ -1,23 +0,0 @@
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using FluentAssertions;
using PowerKit.Extensions;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
public class PlainTextForwardSpecs
{
[Fact]
public async Task I_can_export_a_channel_that_contains_a_forwarded_message()
{
// Act
var document = await ExportWrapper.ExportAsPlainTextAsync(ChannelIds.ForwardTestCases);
// Assert
document
.ReplaceWhiteSpace(' ')
.Should()
.ContainAll("{Forwarded Message}", @"¯\_(ツ)_/¯", "12/28/2025 10:52 PM");
}
}
@@ -1,4 +1,4 @@
using System.IO;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Infrastructure;
@@ -7,7 +7,6 @@ using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Exporting;
using FluentAssertions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -18,7 +17,7 @@ public class SelfContainedSpecs
public async Task I_can_export_a_channel_and_download_all_referenced_assets()
{
// Arrange
using var dir = TempDirectory.Create();
using var dir = TempDir.Create();
var filePath = Path.Combine(dir.Path, "output.html");
// Act
@@ -0,0 +1,19 @@
using System.Text;
namespace DiscordChatExporter.Cli.Tests.Utils.Extensions;
internal static class StringExtensions
{
extension(string str)
{
public string ReplaceWhiteSpace(string replacement = " ")
{
var buffer = new StringBuilder(str.Length);
foreach (var ch in str)
buffer.Append(char.IsWhiteSpace(ch) ? replacement : ch);
return buffer.ToString();
}
}
}
+1 -1
View File
@@ -1,4 +1,4 @@
using AngleSharp.Html.Dom;
using AngleSharp.Html.Dom;
using AngleSharp.Html.Parser;
namespace DiscordChatExporter.Cli.Tests.Utils;
@@ -0,0 +1,36 @@
using System;
using System.IO;
using System.Reflection;
namespace DiscordChatExporter.Cli.Tests.Utils;
internal partial class TempDir(string path) : IDisposable
{
public string Path { get; } = path;
public void Dispose()
{
try
{
Directory.Delete(Path, true);
}
catch (DirectoryNotFoundException) { }
}
}
internal partial class TempDir
{
public static TempDir Create()
{
var dirPath = System.IO.Path.Combine(
System.IO.Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location)
?? Directory.GetCurrentDirectory(),
"Temp",
Guid.NewGuid().ToString()
);
Directory.CreateDirectory(dirPath);
return new TempDir(dirPath);
}
}
@@ -0,0 +1,37 @@
using System;
using System.IO;
using System.Reflection;
namespace DiscordChatExporter.Cli.Tests.Utils;
internal partial class TempFile(string path) : IDisposable
{
public string Path { get; } = path;
public void Dispose()
{
try
{
File.Delete(Path);
}
catch (FileNotFoundException) { }
}
}
internal partial class TempFile
{
public static TempFile Create()
{
var dirPath = System.IO.Path.Combine(
System.IO.Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location)
?? Directory.GetCurrentDirectory(),
"Temp"
);
Directory.CreateDirectory(dirPath);
var filePath = System.IO.Path.Combine(dirPath, Guid.NewGuid() + ".tmp");
return new TempFile(filePath);
}
}
@@ -1,4 +1,4 @@
{
{
"$schema": "https://xunit.net/schema/current/xunit.runner.schema.json",
"methodDisplayOptions": "all",
"methodDisplay": "method"
-1
View File
@@ -15,7 +15,6 @@ WORKDIR /tmp/app
COPY favicon.ico .
COPY NuGet.config .
COPY Directory.Build.props .
COPY Directory.Packages.props .
COPY DiscordChatExporter.Core DiscordChatExporter.Core
COPY DiscordChatExporter.Cli DiscordChatExporter.Cli
@@ -1,8 +1,8 @@
using System;
using System;
using System.Diagnostics.CodeAnalysis;
using System.Threading.Tasks;
using CliFx;
using CliFx.Binding;
using CliFx.Attributes;
using CliFx.Infrastructure;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Utils;
@@ -17,22 +17,23 @@ public abstract class DiscordCommandBase : ICommand
EnvironmentVariable = "DISCORD_TOKEN",
Description = "Authentication token."
)]
public required string Token { get; set; }
public required string Token { get; init; }
[Obsolete("This option doesn't do anything. Kept for backwards compatibility.")]
[CommandOption(
"bot",
'b',
EnvironmentVariable = "DISCORD_TOKEN_BOT",
Description = "This option doesn't do anything. Kept for backwards compatibility."
)]
public bool IsBotToken { get; set; } = false;
public bool IsBotToken { get; init; } = false;
[CommandOption(
"respect-rate-limits",
Description = "Whether to respect advisory rate limits. "
+ "If disabled, only hard rate limits (i.e., 429 responses) will be respected."
+ "If disabled, only hard rate limits (i.e. 429 responses) will be respected."
)]
public bool ShouldRespectRateLimits { get; set; } = true;
public bool ShouldRespectRateLimits { get; init; } = true;
[field: AllowNull, MaybeNull]
protected DiscordClient Discord =>
@@ -1,12 +1,12 @@
using System;
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using CliFx;
using CliFx.Binding;
using CliFx.Attributes;
using CliFx.Exceptions;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Converters;
using DiscordChatExporter.Cli.Commands.Shared;
@@ -37,75 +37,69 @@ public abstract class ExportCommandBase : DiscordCommandBase
get;
// Handle ~/ in paths on Unix systems
// https://github.com/Tyrrrz/DiscordChatExporter/pull/903
set => field = Path.GetFullPath(value);
init => field = Path.GetFullPath(value);
} = Directory.GetCurrentDirectory();
[CommandOption("format", 'f', Description = "Export format.")]
public ExportFormat ExportFormat { get; set; } = ExportFormat.HtmlDark;
public ExportFormat ExportFormat { get; init; } = ExportFormat.HtmlDark;
[CommandOption(
"after",
Description = "Only include messages sent after this date or message ID."
)]
public Snowflake? After { get; set; }
public Snowflake? After { get; init; }
[CommandOption(
"before",
Description = "Only include messages sent before this date or message ID."
)]
public Snowflake? Before { get; set; }
public Snowflake? Before { get; init; }
[CommandOption(
"partition",
'p',
Description = "Split the output into partitions, each limited to the specified "
+ "number of messages (e.g., '100') or file size (e.g., '10mb')."
+ "number of messages (e.g. '100') or file size (e.g. '10mb')."
)]
public PartitionLimit PartitionLimit { get; set; } = PartitionLimit.Null;
public PartitionLimit PartitionLimit { get; init; } = PartitionLimit.Null;
[CommandOption(
"include-threads",
Description = "Which types of threads should be included.",
Converter = typeof(ThreadInclusionModeInputConverter)
Converter = typeof(ThreadInclusionModeBindingConverter)
)]
public ThreadInclusionMode ThreadInclusionMode { get; set; } = ThreadInclusionMode.None;
public ThreadInclusionMode ThreadInclusionMode { get; init; } = ThreadInclusionMode.None;
[CommandOption(
"filter",
Description = "Only include messages that satisfy this filter. "
+ "See the documentation for more info."
)]
public MessageFilter MessageFilter { get; set; } = MessageFilter.Null;
public MessageFilter MessageFilter { get; init; } = MessageFilter.Null;
[CommandOption(
"parallel",
Description = "Limits how many channels can be exported in parallel."
)]
public int ParallelLimit { get; set; } = 1;
[CommandOption(
"reverse",
Description = "Export messages in reverse chronological order (newest first)."
)]
public bool IsReverseMessageOrder { get; set; }
public int ParallelLimit { get; init; } = 1;
[CommandOption(
"markdown",
Description = "Process markdown, mentions, and other special tokens."
)]
public bool ShouldFormatMarkdown { get; set; } = true;
public bool ShouldFormatMarkdown { get; init; } = true;
[CommandOption(
"media",
Description = "Download assets referenced by the export (user avatars, attached files, embedded images, etc.)."
)]
public bool ShouldDownloadAssets { get; set; }
public bool ShouldDownloadAssets { get; init; }
[CommandOption(
"reuse-media",
Description = "Reuse previously downloaded assets to avoid redundant requests."
)]
public bool ShouldReuseAssets { get; set; } = false;
public bool ShouldReuseAssets { get; init; } = false;
[CommandOption(
"media-dir",
@@ -117,33 +111,34 @@ public abstract class ExportCommandBase : DiscordCommandBase
get;
// Handle ~/ in paths on Unix systems
// https://github.com/Tyrrrz/DiscordChatExporter/pull/903
set => field = value is not null ? Path.GetFullPath(value) : null;
init => field = value is not null ? Path.GetFullPath(value) : null;
}
[Obsolete("This option doesn't do anything. Kept for backwards compatibility.")]
[CommandOption(
"dateformat",
Description = "This option doesn't do anything. Kept for backwards compatibility."
)]
public string DateFormat { get; set; } = "MM/dd/yyyy h:mm tt";
public string DateFormat { get; init; } = "MM/dd/yyyy h:mm tt";
[CommandOption(
"locale",
Description = "Locale to use when formatting dates and numbers. "
+ "If not specified, the default system locale will be used."
)]
public string? Locale { get; set; }
public string? Locale { get; init; }
[CommandOption("utc", Description = "Normalize all timestamps to UTC+0.")]
public bool IsUtcNormalizationEnabled { get; set; } = false;
public bool IsUtcNormalizationEnabled { get; init; } = false;
[CommandOption(
"fuck-russia",
EnvironmentVariable = "FUCK_RUSSIA",
Description = "Don't print the Support Ukraine message to the console.",
// Use a converter to accept '1' as 'true' to reuse the existing environment variable
Converter = typeof(TruthyBooleanInputConverter)
Converter = typeof(TruthyBooleanBindingConverter)
)]
public bool IsUkraineSupportMessageDisabled { get; set; } = false;
public bool IsUkraineSupportMessageDisabled { get; init; } = false;
[field: AllowNull, MaybeNull]
protected ChannelExporter Exporter => field ??= new ChannelExporter(Discord);
@@ -165,37 +160,6 @@ public abstract class ExportCommandBase : DiscordCommandBase
throw new CommandException("Option --media-dir cannot be used without --media.");
}
// Make sure the user does not try to export multiple channels into one file.
// Output path must either be a directory or contain template tokens for this to work.
// Validate this up-front, before fetching threads, because thread fetching can take a
// long time and it's frustrating to fail only after it completes.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/799
// https://github.com/Tyrrrz/DiscordChatExporter/issues/917
// https://github.com/Tyrrrz/DiscordChatExporter/issues/1549
var mayExportMultipleChannels =
// Multiple channels were provided explicitly
channels.Count > 1
// Thread inclusion can add more channels to the export
|| ThreadInclusionMode != ThreadInclusionMode.None;
var isValidOutputPath =
// Anything is valid when exporting a single channel
!mayExportMultipleChannels
// When using template tokens, assume the user knows what they're doing
|| OutputPath.Contains('%')
// Otherwise, require an existing directory or an unambiguous directory path
|| Directory.Exists(OutputPath)
|| Path.EndsInDirectorySeparator(OutputPath);
if (!isValidOutputPath)
{
throw new CommandException(
"Attempted to export multiple channels, but the output path is neither a directory nor a template. "
+ "If the provided output path is meant to be treated as a directory, make sure it ends with a slash. "
+ $"Provided output path: '{OutputPath}'."
);
}
var unwrappedChannels = new List<Channel>(channels);
// Unwrap threads
@@ -236,6 +200,28 @@ public abstract class ExportCommandBase : DiscordCommandBase
await console.Output.WriteLineAsync($"Fetched {fetchedThreadsCount} thread(s).");
}
// Make sure the user does not try to export multiple channels into one file.
// Output path must either be a directory or contain template tokens for this to work.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/799
// https://github.com/Tyrrrz/DiscordChatExporter/issues/917
var isValidOutputPath =
// Anything is valid when exporting a single channel
unwrappedChannels.Count <= 1
// When using template tokens, assume the user knows what they're doing
|| OutputPath.Contains('%')
// Otherwise, require an existing directory or an unambiguous directory path
|| Directory.Exists(OutputPath)
|| Path.EndsInDirectorySeparator(OutputPath);
if (!isValidOutputPath)
{
throw new CommandException(
"Attempted to export multiple channels, but the output path is neither a directory nor a template. "
+ "If the provided output path is meant to be treated as a directory, make sure it ends with a slash. "
+ $"Provided output path: '{OutputPath}'."
);
}
// Export
var errorsByChannel = new ConcurrentDictionary<Channel, string>();
var warningsByChannel = new ConcurrentDictionary<Channel, string>();
@@ -281,7 +267,6 @@ public abstract class ExportCommandBase : DiscordCommandBase
Before,
PartitionLimit,
MessageFilter,
IsReverseMessageOrder,
ShouldFormatMarkdown,
ShouldDownloadAssets,
ShouldReuseAssets,
@@ -1,10 +1,10 @@
using System;
using CliFx.Activation;
using System;
using CliFx.Extensibility;
using DiscordChatExporter.Cli.Commands.Shared;
namespace DiscordChatExporter.Cli.Commands.Converters;
internal class ThreadInclusionModeInputConverter : ScalarInputConverter<ThreadInclusionMode>
internal class ThreadInclusionModeBindingConverter : BindingConverter<ThreadInclusionMode>
{
public override ThreadInclusionMode Convert(string? rawValue)
{
@@ -1,9 +1,9 @@
using System.Globalization;
using CliFx.Activation;
using System.Globalization;
using CliFx.Extensibility;
namespace DiscordChatExporter.Cli.Commands.Converters;
internal class TruthyBooleanInputConverter : ScalarInputConverter<bool>
internal class TruthyBooleanBindingConverter : BindingConverter<bool>
{
public override bool Convert(string? rawValue)
{
@@ -2,7 +2,7 @@ using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Binding;
using CliFx.Attributes;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Cli.Utils.Extensions;
@@ -14,23 +14,23 @@ using Spectre.Console;
namespace DiscordChatExporter.Cli.Commands;
[Command("exportall", Description = "Exports all accessible channels.")]
public partial class ExportAllCommand : ExportCommandBase
public class ExportAllCommand : ExportCommandBase
{
[CommandOption("include-dm", Description = "Include direct message channels.")]
public bool IncludeDirectChannels { get; set; } = true;
public bool IncludeDirectChannels { get; init; } = true;
[CommandOption("include-guilds", Description = "Include server channels.")]
public bool IncludeGuildChannels { get; set; } = true;
public bool IncludeGuildChannels { get; init; } = true;
[CommandOption("include-vc", Description = "Include voice channels.")]
public bool IncludeVoiceChannels { get; set; } = true;
public bool IncludeVoiceChannels { get; init; } = true;
[CommandOption(
"data-package",
Description = "Path to the personal data package (ZIP file) requested from Discord. "
+ "If provided, only channels referenced in the dump will be exported."
)]
public string? DataPackageFilePath { get; set; }
public string? DataPackageFilePath { get; init; }
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -1,16 +1,16 @@
using System.Collections.Generic;
using System.Collections.Generic;
using System.Threading.Tasks;
using CliFx.Binding;
using CliFx.Attributes;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Discord.Data;
using PowerKit.Extensions;
using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("export", Description = "Exports one or multiple channels.")]
public partial class ExportChannelsCommand : ExportCommandBase
public class ExportChannelsCommand : ExportCommandBase
{
// TODO: change this to plural (breaking change)
[CommandOption(
@@ -19,7 +19,7 @@ public partial class ExportChannelsCommand : ExportCommandBase
Description = "Channel ID(s). "
+ "If provided with category ID(s), all channels inside those categories will be exported."
)]
public required IReadOnlyList<Snowflake> ChannelIds { get; set; }
public required IReadOnlyList<Snowflake> ChannelIds { get; init; }
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -1,14 +1,14 @@
using System.Threading.Tasks;
using CliFx.Binding;
using System.Threading.Tasks;
using CliFx.Attributes;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Core.Discord.Data;
using PowerKit.Extensions;
using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("exportdm", Description = "Exports all direct message channels.")]
public partial class ExportDirectMessagesCommand : ExportCommandBase
public class ExportDirectMessagesCommand : ExportCommandBase
{
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -1,6 +1,6 @@
using System.Collections.Generic;
using System.Collections.Generic;
using System.Threading.Tasks;
using CliFx.Binding;
using CliFx.Attributes;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Cli.Utils.Extensions;
@@ -11,13 +11,13 @@ using Spectre.Console;
namespace DiscordChatExporter.Cli.Commands;
[Command("exportguild", Description = "Exports all channels within the specified server.")]
public partial class ExportGuildCommand : ExportCommandBase
public class ExportGuildCommand : ExportCommandBase
{
[CommandOption("guild", 'g', Description = "Server ID.")]
public required Snowflake GuildId { get; set; }
public required Snowflake GuildId { get; init; }
[CommandOption("include-vc", Description = "Include voice channels.")]
public bool IncludeVoiceChannels { get; set; } = true;
public bool IncludeVoiceChannels { get; init; } = true;
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -1,31 +1,31 @@
using System;
using System;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Binding;
using CliFx.Attributes;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Cli.Commands.Converters;
using DiscordChatExporter.Cli.Commands.Shared;
using DiscordChatExporter.Core.Discord;
using PowerKit.Extensions;
using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("channels", Description = "Get the list of channels in a server.")]
public partial class GetChannelsCommand : DiscordCommandBase
public class GetChannelsCommand : DiscordCommandBase
{
[CommandOption("guild", 'g', Description = "Server ID.")]
public required Snowflake GuildId { get; set; }
public required Snowflake GuildId { get; init; }
[CommandOption("include-vc", Description = "Include voice channels.")]
public bool IncludeVoiceChannels { get; set; } = true;
public bool IncludeVoiceChannels { get; init; } = true;
[CommandOption(
"include-threads",
Description = "Which types of threads should be included.",
Converter = typeof(ThreadInclusionModeInputConverter)
Converter = typeof(ThreadInclusionModeBindingConverter)
)]
public ThreadInclusionMode ThreadInclusionMode { get; set; } = ThreadInclusionMode.None;
public ThreadInclusionMode ThreadInclusionMode { get; init; } = ThreadInclusionMode.None;
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -1,16 +1,16 @@
using System;
using System;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Binding;
using CliFx.Attributes;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Core.Discord.Data;
using PowerKit.Extensions;
using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("dm", Description = "Gets the list of all direct message channels.")]
public partial class GetDirectChannelsCommand : DiscordCommandBase
public class GetDirectChannelsCommand : DiscordCommandBase
{
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -1,16 +1,16 @@
using System;
using System;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Binding;
using CliFx.Attributes;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Core.Discord.Data;
using PowerKit.Extensions;
using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("guilds", Description = "Gets the list of accessible servers.")]
public partial class GetGuildsCommand : DiscordCommandBase
public class GetGuildsCommand : DiscordCommandBase
{
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -1,13 +1,13 @@
using System;
using System.Threading.Tasks;
using CliFx;
using CliFx.Binding;
using CliFx.Attributes;
using CliFx.Infrastructure;
namespace DiscordChatExporter.Cli.Commands;
[Command("guide", Description = "Explains how to obtain the token, server or channel ID.")]
public partial class GuideCommand : ICommand
public class GuideCommand : ICommand
{
public ValueTask ExecuteAsync(IConsole console)
{
@@ -74,7 +74,7 @@ public partial class GuideCommand : ICommand
using (console.WithForegroundColor(ConsoleColor.DarkCyan))
{
console.Output.WriteLine(
"https://github.com/Tyrrrz/DiscordChatExporter/blob/prime/.docs"
"https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs"
);
}
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Cli.Commands.Shared;
namespace DiscordChatExporter.Cli.Commands.Shared;
public enum ThreadInclusionMode
{
@@ -1,21 +1,26 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<ApplicationIcon>../favicon.ico</ApplicationIcon>
<ApplicationIcon>..\favicon.ico</ApplicationIcon>
<PublishTrimmed>true</PublishTrimmed>
<CopyOutputSymbolsToPublishDirectory>false</CopyOutputSymbolsToPublishDirectory>
</PropertyGroup>
<!-- HACK: Disable trim warnings because they seem to break when the code contains C# 14 extension blocks -->
<PropertyGroup>
<EnableTrimAnalyzer>false</EnableTrimAnalyzer>
<EnableAotAnalyzer>false</EnableAotAnalyzer>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="CliFx" />
<PackageReference Include="CSharpier.MsBuild" PrivateAssets="all" />
<PackageReference Include="Deorcify" />
<PackageReference Include="Gress" />
<PackageReference Include="PowerKit" />
<PackageReference Include="Spectre.Console" />
<PackageReference Include="CliFx" Version="2.3.6" />
<PackageReference Include="CSharpier.MsBuild" Version="1.2.5" PrivateAssets="all" />
<PackageReference Include="Deorcify" Version="1.1.0" PrivateAssets="all" />
<PackageReference Include="Gress" Version="2.1.1" />
<PackageReference Include="Spectre.Console" Version="0.54.0" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="../DiscordChatExporter.Core/DiscordChatExporter.Core.csproj" />
<ProjectReference Include="..\DiscordChatExporter.Core\DiscordChatExporter.Core.csproj" />
</ItemGroup>
</Project>
+30 -2
View File
@@ -1,13 +1,41 @@
using System.Diagnostics.CodeAnalysis;
using System.Threading.Tasks;
using CliFx;
using DiscordChatExporter.Cli.Commands;
using DiscordChatExporter.Cli.Commands.Converters;
using DiscordChatExporter.Core.Exporting.Filtering;
using DiscordChatExporter.Core.Exporting.Partitioning;
namespace DiscordChatExporter.Cli;
public static class Program
{
// Explicit references because CliFx relies on reflection and we're publishing with trimming enabled
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(ExportAllCommand))]
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(ExportChannelsCommand))]
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(ExportDirectMessagesCommand))]
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(ExportGuildCommand))]
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(GetChannelsCommand))]
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(GetDirectChannelsCommand))]
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(GetGuildsCommand))]
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(GuideCommand))]
[DynamicDependency(
DynamicallyAccessedMemberTypes.All,
typeof(ThreadInclusionModeBindingConverter)
)]
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(TruthyBooleanBindingConverter))]
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(PartitionLimit))]
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(MessageFilter))]
public static async Task<int> Main(string[] args) =>
await new CommandLineApplicationBuilder()
.AddCommandsFromThisAssembly()
await new CliApplicationBuilder()
.AddCommand<ExportAllCommand>()
.AddCommand<ExportChannelsCommand>()
.AddCommand<ExportDirectMessagesCommand>()
.AddCommand<ExportGuildCommand>()
.AddCommand<GetChannelsCommand>()
.AddCommand<GetDirectChannelsCommand>()
.AddCommand<GetGuildsCommand>()
.AddCommand<GuideCommand>()
.Build()
.RunAsync(args);
}
@@ -1,4 +1,4 @@
using System;
using System;
using System.Threading.Tasks;
using CliFx.Infrastructure;
using Spectre.Console;
@@ -1,6 +1,6 @@
using System.Text.Json;
using System.Text.Json;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,4 +1,4 @@
using System;
using System;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,9 +1,9 @@
using System;
using System;
using System.IO;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,9 +1,9 @@
using System.Collections.Generic;
using System.Collections.Generic;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,4 +1,4 @@
using System.Collections.Generic;
using System.Collections.Generic;
using System.Linq;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Discord.Data;
// https://discord.com/developers/docs/resources/channel#channel-object-channel-types
public enum ChannelKind
@@ -1,4 +1,4 @@
using System;
using System;
using System.Diagnostics.CodeAnalysis;
using System.Globalization;
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Discord.Data.Common;
namespace DiscordChatExporter.Core.Discord.Data.Common;
public interface IHasId
{
@@ -1,4 +1,4 @@
using System;
using System;
using System.Globalization;
using System.Linq;
@@ -3,8 +3,8 @@ using System.Collections.Generic;
using System.Drawing;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data.Embeds;
@@ -47,7 +47,9 @@ public partial record Embed
var kind =
json.GetPropertyOrNull("type")
?.GetStringOrNull()
.Pipe(s => Enum.ParseOrNull<EmbedKind>(s, true))
?.Pipe(s =>
Enum.TryParse<EmbedKind>(s, true, out var result) ? result : (EmbedKind?)null
)
?? EmbedKind.Rich;
var url = json.GetPropertyOrNull("url")?.GetNonWhiteSpaceStringOrNull();
@@ -56,7 +58,7 @@ public partial record Embed
var color = json.GetPropertyOrNull("color")
?.GetInt32OrNull()
?.Pipe(System.Drawing.Color.FromArgb)
.WithFullAlpha();
.ResetAlpha();
var author = json.GetPropertyOrNull("author")?.Pipe(EmbedAuthor.Parse);
var description = json.GetPropertyOrNull("description")?.GetStringOrNull();
@@ -72,7 +74,7 @@ public partial record Embed
// Under the Discord API model, embeds can only have at most one image.
// Because of that, embeds that are rendered with multiple images on the client
// (e.g., tweet embeds), are exposed from the API as multiple separate embeds.
// (e.g. tweet embeds), are exposed from the API as multiple separate embeds.
// Our embed model is consistent with the user-facing side of Discord, so images
// are stored as an array. The API will only ever return one image, but we deal
// with this by merging related embeds at the end of the message parsing process.
@@ -1,4 +1,4 @@
using System.Text.RegularExpressions;
using System.Text.RegularExpressions;
namespace DiscordChatExporter.Core.Discord.Data.Embeds;
@@ -1,4 +1,4 @@
using System.Text.RegularExpressions;
using System.Text.RegularExpressions;
namespace DiscordChatExporter.Core.Discord.Data.Embeds;
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Discord.Data.Embeds;
namespace DiscordChatExporter.Core.Discord.Data.Embeds;
public partial record YouTubeVideoEmbedProjection(string VideoId)
{
@@ -1,8 +1,8 @@
using System.Text.Json;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -10,14 +10,14 @@ namespace DiscordChatExporter.Core.Discord.Data;
public partial record Emoji(
// Only present on custom emoji
Snowflake? Id,
// Name of a custom emoji (e.g., LUL) or actual representation of a standard emoji (e.g., 🙂)
// Name of a custom emoji (e.g. LUL) or actual representation of a standard emoji (e.g. 🙂)
string Name,
bool IsAnimated
)
{
public bool IsCustomEmoji { get; } = Id is not null;
// Name of a custom emoji (e.g., LUL) or name of a standard emoji (e.g., slight_smile)
// Name of a custom emoji (e.g. LUL) or name of a standard emoji (e.g. slight_smile)
public string Code { get; } = Id is not null ? Name : EmojiIndex.TryGetCode(Name) ?? Name;
public string ImageUrl { get; } =
@@ -1,4 +1,4 @@
using System;
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
@@ -8855,8 +8855,6 @@ internal static class EmojiIndex
["united_nations"] = "🇺🇳",
};
public static IReadOnlyCollection<string> GetAllNames() => _toCodes.Keys;
public static string? TryGetCode(string name) => _toCodes.GetValueOrDefault(name);
public static string? TryGetName(string code) => _fromCodes.GetValueOrDefault(code);
@@ -1,7 +1,7 @@
using System.Text.Json;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,6 +1,6 @@
using System.Text.Json;
using System.Text.Json;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,7 +1,7 @@
using System.Text.Json;
using System.Text.Json;
using System.Text.RegularExpressions;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,9 +1,9 @@
using System.Collections.Generic;
using System.Collections.Generic;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,11 +1,11 @@
using System;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Discord.Data.Embeds;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -27,16 +27,9 @@ public partial record Message(
IReadOnlyList<User> MentionedUsers,
MessageReference? Reference,
Message? ReferencedMessage,
MessageSnapshot? ForwardedMessage,
Interaction? Interaction
) : IHasId
{
public bool IsEmpty { get; } =
string.IsNullOrWhiteSpace(Content)
&& !Attachments.Any()
&& !Embeds.Any()
&& !Stickers.Any();
public bool IsSystemNotification { get; } =
Kind is >= MessageKind.RecipientAdd and <= MessageKind.ThreadCreated;
@@ -45,7 +38,11 @@ public partial record Message(
// App interactions are rendered as replies in the Discord client, but they are not actually replies
public bool IsReplyLike => IsReply || Interaction is not null;
public bool IsForwarded { get; } = Reference?.Kind == MessageReferenceKind.Forward;
public bool IsEmpty { get; } =
string.IsNullOrWhiteSpace(Content)
&& !Attachments.Any()
&& !Embeds.Any()
&& !Stickers.Any();
public IEnumerable<User> GetReferencedUsers()
{
@@ -174,17 +171,7 @@ public partial record Message
var messageReference = json.GetPropertyOrNull("message_reference")
?.Pipe(MessageReference.Parse);
var referencedMessage = json.GetPropertyOrNull("referenced_message")?.Pipe(Parse);
// Currently Discord only supports 1 snapshot per forward
var forwardedMessage = json.GetPropertyOrNull("message_snapshots")
?.EnumerateArrayOrNull()
?.Select(j => j.GetPropertyOrNull("message"))
.WhereNotNull()
.Select(MessageSnapshot.Parse)
.FirstOrDefault();
var interaction = json.GetPropertyOrNull("interaction")?.Pipe(Interaction.Parse);
return new Message(
@@ -204,7 +191,6 @@ public partial record Message
mentionedUsers,
messageReference,
referencedMessage,
forwardedMessage,
interaction
);
}
@@ -1,4 +1,4 @@
using System;
using System;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Discord.Data;
// https://discord.com/developers/docs/resources/channel#message-object-message-types
public enum MessageKind
@@ -13,5 +13,4 @@ public enum MessageKind
GuildMemberJoin = 7,
ThreadCreated = 18,
Reply = 19,
ThreadStarterMessage = 21,
}
@@ -1,23 +1,14 @@
using System.Text.Json;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
// https://discord.com/developers/docs/resources/channel#message-object-message-reference-structure
public record MessageReference(
MessageReferenceKind Kind,
Snowflake? MessageId,
Snowflake? ChannelId,
Snowflake? GuildId
)
public record MessageReference(Snowflake? MessageId, Snowflake? ChannelId, Snowflake? GuildId)
{
public static MessageReference Parse(JsonElement json)
{
var kind =
json.GetPropertyOrNull("type")?.GetInt32OrNull()?.Pipe(t => (MessageReferenceKind)t)
?? MessageReferenceKind.Default;
var messageId = json.GetPropertyOrNull("message_id")
?.GetNonWhiteSpaceStringOrNull()
?.Pipe(Snowflake.Parse);
@@ -30,6 +21,6 @@ public record MessageReference(
?.GetNonWhiteSpaceStringOrNull()
?.Pipe(Snowflake.Parse);
return new MessageReference(kind, messageId, channelId, guildId);
return new MessageReference(messageId, channelId, guildId);
}
}
@@ -1,8 +0,0 @@
namespace DiscordChatExporter.Core.Discord.Data;
// https://discord.com/developers/docs/resources/channel#message-reference-types
public enum MessageReferenceKind
{
Default = 0,
Forward = 1,
}
@@ -1,57 +0,0 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Embeds;
using JsonExtensions.Reading;
namespace DiscordChatExporter.Core.Discord.Data;
// https://docs.discord.com/developers/resources/message#message-snapshot-object
public record MessageSnapshot(
DateTimeOffset Timestamp,
DateTimeOffset? EditedTimestamp,
string Content,
IReadOnlyList<Attachment> Attachments,
IReadOnlyList<Embed> Embeds,
IReadOnlyList<Sticker> Stickers
)
{
public static MessageSnapshot Parse(JsonElement json)
{
var timestamp =
json.GetPropertyOrNull("timestamp")?.GetDateTimeOffsetOrNull()
?? DateTimeOffset.MinValue;
var editedTimestamp = json.GetPropertyOrNull("edited_timestamp")?.GetDateTimeOffsetOrNull();
var content = json.GetPropertyOrNull("content")?.GetStringOrNull() ?? "";
var attachments =
json.GetPropertyOrNull("attachments")
?.EnumerateArrayOrNull()
?.Select(Attachment.Parse)
.ToArray()
?? [];
var embeds =
json.GetPropertyOrNull("embeds")?.EnumerateArrayOrNull()?.Select(Embed.Parse).ToArray()
?? [];
var stickers =
json.GetPropertyOrNull("sticker_items")
?.EnumerateArrayOrNull()
?.Select(Sticker.Parse)
.ToArray()
?? [];
return new MessageSnapshot(
timestamp,
editedTimestamp,
content,
attachments,
embeds,
stickers
);
}
}
@@ -1,5 +1,5 @@
using System.Text.Json;
using PowerKit.Extensions;
using System.Text.Json;
using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,8 +1,8 @@
using System.Drawing;
using System.Drawing;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -18,7 +18,7 @@ public record Role(Snowflake Id, string Name, int Position, Color? Color) : IHas
var color = json.GetPropertyOrNull("color")
?.GetInt32OrNull()
?.Pipe(System.Drawing.Color.FromArgb)
.WithFullAlpha()
.ResetAlpha()
.NullIf(c => c.ToRgb() <= 0);
return new Role(id, name, position, color);
@@ -1,8 +1,8 @@
using System;
using System;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Discord.Data;
public enum StickerFormat
{
@@ -1,7 +1,7 @@
using System.Text.Json;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
+100 -262
View File
@@ -1,4 +1,4 @@
using System;
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
@@ -11,10 +11,10 @@ using System.Threading.Tasks;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Exceptions;
using DiscordChatExporter.Core.Utils;
using DiscordChatExporter.Core.Utils.Extensions;
using Gress;
using JsonExtensions.Http;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord;
@@ -30,8 +30,9 @@ public class DiscordClient(
string url,
TokenKind tokenKind,
CancellationToken cancellationToken = default
) =>
await Http.ResponseResiliencePipeline.ExecuteAsync(
)
{
return await Http.ResponseResiliencePipeline.ExecuteAsync(
async innerCancellationToken =>
{
using var request = new HttpRequestMessage(HttpMethod.Get, new Uri(_baseUri, url));
@@ -58,12 +59,12 @@ public class DiscordClient(
{
var remainingRequestCount = response
.Headers.TryGetValue("X-RateLimit-Remaining")
?.Pipe(s => int.ParseOrNull(s, CultureInfo.InvariantCulture));
?.Pipe(s => int.Parse(s, CultureInfo.InvariantCulture));
var resetAfterDelay = response
.Headers.TryGetValue("X-RateLimit-Reset-After")
?.Pipe(s => double.ParseOrNull(s, CultureInfo.InvariantCulture))
?.Pipe(TimeSpan.FromSeconds);
?.Pipe(s => double.Parse(s, CultureInfo.InvariantCulture))
.Pipe(TimeSpan.FromSeconds);
// If this was the last request available before hitting the rate limit,
// wait out the reset time so that future requests can succeed.
@@ -90,6 +91,7 @@ public class DiscordClient(
},
cancellationToken
);
}
private async ValueTask<TokenKind> ResolveTokenKindAsync(
CancellationToken cancellationToken = default
@@ -159,7 +161,7 @@ public class DiscordClient(
$"""
Request to '{url}' failed: {response
.StatusCode.ToString()
.SeparateWords(' ')
.ToSpaceSeparatedWords()
.ToLowerInvariant()}.
Response content: {await response.Content.ReadAsStringAsync(
cancellationToken
@@ -192,23 +194,6 @@ public class DiscordClient(
return Application.Parse(response);
}
private async ValueTask EnsureMessageContentIntentAsync(
CancellationToken cancellationToken = default
)
{
if (await ResolveTokenKindAsync(cancellationToken) != TokenKind.Bot)
return;
var application = await GetApplicationAsync(cancellationToken);
if (application.IsMessageContentIntentEnabled)
return;
throw new DiscordChatExporterException(
"Provided bot account is missing the MESSAGE_CONTENT privileged intent.",
true
);
}
public async ValueTask<User?> TryGetUserAsync(
Snowflake userId,
CancellationToken cancellationToken = default
@@ -362,7 +347,6 @@ public class DiscordClient(
$"guilds/{guildId}/members/{memberId}",
cancellationToken
);
return response?.Pipe(j => Member.Parse(j, guildId));
}
@@ -411,12 +395,14 @@ public class DiscordClient(
?.GetNonWhiteSpaceStringOrNull()
?.Pipe(Snowflake.Parse);
// It's possible for the parent channel to be inaccessible, despite the
// child channel being accessible.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/1108
var parent = parentId is not null
? await TryGetChannelAsync(parentId.Value, cancellationToken)
: null;
Channel? parent = null;
if (parentId is not null)
{
// It's possible for the parent channel to be inaccessible, despite the
// child channel being accessible.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/1108
parent = await TryGetChannelAsync(parentId.Value, cancellationToken);
}
return Channel.Parse(response.Value, parent);
}
@@ -443,11 +429,6 @@ public class DiscordClient(
.Where(c => before is null || c.MayHaveMessagesBefore(before.Value))
.ToArray();
// Track yielded thread IDs to avoid duplicates that can occur when a thread transitions
// from active to archived between the two separate API calls used to fetch threads.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/1433
var seenThreadIds = new HashSet<Snowflake>();
// User accounts can only fetch threads using the search endpoint
if (await ResolveTokenKindAsync(cancellationToken) == TokenKind.User)
{
@@ -491,9 +472,7 @@ public class DiscordClient(
break;
}
if (seenThreadIds.Add(thread.Id))
yield return thread;
yield return thread;
currentOffset++;
}
@@ -532,12 +511,7 @@ public class DiscordClient(
.Pipe(parentsById.GetValueOrDefault);
if (filteredChannels.Contains(parent))
{
var thread = Channel.Parse(threadJson, parent);
if (seenThreadIds.Add(thread.Id))
yield return thread;
}
yield return Channel.Parse(threadJson, parent);
}
}
@@ -573,14 +547,12 @@ public class DiscordClient(
)
{
var thread = Channel.Parse(threadJson, channel);
yield return thread;
currentBefore = threadJson
.GetProperty("thread_metadata")
.GetProperty("archive_timestamp")
.GetString();
if (seenThreadIds.Add(thread.Id))
yield return thread;
}
if (!response.Value.GetProperty("has_more").GetBoolean())
@@ -592,55 +564,6 @@ public class DiscordClient(
}
}
public async ValueTask<Message?> TryGetMessageAsync(
Snowflake channelId,
Snowflake messageId,
CancellationToken cancellationToken = default
)
{
// Use the regular message listing endpoint with the 'around' parameter instead of the
// dedicated single-message endpoint, because the latter is not accessible to user tokens.
var url = new UrlBuilder()
.SetPath($"channels/{channelId}/messages")
.SetQueryParameter("around", messageId.ToString())
.SetQueryParameter("limit", "1")
.Build();
// Can be null on channels that the user cannot access
var response = await TryGetJsonResponseAsync(url, cancellationToken);
if (response is null)
return null;
// The endpoint returns messages around the requested ID, so make sure to only return
// the message that exactly matches it (it may be absent if it has been deleted).
return response
.Value.EnumerateArray()
.Select(Message.Parse)
.FirstOrDefault(m => m.Id == messageId);
}
private async ValueTask<Message?> TryGetFirstMessageAsync(
Snowflake channelId,
Snowflake? after = null,
CancellationToken cancellationToken = default
)
{
var url = new UrlBuilder()
.SetPath($"channels/{channelId}/messages")
.SetQueryParameter("limit", "1")
.SetQueryParameter("after", (after ?? Snowflake.Zero).ToString())
.Build();
// Can be null on channels that the user cannot access
var response = await TryGetJsonResponseAsync(url, cancellationToken);
if (response is null)
return null;
var message = response.Value.EnumerateArray().Select(Message.Parse).FirstOrDefault();
return message;
}
private async ValueTask<Message?> TryGetLastMessageAsync(
Snowflake channelId,
Snowflake? before = null,
@@ -653,137 +576,8 @@ public class DiscordClient(
.SetQueryParameter("before", before?.ToString())
.Build();
// Can be null on channels that the user cannot access
var response = await TryGetJsonResponseAsync(url, cancellationToken);
if (response is null)
return null;
return response.Value.EnumerateArray().Select(Message.Parse).LastOrDefault();
}
private async IAsyncEnumerable<Message> GetMessagesAsync(
Snowflake channelId,
Snowflake? after,
Snowflake? before,
IProgress<Percentage>? progress,
bool isReverse,
[EnumeratorCancellation] CancellationToken cancellationToken
)
{
// To keep the understanding of message history independent of the fetching direction,
// we'll refer to the two ends of the range as Alpha and Omega.
// Depending on the direction, these are either 'before' and 'after', or 'after' and 'before'.
//
// Chronological order:
// <after> Alpha [----->----->----] Omega <before>
// Reverse chronological order:
// <after> Omega [-----<-----<----] Alpha <before>
// Because Discord API doesn't allow us to provide both 'after' and 'before' parameters
// at the same time, we have to establish at least one end of the boundary manually.
// To do that, we'll fetch the Omega message, which will be the terminal message in the range:
// last message in chronological order, or first message in reverse chronological order.
// This snapshotting also has the side benefit of allowing us to calculate progress by comparing
// the timestamps of the Alpha message, Omega message, and the message being currently processed.
var omegaMessage = !isReverse
? await TryGetLastMessageAsync(channelId, before, cancellationToken)
: await TryGetFirstMessageAsync(channelId, after, cancellationToken);
// If the Omega doesn't exist or falls outside of the range, then there are simply no messages
// satisfying the specified range.
if (
omegaMessage is null
|| (!isReverse && omegaMessage.Timestamp < after?.ToDate())
|| (isReverse && omegaMessage.Timestamp > before?.ToDate())
)
{
yield break;
}
// Persist the Alpha message as soon as we fetch the initial batch of messages.
// This is only used for calculating progress.
var alphaMessage = default(Message);
var currentBoundary = !isReverse ? after ?? Snowflake.Zero : before;
while (true)
{
var url = new UrlBuilder()
.SetPath($"channels/{channelId}/messages")
.SetQueryParameter("limit", "100")
.SetQueryParameter(!isReverse ? "after" : "before", currentBoundary?.ToString())
.Build();
var response = await GetJsonResponseAsync(url, cancellationToken);
var messages = response
.EnumerateArray()
.Select(Message.Parse)
// Messages in batches are always returned from newest to oldest, so reverse if needed
.Pipe(messages => isReverse ? messages : messages.Reverse())
.ToArray();
// Break if there are no messages (can happen if messages are deleted during execution)
if (!messages.Any())
yield break;
// If all messages are empty, make sure that it's not because the bot account doesn't
// have the MESSAGE_CONTENT intent enabled.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/1106#issuecomment-1741548959
if (messages.All(m => m.IsEmpty))
await EnsureMessageContentIntentAsync(cancellationToken);
foreach (var message in messages)
{
// Ensure that we're still in range by checking against the Omega
if (!isReverse ? message.Id > omegaMessage.Id : message.Id < omegaMessage.Id)
{
yield break;
}
alphaMessage ??= message;
// Report progress based on timestamps
if (progress is not null)
{
var fetchedDuration = isReverse
? alphaMessage.Timestamp - message.Timestamp
: message.Timestamp - alphaMessage.Timestamp;
var totalDuration = isReverse
? alphaMessage.Timestamp - omegaMessage.Timestamp
: omegaMessage.Timestamp - alphaMessage.Timestamp;
progress.Report(
Percentage.FromFraction(
// Avoid division by zero if all messages have the exact same timestamp
// (which happens when there's only one message in the channel)
totalDuration > TimeSpan.Zero
? fetchedDuration / totalDuration
: 1
)
);
}
// Some messages, for example thread starter messages, are returned by the API as content-less references.
// Try to resolve them to the actual message so that they appear as they do in the Discord client.
var actualMessage =
message.Kind == MessageKind.ThreadStarterMessage
&& message.Reference?.ChannelId is { } referencedChannelId
&& message.Reference?.MessageId is { } referencedMessageId
? await TryGetMessageAsync(
referencedChannelId,
referencedMessageId,
cancellationToken
)
: null;
yield return actualMessage ?? message;
}
// The new boundary is always determined by the last message in the batch,
// because we order the messages based on fetching direction.
currentBoundary = messages.Last().Id;
}
var response = await GetJsonResponseAsync(url, cancellationToken);
return response.EnumerateArray().Select(Message.Parse).LastOrDefault();
}
public async IAsyncEnumerable<Message> GetMessagesAsync(
@@ -794,41 +588,85 @@ public class DiscordClient(
[EnumeratorCancellation] CancellationToken cancellationToken = default
)
{
await foreach (
var message in GetMessagesAsync(
channelId,
after,
before,
progress,
false,
cancellationToken
)
)
{
yield return message;
}
}
// Get the last message in the specified range, so we can later calculate the
// progress based on the difference between message timestamps.
// This also snapshots the boundaries, which means that messages posted after
// the export started will not appear in the output.
var lastMessage = await TryGetLastMessageAsync(channelId, before, cancellationToken);
if (lastMessage is null || lastMessage.Timestamp < after?.ToDate())
yield break;
public async IAsyncEnumerable<Message> GetMessagesInReverseAsync(
Snowflake channelId,
Snowflake? after = null,
Snowflake? before = null,
IProgress<Percentage>? progress = null,
[EnumeratorCancellation] CancellationToken cancellationToken = default
)
{
await foreach (
var message in GetMessagesAsync(
channelId,
after,
before,
progress,
true,
cancellationToken
)
)
// Keep track of the first message in range in order to calculate the progress
var firstMessage = default(Message);
var currentAfter = after ?? Snowflake.Zero;
while (true)
{
yield return message;
var url = new UrlBuilder()
.SetPath($"channels/{channelId}/messages")
.SetQueryParameter("limit", "100")
.SetQueryParameter("after", currentAfter.ToString())
.Build();
var response = await GetJsonResponseAsync(url, cancellationToken);
var messages = response
.EnumerateArray()
.Select(Message.Parse)
// Messages are returned from newest to oldest, so we need to reverse them
.Reverse()
.ToArray();
// Break if there are no messages (can happen if messages are deleted during execution)
if (!messages.Any())
yield break;
// If all messages are empty, make sure that it's not because the bot account doesn't
// have the Message Content Intent enabled.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/1106#issuecomment-1741548959
if (
messages.All(m => m.IsEmpty)
&& await ResolveTokenKindAsync(cancellationToken) == TokenKind.Bot
)
{
var application = await GetApplicationAsync(cancellationToken);
if (!application.IsMessageContentIntentEnabled)
{
throw new DiscordChatExporterException(
"Provided bot account does not have the Message Content Intent enabled.",
true
);
}
}
foreach (var message in messages)
{
firstMessage ??= message;
// Ensure that the messages are in range
if (message.Timestamp > lastMessage.Timestamp)
yield break;
// Report progress based on timestamps
if (progress is not null)
{
var exportedDuration = (message.Timestamp - firstMessage.Timestamp).Duration();
var totalDuration = (lastMessage.Timestamp - firstMessage.Timestamp).Duration();
progress.Report(
Percentage.FromFraction(
// Avoid division by zero if all messages have the exact same timestamp
// (which happens when there's only one message in the channel)
totalDuration > TimeSpan.Zero
? exportedDuration / totalDuration
: 1
)
);
}
yield return message;
currentAfter = message.Id;
}
}
}
@@ -1,4 +1,4 @@
using System;
using System;
using System.Collections.Generic;
using System.IO.Compression;
using System.Linq;
@@ -1,3 +1,3 @@
namespace DiscordChatExporter.Core.Discord.Dump;
namespace DiscordChatExporter.Core.Discord.Dump;
public record DataDumpChannel(Snowflake Id, string Name);
@@ -1,4 +1,4 @@
using System;
using System;
namespace DiscordChatExporter.Core.Discord;
@@ -1,4 +1,4 @@
using System;
using System;
using System.Diagnostics.CodeAnalysis;
using System.Globalization;
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Discord;
namespace DiscordChatExporter.Core.Discord;
public enum TokenKind
{
@@ -1,15 +1,14 @@
<Project Sdk="Microsoft.NET.Sdk">
<ItemGroup>
<PackageReference Include="AngleSharp" />
<PackageReference Include="AsyncKeyedLock" />
<PackageReference Include="CSharpier.MsBuild" PrivateAssets="all" />
<PackageReference Include="Gress" />
<PackageReference Include="JsonExtensions" />
<PackageReference Include="Polly" />
<PackageReference Include="PowerKit" />
<PackageReference Include="RazorBlade" />
<PackageReference Include="Superpower" />
<PackageReference Include="WebMarkupMin.Core" />
<PackageReference Include="YoutubeExplode" />
<PackageReference Include="AngleSharp" Version="1.4.0" />
<PackageReference Include="AsyncKeyedLock" Version="8.0.1" />
<PackageReference Include="CSharpier.MsBuild" Version="1.2.5" PrivateAssets="all" />
<PackageReference Include="Gress" Version="2.1.1" />
<PackageReference Include="JsonExtensions" Version="1.2.0" />
<PackageReference Include="Polly" Version="8.6.5" />
<PackageReference Include="RazorBlade" Version="0.11.0" />
<PackageReference Include="Superpower" Version="3.1.0" />
<PackageReference Include="WebMarkupMin.Core" Version="2.20.1" />
<PackageReference Include="YoutubeExplode" Version="6.5.6" />
</ItemGroup>
</Project>

Some files were not shown because too many files have changed in this diff Show More