Compare commits

...

1134 Commits

Author SHA1 Message Date
github-actions[bot] 4429fd0450 Release 2025.09.23
Created by: bashonly

:ci skip all
2025-09-23 06:45:05 +00:00
bashonly 2e81e298cd [cleanup] Misc (#14268)
Closes #13427
Authored by: bashonly, seproDev, doe1080

Co-authored-by: sepro <sepro@sepr0.com>
Co-authored-by: doe1080 <98906116+doe1080@users.noreply.github.com>
2025-09-23 06:33:30 +00:00
sepro 7f5d9f8543 [ie/youtube] Force player 0004de42 (#14398)
Closes #14400
Authored by: seproDev
2025-09-23 07:07:28 +02:00
sepro f8750504c2 [ie/twitch:clips] Fix extractor (#14397)
Closes #14396
Authored by: seproDev
2025-09-22 21:42:12 +02:00
sepro 8821682f15 [docs] Establish AI/LLM contribution policy (#14194)
Authored by: seproDev, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-09-21 19:15:16 +02:00
sepro 08d7899683 [cleanup] Deprecate various options (#13821)
Closes #14198, Closes #12909
Authored by: seproDev
2025-09-21 17:10:37 +02:00
sepro 98b6b0d339 [utils] mimetype2ext: Recognize vnd.dlna.mpeg-tts (#14388)
Closes #14386
Authored by: seproDev
2025-09-21 16:29:21 +02:00
sepro bf5d18016b [ie/vk:uservideos] Support alternate URL format (#14376)
Closes #11793
Authored by: seproDev
2025-09-20 01:12:02 +02:00
sepro 4bc19adc87 [ie/ttinglive] Adapt FlexTV extractor to new domain (#14375)
Closes #14342
Authored by: seproDev
2025-09-20 00:38:34 +02:00
sepro b2c01d0498 [ie/applepodcast] Fix extractor (#14372)
Closes #14368
Authored by: seproDev
2025-09-19 22:22:47 +02:00
bashonly e123a48f11 [ie/telecinco] Support browser impersonation (#14351)
Closes #14349
Authored by: bashonly
2025-09-17 04:58:42 +00:00
bashonly 820c6e2445 [ie/mitele] Remove extractor (#14348)
Closes #13535
Authored by: bashonly
2025-09-16 23:03:09 +00:00
thegymguy 677997d84e [ie/xhamster] Fix extractor (#14345)
Fix a1c98226a4

Closes #14145
Authored by: arand, thegymguy

Co-authored-by: arand <183498+arand@users.noreply.github.com>
2025-09-16 22:19:08 +00:00
sepro b81e9272dc [ie/vk] Support vksport URLs (#14341)
Closes #14175
Authored by: seproDev
2025-09-16 19:43:00 +02:00
bashonly df4b4e8ccf [build] Use PyInstaller 6.16 for Windows (#14318)
Authored by: bashonly
2025-09-13 22:50:34 +00:00
sepro f3829463c7 [utils] random_user_agent: Bump versions (#14317)
Authored by: seproDev
2025-09-14 00:32:25 +02:00
bashonly ae3923b6b2 [ci] Improve workflow checks (#14316)
Authored by: bashonly
2025-09-13 21:55:35 +00:00
bashonly 8ab262c66b [cleanup] Remove references to setup.cfg (#14315)
Fix eb4b3a5fc7

Authored by: bashonly
2025-09-13 21:30:06 +00:00
bashonly e2d37bcc8e [build] Refactor Linux build jobs (#14275)
Authored by: bashonly
2025-09-13 21:20:28 +00:00
sepro eb4b3a5fc7 [cleanup] Remove setup.cfg (#14314)
Authored by: seproDev
2025-09-13 21:57:54 +02:00
bashonly 65e90aea29 [cleanup] Remove broken extractors (#14305)
Closes #1466, Closes #2005, Closes #4897, Closes #5118, Closes #8489, Closes #13072
Authored by: bashonly
2025-09-12 08:15:41 +00:00
doe1080 17bfaa53ed [ie/onsen] Add extractor (#10971)
Closes #10902
Authored by: doe1080
2025-09-11 22:51:31 +00:00
doe1080 8cb037c0b0 [ie/smotrim] Rework extractors (#14200)
Closes #9372, Closes #11804, Closes #13900
Authored by: doe1080, swayll

Co-authored-by: Nikolay Fedorov <40500428+swayll@users.noreply.github.com>
2025-09-11 20:59:54 +00:00
doe1080 7d9e48b22a [ie/tunein] Fix extractors (#13981)
Authored by: doe1080
2025-09-11 20:42:01 +00:00
sepro f5cb721185 [ie/loco] Fix extractor (#14256)
Closes #14255
Authored by: seproDev
2025-09-11 21:32:35 +02:00
bashonly 83b8409366 [ci] Test with Python 3.14 (#13468)
Authored by: bashonly
2025-09-11 15:35:55 +00:00
bashonly ba80446855 [cleanup] Bump ruff to 0.13.x (#14293)
Authored by: bashonly
2025-09-11 15:18:01 +00:00
bashonly 22ea0688ed [ci] Bump actions/setup-python to v6 (#14282)
Authored by: bashonly
2025-09-10 21:29:12 +00:00
bashonly 5c1abcdc49 [ie/tiktok:live] Fix room ID extraction (#14287)
Closes #9418
Authored by: bashonly
2025-09-10 21:26:27 +00:00
doe1080 3d9a88bd8e [ie/pixivsketch] Remove extractors (#14196)
Authored by: doe1080
2025-09-10 21:22:10 +00:00
doe1080 9def9a4b0e [ie/newspicks] Warn when only preview is available (#14197)
Closes #14137
Authored by: doe1080
2025-09-10 21:20:03 +00:00
bashonly 679587dac7 [ie/vimeo] Fix login error handling (#14280)
Closes #14279
Authored by: bashonly
2025-09-10 18:39:07 +00:00
Will Smillie a1c98226a4 [ie/xhamster] Fix extractor (#14286)
Closes #14145
Authored by: nicolaasjan, willsmillie

Co-authored-by: nicolaasjan <14093220+nicolaasjan@users.noreply.github.com>
2025-09-10 18:17:24 +00:00
bashonly c8ede5f34d [build] Use new PyInstaller builds for Windows (#14273)
Authored by: bashonly
2025-09-08 22:44:36 +00:00
bashonly a183837ec8 [test:utils] Fix sanitize_path test for Windows CPython 3.11 (#13878)
Authored by: Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.dev>
2025-09-07 20:43:39 +00:00
Sipherdrakon 067062bb87 [ie/10play] Fix extractor (#14242)
Closes #14212
Authored by: Sipherdrakon
2025-09-07 21:09:23 +02:00
bashonly 8597a4331e [build] Fix cache warmer (#14261)
Fix 50136eeeb3

Authored by: bashonly
2025-09-07 00:57:20 +00:00
sepro 48a214bef4 [build] Use SPDX license identifier (#14260)
Authored by: cdce8p

Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
2025-09-07 02:02:20 +02:00
Mozi 6a763a55d8 [compat] Add compat_datetime_from_timestamp (#11902)
Authored by: pzhlkj6612, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-09-07 01:48:24 +02:00
sepro e6e6b51214 [docs] Clarify license of PyInstaller-bundled executables (#14257)
Closes #348

Authored by: seproDev
2025-09-07 01:17:02 +02:00
bashonly 7c9b10ebc8 [ci] Test and lint workflows (#14249)
Authored by: bashonly
2025-09-06 22:28:11 +00:00
bashonly cd94e70040 [build] Post-release workflow cleanup (#14250)
Authored by: bashonly
2025-09-06 05:41:43 +00:00
github-actions[bot] 7c27965ff6 Release 2025.09.05
Created by: bashonly

:ci skip all
2025-09-05 22:46:00 +00:00
bashonly 50136eeeb3 [build] Overhaul Linux builds and refactor release workflow (#13997)
- Use `manylinux-shared` images for Linux builds
- Discontinue `yt-dlp_linux_armv7l`/`linux_armv7l_exe` release binary
- Add `yt-dlp_linux_armv7l.zip`/`linux_armv7l_dir` release binary
- Add `yt-dlp_musllinux` and `yt-dlp_musllinux_aarch64` release binaries
- Migrate `linux_exe` build strategy from staticx+musl to manylinux2014/glibc2.17
- Rewrite release.yml's "unholy bash monstrosity" as devscripts/setup_variables.py

Closes #10072, Closes #10630, Closes #10578, Closes #13976, Closes #13977, Closes #14106
Authored by: bashonly
2025-09-05 22:38:20 +00:00
Chase Ryan 603acdff07 [ie/charlierose] Fix extractor (#14231)
Authored by: gitchasing
2025-09-05 23:28:52 +02:00
sepro d925e92b71 [ie/vevo] Restore extractors (#14203)
Partially reverts 6f4c1bb593

Authored by: seproDev
2025-08-31 00:41:52 +02:00
sepro ed24640943 [ie/lrt] Fix extractors (#14193)
Closes #13501
Authored by: seproDev
2025-08-30 00:28:44 +02:00
sepro 76bb46002c Fix --id deprecation warning (#14190)
Authored by: seproDev
2025-08-29 22:06:53 +02:00
InvalidUsernameException 1e28f6bf74 [ie/kick:vod] Support ongoing livestream VODs (#14154)
Authored by: InvalidUsernameException
2025-08-28 01:26:49 +00:00
garret1317 0b51005b48 [ie/ITVBTCC] Fix extractor (#14161)
Closes #14156
Authored by: garret1317
2025-08-28 01:19:25 +00:00
Abdulmohsen 223baa81f6 [ie/tver] Extract more metadata (#14165)
Authored by: arabcoders
2025-08-28 01:18:10 +00:00
Gegham Zakaryan 18fe696df9 [ie/googledrive] Fix subtitles extraction (#14139)
Authored by: zakaryan2004
2025-08-28 01:12:08 +00:00
github-actions[bot] 487a90c8ef Release 2025.08.27
Created by: bashonly

:ci skip all
2025-08-27 23:56:39 +00:00
bashonly 8cd37b85d4 [ie/youtube] Use alternative tv user-agent when authenticated (#14169)
Authored by: bashonly
2025-08-27 23:00:03 +00:00
bashonly 5c7ad68ff1 [ie/youtube] Deprioritize web_safari m3u8 formats (#14168)
Authored by: bashonly
2025-08-27 22:31:51 +00:00
sepro 1ddbd033f0 [ie/generic] Simplify invalid URL error message (#14167)
Authored by: seproDev
2025-08-27 23:27:57 +02:00
sepro fec30c56f0 [ie/generic] Use https as fallback protocol (#14160)
Authored by: seproDev
2025-08-27 22:25:35 +02:00
sepro d6950c27af [ie/skeb] Support wav files (#14147)
Closes #14146
Authored by: seproDev
2025-08-27 15:34:44 +02:00
bashonly 3bd9154412 [ie/youtube] Player client maintenance (#14135)
- Prioritize `tv_simply` over `tv` in default logged-out clients
- Revert `tv` client user-agent to work around 403 errors

Authored by: bashonly
2025-08-23 23:45:29 +00:00
bashonly 8f4a908300 [ie/youtube] Add tcc player JS variant (#14134)
Authored by: bashonly
2025-08-23 23:43:50 +00:00
github-actions[bot] f1ba9f4ddb Release 2025.08.22
Created by: bashonly

:ci skip all
2025-08-22 23:59:02 +00:00
bashonly 5c8bcfdbc6 [ie/youtube] Optimize playback wait times (#14124)
Authored by: bashonly
2025-08-22 23:53:28 +00:00
bashonly 895e762a83 [ie/youtube] Replace ios with tv_simply in default clients (#14123)
Also:
- Add `web_safari` to default logged-in clients
- Add `web_creator` to default premium clients
- Flag `ios` HLS formats as requiring PO token

Closes #13702
Authored by: bashonly, coletdjnz

Co-authored-by: coletdjnz <coletdjnz@protonmail.com>
2025-08-22 23:49:54 +00:00
bashonly 39b7b8ddc7 [ie/youtube] Improve tv client context (#14122)
Closes #12563
Authored by: bashonly
2025-08-22 23:44:32 +00:00
bashonly 526410b4af [cookies] Fix f29acc4a6e
Authored by: bashonly
2025-08-21 22:13:45 -05:00
bashonly f29acc4a6e [cookies] Fix --cookies-from-browser with Firefox 142+ (#14114)
Ref: https://github.com/mozilla-firefox/firefox/commit/5869af852cd20425165837f6c2d9971f3efba83d
Related: 28b68f6875

Closes #13559, Closes #14113
Authored by: bashonly, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.dev>
2025-08-21 23:47:18 +00:00
zhallgato 4dbe96459d [ie/mediaklikk] Fix extractor (#13975)
Closes #14091
Authored by: zhallgato
2025-08-21 21:28:12 +00:00
bashonly a03c37b44e [ie/youtube] Update tv client config (#14101)
Authored by: seproDev
    
Co-authored-by: sepro <sepro@sepr0.com>
2025-08-20 19:46:10 +00:00
doe1080 fcea3edb5c [ie/steam] Fix extractors (#14093)
Authored by: doe1080
2025-08-20 15:44:45 +00:00
bashonly 415b6d9ca8 [build] Post-release workflow cleanup (#14090)
Authored by: bashonly
2025-08-20 06:17:45 +00:00
github-actions[bot] 575753b9f3 Release 2025.08.20
Created by: bashonly

:ci skip all
2025-08-20 02:51:33 +00:00
bashonly c2fc4f3e7f [cleanup] Misc (#13991)
Authored by: bashonly
2025-08-20 02:42:34 +00:00
bashonly 07247d6c20 [build] Add Windows ARM64 builds (#14003)
Adds yt-dlp_arm64.exe and yt-dlp_win_arm64.zip to release assets

Closes #13849
Authored by: bashonly
2025-08-20 02:28:00 +00:00
bashonly f63a7e41d1 [ie/youtube] Add playback_wait extractor-arg
Authored by: bashonly
2025-08-19 21:22:00 -05:00
bashonly 7b8a8abb98 [ie/francetv:site] Fix extractor (#14082)
Closes #14072
Authored by: bashonly
2025-08-20 01:35:32 +00:00
bashonly a97f4cb57e [ie/youtube] Handle required preroll waiting period (#14081)
Authored by: bashonly
2025-08-19 20:06:53 -05:00
bashonly d154dc3dcf [ie/youtube] Remove default player params (#14081)
Closes #13930
Authored by: bashonly
2025-08-19 20:06:53 -05:00
bashonly 438d3f06b3 [fd] Support available_at format field (#13980)
Authored by: bashonly
2025-08-20 00:38:48 +00:00
CasperMcFadden95 74b4b3b005 [ie/faulio] Add extractor (#13907)
Authored by: CasperMcFadden95
2025-08-19 23:27:31 +00:00
e2dk4r 36e873822b [ie/puhutv] Fix playlists extraction (#11955)
Closes #5691
Authored by: e2dk4r
2025-08-19 23:20:07 +00:00
Arseniy D. d3d1ac8eb2 [ie/steam] Fix extractor (#14008)
Closes #14000
Authored by: AzartX47
2025-08-19 23:14:20 +00:00
doe1080 86d74e5cf0 [ie/medialaan] Rework extractors (#14015)
Fixes MedialaanIE and VTMIE

Authored by: doe1080
2025-08-19 23:10:17 +00:00
Junyi Lou 6ca9165648 [ie/bilibili] Handle Bangumi redirection (#14038)
Closes #13924
Authored by: junyilou, grqz

Co-authored-by: _Grqz <173015200+grqz@users.noreply.github.com>
2025-08-19 23:06:49 +00:00
Pierre 82a1390204 [ie/svt] Extract forced subs under separate lang code (#14062)
Closes #14020
Authored by: PierreMesure
2025-08-19 22:57:46 +00:00
Runar Saur Modahl 7540aa1da1 [ie/NRKTVEpisode] Fix extractor (#14065)
Closes #14054
Authored by: runarmod
2025-08-19 22:45:55 +00:00
bashonly 35da8df4f8 [utils] Add improved jwt_encode function (#14071)
Also deprecates `jwt_encode_hs256`

Authored by: bashonly
2025-08-19 22:36:00 +00:00
bashonly 8df121ba59 [ie/mtv] Overhaul extractors (#14052)
Adds SouthParkComBrIE and SouthParkCoUkIE

Removes these extractors:
 - CMTIE: migrated to Paramount+
 - ComedyCentralTVIE: migrated to Paramount+
 - MTVDEIE: migrated to Paramount+
 - MTVItaliaIE: migrated to Paramount+
 - MTVItaliaProgrammaIE: migrated to Paramount+
 - MTVJapanIE: migrated to JP Services
 - MTVServicesEmbeddedIE: dead domain
 - MTVVideoIE: migrated to Paramount+
 - NickBrIE: redirects to landing page w/o any videos
 - NickDeIE: redirects to landing page w/o any videos
 - NickRuIE: redirects to landing page w/o any videos
 - BellatorIE: migrated to PFL
 - ParamountNetworkIE: migrated to Paramount+
 - SouthParkNlIE: site no longer exists
 - TVLandIE: migrated to Paramount+

Closes #169, Closes #1711, Closes #1712, Closes #2621, Closes #3167, Closes #3893, Closes #4552, Closes #4702, Closes #4928, Closes #5249, Closes #6156, Closes #8722, Closes #9896, Closes #10168, Closes #12765, Closes #13446, Closes #14009

Authored by: bashonly, doe1080, Randalix, seproDev

Co-authored-by: doe1080 <98906116+doe1080@users.noreply.github.com>
Co-authored-by: Randalix <23729538+Randalix@users.noreply.github.com>
Co-authored-by: sepro <sepro@sepr0.com>
2025-08-19 20:46:11 +00:00
bashonly 471a2b60e0 [ie/tiktok:user] Improve infinite loop prevention (#14077)
Fix edf55e8184

Closes #14076
Authored by: bashonly
2025-08-19 20:39:33 +00:00
bashonly df0553153e [ie/youtube] Default to main player JS variant (#14079)
Authored by: bashonly
2025-08-19 19:28:15 +00:00
bashonly 7bc53ae799 [ie/youtube] Extract title and description from initial data (#14078)
Closes #13604
Authored by: bashonly
2025-08-19 19:27:17 +00:00
bashonly d8200ff0a4 [ie/vimeo:album] Support embed-only and non-numeric albums (#14021)
Authored by: bashonly
2025-08-18 22:09:35 +00:00
bashonly 0f6b915822 [ie/vimeo:event] Fix extractor (#14064)
Closes #14059
Authored by: bashonly
2025-08-18 21:09:25 +00:00
doe1080 374ea049f5 [ie/niconico:live] Support age-restricted streams (#13549)
Authored by: doe1080
2025-08-18 17:43:40 +00:00
doe1080 6f4c1bb593 [cleanup] Remove dead extractors (#13996)
Removes ArkenaIE, PladformIE, VevoIE, VevoPlaylistIE

Authored by: doe1080
2025-08-18 16:34:32 +00:00
doe1080 c22660aed5 [ie/adobetv] Fix extractor (#13917)
Removes AdobeTVChannelIE, AdobeTVEmbedIE, AdobeTVIE, AdobeTVShowIE

Authored by: doe1080
2025-08-18 16:32:26 +00:00
bashonly 404bd889d0 [ie/weibo] Support more URLs and --no-playlist (#14035)
Authored by: bashonly
2025-08-16 23:02:04 +00:00
bashonly edf55e8184 [ie/tiktok:user] Avoid infinite loop during extraction (#14032)
Closes #14031
Authored by: bashonly
2025-08-16 22:57:14 +00:00
bashonly 8a8861d538 [ie/youtube:tab] Fix playlists tab extraction (#14030)
Closes #14028
Authored by: bashonly
2025-08-16 22:55:21 +00:00
sepro 70f5669951 Warn against using -f mp4 (#13915)
Authored by: seproDev
2025-08-17 00:35:46 +02:00
doe1080 6ae3543d5a [ie] _rta_search: Do not assume age_limit is 0 (#13985)
Authored by: doe1080
2025-08-16 04:28:58 +00:00
doe1080 770119bdd1 [ie] Extract avif storyboard formats from MPD manifests (#14016)
Authored by: doe1080
2025-08-16 03:32:21 +00:00
Arseniy D. 8e3f8065af [ie/weibo] Fix extractors (#14012)
Closes #14012
Authored by: AzartX47, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-08-16 03:07:35 +00:00
bashonly aea85d525e [build] Discontinue darwin_legacy_exe support (#13860)
* Removes "yt-dlp_macos_legacy" from release assets
* Discontinues executable support for macOS < 10.15

Closes #13856
Authored by: bashonly
2025-08-13 22:02:58 +00:00
bashonly f2919bd28e [ie/youtube] Add es5 and es6 player JS variants (#14005)
Authored by: bashonly
2025-08-12 23:24:31 +00:00
bashonly 681ed2153d [build] Bump PyInstaller version to 6.15.0 for Windows (#14002)
Authored by: bashonly
2025-08-12 23:17:13 +00:00
bashonly bdeb3eb3f2 [pp/XAttrMetadata] Only set "Where From" attribute on macOS (#13999)
Fix 3e918d825d

Closes #14004
Authored by: bashonly
2025-08-12 07:58:22 +00:00
github-actions[bot] b7de89c910 Release 2025.08.11
Created by: bashonly

:ci skip all
2025-08-11 03:54:46 +00:00
sepro 5e4ceb35cf [cleanup] Misc (#13852)
Closes #13815
Authored by: seproDev, injust, bashonly

Co-authored-by: Justin Su <injustsu@gmail.com>
Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-08-11 03:18:28 +00:00
bashonly e98695549e [rh:curl_cffi] Support curl_cffi 0.11.x, 0.12.x, 0.13.x (#13989)
Authored by: bashonly
2025-08-11 03:16:07 +00:00
bashonly bf366517ef [ie/youtube] Update player params (#13979)
Closes #13930
Authored by: bashonly
2025-08-10 07:33:45 +00:00
bashonly c76ce28e06 Deprecate linux_armv7l_exe support (#13978)
Ref: https://github.com/yt-dlp/yt-dlp/issues/13976

Authored by: bashonly
2025-08-10 06:53:10 +00:00
Simon Sawicki e8d49b1c7f [ie/motherless] Fix extractor (#13960)
Authored by: Grub4K
2025-08-07 21:04:30 -07:00
Sojiroh a6df5e8a58 [ie/YandexDisk] Support 360 URLs (#13935)
Closes #13887
Authored by: Sojiroh
2025-08-07 23:16:55 +02:00
bashonly e8d2807296 [ie/digitalconcerthall] Fix formats extraction (#13948)
Closes #13925
Authored by: bashonly
2025-08-07 00:03:44 +00:00
bashonly fe53ebe5b6 [fd/dash] Re-extract if using --load-info-json with --live-from-start (#13922)
Closes #13906
Authored by: bashonly
2025-08-06 20:08:34 +00:00
sepro 662af5bb83 Warn when yt-dlp is severely outdated (#13937)
Authored by: seproDev
2025-08-06 21:14:45 +02:00
bashonly 8175f3738f [rh:requests] Bump minimum required version of urllib3 to 2.0.2 (#13939)
- urllib3 1.26.x gives unexpected results with partial reads: https://github.com/urllib3/urllib3/issues/2128
- urllib3 2.0.0 and 2.0.1 were yanked from PyPI: https://github.com/urllib3/urllib3/issues/3009

Closes #13927
Authored by: bashonly
2025-08-06 19:00:53 +00:00
sepro 1e0c77ddcc [pp/XAttrMetadata] Don't write "Where from" on Windows (#13944)
Fix 3e918d825d

Closes #13942
Authored by: seproDev
2025-08-06 16:52:34 +02:00
sepro e651a53a2f Revert f799a4b472 2025-08-05 22:02:13 +02:00
sepro f799a4b472 [ie/youtube] Update tv client config (#13934)
Closes #13930
Authored by: seproDev
2025-08-05 18:47:37 +02:00
coletdjnz 38c2bf4026 [ie/youtube] Add player params to mweb client (#13914)
Authored by: coletdjnz
2025-08-03 13:07:06 +12:00
Iuri Campos 6ff135c319 [ie/shiey] Add extractor (#13354)
Closes #12129
Authored by: iribeirocampos
2025-08-03 00:05:40 +02:00
JChris246 cd31c319e3 [ie/fc2] Fix old video support (#12633)
Closes #11778
Authored by: JChris246, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-08-02 23:37:35 +02:00
u-spec-png 6539ee1947 [ie/N1Info:article] Fix extractor (#13865)
Authored by: u-spec-png
2025-08-02 20:10:40 +00:00
CasperMcFadden95 43dedbe639 [ie/RoyaLive] Support en URLs (#13908)
Authored by: CasperMcFadden95
2025-08-02 19:59:30 +00:00
doe1080 05e553e9d1 [ie/niconico] Fix error handling & improve metadata extraction (#13240)
Closes #13338
Authored by: doe1080
2025-08-02 19:55:08 +00:00
doe1080 1c6068af99 [cleanup] Move embed tests to dedicated extractors (#13782)
Authored by: doe1080
2025-08-01 20:50:20 +00:00
garret1317 71f30921a2 [ie/tbsjp] Fix extractor (#13485)
Closes #13484
Authored by: garret1317
2025-07-31 20:33:05 +00:00
Abdulmohsen 121647705a [ie/TVer] Support --ignore-no-formats-error when geo-blocked (#13598)
Authored by: arabcoders
2025-07-30 23:23:06 +00:00
bashonly 70d7687487 [ie/TVer] Extract Streaks API info (#13885)
Closes #13874
Authored by: bashonly
2025-07-30 23:15:59 +00:00
bashonly 42ca3d601e [ie/archive.org] Fix metadata extraction (#13880)
Closes #13881
Authored by: bashonly
2025-07-30 06:11:09 +00:00
bashonly 62e2a9c0d5 [ci] Bump supported PyPy version to 3.11 (#13877)
Ref: https://pypy.org/posts/2025/07/pypy-v7320-release.html

Authored by: bashonly
2025-07-29 21:31:35 +00:00
bashonly 28b68f6875 [cookies] Load cookies with float expires timestamps (#13873)
Authored by: bashonly
2025-07-29 19:47:28 +00:00
fries1234 682334e4b3 [ie/tvw:news] Add extractor (#12907)
Authored by: fries1234
2025-07-27 22:26:33 +02:00
Florentin Le Moal b831406a1d [ie/rtve.es:program] Add extractor
Authored by: meGAmeS1, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-07-27 21:52:05 +02:00
bashonly 23c658b9cb Raise minimum recommended Python version to 3.10 (#13859)
Ref: https://github.com/yt-dlp/yt-dlp/issues/13858

Authored by: bashonly
2025-07-26 22:59:02 +00:00
bashonly cc5a5caac5 Deprecate darwin_legacy_exe support (#13857)
Ref: https://github.com/yt-dlp/yt-dlp/issues/13856

Authored by: bashonly
2025-07-26 22:12:53 +00:00
bashonly 66aa21dc5a [build] Use macos-14 runner for macos builds (#13814)
Ref: https://github.blog/changelog/2025-07-11-upcoming-changes-to-macos-hosted-runners-macos-latest-migration-and-xcode-support-policy-updates/#macos-13-is-closing-down

Authored by: bashonly
2025-07-26 19:39:54 +00:00
Tom Hebb 57186f958f [fd/hls] Fix --hls-split-continuity support (#13321)
Authored by: tchebb
2025-07-26 20:43:38 +02:00
CasperMcFadden95 daa1859be1 [ie/FaulioLive] Support Bahry TV (#13850)
Authored by: CasperMcFadden95
2025-07-26 20:11:57 +02:00
c-basalt e8c2bf798b [ie/neteasemusic] Support XFF (#11044)
Closes #11043
Authored by: c-basalt
2025-07-26 20:02:56 +02:00
doe1080 1fe83b0111 [ie/eagleplatform] Remove extractors (#13469)
Authored by: doe1080
2025-07-26 17:34:22 +02:00
InvalidUsernameException 30302df22b [ie/sportdeuschland] Support embedded player URLs (#13833)
Closes #13766
Authored by: InvalidUsernameException
2025-07-25 22:22:32 +00:00
CasperMcFadden95 3e609b2ced [ie/FaulioLive] Add extractor (#13421)
Authored by: CasperMcFadden95, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-07-25 23:33:49 +02:00
bashonly d399505fdf [fd/external] Work around ffmpeg's file: URL handling (#13844)
Closes #13781
Authored by: bashonly
2025-07-25 19:44:39 +00:00
sepro 61d4cd0bc0 [ie/PlyrEmbed] Add extractor (#13836)
Closes #13827
Authored by: seproDev
2025-07-25 20:55:41 +02:00
doe1080 4385480795 [utils] parse_resolution: Support width-only pattern (#13802)
Authored by: doe1080
2025-07-25 20:41:21 +02:00
Barry van Oudtshoorn 485de69dbf [ie/Parlview] Rework extractor (#13788)
Closes #13787
Authored by: barryvan
2025-07-25 06:00:31 +02:00
ischmidt20 0adeb1e54b [ie/tbs] Fix truTV support (#9683)
Closes #3400
Authored by: ischmidt20, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-07-24 22:35:48 +00:00
bashonly afaf60d9fd [ie/vimeo] Fix login support and require authentication (#13823)
Closes #13822
Authored by: bashonly
2025-07-23 23:27:20 +00:00
Atsushi2965 7e3f48d64d [pp/EmbedThumbnail] Fix ffmpeg args for embedding in mp3 (#13720)
Authored by: atsushi2965
2025-07-22 21:55:00 +00:00
bashonly 59765ecbc0 [ie/sproutvideo] Fix extractor (#13813)
Authored by: bashonly
2025-07-22 21:46:46 +00:00
bashonly c59ad2b066 [utils] random_user_agent: Bump versions (#13543)
Closes #5362
Authored by: bashonly
2025-07-22 21:34:03 +00:00
Simon Sawicki eed94c7306 [utils] Add WINDOWS_VT_MODE to globals (#12460)
Authored by: Grub4K
2025-07-22 20:10:51 +02:00
Roland Crosby 3e918d825d [pp/XAttrMetadata] Add macOS "Where from" attribute (#12664)
Authored by: rolandcrosby
2025-07-22 19:50:42 +02:00
github-actions[bot] 035b1ece8f Release 2025.07.21
Created by: bashonly

:ci skip all
2025-07-21 23:47:12 +00:00
sepro 9951fdd0d0 [cleanup] Misc (#13595)
Closes #10853, Closes #12436, Closes #13314, Closes #13609
Authored by: seproDev, InvalidUsernameException, doe1080, hseg, bashonly, adamralph

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
Co-authored-by: InvalidUsernameException <InvalidUsernameException@users.noreply.github.com>
Co-authored-by: gesh <gesh@gesh.uni.cx>
Co-authored-by: Adam Ralph <adam@adamralph.com>
Co-authored-by: doe1080 <98906116+doe1080@users.noreply.github.com>
2025-07-21 23:43:30 +00:00
Simon Sawicki 959ac99e98 Fix --exec placeholder expansion on Windows
See https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-45hg-7f49-5h56 for more details

Authored by: Grub4K
2025-07-21 18:19:46 -05:00
bashonly d88b304d44 [ie/patreon:campaign] Fix extractor (#13712)
Closes #13622
Authored by: bashonly
2025-07-21 23:15:31 +00:00
bashonly b15aa8d772 [ie/BiliBiliBangumi] Fix extractor (#13800)
Closes #13795
Authored by: bashonly
2025-07-21 23:11:58 +00:00
c-basalt d3edc5d52a [ie/bilibili] Pass newer user-agent with API requests (#13736)
Closes #12887
Authored by: c-basalt
2025-07-21 23:04:43 +00:00
doe1080 060c6a4501 [ie/skeb] Rework extractor (#13593)
Closes #7440
Authored by: doe1080
2025-07-21 22:32:10 +00:00
doe1080 6be26626f7 [utils] unified_timestamp: Return int values (#13796)
Authored by: doe1080
2025-07-21 21:59:13 +00:00
bashonly ef103b2d11 [ie/hotstar] Fix error handling (#13793)
Fix 7e0af2b1f0

Closes #13790
Authored by: bashonly
2025-07-21 19:09:52 +00:00
bashonly 3e49bc8a1b Make extractor-designated impersonation override --impersonate (#13792)
Fix 32809eb2da

Authored by: bashonly
2025-07-21 18:42:21 +00:00
bashonly 2ac3eb9837 Fix ImpersonateTarget sanitization (#13791)
Fix 32809eb2da

Authored by: bashonly
2025-07-21 18:41:00 +00:00
bashonly 8820101aa3 [ie/youtube] Use impersonation for downloading subtitles (#13786)
Closes #13770
Authored by: bashonly
2025-07-20 23:22:04 +00:00
bashonly a4561c7a66 [rh:requests] Refactor default headers (#13785)
Authored by: bashonly
2025-07-20 23:20:58 +00:00
bashonly 32809eb2da Allow extractors to designate formats/subtitles for impersonation (#13778)
Authored by: bashonly
2025-07-20 23:05:43 +00:00
WouterGordts f9dff95cb1 [ie/bandcamp] Extract tags (#13480)
Authored by: WouterGordts
2025-07-20 20:12:40 +00:00
Tim 790c286ce3 [ie/10play] Support new site domain (#13611)
Closes #13577
Authored by: Georift
2025-07-20 20:00:44 +00:00
bashonly 87e3dc8c7f [ie/mlbtv] Make formats downloadable with ffmpeg (#13761)
Authored by: bashonly
2025-07-20 19:57:20 +00:00
R0hanW 1a8474c3ca [ie/PlayerFm] Add extractor (#13016)
Closes #4518
Authored by: R0hanW
2025-07-19 01:38:52 +02:00
bashonly 09982bc33e [ie/dangalplay] Support other login regions (#13768)
Authored by: bashonly
2025-07-18 23:24:52 +00:00
Víctor Schmidt c8329fc572 [ie/rai] Fix formats extraction (#13572)
Closes #13548
Authored by: moonshinerd, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-07-18 22:43:04 +00:00
bashonly 1f27a9f8ba [core] Warn when skipping formats (#13090)
Authored by: bashonly
2025-07-18 21:59:50 +00:00
bashonly 4919051e44 [core] Don't let format testing alter the return code (#13767)
Closes #13750
Authored by: bashonly
2025-07-18 21:55:02 +00:00
bashonly 5f951ce929 [ie/aenetworks] Support new URL formats (#13747)
Closes #13745
Authored by: bashonly
2025-07-18 20:06:02 +00:00
bashonly 28bf46b7da [utils] urlhandle_detect_ext: Use x-amz-meta-file-type headers (#13749)
Authored by: bashonly
2025-07-18 19:46:06 +00:00
bashonly b8abd255e4 [utils] mimetype2ext: Always parse flac from audio/flac (#13748)
Authored by: bashonly
2025-07-18 19:43:40 +00:00
bashonly c1ac543c81 [ie/soundcloud] Always extract original format extension (#13746)
Closes #13743
Authored by: bashonly
2025-07-16 23:19:58 +00:00
flanter21 dcc4cba39e [ie/blackboardcollaborate] Support subtitles and authwalled videos (#12473)
Authored by: flanter21
2025-07-16 23:17:48 +00:00
Nikolay Fedorov 3a84be9d16 [ie/TheHighWire] Add extractor (#13505)
Closes #13364
Authored by: swayll
2025-07-14 19:01:53 +00:00
rdamas d42a6ff0c4 [ie/archive.org] Fix extractor (#13706)
Closes #13704
Authored by: rdamas
2025-07-14 18:55:52 +00:00
bashonly ade876efb3 [ie/francetv] Improve error handling (#13726)
Closes #13324
Authored by: bashonly
2025-07-14 17:25:45 +00:00
bashonly 7e0af2b1f0 [ie/hotstar] Improve error handling (#13727)
Authored by: bashonly
2025-07-14 17:24:52 +00:00
doe1080 d57a0b5aa7 [ie/noovo] Remove extractor (#13429)
Authored by: doe1080
2025-07-14 01:12:00 +02:00
doe1080 6fb3947c0d [ie/bellmedia] Remove extractor (#13429)
Authored by: doe1080
2025-07-14 01:12:00 +02:00
doe1080 9f54ea3898 [ie/ctv] Remove extractor (#13429)
Authored by: doe1080
2025-07-14 01:12:00 +02:00
chauhantirth 07d1d85f63 [ie/hotstar] Fix support for free accounts (#13700)
Fixes b5bd057fe8

Closes #13600
Authored by: chauhantirth
2025-07-13 22:35:26 +00:00
doe1080 5d693446e8 [ie/limelight] Remove extractors (#13267)
Authored by: doe1080
2025-07-14 00:10:59 +02:00
doe1080 23e9389f93 [ie/bandaichannel] Remove extractor (#13152)
Closes #8829
Authored by: doe1080
2025-07-13 23:53:47 +02:00
doe1080 6d39c420f7 [ie/JoqrAg] Remove extractor (#13152)
Authored by: doe1080
2025-07-13 23:53:47 +02:00
barsnick 85c3fa1925 [ie/RaiSudtirol] Support alternative domain (#13718)
Authored by: barsnick
2025-07-13 23:35:10 +02:00
Povilas Balzaravičius b4b4486eff [ie/LRTRadio] Fix extractor (#13717)
Authored by: Pawka
2025-07-13 21:24:37 +00:00
Frank Cai 630f3389c3 [ie/UnitedNationsWebTv] Add extractor (#13538)
Closes #2675
Authored by: averageFOSSenjoyer
2025-07-13 23:16:01 +02:00
bashonly a6db1d297a [ie/vimeo] Handle age-restricted videos (#13719)
Closes #13716
Authored by: bashonly
2025-07-13 21:09:39 +00:00
ShockedPlot7560 0f33950c77 [ie/mixlr] Add extractors (#13561)
Authored by: ShockedPlot7560, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-07-13 01:35:51 +02:00
bashonly b5fea53f20 [ie] Rework _search_nextjs_v13_data helper (#13711)
Fix 5245231e4a

Authored by: bashonly
2025-07-12 23:12:05 +00:00
bashonly 5245231e4a [ie] Add _search_nextjs_v13_data helper (#13398)
* Fixes FranceTVSiteIE livestream extraction
* Fixes GoPlayIE metadata extraction

Authored by: bashonly
2025-07-12 22:12:46 +00:00
Lyuben Ivanov 3ae61e0f31 [ie/BTVPlus] Add extractor (#13541)
Authored by: bubo
2025-07-12 21:56:11 +02:00
bashonly a5d697f62d [ie/vimeo] Fix extractor (#13692)
Closes #13180, Closes #13689
Authored by: bashonly
2025-07-12 19:23:22 +00:00
coletdjnz 6e5bee418b [ie/youtube] Ensure context params are consistent for web clients (#13701)
Authored by: coletdjnz
2025-07-12 13:44:27 +12:00
coletdjnz 5b57b72c1a [ie/youtube] Do not require PO Token for premium accounts (#13640)
Authored by: coletdjnz
2025-07-11 18:54:01 +12:00
doe1080 2aaf1aa71d [ie/newspicks] Fix extractor (#13612)
Closes #10472
Authored by: doe1080
2025-07-09 22:21:47 +00:00
Nikolay Fedorov 7b4c96e089 [ie/mir24.tv] Add extractor (#13651)
Closes #13365
Authored by: swayll
2025-07-09 22:16:33 +00:00
bashonly 0b359b184d [ie/9gag] Support browser impersonation (#13678)
Closes #10837
Authored by: bashonly
2025-07-09 21:58:19 +00:00
bashonly 805519bfaa [jsinterp] Fix undefined variable name caching (#13677)
Fix b342d27f3f

Authored by: bashonly
2025-07-09 20:45:47 +00:00
coletdjnz aa9f1f4d57 [ie/youtube] Log bad playability statuses of player responses (#13647)
Authored by: coletdjnz
2025-07-09 18:29:54 +12:00
InvalidUsernameException fd36b8f31b [test:download] Support playlist_maxcount (#13433)
Authored by: InvalidUsernameException
2025-07-08 04:19:03 +00:00
barsnick 99093e96fd [devscripts] Fix filename/directory Bash completions (#13620)
Closes #13619
Authored by: barsnick
2025-07-08 04:18:15 +00:00
garret1317 7c49a93788 [ie/NhkRadiru] Fix metadata extraction (#12708)
Authored by: garret1317
2025-07-08 03:55:19 +00:00
bashonly 884f35d54a [ie/BiliBiliBangumi] Fix geo-block detection (#13667)
Closes #13634
Authored by: bashonly
2025-07-08 03:54:27 +00:00
bashonly c23d837b65 [ie/youtube:tab] Fix subscriptions feed extraction (#13665)
Adds support for LOCKUP_CONTENT_TYPE_VIDEO view models

Closes #13658
Authored by: bashonly
2025-07-07 20:25:34 +00:00
bashonly a7113722ec [fd/hls] Do not fall back to ffmpeg when native is required (#13655)
Authored by: bashonly
2025-07-06 22:14:22 +00:00
bashonly 0e68332bcb [ie/youtube] Fix subtitles extraction (#13659)
Fixes regression introduced in 2ba5391cd6

Closes #13654
Authored by: bashonly
2025-07-06 22:07:21 +00:00
bashonly 422cc8cb2f [ie/twitch] Improve error handling (#13618)
Authored by: bashonly
2025-07-06 22:03:34 +00:00
bashonly fca94ac5d6 [ie/youtube] Extract global nsig helper functions (#13639)
Authored by: bashonly, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-07-05 18:23:15 -05:00
bashonly b342d27f3f [jsinterp] Cache undefined variable names (#13639)
Authored by: bashonly
2025-07-05 18:23:15 -05:00
bashonly b6328ca050 [jsinterp] Fix variable scoping (#13639)
Authored by: bashonly, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-07-05 18:23:15 -05:00
bashonly 0b41746964 [ie/sproutvideo] Fix extractor (#13610)
Closes #13606
Authored by: bashonly
2025-07-02 13:21:06 +00:00
Simon Sawicki c316416b97 [rh:requests] Do not allocate 2GB on read (#13603)
Fixes c2ff2dbaec

Authored by: Grub4K
2025-07-02 01:42:00 +02:00
Simon Sawicki e99c0b838a [ie] Detect invalid m3u8 playlist data (#13601)
Authored by: Grub4K
2025-07-02 00:32:32 +02:00
Simon Sawicki c2ff2dbaec [rh:requests] Work around partial read dropping data (#13599)
Authored by: Grub4K
2025-07-02 00:12:43 +02:00
sepro ca5cce5b07 [cleanup] Bump ruff to 0.12.x (#13596)
Authored by: seproDev
2025-07-01 21:17:11 +02:00
sepro f3008bc5f8 No longer enable --mtime by default (#12781)
Closes #12780
Authored by: seproDev
2025-07-01 13:23:53 +02:00
github-actions[bot] 30fa54280b Release 2025.06.30
Created by: bashonly

:ci skip all
2025-06-30 23:47:20 +00:00
bashonly b018784498 [cleanup] Misc (#13590)
Authored by: bashonly
2025-06-30 23:44:42 +00:00
bashonly 11b9416e10 [ie/sproutvideo] Support browser impersonation (#13589)
Closes #13576
Authored by: bashonly
2025-06-30 23:37:56 +00:00
Clark 35fc33fbc5 [ie/sauceplus] Add extractor (#13567)
Authored by: ceandreasen, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-06-30 23:25:28 +00:00
helpimnotdrowning b16722ede8 [ie/kick] Support subscriber-only content (#13550)
Closes #13442
Authored by: helpimnotdrowning
2025-06-30 23:24:04 +00:00
bashonly 500761e41a [ie] Fix m3u8 playlist data corruption (#13588)
Revert 7b81634fb1

Closes #13581
Authored by: bashonly
2025-06-30 23:06:22 +00:00
bashonly 2ba5391cd6 [ie/youtube] Fix premium formats extraction (#13586)
Fix ff6f94041a

Closes #13545
Authored by: bashonly
2025-06-30 23:02:59 +00:00
bashonly e9f157669e [ie/hotstar] Fix formats extraction (#13585)
Fix b5bd057fe8

Authored by: bashonly
2025-06-30 19:19:43 +00:00
sepro 958153a226 [jsinterp] Fix extract_object (#13580)
Fixes sig extraction for YouTube player `e12fbea4`

Authored by: seproDev
2025-06-30 15:50:33 +02:00
bashonly 1b88384634 [ci] Add signature tests (#13582)
Authored by: bashonly
2025-06-30 13:05:52 +00:00
Simon Sawicki 7b81634fb1 [ie] Detect invalid m3u8 playlist data (#13563)
Authored by: Grub4K
2025-06-29 18:49:27 +02:00
bashonly 7e2504f941 [ie/jiocinema] Remove extractors (#13565)
Closes #10123, Closes #10144, Closes #10225, Closes #10240, Closes #10508
Authored by: bashonly
2025-06-28 23:32:21 +00:00
bashonly 4bd9a7ade7 [ie/hotstar:series] Fix extractor (#13564)
* Removes HotStarSeasonIE and HotStarPlaylistIE

Authored by: bashonly
2025-06-28 23:30:51 +00:00
chauhantirth b5bd057fe8 [ie/hotstar] Fix extractor (#13530)
Closes #11195
Authored by: chauhantirth, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-06-28 02:29:43 +00:00
bashonly 5e292baad6 [ie/hotstar] Raise for login required (#10405)
Closes #10366
Authored by: bashonly
2025-06-27 22:31:06 +00:00
bashonly 0a6b104489 [ie/hotstar] Fix metadata extraction (#13560)
Closes #7946
Authored by: bashonly
2025-06-27 22:29:37 +00:00
doe1080 06c1a8cdff [ie/niconico:live] Fix extractor and downloader (#13158)
Authored by: doe1080
2025-06-26 17:45:03 +00:00
c-basalt 99b85ac102 [ie/BilibiliSpaceVideo] Extract hidden-mode collections as playlists (#13533)
Closes #13435
Authored by: c-basalt
2025-06-26 17:42:41 +00:00
github-actions[bot] eff0759705 Release 2025.06.25
Created by: bashonly

:ci skip all
2025-06-25 23:53:38 +00:00
Anton Larionov 1838a1ce5d [ie/mave] Add extractor (#13380)
Authored by: anlar
2025-06-25 23:51:20 +00:00
doe1080 2600849bad [ie/huya:live] Fix extractor (#13520)
Authored by: doe1080
2025-06-25 23:37:49 +00:00
D Trombett 3bd3029160 [ie/tv8.it] Support slugless URLs (#13478)
Authored by: DTrombett
2025-06-25 23:26:23 +00:00
D Trombett a4ce4327c9 [ie/SportDeutschland] Fix extractor (#13519)
Closes #13518
Authored by: DTrombett
2025-06-25 23:24:39 +00:00
Cæsim c57412d1f9 [ie/lsm] Fix extractors (#13126)
Closes #12298
Authored by: Caesim404
2025-06-25 19:24:20 +00:00
bashonly 5b559d0072 [ie/sproutvideo] Fix extractor (#13544)
Closes #13540
Authored by: bashonly
2025-06-25 19:02:37 +00:00
bashonly 8f94b76cbf [ie/youtube] Check any ios m3u8 formats prior to download (#13524)
Closes #13511
Authored by: bashonly
2025-06-25 18:32:57 +00:00
bashonly ff6f94041a [ie/youtube] Improve player context payloads (#13539)
Closes #12563
Authored by: bashonly
2025-06-25 17:10:00 +00:00
Simon Sawicki 73bf102116 [test] traversal: Fix morsel tests for Python 3.14 (#13471)
Authored by: Grub4K
2025-06-17 09:45:19 +02:00
doe1080 1722c55400 [ie/hypergryph] Improve metadata extraction (#13415)
Closes #13384
Authored by: doe1080, eason1478

Co-authored-by: eason1478 <134664337+eason1478@users.noreply.github.com>
2025-06-12 23:25:08 +00:00
doe1080 e6bd4a3da2 [ie/brightcove:new] Improve metadata extraction (#13461)
Authored by: doe1080
2025-06-12 23:16:48 +00:00
bashonly 51887484e4 [ie] Add _search_nuxt_json helper (#13386)
* Adds InfoExtractor._search_nuxt_json for webpage extraction
* Adds InfoExtractor._resolve_nuxt_array for direct use with payload JSON
* Adds yt_dlp.utils.jslib module for Python solutions to common JavaScript libraries
* Adds devalue.parse and devalue.parse_iter to jslib utils

Ref:
* https://github.com/nuxt/nuxt/commit/9e503be0f2a24f4df72a3ccab2db4d3e63511f57
* https://github.com/Rich-Harris/devalue/blob/f3fd2aa93d79f21746555671f955a897335edb1b/src/parse.js

Authored by: bashonly, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.dev>
2025-06-12 22:15:01 +00:00
github-actions[bot] ba090caeaa Release 2025.06.09
Created by: bashonly

:ci skip all
2025-06-09 23:41:52 +00:00
bashonly 339614a173 [cleanup] Misc (#13278)
Authored by: bashonly
2025-06-09 23:39:00 +00:00
nullpos aa863ddab9 [ie/cu.ntv.co.jp] Fix extractor (#13302)
Closes #10976
Authored by: nullpos, doe1080

Co-authored-by: doe1080 <98906116+doe1080@users.noreply.github.com>
2025-06-08 00:45:32 +00:00
InvalidUsernameException db162b76f6 [ie/zdf] Fix language extraction and format sorting (#13313)
Closes #13118
Authored by: InvalidUsernameException
2025-06-08 00:10:01 +00:00
doe1080 e3c605a61f [ie/sr:mediathek] Improve metadata extraction (#13294)
Authored by: doe1080
2025-06-08 00:06:57 +00:00
doe1080 97ddfefeb4 [ie/nobelprize] Fix extractor (#13205)
Authored by: doe1080
2025-06-08 00:04:32 +00:00
doe1080 a8bf0011bd [ie/startrek] Fix extractor (#13188)
Authored by: doe1080
2025-06-07 23:16:31 +00:00
c-basalt 13e5516271 [ie/BiliBiliBangumi] Fix extractor (#13416)
Closes #13121
Authored by: c-basalt
2025-06-07 23:14:57 +00:00
bashonly 03dba2012d [ie/telecinco] Fix extractor (#13379)
Closes #13378
Authored by: bashonly
2025-06-06 22:02:26 +00:00
bashonly 5d96527be8 [ie/stacommu] Avoid partial stream formats (#13412)
Authored by: bashonly
2025-06-06 21:53:30 +00:00
gamer191 1fd0e88b67 [ie/youtube] Add tv_simply player client (#13389)
Authored by: gamer191
2025-06-06 21:50:36 +00:00
gamer191 231349786e [ie/youtube] Extract srt subtitles (#13411)
Closes #1734
Authored by: gamer191
2025-06-06 19:32:03 +00:00
Sipherdrakon f37d599a69 [ie/aenetworks] Fix playlist extractors (#13408)
Fix 41952255d1

Authored by: Sipherdrakon
2025-06-06 09:50:21 +00:00
Simon Sawicki 9e38b273b7 [ie/youtube] Rework nsig function name extraction (#13403)
Closes #13401

Authored by: Grub4K
2025-06-05 23:50:58 +02:00
doe1080 4e7c1ea346 [ie/umg:de] Rework extractor (#13373)
Authored by: doe1080
2025-06-03 19:20:46 +00:00
barsnick e1b6062f8c [ie/svt:play] Fix extractor (#13329)
Closes #13312
Authored by: barsnick, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-06-03 02:29:03 +00:00
bashonly c723c4e5e7 [ie/vimeo] Extract subtitles from player subdomain (#13350)
Closes #12198
Authored by: bashonly
2025-06-01 23:20:29 +00:00
bashonly 148a1eb4c5 [ie/odnoklassniki] Detect and raise when login is required (#13361)
Closes #13360
Authored by: bashonly
2025-06-01 23:18:24 +00:00
bashonly 85c8a405e3 [ie] Improve JSON LD thumbnails extraction (#13368)
Authored by: bashonly, doe1080

Co-authored-by: doe1080 <98906116+doe1080@users.noreply.github.com>
2025-06-01 23:09:47 +00:00
Sipherdrakon 943083edcd [ie/adobepass] Fix Philo MSO authentication (#13335)
Closes #2603
Authored by: Sipherdrakon
2025-06-01 17:26:33 +00:00
bashonly 3fe72e9eea [ie/weverse] Support login with oauth refresh tokens (#13284)
Closes #7806
Authored by: bashonly
2025-05-30 23:20:59 +00:00
bashonly d30a49742c [ie/youtube] Improve signature extraction debug output (#13327)
Authored by: bashonly
2025-05-30 23:16:47 +00:00
bashonly 6d265388c6 [ie/10play] Fix extractor (#13349)
Closes #12337
Authored by: bashonly
2025-05-30 22:51:25 +00:00
bashonly a9b3700698 [test:postprocessors] Remove binary thumbnail test data (#13341)
Authored by: bashonly
2025-05-30 22:48:48 +00:00
bashonly 201812100f [build] Fix macOS requirements caching (#13328)
Authored by: bashonly
2025-05-28 18:13:48 +00:00
bashonly cc749a8a3b [build] Exclude pkg_resources from being collected (#13320)
Closes #13311
Authored by: bashonly
2025-05-27 23:11:58 +00:00
bashonly f7bbf5a617 [ie/youtube] nsig code improvements and cleanup (#13280)
Authored by: bashonly
2025-05-26 22:54:43 +00:00
Brian b5be29fa58 [ie/youtube] Fix --mark-watched support (#13222)
Closes #11532
Authored by: iednod55, brian6932

Co-authored-by: iednod55 <210167282+iednod55@users.noreply.github.com>
2025-05-26 22:31:22 +00:00
bashonly 6121559e02 [ie/vice] Mark extractors as broken (#13131)
Authored by: bashonly
2025-05-26 15:57:19 -05:00
Max 2e5bf002da [ie/go] Fix provider-locked content extraction (#13131)
Closes #1770, Closes #8073
Authored by: maxbin123, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-05-26 15:57:19 -05:00
Max 6693d66033 [ie/aenetworks] Fix provider-locked content extraction (#13131)
Authored by: maxbin123
2025-05-26 15:57:19 -05:00
Max b094747e93 [ie/WatchESPN] Fix provider-locked content extraction (#13131)
Closes #4662
Authored by: maxbin123
2025-05-26 15:57:19 -05:00
bashonly 98f8eec956 [ie/brightcove:new] Adapt to new AdobePass requirement (#13131)
Authored by: bashonly
2025-05-26 15:57:19 -05:00
bashonly 0daddc780d [ie/turner] Adapt extractors to new AdobePass flow (#13131)
Authored by: bashonly
2025-05-26 15:57:19 -05:00
bashonly 2d7949d564 [ie/nbc] Rework and adapt extractors to new AdobePass flow (#13131)
Closes #1032, Closes #10874, Closes #11148, Closes #12432
Authored by: bashonly
2025-05-26 15:57:19 -05:00
bashonly ed108b3ea4 [ie/theplatform] Improve metadata extraction (#13131)
Authored by: bashonly
2025-05-26 15:57:19 -05:00
Max eee90acc47 [ie/adobepass] Add Fubo MSO (#13131)
Closes #8287
Authored by: maxbin123
2025-05-26 15:57:19 -05:00
Max 711c5d5d09 [ie/adobepass] Rework to require software statement (#13131)
* Also removes broken cookie support

Closes #11811
Authored by: maxbin123, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-05-26 15:57:19 -05:00
bashonly 89c1b349ad [ie/adobepass] Validate login URL before sending credentials (#13131)
Authored by: bashonly
2025-05-26 15:57:19 -05:00
bashonly 0ee1102268 [ie/adobepass] Always add newer user-agent when required (#13131)
Fix dcfeea4dd5

Closes #516
Authored by: bashonly
2025-05-26 15:57:19 -05:00
doe1080 7794374de8 [ie/twitter:broadcast] Support events URLs (#13248)
Closes #12989
Authored by: doe1080
2025-05-23 19:25:56 +00:00
bashonly 538eb30567 [ie/podchaser] Fix extractor (#13271)
Closes #13269
Authored by: bashonly
2025-05-23 17:42:24 +00:00
doe1080 f8051e3a61 [ie/toutiao] Add extractor (#13246)
Closes #12125
Authored by: doe1080
2025-05-23 17:29:55 +00:00
bashonly 52f9729c9a [ie/twitcasting] Fix password-protected livestream support (#13097)
Closes #13096
Authored by: bashonly
2025-05-23 12:58:53 +00:00
bashonly 1a8a03ea8d [ie/patreon] Fix referer header used for embeds (#13276)
Fix e0d6c08229

Closes #13263
Authored by: bashonly
2025-05-23 12:53:36 +00:00
bashonly e0d6c08229 [ie/patreon] Fix m3u8 formats extraction (#13266)
Closes #13263
Authored by: bashonly
2025-05-22 22:42:42 +00:00
bashonly 53ea743a9c [ie/youtube] Fix automatic captions for some client combinations (#13268)
Fix 32ed5f107c

Authored by: bashonly
2025-05-22 22:41:31 +00:00
github-actions[bot] 415b4c9f95 Release 2025.05.22
Created by: bashonly

:ci skip all
2025-05-22 09:49:11 +00:00
bashonly 7977b329ed [cleanup] Misc (#13166)
Authored by: bashonly
2025-05-22 09:33:11 +00:00
Matt Broadway e491fd4d09 [cookies] Fix Linux desktop environment detection (#13197)
Closes #12885
Authored by: mbway
2025-05-22 09:22:11 +00:00
bashonly 32ed5f107c [ie/youtube] Add PO token support for subtitles (#13234)
Closes #13075
Authored by: bashonly, coletdjnz

Co-authored-by: coletdjnz <coletdjnz@protonmail.com>
2025-05-22 09:13:42 +00:00
sepro 167d7a9f0f [jsinterp] Fix increment/decrement evaluation (#13238)
Closes #13241
Authored by: seproDev, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-05-21 22:27:07 +00:00
garret1317 83fabf3524 [ie/xinpianchang] Fix extractor (#13245)
Closes #12737
Authored by: garret1317
2025-05-21 18:29:35 +00:00
bashonly 00b1bec552 [ie/twitch] Support --live-from-start (#13202)
Closes #10520
Authored by: bashonly
2025-05-20 21:53:54 +00:00
Yasin Özel c7e575e316 [ie/youtube] Fix geo-restriction error handling (#13217)
Authored by: yozel
2025-05-20 21:39:27 +00:00
Subrat Lima 31e090cb78 [ie/picarto:vod] Support /profile/ video URLs (#13227)
Closes #13192
Authored by: subrat-lima
2025-05-20 21:37:21 +00:00
bashonly 545c1a5b6f [ie/vimeo:event] Add extractor (#13216)
Closes #1608
Authored by: bashonly
2025-05-20 18:28:34 +00:00
bashonly f569be4602 [ie/niconico] Fix error handling (#13236)
Closes #11430
Authored by: bashonly
2025-05-20 18:25:27 +00:00
coletdjnz 2685654a37 [ie/youtube] Add a PO Token Provider Framework (#12840)
https://github.com/yt-dlp/yt-dlp/tree/master/yt_dlp/extractor/youtube/pot/README.md

Authored by: coletdjnz
2025-05-18 13:45:26 +12:00
Povilas Balzaravičius abf58dcd6a [ie/LRTRadio] Fix extractor (#13200)
Authored by: Pawka
2025-05-17 20:37:00 +00:00
Geoffrey Frogeye 20f288bdc2 [ie/nebula] Support --mark-watched (#13120)
Authored by: GeoffreyFrogeye
2025-05-16 23:24:30 +00:00
bashonly f475e8b529 [ie/once] Remove extractor (#13164)
Authored by: bashonly
2025-05-16 23:16:58 +00:00
bashonly 41c0a1fb89 [ie/1tv] Fix extractor (#13168)
Closes #13167
Authored by: bashonly
2025-05-16 23:16:03 +00:00
Jan Baier a7d9a5eb79 [ie/iprima] Fix login support (#12937)
Closes #12387
Authored by: baierjan
2025-05-16 23:04:24 +00:00
Subrat Lima 586b557b12 [ie/jiosaavn:artist] Add extractor (#12803)
Closes #10823

Authored by: subrat-lima
2025-05-11 03:01:13 -05:00
Subrat Lima 317f4b8006 [ie/jiosaavn:show:playlist] Add extractor (#12803)
Closes #12766

Authored by: subrat-lima
2025-05-11 03:01:13 -05:00
Subrat Lima 6839276496 [ie/jiosaavn:show] Add extractor (#12803)
Closes #12766

Authored by: subrat-lima
2025-05-11 03:01:13 -05:00
bashonly cbcfe6378d [ie/sprout] Remove extractor (#13149)
Authored by: bashonly
2025-05-10 23:22:53 +00:00
bashonly 7dbb47f84f [ie/cartoonnetwork] Remove extractor (#13148)
Authored by: bashonly
2025-05-10 23:22:38 +00:00
bashonly 464c84fedf [ie/amcnetworks] Fix extractor (#13147)
Authored by: bashonly
2025-05-10 23:15:12 +00:00
doe1080 7a7b85c901 [ie/niconico:live] Fix extractor (#13045)
Authored by: doe1080
2025-05-10 22:46:28 +00:00
v3DJG6GL d880e06080 [ie/playsuisse] Improve metadata extraction (#12466)
Authored by: v3DJG6GL
2025-05-10 22:37:04 +00:00
bashonly ded11ebc9a [ie/youtube] Extract media_type for all videos (#13136)
Authored by: bashonly
2025-05-10 22:33:57 +00:00
diman8 ea8498ed53 [ie/SVTPage] Fix extractor (#12957)
Closes #13142
Authored by: diman8
2025-05-10 08:53:59 +00:00
bashonly b26bc32579 [ie/nytimesarticle] Fix extraction (#13104)
Closes #13098
Authored by: bashonly
2025-05-06 20:32:41 +00:00
bashonly f123cc83b3 [ie/wat.tv] Improve error handling (#13111)
Closes #8191
Authored by: bashonly
2025-05-05 15:03:07 +00:00
bashonly 0feec6dc13 [ie/youtube] Add web_embedded client for age-restricted videos (#13089)
Authored by: bashonly
2025-05-03 20:11:40 +00:00
bashonly 1d0f6539c4 [ie/bitchute] Fix extractor (#13081)
Closes #13080
Authored by: bashonly
2025-05-03 19:31:33 +00:00
bashonly 17cf9088d0 [build] Bump PyInstaller to v6.13.0 (#13082)
Ref: https://github.com/yt-dlp/yt-dlp/issues/10294

Authored by: bashonly
2025-05-03 17:10:31 +00:00
bashonly 9064d2482d [build] Bump run-on-arch-action to v3 (#13088)
Authored by: bashonly
2025-05-03 17:08:24 +00:00
Abdulmohsen 8f303afb43 [ie/youtube] Fix --live-from-start support for premieres (#13079)
Closes #8543
Authored by: arabcoders
2025-05-03 15:23:28 +00:00
bashonly 5328eda882 [ie/weverse] Fix live extraction (#13084)
Closes #12883
Authored by: bashonly
2025-05-03 07:19:52 +00:00
github-actions[bot] b77e5a553a Release 2025.04.30
Created by: bashonly

:ci skip all
2025-04-30 23:24:48 +00:00
sepro 505b400795 [cleanup] Misc (#12844)
Authored by: seproDev, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-04-30 23:01:25 +00:00
bashonly 74fc2ae12c [ie/youtube] Do not strictly deprioritize missing_pot formats (#13061)
Deprioritization was redundant; they're already hidden behind an extractor-arg

Authored by: bashonly
2025-04-30 22:51:40 +00:00
InvalidUsernameException 7be14109a6 [ie/zdf] Fix extractors (#12779)
Closes #12647
Authored by: InvalidUsernameException, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-04-30 22:27:42 +00:00
bashonly 61c9a938b3 [ie/youtube] Cache signature timestamps (#13047)
Closes #12825
Authored by: bashonly
2025-04-30 01:15:17 +00:00
bashonly fd8394bc50 [ie/youtube] Improve warning for SABR-only/SSAP player responses (#13049)
Ref: https://github.com/yt-dlp/yt-dlp/issues/12482

Authored by: bashonly
2025-04-30 01:13:35 +00:00
bashonly 22ac81a069 [ie/vimeo] Extract from mobile API (#13034)
Closes #12974
Authored by: bashonly
2025-04-29 16:45:54 +00:00
doe1080 25cd7c1ecb [ie/niconico] Fix login support (#13008)
Authored by: doe1080
2025-04-28 22:42:01 +00:00
bashonly 28f04e8a5e [ie/reddit] Support --ignore-no-formats-error (#12993)
Closes #12987
Authored by: bashonly
2025-04-28 22:31:34 +00:00
sepro a3e91df30a [ie/TV2DK] Fix extractor (#12945)
Closes #10334
Authored by: seproDev, bashonly

Co-authored-by: bashonly <bashonly@protonmail.com>
2025-04-29 00:21:54 +02:00
bashonly 80736b9c90 [ie/bpb] Fix formats extraction (#13015)
Closes #13011
Authored by: bashonly
2025-04-28 22:20:39 +00:00
Sergei Zharkov 1ae6bff564 [ie/twitch:clips] Fix uploader metadata extraction (#13022)
Fix 61046c3161

Authored by: 1271
2025-04-28 22:19:14 +00:00
sepro b37ff4de5b [ie/linkedin:events] Add extractor (#12926)
Authored by: seproDev, bashonly

Co-authored-by: bashonly <bashonly@protonmail.com>
2025-04-28 22:58:30 +02:00
Simon Sawicki 3690e91265 [ci] Add file mode test to code check (#13036)
Authored by: Grub4K
2025-04-28 21:21:06 +02:00
coletdjnz 8cb08028f5 [ie/youtube] Detect and warn when account cookies are rotated (#13014)
Related: https://github.com/yt-dlp/yt-dlp/issues/8227

Authored by: coletdjnz
2025-04-27 12:16:34 +12:00
bashonly 1cf39ddf3d [ie/twitter] Fix extraction when logged-in (#13024)
Closes #13010
Authored by: bashonly
2025-04-26 22:39:29 +00:00
bashonly c2d6659d10 [ie/youtube] Detect player JS variants for any locale (#13003)
Authored by: bashonly
2025-04-26 22:08:34 +00:00
coletdjnz 26feac3dd1 [ie/youtube] Add context to video request rate limit error (#12958)
Related: https://github.com/yt-dlp/yt-dlp/issues/11426

Authored by: coletdjnz
2025-04-25 16:11:07 +12:00
doe1080 70599e53b7 [ie/twitter:spaces] Improve metadata extraction (#12911)
Authored by: doe1080
2025-04-25 03:42:17 +00:00
doe1080 8d127b18f8 [fd/NiconicoDmc] Remove downloader (#12916)
Authored by: doe1080
2025-04-24 15:20:25 -05:00
doe1080 7d05aa99c6 [ie/niconico] Remove DMC formats support (#12916)
Authored by: doe1080
2025-04-24 15:20:25 -05:00
bashonly 36da6360e1 [ie/mlbtv] Fix device ID caching (#12980)
Authored by: bashonly
2025-04-24 19:18:22 +00:00
bashonly e7e3b7a55c [ie/dacast] Support tokenized URLs (#12979)
Authored by: bashonly
2025-04-24 19:10:34 +00:00
D Trombett dce8234624 [ie/RaiPlay] Fix DRM detection (#12971)
Closes #12969
Authored by: DTrombett
2025-04-24 18:26:35 +00:00
sepro 2381881fe5 [ie/vk] Fix uploader extraction (#12985)
Closes #12967
Authored by: seproDev
2025-04-23 14:31:20 +00:00
Sergey B (Troex Nevelin) 741fd809bc [ie/GetCourseRu] Fix extractors (#12943)
Closes #12941
Authored by: troex
2025-04-23 00:14:42 +00:00
bashonly 34a061a295 [ie/generic] Fix MPD extraction for file:// URLs (#12978)
Fix 5086d4aed6
Authored by: bashonly
2025-04-23 00:06:35 +00:00
bashonly 9032f98136 [ie/cda] Fix formats extraction (#12975)
Closes #12962
Authored by: bashonly
2025-04-23 00:00:41 +00:00
bashonly de271a06fd [ie/twitcasting] Fix livestream extraction (#12977)
Closes #12966
Authored by: bashonly
2025-04-22 23:54:41 +00:00
bashonly d596824c2f [ie/vimeo] Fix API extraction (#12976)
Closes #12974
Authored by: bashonly
2025-04-22 23:47:38 +00:00
sepro 88eb1e7a9a Add --preset-alias option (#12839)
Authored by: seproDev, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2025-04-19 22:08:34 +02:00
sepro f5a37ea40e [ie/loco] Fix extractor (#12934)
Closes #12930
Authored by: seproDev
2025-04-19 02:02:09 +02:00
Florentin Le Moal f07ee91c71 [ie/rtve] Rework extractors (#10388)
Closes #1346, Closes #5756
Authored by: meGAmeS1, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-04-19 01:47:14 +02:00
fries1234 ed8ad1b4d6 [ie/tvw:tvchannels] Add extractor (#12721)
Authored by: fries1234
2025-04-19 01:35:47 +02:00
Florentin Le Moal 839d643253 [ie/AtresPlayer] Rework extractor (#11424)
Closes #996, Closes #1165
Authored by: meGAmeS1, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-04-18 22:12:31 +02:00
香芋奶茶 f5736bb35b [ie/AbemaTV] Fix thumbnail extraction (#12859)
Closes #12858
Authored by: Kiritomo
2025-04-18 21:12:27 +02:00
sepro 9d26daa04a [ie/panopto] Fix formats extraction (#12925)
Closes #11042
Authored by: seproDev
2025-04-18 21:09:41 +02:00
sepro 73a26f9ee6 [ie/linkedin] Support feed URLs (#12927)
Closes #6104
Authored by: seproDev
2025-04-18 21:08:13 +02:00
sepro 4e69a626cc [ie/tvp:vod] Improve _VALID_URL (#12923)
Closes #12917
Authored by: seproDev
2025-04-18 21:05:01 +02:00
pj47x 77aa15e98f [ie/manyvids] Fix extractor (#10907)
Closes #8268
Authored by: pj47x
2025-04-18 18:38:58 +00:00
Michał Walenciak cb271d445b [ie/CDAFolder] Extend _VALID_URL (#12919)
Closes #12918
Authored by: Kicer86, fireattack

Co-authored-by: fireattack <human.peng@gmail.com>
2025-04-18 18:32:38 +00:00
doe1080 ceab4d5ed6 [networking] Add PATCH request shortcut (#12884)
Authored by: doe1080
2025-04-18 11:46:19 +12:00
leeblackc ed6c6d7eef [ie/youtube] Add extractor arg to skip "initial_data" request (#12865)
Closes https://github.com/yt-dlp/yt-dlp/issues/12826

Authored by: leeblackc
2025-04-18 11:42:08 +12:00
coletdjnz f484c51599 [ie/youtube] Add warning on video captcha challenge (#12939)
Authored by: coletdjnz
2025-04-18 11:40:39 +12:00
coletdjnz 72ba487930 [ie/youtube:tab] Extract continuation from empty page (#12938)
Fixes https://github.com/yt-dlp/yt-dlp/issues/12933 https://github.com/yt-dlp/yt-dlp/issues/8206

Authored by: coletdjnz
2025-04-18 11:34:30 +12:00
Subrat Lima 74e90dd9b8 [ie/LRTRadio] Add extractor (#12801)
Closes #12745
Authored by: subrat-lima
2025-04-06 23:26:44 +00:00
Snack 1d45e30537 [ie/niconico:live] Fix extractor (#12809)
Closes #12365
Authored by: Snack-X
2025-04-06 23:24:58 +00:00
Frank Aurich 3c1c75ecb8 [ie/kika] Add playlist extractor (#12832)
Closes #3658
Authored by: 1100101
2025-04-06 21:04:24 +02:00
J.Luis 7faa18b83d [ie/ivoox] Add extractor (#12768)
Authored by: NeonMan, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-04-06 20:48:07 +02:00
doe1080 a473e59233 [utils] url_or_none: Support WebSocket URLs (#12848)
Authored by: doe1080
2025-04-06 20:46:08 +02:00
sepro 45f01de00e [utils] _yield_json_ld: Make function less fatal (#12855)
Authored by: seproDev
2025-04-06 20:31:00 +02:00
WouterGordts db6d1f145a [ie/mixcloud] Refactor extractor (#12830)
Authored by: WouterGordts, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-04-06 19:51:08 +02:00
sepro a3f2b54c25 [ie/dzen.ru] Rework extractors (#12852)
Closes #5523, Closes #10818, Closes #11385, Closes #11470
Authored by: seproDev
2025-04-06 17:41:48 +02:00
LN Liberda 91832111a1 [ie/TokFMPodcast] Fix formats extraction (#12842)
Authored by: selfisekai
2025-04-06 17:05:43 +02:00
Ben Faerber 425017531f [ie/parti] Add extractors (#12769)
Closes #11434
Authored by: benfaerber
2025-04-05 22:09:53 +02:00
sepro 58d0c83457 [ie/rumble] Improve format extraction (#12838)
Closes #12837
Authored by: seproDev
2025-04-05 20:29:57 +02:00
sepro 4ebf41309d [ie/CrowdBunker] Make format extraction non-fatal (#12836)
Authored by: seproDev
2025-04-05 19:49:51 +02:00
CasperMcFadden95 e1847535e2 [ie/RoyaLive] Add extractor (#12817)
Authored by: CasperMcFadden95
2025-04-03 21:02:24 +02:00
sepro 5361a7c6e2 [ie/vk] Fix chapters extraction (#12821)
Fix 05c8023a27

Authored by: seproDev
2025-04-03 19:55:36 +02:00
github-actions[bot] 349f36606f Release 2025.03.31
Created by: bashonly

:ci skip all
2025-03-31 21:54:27 +00:00
bashonly 5e457af57f [cleanup] Misc (#12802)
Authored by: bashonly
2025-03-31 21:38:21 +00:00
DmitryScaletta 61046c3161 [ie/twitch:clips] Extract portrait formats (#12763)
Authored by: DmitryScaletta
2025-03-31 21:21:14 +00:00
bashonly 07f04005e4 [ie/youtube] Add player_js_variant extractor-arg (#12767)
- Always distinguish between different JS variants' code/functions
- Change naming scheme for nsig and sigfuncs in disk cache

Authored by: bashonly
2025-03-31 19:45:48 +00:00
bashonly e465b078ea [ie/on24] Support mainEvent URLs (#12800)
Closes #12782
Authored by: bashonly
2025-03-31 19:25:10 +00:00
bashonly d63696f23a [ie/MicrosoftLearnEpisode] Extract more formats (#12799)
Closes #12798
Authored by: bashonly
2025-03-31 19:21:44 +00:00
Muhammad Labeeb bb321cfdc3 [ie/francaisfacile] Add extractor (#12787)
Authored by: mlabeeb03
2025-03-31 19:06:33 +00:00
Miroslav Bendík 5fc521cbd0 [ie/stvr] Rename extractor from RTVS to STVR (#12788)
Authored by: mireq
2025-03-31 19:04:52 +00:00
bashonly f033d86b96 [ie/mlbtv] Fix radio-only extraction (#12792)
Authored by: bashonly
2025-03-30 23:28:14 +00:00
bashonly 9a1ec1d36e [ie/generic] Validate response before checking m3u8 live status (#12784)
Closes #12744
Authored by: bashonly
2025-03-30 23:02:59 +00:00
bashonly 2956035912 [ie/sbs] Fix subtitles extraction (#12785)
Closes #12783
Authored by: bashonly
2025-03-30 22:54:55 +00:00
sepro 22e34adbd7 Add --compat-options 2024 (#12789)
Authored by: seproDev
2025-03-31 00:38:46 +02:00
coletdjnz 6a6d97b2cb [ie/youtube:tab] Fix playlist continuation extraction (#12777)
Fixes https://github.com/yt-dlp/yt-dlp/issues/12759

Authored by: coletdjnz
2025-03-29 11:13:09 +13:00
github-actions[bot] 3ddbebb3c6 Release 2025.03.27
Created by: bashonly

:ci skip all
2025-03-27 23:45:56 +00:00
bashonly 48be862b32 [ie/youtube] Make signature and nsig extraction more robust (#12761)
Authored by: bashonly, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-03-27 22:31:01 +00:00
bashonly a8b9ff3c2a [jsinterp] Fix nested attributes and object extraction (#12760)
Authored by: bashonly, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-03-27 22:28:30 +00:00
github-actions[bot] 6eaa574c82 Release 2025.03.26
Created by: bashonly

:ci skip all
2025-03-26 00:04:51 +00:00
sepro ecee97b4fa [ie/youtube] Only cache nsig code on successful decoding (#12750)
Authored by: seproDev, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-03-25 23:47:45 +00:00
sepro a550dfc904 [ie/youtube] Fix signature and nsig extraction for player 4fcd6e4a (#12748)
Closes #12746
Authored by: seproDev
2025-03-25 23:40:58 +00:00
github-actions[bot] 336b33e72f Release 2025.03.25
Created by: bashonly

:ci skip all
2025-03-25 00:07:18 +00:00
sepro 9dde546e7e [cleanup] Misc (#12694)
Authored by: seproDev
2025-03-25 00:05:02 +00:00
Abdulmohsen 66e0bab814 [ie/TVer] Fix extractor (#12659)
Closes #12643, Closes #12282
Authored by: arabcoders, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-03-25 00:00:22 +00:00
doe1080 801afeac91 [ie/streaks] Add extractor (#12679)
Authored by: doe1080
2025-03-24 23:12:09 +00:00
bashonly 86ab79e1a5 [ie] Fix sorting of HLS audio formats by GROUP-ID (#12714)
Closes #11178
Authored by: bashonly
2025-03-24 22:38:22 +00:00
Subrat Lima 3396eb50dc [ie/17live:vod] Add extractor (#12723)
Closes #12570
Authored by: subrat-lima
2025-03-24 22:26:45 +00:00
fireattack 5086d4aed6 [ie/generic] Fix MPD base URL parsing (#12718)
Closes #12709
Authored by: fireattack
2025-03-24 22:24:09 +00:00
sepro 9491b44032 [utils] js_to_json: Make function less fatal (#12715)
Authored by: seproDev
2025-03-24 22:28:47 +01:00
doe1080 b7fbb5a0a1 [ie/vrsquare] Add extractors (#12515)
Authored by: doe1080
2025-03-24 22:28:09 +01:00
bashonly 4054a2b623 [ie/youtube] Fix PhantomJS nsig fallback (#12728)
Also fixes the NSigDeno plugin

Closes #12724
Authored by: bashonly
2025-03-24 21:22:25 +00:00
bashonly b9c979461b [ie/youtube] Fix signature and nsig extraction for player 363db69b (#12725)
Closes #12724
Authored by: bashonly
2025-03-24 21:18:51 +00:00
bashonly 9d5e6de2e7 [ie/9now.com.au] Fix extractor (#12702)
Closes #12591
Authored by: bashonly
2025-03-23 16:35:46 +00:00
Simon Sawicki 9bf23902ce [rh:curl_cffi] Support curl_cffi 0.10.x (#12670)
Authored by: Grub4K
2025-03-23 00:15:20 +01:00
sepro be5af3f9e9 [ie/deezer] Remove extractors (#12704)
Authored by: seproDev
2025-03-22 22:53:20 +01:00
sepro fe4f14b836 [ie/viki] Remove extractors (#12703)
Closes #2907, Closes #2869
Authored by: seproDev
2025-03-22 22:34:07 +01:00
Simon Sawicki b872ffec50 [core] Fix attribute error on failed VT init (#12696)
Authored by: Grub4K
2025-03-22 21:03:28 +01:00
bashonly e2dfccaf80 [ie/chzzk:video] Fix extraction (#12692)
Closes #12487
Authored by: dirkf, bashonly

Co-authored-by: dirkf <fieldhouse@gmx.net>
2025-03-22 16:44:05 +00:00
github-actions[bot] b4488a9e12 Release 2025.03.21
Created by: bashonly

:ci skip all
2025-03-21 23:49:09 +00:00
Simon Sawicki f36e4b6e65 [cleanup] Misc (#12526)
Authored by: Grub4K, seproDev, gamer191, dirkf

Co-authored-by: sepro <sepro@sepr0.com>
2025-03-21 23:41:56 +00:00
D Trombett 983095485c [ie/loco] Add extractor (#12667)
Closes #12496
Authored by: DTrombett
2025-03-21 23:24:13 +00:00
Michaël De Boey bbada3ec07 [ie/ketnet] Remove extractor (#12628)
Authored by: MichaelDeBoey
2025-03-21 23:19:36 +00:00
Michiel Sikma 8305df0001 [ie/soop] Fix timestamp extraction (#12609)
Closes #12606
Authored by: msikma
2025-03-21 23:16:30 +00:00
bashonly 7223d29569 [ie/mitele] Fix extractor (#12689)
Closes #12655
Authored by: bashonly
2025-03-21 23:14:46 +00:00
bashonly f5fb2229e6 [ie/BilibiliPlaylist] Fix extractor (#12690)
Closes #12651
Authored by: bashonly
2025-03-21 23:04:58 +00:00
JChris246 89a68c4857 [ie/jamendo] Fix thumbnail extraction (#12622)
Closes #11779
Authored by: JChris246, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-03-21 23:04:34 +00:00
sepro 9b868518a1 [ie/youtube] Fix nsig and signature extraction for player 643afba4 (#12684)
Closes #12677, Closes #12682
Authored by: seproDev, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-03-21 20:58:10 +00:00
D Trombett 2ee3a0aff9 [ie/tv8.it] Add live and playlist extractors (#12569)
Closes #12542
Authored by: DTrombett
2025-03-16 23:10:16 +01:00
Arc8ne 01a8be4c23 [ie/Canalsurmas] Add extractor (#12497)
Closes #5516
Authored by: Arc8ne
2025-03-16 23:03:10 +01:00
Refael Ackermann ebac65aa9e [ie/NBCStations] Fix extractor (#12534)
Authored by: refack
2025-03-16 21:41:32 +00:00
thedenv 4815dac131 [ie/msn] Rework extractor (#12513)
Closes #3225
Authored by: thedenv, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-03-16 19:54:46 +01:00
Simon Sawicki 95f8df2f79 [networking] Always add unsupported suffix on version mismatch (#12626)
Authored by: Grub4K
2025-03-16 12:45:44 +01:00
coletdjnz e67d786c7c [ie/youtube] Warn on DRM formats (#12593)
Authored by: coletdjnz
2025-03-16 10:28:16 +13:00
sepro d9a53cc1e6 [ie/reddit] Truncate title (#12567)
Authored by: seproDev
2025-03-15 22:16:00 +01:00
sepro 83b119dadb [ie/tiktok] Truncate title (#12566)
Authored by: seproDev
2025-03-15 22:15:29 +01:00
sepro 06f6de78db [ie/twitter] Truncate title (#12560)
Authored by: seproDev
2025-03-15 22:15:03 +01:00
sepro 3380febe99 [ie/youtube] Player client maintenance (#12603)
Authored by: seproDev
2025-03-15 21:57:56 +01:00
rysson be0d819e11 [ie/cda] Fix login support (#12552)
Closes #10306
Authored by: rysson
2025-03-15 21:47:50 +01:00
Michaël De Boey df9ebeec00 [ie/vrtmax] Rework extractor (#12479)
Closes #7997, Closes #8174, Closes #9375
Authored by: MichaelDeBoey, bergoid, seproDev

Co-authored-by: bergoid <bergoid@users.noreply.github.com>
Co-authored-by: sepro <sepro@sepr0.com>
2025-03-15 21:29:22 +01:00
fireattack 17504f2535 [ie/openrec] Fix _VALID_URL (#12608)
Authored by: fireattack
2025-03-15 17:14:01 +01:00
coletdjnz 4432a9390c [ie/youtube] Split into package (#12557)
Authored by: coletdjnz
2025-03-13 17:37:33 +13:00
sepro 05c8023a27 [ie/vk] Improve metadata extraction (#12510)
Closes #12509
Authored by: seproDev
2025-03-07 22:14:38 +01:00
bashonly bd0a668169 [ie/pinterest] Fix extractor (#12538)
Closes #12529
Authored by: mikf

Co-authored-by: =?UTF-8?q?Mike=20F=C3=A4hrmann?= <mike_faehrmann@web.de>
2025-03-05 06:38:23 +00:00
bashonly b8b4754704 [ie/twitter] Fix syndication token generation (#12537)
Fix 14cd7f3443

Authored by: bashonly
2025-03-05 06:22:52 +00:00
u-spec-png 9d70abe4de [ie/N1] Fix extraction of newer articles (#12514)
Authored by: u-spec-png
2025-03-04 01:51:23 +01:00
sepro 8eb9c1bf3b [ie/RTP] Rework extractor (#11638)
Closes #4661, Closes #10393, Closes #11244
Authored by: seproDev, vallovic, red-acid, pferreir, somini

Co-authored-by: vallovic <vallovic@gmail.com>
Co-authored-by: red-acid <161967284+red-acid@users.noreply.github.com>
Co-authored-by: Pedro Ferreira <pedro@dete.st>
Co-authored-by: somini <dev@somini.xyz>
2025-03-04 00:46:18 +01:00
fries1234 42b7440963 [ie/tvw] Add extractor (#12271)
Authored by: fries1234
2025-03-03 23:25:30 +01:00
sepro 172d5fcd77 [ie/MagellanTV] Fix extractor (#12505)
Closes #12498
Authored by: seproDev
2025-03-03 22:55:03 +01:00
Simon Sawicki 7d18fed8f1 [networking] Add keep_header_casing extension (#11652)
Authored by: coletdjnz, Grub4K

Co-authored-by: coletdjnz <coletdjnz@protonmail.com>
2025-03-03 00:10:01 +01:00
coletdjnz 79ec2fdff7 [ie/youtube] Warn on missing formats due to SSAP (#12483)
See https://github.com/yt-dlp/yt-dlp/issues/12482

Authored by: coletdjnz
2025-02-28 19:33:31 +13:00
sepro 3042afb5fe [ie/CultureUnplugged] Extend _VALID_URL (#12486)
Closes #12477
Authored by: seproDev
2025-02-26 19:39:50 +01:00
sepro ad60137c14 [ie/Dailymotion] Improve embed detection (#12464)
Closes #12453
Authored by: seproDev
2025-02-26 19:36:33 +01:00
4ft35t 0bb3978862 [ie/weibo] Support playlists (#12284)
Closes #12283
Authored by: 4ft35t
2025-02-23 19:16:06 +00:00
XPA 7508e34f20 [ie/niconico] Fix format sorting (#12442)
Authored by: xpadev-net
2025-02-23 19:07:08 +00:00
bashonly 9807181cfb [ie/lbry] Make m3u8 format extraction non-fatal (#12463)
Closes #12459
Authored by: bashonly
2025-02-23 18:24:48 +00:00
bashonly 7126b47260 [ie/lbry] Raise appropriate error for non-media files (#12462)
Closes #12182
Authored by: bashonly
2025-02-23 17:59:22 +00:00
bashonly eb1417786a [ie/gem.cbc.ca] Fix login support (#12414)
Closes #12406
Authored by: bashonly
2025-02-23 09:56:47 +00:00
bashonly 6933f5670c [ie/playsuisse] Fix login support (#12444)
Closes #12425
Authored by: bashonly
2025-02-23 09:22:51 +00:00
Alexander Seiler 26a502fc72 [ie/azmedien] Fix extractor (#12375)
Authored by: goggle
2025-02-23 09:14:35 +00:00
Ben Faerber 652827d5a0 [ie/softwhiteunderbelly] Add extractor (#12281)
Authored by: benfaerber
2025-02-23 09:11:58 +00:00
Pedro Belo 0e1697232f [ie/globo] Fix subtitles extraction (#12270)
Authored by: pedro
2025-02-23 08:57:27 +00:00
Kenshin9977 9f77e04c76 Fix external downloader availability when using --ffmpeg-location (#12318)
This fix is only applicable to the CLI option

Authored by: Kenshin9977
2025-02-23 08:50:43 +00:00
Simon Sawicki c034d65548 Fix lazy extractor state (Fix 4445f37a7a) (#12452)
Authored by: coletdjnz, Grub4K, pukkandan
2025-02-23 09:44:27 +01:00
bashonly 480125560a [ie/instagram] Improve error handling (#12410)
Closes #5967, Closes #6294, Closes #7328, Closes #8452
Authored by: bashonly
2025-02-23 08:35:22 +00:00
bashonly a59abe0636 [ie/instagram] Fix extraction of older private posts (#12451)
Authored by: bashonly
2025-02-23 08:31:00 +00:00
Chris Ellsworth a90641c836 [ie/instagram] Add app_id extractor-arg (#12359)
Authored by: chrisellsworth
2025-02-23 08:16:04 +00:00
fireattack 65c3c58c0a [ie/instagram:story] Support --no-playlist (#12397)
Closes #12395
Authored by: fireattack
2025-02-23 07:24:21 +00:00
bashonly 99ea297875 [ie/tiktok] Improve error handling (#12445)
Closes #8678
Authored by: bashonly
2025-02-23 06:53:13 +00:00
bashonly 6deeda5c11 [ie/soundcloud] Fix thumbnail extraction (#12447)
Closes #11835, Closes #12435
Authored by: bashonly
2025-02-23 06:20:53 +00:00
Refael Ackermann 7f3006eb0c [ie/wsj] Support opinion URLs and impersonation (#12431)
Authored by: refack
2025-02-23 00:40:53 +00:00
coletdjnz 4445f37a7a [core] Load plugins on demand (#11305)
- Adds `--no-plugin-dirs` to disable plugin loading
- `--plugin-dirs` now supports post-processors

Authored by: coletdjnz, Grub4K, pukkandan
2025-02-23 11:00:46 +13:00
sepro 3a1583ca75 [ie/BunnyCdn] Add extractor (#11586)
Also adds BunnyCdnFD

Authored by: seproDev, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2025-02-21 22:39:41 +01:00
Simon Sawicki a3e0c7d3b2 [test] Show all differences for expect_value and expect_dict (#12334)
Authored by: Grub4K
2025-02-21 21:29:07 +01:00
Simon Sawicki f7a1f2d813 [core] Support emitting ConEmu progress codes (#10649)
Authored by: Grub4K
2025-02-20 20:33:31 +01:00
bashonly 9deed13d7c [ie/soundcloud] Extract tags (#12420)
Authored by: bashonly
2025-02-20 15:51:08 +00:00
bashonly c2e6e1d5f7 [ie/niconico:live] Fix thumbnail extraction (#12419)
Closes #12417
Authored by: bashonly
2025-02-20 15:39:06 +00:00
github-actions[bot] 9c3e8b1696 Release 2025.02.19
Created by: bashonly

:ci skip all
2025-02-19 02:42:18 +00:00
bashonly 4985a40417 [cleanup] Misc (#12238)
Authored by: StefanLobbenmeier, dirkf, Grub4K

Co-authored-by: Stefan Lobbenmeier <Stefan.Lobbenmeier@gmail.com>
Co-authored-by: dirkf <fieldhouse@gmx.net>
Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2025-02-19 02:29:29 +00:00
sepro 01a63629a2 [docs] Add note to supportedsites.md (#12382)
Authored by: seproDev
2025-02-19 02:27:49 +00:00
bashonly be69468752 [fd/hls] Support --write-pages for m3u8 media playlists (#12333)
Authored by: bashonly
2025-02-19 02:23:42 +00:00
bashonly 5271ef48c6 [ie/gem.cbc.ca] Fix extractors (#12404)
Does not fix broken login support

Closes #11848
Authored by: bashonly, dirkf

Co-authored-by: dirkf <fieldhouse@gmx.net>
2025-02-19 02:20:50 +00:00
coletdjnz d48e612609 [ie/youtube] Retry on more critical requests (#12339)
Authored by: coletdjnz
2025-02-19 00:39:51 +00:00
bashonly 5c4c2ddfaa [ie/francetvinfo.fr] Fix extractor (#12402)
Closes #12366
Authored by: bashonly
2025-02-19 00:28:34 +00:00
bashonly ec17fb16e8 [ie/youtube] nsig workaround for tce player JS (#12401)
Closes #12398
Authored by: bashonly
2025-02-19 00:24:12 +00:00
bashonly e7882b682b [ie/3sat] Fix extractor (#12403)
Fix 241ace4f10

Closes #12391
Authored by: bashonly
2025-02-19 00:19:02 +00:00
bashonly 6ca23ffaa4 [ie/reddit] Bypass gated subreddit warning (#12335)
Closes #12331
Authored by: bashonly
2025-02-11 21:32:25 +00:00
Laurent FAVOLE f53553087d [ie/Digiview] Add extractor (#9902)
Authored by: lfavole
2025-02-11 21:04:20 +01:00
bashonly 4ecb833472 [misc] Clarify that the issue template cannot be removed (#12332)
Fix 517ddf3c3f

Authored by: bashonly
2025-02-11 00:40:21 +00:00
Mozi 2081634474 [test:download] Validate and sort info dict fields (#12299)
Authored by: pzhlkj6612, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-02-10 23:22:21 +00:00
bashonly c987be0acb [fd/hls] Support hls_media_playlist_data format field (#12322)
Authored by: bashonly
2025-02-10 23:08:10 +00:00
Patrick Robertson 14cd7f3443 [ie/twitter] Fix syndication token generation (#12107)
Authored by: pjrobertson, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2025-02-10 19:00:00 +00:00
sepro 4ca8c44a07 [jsinterp] Improve zeroise (#12313)
Authored by: seproDev
2025-02-09 22:37:23 +01:00
Stefan Lobbenmeier 241ace4f10 [ie/zdf] Extract more metadata (#9565)
Closes #9564
Authored by: StefanLobbenmeier
2025-02-09 19:19:28 +00:00
bashonly 1295bbedd4 [ie/francetv:site] Fix livestream extraction (#12316)
Closes #12310
Authored by: bashonly
2025-02-09 02:21:48 +00:00
Julien Valentin 19edaa44fc [ie/generic] Extract live_status for DASH manifest URLs (#12256)
* Also removes the content-type check for dash+xml/mpd.
This was added in cf1f13b817,
but is a no-op since the regex pattern was never changed accordingly.
And it looks like it was unwanted anyways per 28ad7df65d

Closes #12255
Authored by: mp3butcher
2025-02-08 23:28:54 +00:00
entourage8 10b7ff68e9 [fd/hls] Fix BYTERANGE logic (#11972)
Closes #3578, Closes #3810, Closes #9400
Authored by: entourage8
2025-02-08 21:43:12 +00:00
Simon Sawicki 0d9f061d38 [jsinterp] Add js_number_to_string (#12110)
Authored by: Grub4K
2025-02-08 18:48:36 +01:00
sepro 517ddf3c3f [misc] Improve Issue/PR templates (#11499)
Authored by: seproDev
2025-02-08 17:00:38 +01:00
bashonly 03c3d70577 [ie/cwtv:movie] Add extractor (#12227)
Closes #12113
Authored by: bashonly
2025-01-30 19:58:10 +00:00
dove f8d0161455 [ie/globo] Fix extractor (#11795)
Closes #9512, Closes #11541, Closes #11772
Authored by: slipinthedove, YoshiTabletopGamer

Co-authored-by: YoshiTabletopGamer <88633614+YoshiTabletopGamer@users.noreply.github.com>
2025-01-29 23:55:40 +00:00
alard d59f14a0a7 [ie/goplay] Fix extractor (#12237)
Authored by: alard
2025-01-29 23:38:36 +00:00
bashonly 817483ccc6 [ie/francetv:site] Fix extractor (#12236)
Closes #12209
Authored by: bashonly
2025-01-29 23:23:29 +00:00
bashonly 861aeec449 [ie/dropbox] Fix extraction (#12228)
Closes #12109
Authored by: bashonly
2025-01-29 16:56:06 +00:00
barsnick 57c717fee4 [ie/acast] Support shows.acast.com URLs (#12223)
Authored by: barsnick
2025-01-28 23:41:02 +00:00
Roland Hieber 9fb8ab2ff6 [ie/pbs] Support www.thirteen.org URLs (#11191)
Authored by: rohieb
2025-01-28 23:38:26 +00:00
arantius 18a28514e3 [ie/cwtv] Fix extractor (#12207)
Closes #12108
Authored by: arantius
2025-01-28 23:26:37 +00:00
github-actions[bot] 5ff7a43623 Release 2025.01.26
Created by: bashonly

:ci skip all
2025-01-26 03:54:22 +00:00
sepro 3b45319344 [cleanup] Misc (#12194)
Closes #12098, Closes #12133
Authored by: seproDev, bashonly, lonble, pjrobertson

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
Co-authored-by: Lonble <74650029+lonble@users.noreply.github.com>
Co-authored-by: Patrick Robertson <robertson.patrick@gmail.com>
2025-01-26 03:32:10 +00:00
nosoop 421bc72103 [ie/youtube] Extract media_type for livestreams (#11605)
Closes #11563
Authored by: nosoop
2025-01-26 03:27:12 +00:00
FestplattenSchnitzel d4f5be1735 [ie/ViMP:Playlist] Add support for tags (#11688)
Authored by: FestplattenSchnitzel
2025-01-26 03:20:42 +00:00
bashonly 797d2472a2 [ie/TheaterComplexTownPPV] Support live URLs (#11720)
Closes #11718
Authored by: bashonly
2025-01-26 03:12:32 +00:00
knackku 3b99a0f0e0 [ie/xhamster] Various improvements (#11738)
Closes #7620
Authored by: knackku
2025-01-26 03:10:24 +00:00
middlingphys c709cc41cb [ie/abematv] Support season extraction (#11771)
Closes #10602
Authored by: middlingphys
2025-01-26 03:05:40 +00:00
invertico 4850ce91d1 [ie/redgifs] Support /ifr/ URLs (#11805)
Authored by: invertico
2025-01-26 02:40:05 +00:00
msm595 e2e73b5c65 [ie/patreon] Extract attachment filename as alt_title (#12000)
Authored by: msm595
2025-01-26 02:36:16 +00:00
krandor 13825ab778 [ie/pbs] Fix extractor (#12024)
Closes #8703, Closes #9740, Closes #11514
Authored by: dirkf, krandor, n10dollar

Co-authored-by: dirkf <fieldhouse@gmx.net>
Co-authored-by: Neil <ntendolkar@berkeley.edu>
2025-01-26 02:25:35 +00:00
test20140 bc88b904cd [ie/niconico:series] Fix extractor (#11822)
Closes #7320, Closes #12001
Authored by: test20140
2025-01-26 01:47:15 +00:00
kibaa 76ac023ff0 [ie/youtube:tab] Improve shorts title extraction (#11991) (#11997)
Closes #11991
Authored by: d3d9, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-01-26 01:23:29 +00:00
kclauhk b3007c44cd [ie/naver] Fix m3u8 formats extraction (#12037)
Closes #11953
Authored by: kclauhk
2025-01-26 01:16:26 +00:00
N/Ame 78912ed9c8 [ie/bilibili] Support space video list extraction without login (#12089)
Closes #12007
Authored by: grqz
2025-01-26 00:56:36 +00:00
InvalidUsernameException bb69f5dab7 [ie/zdf] Fix extractors (#11041)
Closes #4782, Closes #10672
Authored by: InvalidUsernameException
2025-01-26 00:29:57 +00:00
gavin 6d304133ab [ie/soundcloud] Extract more metadata (#11945)
Authored by: 7x11x13
2025-01-25 22:52:48 +00:00
Jixun 9ff330948c [ie/vimeo] Fix thumbnail extraction (#12142)
Closes #11931
Authored by: jixunmoe
2025-01-25 21:42:34 +00:00
Simon Sawicki fc12e724a3 [utils] sanitize_path: Fix some incorrect behavior (#11923)
Authored by: Grub4K
2025-01-25 22:32:00 +01:00
Konstantin Kulakov 61ae5dc34a [ie/1tv] Support sport1tv.ru domain (#11889)
Closes #11894
Authored by: kvk-2015
2025-01-25 22:21:45 +01:00
c-basalt 4651679104 [ie/bilibili] Support space /lists/ URLs (#11964)
Closes #11959
Authored by: c-basalt
2025-01-25 20:56:30 +00:00
sepro ff44ed5306 [ie/crunchyroll] Remove extractors (#12195)
Closes #2561, Closes #5869, Closes #6278, Closes #7099, Closes #7414, Closes #7465, Closes #7976, Closes #8235, Closes #9867, Closes #10207
Authored by: seproDev
2025-01-25 20:57:08 +01:00
doe1080 cdcf1e8672 [ie/funimation] Remove extractors (#12167)
Closes #1569, Closes #2255, Closes #2517, Closes #2723, Closes #4318, Closes #4345, Closes #5326, Closes #6575, Closes #8644
Authored by: doe1080
2025-01-25 20:29:24 +01:00
Dioarya f7d071e8aa [core] Fix float comparison values in format filters (#11880)
Closes #10115
Authored by: Dioarya, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-01-25 19:12:56 +00:00
Boof 45732e2590 [ie/nrk] Fix extraction (#12193)
Closes #12192
Authored by: hexahigh
2025-01-25 18:24:04 +00:00
gavin 7bfb4f72e4 [ie/soundcloud:user] Add /comments page support (#11999)
Authored by: 7x11x13
2025-01-25 18:48:06 +01:00
Subrat Lima 5d904b077d [ie/subsplash] Add extractors (#11054)
Closes #10922
Authored by: subrat-lima, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-01-25 18:14:45 +01:00
Roman e7cc02b14d [ie/GoodGame] Fix extractor (#12173)
Authored by: NecroRomnt
2025-01-25 18:10:44 +01:00
bashonly f0d4b8a5d6 [ie/youtube] Restore convenience workarounds (#12181)
Authored by: bashonly
2025-01-25 16:18:15 +00:00
coletdjnz 6b91d232e3 [ie/youtube] Use different PO token for GVS and Player (#12090)
Authored by: coletdjnz
2025-01-25 13:17:37 +13:00
Antoine Bollengier de82acf876 [ie/youtube] Update ios player client (#12155)
Authored by: b5i
2025-01-23 22:52:32 +00:00
coletdjnz 326fb1ffaf [ie/youtube] Download tv client Innertube config (#12168)
Authored by: coletdjnz
2025-01-23 18:26:02 +13:00
August Wikerfors ccda63934d [ie/Bluesky] Prefer source format (#12154)
Authored by: 0x9fff00
2025-01-21 22:59:39 +01:00
finch71 9676b05715 [ie/BiliBiliDynamic] Add extractor (#11838)
Closes #11726
Authored by: finch71, grqz

Co-authored-by: N/Ame <173015200+grqz@users.noreply.github.com>
2025-01-20 21:45:04 +01:00
sepro f9f24ae376 [ie/XiaoHongShu] Extract more formats (#12147)
Authored by: seproDev
2025-01-20 19:55:30 +01:00
kclauhk af2c821d74 [ie/piramidetv] Add extractors (#10777)
Closes #10706, Closes #10708
Authored by: kclauhk, HobbyistDev, seproDev

Co-authored-by: HobbyistDev <tesutonihon4@gmail.com>
Co-authored-by: sepro <sepro@sepr0.com>
2025-01-20 16:26:05 +01:00
Paul Wise 1ef3ee7500 [ie/nest] Add extractors (#11747)
Authored by: pabs3, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-01-20 16:13:24 +01:00
subsense 20c765d023 [ie/eggs] Add extractors (#11904)
Closes #11843
Authored by: subsense, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-01-20 16:08:11 +01:00
cotko 3fc4608656 [ie/rtvslo.si:show] Extract more metadata (#12136)
Authored by: cotko
2025-01-20 07:53:21 +01:00
Grabien 68221ecc87 [ie/senategov] Fix extractors (#9361)
Authored by: Grabien, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-01-20 00:01:22 +01:00
sepro de30f652ff [ie/LBRY] Support signed URLs (#12138)
Authored by: seproDev
2025-01-19 17:52:31 +01:00
Boof 89198bb23b [ie/nrk] Extract more formats (#12069)
Closes #12053
Authored by: hexahigh
2025-01-19 14:13:40 +01:00
4ft35t a567f97b62 [ie/Weibo] Extend _VALID_URL (#12088)
Closes #12086
Authored by: 4ft35t
2025-01-19 14:10:36 +01:00
bashonly 1643686104 [ie/dropout] Fix extraction (#12102)
Closes #12103
Authored by: bashonly
2025-01-16 02:40:13 +00:00
github-actions[bot] bbc7591d3b Release 2025.01.15
Created by: bashonly

:ci skip all
2025-01-15 23:50:41 +00:00
bashonly c8541f8b13 [ie/youtube] Do not use web_creator as a default client (#12087)
Closes #12085
Authored by: bashonly
2025-01-15 18:21:56 +00:00
github-actions[bot] a3c0321825 Release 2025.01.12
Created by: bashonly

:ci skip all
2025-01-12 23:35:35 +00:00
Simon Sawicki dade5e35c8 [cleanup] Misc (#11915)
Authored by: grqz, Grub4K, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
Co-authored-by: N/Ame <173015200+grqz@users.noreply.github.com>
2025-01-12 23:24:22 +00:00
Allen e2ef4fece6 [ie/vine] Remove extractors (#11700)
Authored by: allendema
2025-01-12 19:43:16 +01:00
Mozi 1f489f4a45 [ie/DrTalks] Add extractor (#10831)
Closes #6390
Authored by: pzhlkj6612, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-01-12 19:42:02 +01:00
coletdjnz 75079f4e3f [ie/youtube] Refactor cookie auth (#11989)
Authored by: coletdjnz
2025-01-12 15:02:57 +13:00
coletdjnz 712d2abb32 [ie/youtube] Use tv instead of mweb client by default (#12059)
Authored by: coletdjnz
2025-01-12 15:01:13 +13:00
bashonly 8346b54915 Fix filename sanitization with --no-windows-filenames (#11988)
Fix bug in 6fc85f617a

Closes #11987
Authored by: bashonly
2025-01-11 19:05:23 +00:00
Paul Storkman 1f4e1e85a2 [core] Validate retries values are non-negative (#11927)
Closes #11926
Authored by: Strkmn
2025-01-11 19:51:16 +01:00
HobbyistDev 763ed06ee6 [ie/XiaoHongShu] Extend _VALID_URL (#11806)
Closes #11797
Authored by: HobbyistDev
2025-01-11 18:25:18 +01:00
voidptr_t 3c14e9191f [ie/PlVideo] Add extractor (#10657)
Closes #10311
Authored by: Sanceilaks, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2025-01-11 15:39:31 +01:00
coletdjnz 0b6b7742c2 [ie/youtube] Fix DASH formats incorrectly skipped in some situations (#11910)
Closes https://github.com/yt-dlp/yt-dlp/issues/11907
Authored by: coletdjnz
2024-12-26 14:19:17 +13:00
github-actions[bot] 3905f64920 Release 2024.12.23
Created by: bashonly

:ci skip all
2024-12-23 23:47:20 +00:00
bashonly 65cf46cddd [ie/youtube] Player client maintenance (#11893)
Closes #11867
Authored by: bashonly
2024-12-23 23:26:35 +00:00
coletdjnz 9f42e68a74 [ie/youtube] Skip iOS formats that require PO Token (#11890)
Partial fix for https://github.com/yt-dlp/yt-dlp/issues/11868

Authored by: coletdjnz
2024-12-24 12:03:28 +13:00
pukkandan 6fc85f617a Don't sanitize filename on Unix when --no-windows-filenames (#9591)
Closes #4547, Closes #8464
Authored by: pukkandan
2024-12-23 15:57:25 +05:30
bashonly d298693b1b [ie/soundcloud] Various fixes (#11820)
- Fix original/download formats so that they are considered bestaudio
- Raise appropriate error if track is DRM-protected

Authored by: bashonly
2024-12-15 20:16:04 +00:00
bashonly 09a6c68712 [ie/youtube] Add age-gate workaround for some embeddable videos (#11821)
Closes #11296
Authored by: bashonly
2024-12-15 20:09:48 +00:00
bashonly 1a8851b689 [ie/youtube] Fix uploader_id extraction (#11818)
Closes #11816
Authored by: bashonly
2024-12-15 20:07:18 +00:00
bashonly b91c3925c2 [update] Check 64-bitness when upgrading ARM builds (#11819)
Closes #11813
Authored by: bashonly
2024-12-15 19:55:30 +00:00
bashonly 3d3ee458c1 [update] Fix endless update loop for linux_exe builds (#11827)
Closes #11808
Authored by: bashonly
2024-12-15 19:47:50 +00:00
github-actions[bot] 2037a6414f Release 2024.12.13
Created by: bashonly

:ci skip all
2024-12-13 10:35:40 +00:00
sepro 5421669626 [cleanup] Make more playlist entries lazy (#11763)
Authored by: seproDev
2024-12-13 10:25:29 +00:00
bashonly dc3c4fddcc [ie/youtube] Prioritize original language over auto-dubbed audio (#11803)
Closes #11753
Authored by: bashonly
2024-12-13 10:21:48 +00:00
bashonly 5460cd9189 [ie/youtube] Fix signature function extraction for 2f1832d2 (#11801)
Closes #11798
Authored by: bashonly
2024-12-13 09:43:08 +00:00
Crypto90 f6c73aad5f [ie/youtube:search_url] Fix playlist searches (#11782)
Closes #11666
Authored by: Crypto90
2024-12-12 13:54:11 +00:00
Pew d5e2a379f2 [ie/youtube] Fix release_date extraction (#11759)
Authored by: MutantPiggieGolem1
2024-12-12 13:46:52 +00:00
bashonly bc262bcad4 [ie/patreon:campaign] Support /c/ URLs (#11756)
Closes #11755
Authored by: bashonly
2024-12-12 13:44:19 +00:00
bashonly f4d3e9e6dc [ie/soundcloud] Fix extraction (#11777)
Authored by: bashonly
2024-12-12 13:39:38 +00:00
github-actions[bot] 6fef824025 Release 2024.12.06
Created by: bashonly

:ci skip all
2024-12-06 16:07:07 +00:00
bashonly 4bd2655398 [ie/youtube] Raise if n function returns input value (#11752)
Improve a95ee6d880

Authored by: bashonly
2024-12-06 15:58:44 +00:00
bashonly a95ee6d880 [ie/youtube] Fix n sig extraction for player 3bb1f723 (#11750)
Closes #11744
Authored by: bashonly
2024-12-06 15:35:18 +00:00
bashonly 4c85ccd136 [ie/youtube] Fix signature function extraction (#11751)
Closes #11748
Authored by: bashonly
2024-12-06 15:34:13 +00:00
bashonly 2feb28028e [ie/soundcloud] Fix formats extraction (#11742)
Authored by: bashonly
2024-12-06 15:02:30 +00:00
N/Ame fca3eb5f8b [ie/bilibili] Fix HD formats extraction (#11734)
Fixes dc16876480

Closes #10554
Authored by: grqz
2024-12-04 23:11:55 +00:00
bashonly 2e49c789d3 [ie/youtube] Player client maintenance (#11724)
Closes #11686
Authored by: bashonly
2024-12-04 22:33:14 +00:00
wesson09 354cb4026c [cookies] Add --cookies-from-browser support for MS Store Firefox (#11731)
Authored by: wesson09
2024-12-04 18:41:58 +01:00
github-actions[bot] cfa76f35d2 Release 2024.12.03
Created by: bashonly

:ci skip all
2024-12-03 20:30:33 +00:00
bashonly 2b67ac300a [cleanup] Misc (#11716)
Authored by: bashonly, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2024-12-03 20:22:21 +00:00
bashonly c038a7b187 [ie/vk] Fix extractors (#11715)
Closes #5832, Closes #11471, Closes #11646, Closes #11670
Authored by: bashonly
2024-12-03 14:28:43 +00:00
Link a13a336aa6 [ie/bilibili] Fix subtitles and chapters extraction (#11708)
Authored by: xiaomac
2024-12-03 04:08:46 +00:00
N/Ame dc16876480 [ie/bilibili] Always try to extract HD formats (#10559)
Closes #10554
Authored by: grqz
2024-12-03 03:44:03 +00:00
N/Ame f05a1cd149 [ie/bilibili] Fix supporter-only video extraction (#11711)
Fix bug in 239f5f36fe
Closes #11702
Authored by: grqz, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-12-03 01:19:22 +00:00
sepro d8fb349086 [cleanup] Bump ruff to 0.8.x (#11608)
Authored by: seproDev
2024-12-02 16:29:30 +01:00
sepro 2bea793632 [ie/MicrosoftEmbed] Make format extraction non fatal (#11654)
Authored by: seproDev
2024-12-02 16:22:16 +01:00
Elan Ruusamäe 62cba8a1be [ie/duoplay] Fix extractor (#11588)
Authored by: glensc, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-12-01 22:33:11 +00:00
N/Ame 239f5f36fe [ie/bilibili] Fix extractor (#11667)
Closes #11665
Authored by: grqz
2024-12-01 21:55:18 +00:00
bashonly 0d146c1e36 [ie/youtube] Adjust player clients for site changes (#11663)
Closes #11640
Authored by: bashonly
2024-12-01 15:25:09 +00:00
DarkZeros cd0f934604 [ie/mitele] Fix extractor (#11683)
Closes #11690
Authored by: DarkZeros
2024-12-01 14:21:57 +00:00
N/Ame 360aed810a [ie/instagram] Support share URLs (#11677)
Closes #11630
Authored by: grqz
2024-12-01 14:16:50 +00:00
bashonly 00dcde7286 [ie/dropbox] Fix password-protected video extraction (#11636)
Closes #11634
Authored by: bashonly
2024-11-27 01:47:28 +00:00
bashonly 910ecc4229 [ie/tiktok] Deprioritize animated thumbnails (#11645)
Closes #11641
Authored by: bashonly
2024-11-27 00:45:01 +00:00
bashonly 0a0d80800b [ie/dacast] Fix HLS AES formats extraction (#11644)
Closes #11643
Authored by: bashonly
2024-11-26 23:18:48 +00:00
Simon Sawicki e0500cbf79 [ie] Handle fragmented formats in _remove_duplicate_formats (#11637)
Authored by: Grub4K
2024-11-27 00:05:07 +01:00
Jakob Kruse 4b5eec0aaa [ie/chaturbate] Fix support for non-public streams (#11624)
Fix bug in 720b3dc453

Closes #11623
Authored by: jkruse
2024-11-24 22:20:30 +00:00
sepro fe70f20aed [ie/youtube:tab] Fix playlists tab extraction (#11615)
Closes #11524
Authored by: seproDev
2024-11-23 22:46:50 +01:00
coletdjnz c7316373c0 [rh:websockets] Support websockets 14.0+ (#11616)
Authored by: coletdjnz
2024-11-24 10:30:00 +13:00
N/Ame e0f1ae813b [ie/facebook] Support more groups URLs (#11576)
Authored by: grqz
2024-11-23 19:47:37 +00:00
sepro 7d6c259a03 Add playlist_webpage_url field (#11613)
Closes #10827
Authored by: seproDev
2024-11-23 20:42:35 +01:00
gitninja1234 16336c51d0 [ie/stripchat] Fix extractor (#11596)
Closes #11587
Authored by: gitninja1234
2024-11-23 19:40:45 +00:00
bashonly ccf0a6b86b [cleanup] Misc (#11574)
Authored by: bashonly, pzhlkj6612

Co-authored-by: Mozi <29089388+pzhlkj6612@users.noreply.github.com>
2024-11-23 18:51:51 +00:00
github-actions[bot] f919729538 Release 2024.11.18
Created by: bashonly

:ci skip all
2024-11-18 05:45:05 +00:00
bashonly 7ea2787920 [ie/reddit] Improve error handling (#11573)
Authored by: bashonly
2024-11-18 05:36:38 +00:00
bashonly f7257588bd [ie/digitalconcerthall] Support login with access/refresh tokens (#11571)
Removes broken support for login with email and password
Removes obsolete `prefer_combined_hls` extractor-arg

Closes #11404, Closes #11436
Authored by: bashonly
2024-11-18 05:16:17 +00:00
bashonly da252d9d32 [cleanup] Misc (#11554)
Closes #6884
Authored by: bashonly, Grub4K, seproDev

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
Co-authored-by: sepro <sepro@sepr0.com>
2024-11-17 23:25:05 +00:00
gillux e079ffbda6 [ie/litv] Fix extractor (#11071)
Authored by: jiru
2024-11-17 21:37:15 +00:00
bashonly 2009cb27e1 [ie/SonyLIVSeries] Add sort_order extractor-arg (#11569)
Authored by: bashonly
2024-11-17 21:16:22 +00:00
Jackson Humphrey f351440f1d [ie/ctvnews] Fix extractor (#11534)
Closes #8689
Authored by: jshumphrey, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-11-17 21:06:50 +00:00
qbnu f9d98509a8 [ie/ctvnews] Fix playlist ID extraction (#8892)
Authored by: qbnu
2024-11-17 19:35:10 +00:00
sepro 37cd7660ea [ie/youtube:tab] Fix podcasts tab extraction (#11567)
Authored by: seproDev
2024-11-17 19:46:04 +01:00
ChocoLZS d867f99622 [ie/PiaLive] Add extractor (#10811)
Authored by: ChocoLZS
2024-11-17 19:41:57 +01:00
doe1080 10fc719bc7 [cleanup] Remove dead extractors (#11566)
- Removes MildomClipIE, MildomIE, MildomUserVodIE, MildomVodIE
- Removes PokemonIE, PokemonWatchIE
- Removes VeohIE, VeohUserIE

Closes #3373, Closes #7059
Authored by: doe1080
2024-11-17 16:22:40 +00:00
krichbanana eb15fd5a32 [ie/kenh14] Add extractor (#3996)
Closes #3937
Authored by: krichbanana, pzhlkj6612

Co-authored-by: Mozi <29089388+pzhlkj6612@users.noreply.github.com>
2024-11-17 14:12:26 +00:00
sepro 7cecd299e4 [ie/chaturbate] Don't break embed detection (#11565)
Bugfix for 720b3dc453

Authored by: seproDev
2024-11-17 13:32:12 +01:00
bashonly 52c0ffe40a [ie/youtube] Remove broken OAuth support (#11558)
Closes #11462
Authored by: bashonly
2024-11-16 23:40:21 +00:00
sepro 637d62a3a9 [ie/youtube] Player client maintenance (#11528)
Authored by: bashonly, seproDev

Co-authored-by: bashonly <bashonly@protonmail.com>
2024-11-17 00:31:04 +01:00
sepro f95a92b3d0 [cleanup] Deprecate more compat functions (#11439)
Authored by: seproDev
2024-11-17 00:24:11 +01:00
Jackson Humphrey 1d253b0a27 [ie/patreon] Fix comments extraction (#11530)
Closes #11483
Authored by: jshumphrey, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-11-16 20:02:14 +00:00
powergold1 720b3dc453 [ie/chaturbate] Extract from API and support impersonation (#11555)
Closes #6546, Closes #10359
Authored by: powergold1
2024-11-16 19:55:40 +00:00
Jackson Humphrey d215fba7ed [ie/RedGifsUser] Fix extraction (#11531)
Closes #7382, Closes #9131
Authored by: jshumphrey
2024-11-16 19:50:17 +00:00
Jackson Humphrey 8388ec256f [ie/spankbang] Support browser impersonation (#11542)
Closes #6545
Authored by: jshumphrey
2024-11-16 19:48:47 +00:00
sepro 6365e92589 [ie/bandlab] Add extractors (#11535)
Closes #7750
Authored by: seproDev
2024-11-16 17:56:43 +01:00
Alessandro Campolo 70c55cb08f [ie/RadioRadicale] Add extractor (#5607)
Authored by: a13ssandr0, pzhlkj6612

Co-authored-by: Mozi <29089388+pzhlkj6612@users.noreply.github.com>
2024-11-16 13:56:15 +01:00
bashonly c699bafc50 [ie/soop] Fix thumbnail extraction (#11545)
Closes #11537

Authored by: bashonly
2024-11-15 22:51:55 +00:00
bashonly eb64ae7d5d [ie] Allow ext override for thumbnails (#11545)
Authored by: bashonly
2024-11-15 22:51:55 +00:00
Simon Sawicki c014fbcddc [utils] subs_list_to_dict: Add lang default parameter (#11508)
Authored by: Grub4K
2024-11-15 23:25:52 +01:00
Simon Sawicki 39d79c9b9c [utils] Fix join_nonempty, add **kwargs to unpack (#11559)
Authored by: Grub4K
2024-11-15 22:06:15 +01:00
Jackson Humphrey f2a4983df7 [ie/archive.org] Fix comments extraction (#11527)
Closes #11526
Authored by: jshumphrey
2024-11-12 23:26:18 +00:00
bashonly bacc31b05a [ie/facebook] Fix formats extraction (#11513)
Closes #11497
Authored by: bashonly
2024-11-12 23:23:10 +00:00
manav_chaudhary a9f85670d0 [ie/Chaturbate] Support alternate domains (#10595)
Closes #10594
Authored by: manavchaudhary1
2024-11-11 23:41:56 +01:00
Sam 6b43a8d84b [ie/goplay] Fix extractor (#11466)
Closes #10857
Authored by: SamDecrock, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-11-11 22:03:31 +00:00
Hugo 2db8c2e7d5 [ie/CloudflareStream] Avoid extraction via videodelivery.net (#11478)
Closes #11477
Authored by: hugovdev
2024-11-11 22:00:05 +00:00
bashonly f9c8deb4e5 [build] Bump PyInstaller version pin to >=6.11.1 (#11507)
Authored by: bashonly
2024-11-11 21:19:03 +00:00
Sakura286 0ec9bfed4d [ie/MixchMovie] Add extractor (#10897)
Closes #10765
Authored by: Sakura286
2024-11-11 21:40:29 +01:00
Subrat Lima c673731061 [ie/spreaker] Support podcast and feed pages (#10968)
Closes #10925
Authored by: subrat-lima
2024-11-11 20:08:18 +01:00
sepro e398217aae [ie/rutube] Rework extractors (#11480)
Closes #9694, Closes #10104, Closes #11117, Closes #11415, Closes #11476
Authored by: seproDev
2024-11-11 18:44:53 +01:00
Julio Napurí c39016f66d [ie/spreaker] Support episode pages and access keys (#11489)
Authored by: julionc
2024-11-11 18:42:05 +01:00
sepro b83ca24eb7 [core] Catch broken Cryptodome installations (#11486)
Authored by: seproDev
2024-11-10 00:53:49 +01:00
bashonly 240a7d43c8 [build] Pin websockets version to >=13.0,<14 (#11488)
websockets 14.0 causes CI test failures (a lot more of them)

Authored by: bashonly
2024-11-09 23:46:47 +00:00
bashonly f13df591d4 [build] Enable attestations for trusted publishing (#11420)
Reverts 428ffb75aa

Authored by: bashonly
2024-11-09 23:26:02 +00:00
Steve Ovens be3579aaf0 [ie/GameDevTV] Add extractor (#11368)
Authored by: stratus-ss, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-11-06 21:58:44 +00:00
bashonly 85fdc66b6e [ie/adobepass] Fix provider requests (#11472)
Fix bug in dcfeea4dd5

Closes #11469
Authored by: bashonly
2024-11-06 21:26:05 +00:00
github-actions[bot] 282e19db82 Release 2024.11.04
Created by: bashonly

:ci skip all
2024-11-04 00:45:21 +00:00
sepro 197d0b03b6 [cleanup] Misc (#11347)
Closes #11361
Authored by: avagordon01, bashonly, grqz, Grub4K, seproDev

Co-authored-by: Ava Gordon <avagordon01@gmail.com>
Co-authored-by: bashonly <bashonly@protonmail.com>
Co-authored-by: N/Ame <173015200+grqz@users.noreply.github.com>
Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2024-11-04 01:33:21 +01:00
lauren n. liberda b03267bf06 [ie/Tumblr] Support more URLs (#6057)
Closes #5893
Authored by: selfisekai, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2024-11-04 01:26:46 +01:00
Evgeny Zislis 4613096f2e [cookies] Support chrome table version 24 (#11425)
Closes #6564
Authored by: kesor, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2024-11-04 00:59:57 +01:00
bashonly 838f4385de [ie/nfl] Fix extractors (#11409)
Authored by: bashonly
2024-11-03 23:53:26 +00:00
sepro d135823137 [ie/Dailymotion] Support shortened URLs (#11374)
Authored by: seproDev, bashonly
Co-authored-by: bashonly <bashonly@protonmail.com>
2024-11-04 00:27:54 +01:00
chris 59f8dd8239 [ie/ARDMediathek] Extract chapters (#11442)
Authored by: iw0nderhow
2024-11-04 00:11:41 +01:00
Dong Heon Hee 9c6534da81 [ie/chzzk:video] Fix extraction (#11228)
Closes #11226
Authored by: hui1601
2024-11-03 23:08:41 +00:00
Mozi a403dcf9be [ie/Dailymotion] Improve embed extraction (#10843)
Closes #8848, Closes #9432
Authored by: pzhlkj6612, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-11-03 23:02:48 +00:00
sepro 754940e9a5 [ie/bfmtv] Fix extractors (#11444)
Authored by: seproDev
2024-11-03 21:19:35 +01:00
sepro beae2db127 [aes] Fix GCM pad length calculation (#11438)
Closes #10169
Authored by: seproDev
2024-11-03 21:03:09 +01:00
sepro 3945677a75 [core] Prioritize AV1 (#11153)
Authored by: seproDev
2024-11-03 20:39:10 +01:00
bashonly b103aca24d [utils] Fix and improve find_element and find_elements (#11443)
Fix d710a6ca7c

Authored by: bashonly, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2024-11-03 18:19:45 +00:00
Willow 5c7a5aaab2 [ie/Bluesky] Add extractor (#11055)
Closes #10987
Authored by: MellowKyler, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2024-11-03 09:07:25 +01:00
Simon Sawicki 422195ec70 [utils] Allow partial application for even more functions (#11437)
Fixes b6dc2c49e8

Authored by: Grub4K
2024-11-02 21:42:00 +01:00
Nicolas F. a6783a3b99 [ie/yle_areena] Support live events (#11358)
Authored by: CounterPillow, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-10-31 23:23:42 +00:00
bashonly 428ffb75aa [build] Disable attestations for trusted publishing (#11418)
Currently does not work with reusable workflows, e.g. release-nightly.yml calling release.yml

Ref: https://github.com/pypa/gh-action-pypi-publish/releases/tag/v1.11.0
     https://github.com/pypa/gh-action-pypi-publish/discussions/255
     https://github.com/pypi/warehouse/issues/11096

Authored by: bashonly
2024-10-31 09:00:08 +00:00
bashonly b6dc2c49e8 [utils] Allow partial application for more functions (#11391)
Also adds the `trim_str` traversal helper

Authored by: bashonly, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2024-10-30 21:53:41 +00:00
bashonly 76802f4613 [ie/twitter] Remove cookies migration workaround (#11392)
Closes #11338
Authored by: bashonly
2024-10-30 19:26:28 +00:00
bashonly d569a88452 [ie/youtube] Adjust OAuth refresh token handling (#11414)
Removes support for using '' as an empty password in netrc, e.g.:
machine youtube login oauth password ''

Double-quotes ("") are valid and must be used instead, e.g.:
machine youtube login oauth password ""

Authored by: bashonly
2024-10-30 18:58:50 +00:00
bashonly 88402b714e Fix --netrc empty string parsing for Python <=3.10 (#11414)
Ref: https://github.com/python/cpython/commit/15409c720be0503131713e3d3abc1acd0da07378

Closes #11413
Authored by: bashonly, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2024-10-30 18:58:50 +00:00
bashonly 5bc5fb2835 Allow thumbnails with .jpe extension (#11408)
Fix 5ce582448e

Closes #11407
Authored by: bashonly
2024-10-29 23:25:46 +00:00
bashonly f93c16395c [utils] Fix find_element by class (#11402)
Fix d710a6ca7c

Authored by: bashonly
2024-10-29 23:24:17 +00:00
sepro f101e5d34c [ie/Soundcloud] Extract artists (#11377)
Closes #11375
Authored by: seproDev
2024-10-28 12:08:46 +01:00
JAB 330335386d [ie/ccma] Support new 3cat.cat domain (#11222)
Closes #11163
Authored by: JoseAngelB
2024-10-27 23:18:25 +00:00
bashonly 0a3991edae [devscripts] make_changelog: Parse full commit message for fixes (#11366)
Authored by: Grub4K, bashonly

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2024-10-27 23:00:02 +00:00
Simon Sawicki 5c880ef42e [core] Populate format sorting fields before dependent fields (#11353)
Authored by: Grub4K
2024-10-27 00:17:26 +02:00
bashonly 21cdcf03a2 [ie] Resolve language to ISO639-2 for ISM formats (#11359)
Closes #11356
Authored by: bashonly
2024-10-26 18:02:21 +00:00
Wehzuri 6abef74232 [ie/NiconicoUser] Fix extractor (#11324)
Partially addresses #10084

Authored by: Wesley107772
2024-10-26 13:52:09 +00:00
kylegustavo 9acf79c91a [ie/CNN] Fix extractor (#10185)
Closes #2640, Closes #9719
Authored by: kylegustavo, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
2024-10-26 06:06:28 +02:00
bashonly 57212a5f97 [ie/vimeo] Fix API retries (fix c1c9bb4adb) (#11351)
Authored by: bashonly
2024-10-25 18:37:18 +00:00
bashonly c29f5a7fae [ie/generic] Do not impersonate by default (fix edfd095b19) (#11336)
Closes #11335
Authored by: bashonly
2024-10-24 23:11:48 +00:00
kclauhk ec9b25043f [ie/facebook] Fix formats extraction (#11343)
Closes #11337
Authored by: kclauhk
2024-10-24 16:36:09 +00:00
bashonly 914af9a0cf Expand paths in --plugin-dirs (fix 0f593dca9f) (#11334)
Authored by: bashonly
2024-10-23 21:53:51 +00:00
bashonly c998238c2e [cleanup] Misc (#11328)
Closes #11307
Authored by: bashonly, KBelmin

Co-authored-by: Belminho <102482175+KBelmin@users.noreply.github.com>
2024-10-23 21:27:26 +00:00
bashonly d784464399 Remove Python 3.8 support (#11321)
Closes #10086
Authored by: bashonly
2024-10-23 06:33:50 +00:00
bashonly dd2e244469 [build] Use Ubuntu 20.04 and Python 3.9 for Linux ARM builds (#8638)
Authored by: bashonly
2024-10-22 13:09:43 -05:00
bashonly ea9e35d85f [cleanup] Misc (#11311)
Authored by: bashonly
2024-10-22 06:03:30 +00:00
bashonly 87884f1558 [build] Move optional dependencies to the default group (#11255)
Closes #11221
Authored by: bashonly
2024-10-22 05:40:25 +00:00
github-actions[bot] 2a246749ec Release 2024.10.22
Created by: bashonly

:ci skip all
2024-10-22 05:03:55 +00:00
Simon Sawicki 67adeb7bab [cleanup] Misc (#11216)
- Add Python 3.13 to CI, finalize 3.13 support
- Remove Python 3.8 from CI in preparation for removing 3.8 support
- Document that PyPy3.8 and PyPy3.9 are no longer supported
- Usual documentation fixes and code cleanup

Closes #8248, Closes #11146, Closes #11149, Closes #11211
Authored by: Grub4K, grqz, DTrombett, KarboniteKream, bashonly, mikkovedru, seproDev

Co-authored-by: N/Ame <173015200+grqz@users.noreply.github.com>
Co-authored-by: DTrombett <d@trombett.org>
Co-authored-by: =?UTF-8?q?Klemen=20Ko=C5=A1ir?= <klemen.kosir@kream.io>
Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
Co-authored-by: Mikko Vedru <mikko.vedru@gmail.com>
Co-authored-by: sepro <sepro@sepr0.com>
2024-10-22 04:50:35 +00:00
bashonly a886cf3e90 [build] Migrate py2exe builds to win_exe (#11256)
This commit removes py2exe support

Closes #10087
Authored by: bashonly
2024-10-22 04:20:20 +00:00
Allen e68b4c19af [ie/tubitv] Strip extra whitespace from titles (#10795)
Closes #10794
Authored by: allendema
2024-10-22 03:54:41 +00:00
coletdjnz b8635c1d47 [ie/youtube] Support logging in with OAuth (#11001)
See: https://github.com/yt-dlp/yt-dlp/wiki/Extractors#logging-in-with-oauth

Authored by: coletdjnz
2024-10-22 03:46:53 +00:00
63427083dev 46fe60ff19 [ie/afreecatv] Adapt extractors to new sooplive.co.kr domain (#11266)
Closes #11253
Authored by: 63427083, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-10-21 23:42:45 +00:00
DarkZeros 0b7ec08816 [ie/telecinco] Fix extractors (#11142)
Closes #10986, Closes #11106
Authored by: DarkZeros, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-10-21 21:18:12 +00:00
David Skrundz 40054cb4a7 [ie/gem.cbc.ca] Fix formats extraction (#11196)
Also extracts `timestamp` and `release_timestamp` as seconds instead of milliseconds

Authored by: DavidSkrundz
2024-10-21 18:56:43 +00:00
bashonly fed53d70bd [ie/youtube] Remove broken android_producer client (#11297)
Authored by: bashonly
2024-10-21 18:39:58 +00:00
bashonly ec2f4bf082 [ie/youtube] Remove broken age-restriction workaround (#11297)
Closes #11296
Authored by: bashonly
2024-10-21 18:39:58 +00:00
sepro 87408ccfd7 [ie/imgur] Fix thumbnail extraction (#11298)
Authored by: seproDev
2024-10-20 23:18:11 +02:00
Paweł Kotiuk c4d95f67dd [ie/cda] Support folders (#10786)
Closes #5429
Authored by: pktiuk
2024-10-20 23:16:22 +02:00
Deer-Spangle 5af774d7a3 [ie/imgur] Support new URL format (#11075)
Authored by: Deer-Spangle
2024-10-20 22:58:53 +02:00
Imran Hussain 0f593dca9f Add option --plugin-dirs (#11277)
Closes #3260
Authored by: imranh2, coletdjnz

Co-authored-by: coletdjnz <coletdjnz@protonmail.com>
2024-10-20 17:10:26 +00:00
sepro 8de431ec97 [ie/Funk] Extend _VALID_URL (#11269)
Authored by: seproDev
2024-10-20 15:18:15 +02:00
rubyevadestaxes 679c68240a [ie/twitter:spaces] Allow extraction when not logged in (#11289)
Closes #11288
Authored by: rubyevadestaxes
2024-10-19 21:51:47 +00:00
bashonly 3148c1822f [ie/substack] Resolve podcast file extensions (#11275)
Closes #4601
Authored by: bashonly
2024-10-19 21:41:14 +00:00
bashonly 7af1ddaaf2 [ie/youtube] Fix comment_count extraction (#11274)
Authored by: bashonly
2024-10-19 21:40:20 +00:00
bashonly fbc66e3ab3 [utils] Popen: Reset PyInstaller environment (#11258)
- Forces spawning independent subprocesses for exes bundled with PyInstaller>=6.10
- Fixes regression introduced in fb8b7f226d
- Ref: https://pyinstaller.org/en/v6.10.0/CHANGES.html#incompatible-changes

Closes #11259
Authored by: bashonly, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2024-10-16 03:53:53 +00:00
bashonly 64d84d75ca [build] Use macos-13 image for macOS builds (#11236)
Authored by: bashonly
2024-10-15 07:07:42 +00:00
bashonly dcfeea4dd5 [ie/adobepass] Use newer user-agent for provider redirect request (#11250)
Closes #10848
Authored by: bashonly
2024-10-14 22:19:26 +00:00
Mozi cba7868502 [ie/reddit] Detect and raise when login is required (#11202)
Closes #10924
Authored by: pzhlkj6612
2024-10-13 06:27:01 +00:00
Simon Sawicki d710a6ca7c Add extractor helpers (#10653)
Authored by: Grub4K
2024-10-13 05:14:32 +02:00
Simon Sawicki 85b87c991a [utils] sanitize_path: Reimplement function (#11198)
Authored by: Grub4K
2024-10-13 04:10:12 +02:00
Simon Sawicki 16eb28026a [test] Allow running tests explicitly (#11203)
Authored by: Grub4K
2024-10-13 04:01:26 +02:00
Simon Sawicki 1a830394a2 [build] make_lazy_extractors: Force running without plugins (#11205)
Authored by: Grub4K
2024-10-13 03:50:31 +02:00
Simon Sawicki edfd095b19 [ie/generic] Impersonate browser by default (#11206)
Also adds `impersonate` extractor arg

Authored by: Grub4K
2024-10-13 03:42:43 +02:00
Simon Sawicki c5f0f58efd [cookies] Fix compatibility for Python <=3.9 in traceback
Authored by: Grub4K
2024-10-13 03:38:09 +02:00
bashonly babb709605 [ie/patreon:campaign] Stricter URL matching (#11235)
Redefinition of suitable() is no longer necessary

Closes #11233
Authored by: bashonly
2024-10-12 23:23:03 +00:00
bashonly 9d43dcb2c5 [ie/cwtv] Fix extraction (#11230)
Closes #9935
Authored by: bashonly
2024-10-12 17:47:37 +00:00
444995 f433871424 [ie/drtv] Fix extractor (#11141)
Closes #11137
Authored by: 444995
2024-10-11 22:39:00 +00:00
vvto33 ceaea731b6 [ie/TVer] Support series URLs (#9507)
Also improve thumbnails extraction

Authored by: vvto33, pzhlkj6612

Co-authored-by: Mozi <29089388+pzhlkj6612@users.noreply.github.com>
2024-10-10 21:42:34 +00:00
bashonly 5310fa87f6 [ie/weverse] Fix extractor (#11215)
Closes #11213
Authored by: bashonly
2024-10-10 21:29:22 +00:00
github-actions[bot] 983c58fb7a Release 2024.10.07
Created by: bashonly

:ci skip all
2024-10-07 23:41:00 +00:00
bashonly 1a176d874e [cleanup] Misc
Authored by: bashonly
2024-10-07 18:33:33 -05:00
poyhen 079a7bc334 [ie/instagram] Do not hardcode user-agent (#11155)
Closes #10700
Authored by: poyhen
2024-10-07 23:28:08 +00:00
tetra cf85cba5d9 [ie/instagram] Fix extractor (#11156)
Closes #11151
Authored by: tetra-fox
2024-10-07 23:25:54 +00:00
kclauhk 4b7bec66d8 [ie/cwtv] Fix extractor (#11135)
Closes #11131
Authored by: kclauhk
2024-10-07 23:24:31 +00:00
BallzCrasher ccb23e1bac [ie/noodlemagazine] Fix extractor (#11144)
Closes #9936
Authored by: BallzCrasher
2024-10-07 23:23:48 +00:00
Eric Lammerts 3a193346ee [pp/XAttrMetadata] Try to write each attribute (#11115)
Authored by: eric321
2024-10-07 23:17:55 +00:00
sepro de2062753a [ie/youtube] Change default player clients to ios,mweb (#11190)
Closes #11165, Closes #11185
Authored by: seproDev
2024-10-07 23:12:00 +00:00
Simon Sawicki e59c82a74c [cookies] Fix cookie load error handling (#11140)
Authored by: Grub4K
2024-10-01 02:13:48 +02:00
bashonly f91645acea [ie/patreon] Extract all m3u8 formats for locked posts (#11138)
Closes #11125
Authored by: bashonly
2024-09-30 22:42:30 +00:00
Simon Sawicki b31b81d85f [ci] Rerun failed tests (#11143) 2024-10-01 00:33:17 +02:00
Corey Wright 6328e2e67a [ie/ApplePodcasts] Fix extractor (#10903)
Closes #10809
Authored by: coreywright
2024-09-29 23:03:39 +02:00
github-actions[bot] 5945fc1945 Release 2024.09.27
Created by: bashonly

:ci skip all
2024-09-27 23:01:13 +00:00
bashonly c6387abc1a [cleanup] Misc (#10807)
Closes #10751, Closes #10769, Closes #10791
Authored by: bashonly, Codenade, pzhlkj6612, seproDev, coletdjnz, grqz, Grub4K

Co-authored-by: Codenade <amadeus.dorian04@gmail.com>
Co-authored-by: Mozi <29089388+pzhlkj6612@users.noreply.github.com>
Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
Co-authored-by: coletdjnz <coletdjnz@protonmail.com>
Co-authored-by: N/Ame <173015200+grqz@users.noreply.github.com>
Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2024-09-27 22:46:22 +00:00
bashonly cca534cd9e Raise minimum recommended Python version to 3.9 (#11098)
Authored by: bashonly
2024-09-27 22:30:31 +00:00
kclauhk 7509d692b3 [ie/loom] Fix m3u8 formats extraction (#10760)
Closes #10737
Authored by: kclauhk
2024-09-27 22:28:22 +00:00
ndyanx 63da31b3b2 [ie/dropbox] Fix password-protected video support (#10735)
Also adds thumbnail extraction

Closes #9864
Authored by: ndyanx
2024-09-27 22:05:22 +00:00
rakslice 8f4ea14680 Fix format sorting bug with vp9.2 vcodec (#10884)
Authored by: rakslice
2024-09-27 21:32:39 +00:00
fireattack a1b4ac2b8e [ie/vimeo] Fix HLS audio format sorting (#11082)
Closes #10854
Authored by: fireattack
2024-09-27 20:57:57 +00:00
Kieran c08e0b20b5 Allow none arg to negate --convert-subs and --convert-thumbnails (#11066)
Authored by: kieraneglin
2024-09-27 20:52:41 +00:00
bashonly 0aa4426e9a [ie/kick:clips] Support new URL format (#11107)
Closes #11105
Authored by: bashonly
2024-09-27 16:38:40 +00:00
bashonly 48d629d461 [ie/YleAreena] Support podcasts (#11104)
Closes #10840
Authored by: bashonly
2024-09-27 16:38:08 +00:00
bashonly 7f909046f4 [ie/abc.net.au:iview:showseries] Fix extraction (#11101)
Closes #10475
Authored by: bashonly
2024-09-27 16:37:16 +00:00
bashonly eabb4680fd [ie/niconico] Fix m3u8 formats extraction (#11103)
Closes #10724
Authored by: bashonly
2024-09-26 23:27:16 +00:00
bashonly 1d84b780cf [ie/youtube:clip] Prioritize https formats (#11102)
Closes #10856
Authored by: bashonly
2024-09-26 23:26:10 +00:00
bashonly 9f5c9a9089 [ie/wistia] Support password-protected videos (#11100)
Closes #10914
Authored by: bashonly
2024-09-26 23:21:03 +00:00
bashonly a2000bc857 [ie/bilibili] Fix chapters and subtitles extraction (#11099)
Closes #11089
Authored by: bashonly
2024-09-26 23:20:14 +00:00
diman8 5a8a05aebb [ie/SVTPage] Fix extractor (#11010)
Authored by: diman8
2024-09-26 16:57:00 +00:00
tony-hn ad0b857f45 [ie/RumbleChannel] Fix extractor (#11049)
Closes #10833
Authored by: tony-hn
2024-09-26 16:53:52 +00:00
N/Ame 124f058b54 [ie/Germanupa] Add extractor (#10538)
Closes #10527
Authored by: grqz
2024-09-26 18:39:48 +02:00
sepro 416686ed0c [ie/ertgr] Fix video extraction (#11091)
Closes #8955
Authored by: seproDev
2024-09-26 18:35:19 +02:00
sepro b37417e4f9 [ie/SnapchatSpotlight] Add extractor (#11030)
Closes #1797
Authored by: seproDev
2024-09-26 18:32:51 +02:00
Mozi 28b0ecba2a [ie/Mojevideo] Add extractor (#11019)
Closes #8159
Authored by: 04-pasha-04, pzhlkj6612

Co-authored-by: pasha <pasha.syd04@gmail.com>
2024-09-26 18:29:21 +02:00
szantnerb e2b3634e29 [ie/mediaklikk] Fix extractor (#11083)
Closes #11061
Authored by: szantnerb
2024-09-26 18:23:26 +02:00
bashonly fb8b7f226d [build] Bump PyInstaller version pin to >=6.10.0 (#10709)
Authored by: bashonly
2024-09-25 23:07:17 +00:00
sepro b397a64691 [cookies] Improve error message for Windows --cookies-from-browser chrome issue (#11090)
Authored by: seproDev
2024-09-25 23:13:54 +02:00
bashonly 5bb1aa04da [networking] Pin curl-cffi version to < 0.7.2 (#11092)
Ref: https://github.com/lexiforest/curl_cffi/issues/394

Authored by: bashonly
2024-09-25 20:59:20 +00:00
bashonly fa2be9a7c6 [ie/youtube] Fix format_note (Bugfix for 3a3bd00037) (#11028)
Authored by: bashonly
2024-09-24 22:12:02 +00:00
bashonly 3ad0b7f422 [ie/tiktok] Fix web formats extraction (#11074)
Closes #11034
Authored by: bashonly
2024-09-24 22:10:42 +00:00
1-Byte 4a9bc8c363 [ie/NZZ] Fix extractor (#10461)
Closes #5653
Authored by: 1-Byte
2024-09-17 21:17:05 +02:00
Khaoklong51 a06bb58679 [ie/BiliIntl] Fix referer header (#11003)
Closes #10996
Authored by: Khaoklong51
2024-09-14 16:19:17 +00:00
bashonly a555389c9b [ie/HGTVDe] Fix extractor (#10992)
Closes #10984
Authored by: bashonly, rdamas

Co-authored-by: Robert Damas <robert.damas@byom.de>
2024-09-14 00:23:22 +00:00
N/Ame 173d54c151 [ie/kick:vod] Support new URL format (#10988)
Closes #10975
Authored by: grqz, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-09-14 00:21:07 +00:00
Oto Valek 4a27b8f092 [ie/IPrima] Fix zoom URL support (#10959)
Closes #6100
Authored by: otovalek
2024-09-14 00:19:03 +00:00
sepro 41a241ca6f [ie/Sen] Add extractor (#10952)
Closes #10951
Authored by: seproDev
2024-09-14 00:16:34 +00:00
sepro 3aa0156e05 [ie/Xinpianchang] Fix extractor (#10950)
Authored by: seproDev
2024-09-14 00:15:07 +00:00
sepro 300c91274f [ie/Servus] Fix extractor (#10944)
Closes #10941
Authored by: seproDev
2024-09-14 00:14:09 +00:00
aarubui d8d473002b [ie/tenplay] Fix extractor (#10928)
Closes #10926
Authored by: aarubui
2024-09-14 00:09:15 +00:00
naglis 36f9e602ad [ie/screenrec] Add extractor (#10917)
Closes #9780
Authored by: naglis
2024-09-13 23:27:10 +00:00
ischmidt20 7adff8caf1 [ie/WatchESPN] Improve auth support (#10910)
Authored by: ischmidt20
2024-09-13 23:25:12 +00:00
naglis fa83d0b36b [ie/LnkGo] Remove extractor (#10904)
Authored by: naglis
2024-09-13 23:23:19 +00:00
Sahil Singh c8c078fe28 [ie/pinterest] Extend _VALID_URL (#10867)
Closes #10850
Authored by: sahilsinghss73, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-09-13 23:22:14 +00:00
bashonly 325001317d [ie] Handle decode errors when reading responses (#10868)
Authored by: bashonly
2024-09-13 23:20:17 +00:00
bashonly cc85596d5b [utils] mimetype2ext: Recognize aacp as aac (#10860)
Authored by: bashonly
2024-09-13 23:19:18 +00:00
Leng 0e1b941c6b [ie/facebook:reel] Improve metadata extraction
Closes #9057, Closes #10824
Authored by: lengzuo
2024-09-13 23:18:13 +00:00
Xingchen Song(宋星辰) 3dfd720d09 [ie/ximalaya] Add VIP support (#10832)
Closes #6928
Authored by: xingchensong, seproDev

Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
2024-09-13 23:16:34 +00:00
hugepower 25c1cdaa26 [ie/huya:video] Add extractor (#10686)
Closes #10679
Authored by: hugepower
2024-09-13 23:12:38 +00:00
Cosmin Tanislav d02df303d8 [ie/RTP] Support more subpages (#10787)
Authored by: Demon000
2024-09-13 23:09:52 +00:00
Scott Robinson 5d0176547f [ie/Bandcamp:user] Fix extraction (#10328)
Authored by: quad, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-09-13 23:02:54 +00:00
sepro 409f8e9e3b [ie] Fix JW Player format parsing (#10956)
Authored by: seproDev
2024-09-13 22:54:41 +00:00
Deukhoofd b4760c778d [ie/beacon] Add extractor (#9901)
Authored by: Deukhoofd
2024-09-13 22:50:15 +00:00
sepro 9431777b4c [ie/youtube:tab] Fix shorts tab extraction (#10938)
Closes #10936
Authored by: seproDev
2024-09-13 22:46:44 +00:00
sepro 3a3bd00037 [ie/youtube] Add po_token, visitor_data, data_sync_id extractor args (#10648)
Authored by:  seproDev, coletdjnz, bashonly
2024-09-13 22:51:58 +12:00
coletdjnz d1c4d88b2d [networking] Fix handler not being added to RequestError (#10955)
Authored by: coletdjnz
2024-09-08 19:32:44 +12:00
sepro 46f4c80bc3 [ie/SampleFocus] Fix extractor (#10947)
Closes #10945
Authored by: seproDev
2024-09-07 17:06:12 +02:00
sepro 0fba08485b [ie/khanacademy] Fix extractor (#10913)
Closes #10912
Authored by: seproDev
2024-09-05 20:47:14 +02:00
Simon Sawicki b6200bdcf3 [ci] Add comment sanitization workflow (#10915)
Co-authored-by: bashonly <bashonly@protonmail.com>
Authored by: bashonly, Grub4K
2024-09-05 20:06:15 +02:00
sepro e8e6a982a1 [ie/vimeo] Fix login detection (bugfix for 4115c24d15) (#10906)
Authored by: seproDev
2024-09-02 21:20:37 +02:00
bashonly 7e41628ff5 [build] Pin delocate version for macos (#10901)
Authored by: bashonly
2024-09-01 23:56:50 +00:00
Frank Aurich e6f48ca808 [ie/KiKA] Add extractor (#5788)
Authored by: 1100101
2024-09-02 01:28:51 +02:00
bashonly 4115c24d15 [ie/vimeo] Always try to extract original format (#10721)
Closes #9163
Authored by: bashonly
2024-09-01 23:25:36 +00:00
bashonly ad9a8115aa [ci] Add issue tracker anti-spam protection (#10861)
Authored by: bashonly
2024-08-28 08:01:51 +00:00
Mozi 41be32e78c [ie/Rutube] Support livestreams (#10844)
Closes #4418, Closes #4594
Authored by: pzhlkj6612
2024-08-26 23:17:25 +02:00
Mozi e978c312d6 [ie/Vidflex] Add extractor (#10002)
Closes #1377
Authored by: pzhlkj6612
2024-08-26 22:56:36 +02:00
coletdjnz 6f9e653743 [rh:websockets] Upgrade websockets to 13.0 (#10815)
Fixes CI hanging

Authored by: coletdjnz
2024-08-21 19:17:26 +12:00
sepro f0bb28504c [ie/Eurosport] Support local URL variants (#10785)
Authored by: seproDev
2024-08-20 00:12:42 +02:00
bashonly bef1d4d6fc [ie/twitter:spaces] Support video spaces (#10789)
Authored by: bashonly
2024-08-19 15:38:19 +00:00
garret1317 c8d096c5ce [ie/radiko] Extract unique id values (#10726)
Authored by: garret1317
2024-08-19 15:22:19 +00:00
Mozi a7d3235c84 [ie/asobistage] Support redirected URLs (#10768)
Authored by: pzhlkj6612
2024-08-18 16:50:06 +00:00
kclauhk d62fef7e07 [ie/facebook:ads] Fix extractor (#10704)
Closes #10701
Authored by: kclauhk
2024-08-15 19:53:37 +00:00
Christopher Schreiner cc88a54bb1 [ie/adn] Fix extractors (#10749)
Closes #10748
Authored by: infanf
2024-08-15 19:50:08 +00:00
N/Ame b43bd86485 [ie/bilibili] Fix festival URL support (#10740)
Closes #10739
Authored by: grqz, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-08-15 19:33:41 +00:00
Hank Brown 232e6db30c [ie/PatreonCampaign] Support API URLs (#10734)
Closes #10733
Authored by: hibes, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-08-13 23:26:55 +00:00
bashonly 49f3741a82 [ie/youtube] Support excluding player_clients in extractor-arg (#10710)
Closes #10699
Authored by: bashonly
2024-08-12 09:12:46 +00:00
github-actions[bot] a065086640 Release 2024.08.06
Created by: bashonly

:ci skip all :ci run dl
2024-08-06 03:03:12 +00:00
bashonly 4d92312083 [ie/niconico] Fix extractor (#10677)
Closes #10662
Authored by: bashonly
2024-08-06 02:50:06 +00:00
scribblemaniac fc5eecfa31 [ie/gem.cbc.ca:live] Fix extractor (#10565)
Authored by: scribblemaniac, bashonly
2024-08-06 01:02:21 +00:00
bashonly 406f4c2e47 [ie/youtube] Change default player clients to ios,web_creator (#10674)
Closes #10660
Authored by: bashonly
2024-08-05 23:26:50 +00:00
sepro c86891eb94 [ie/youtube] Fix n function name extraction for player b12cc44b (#10668)
Authored by: seproDev
2024-08-05 20:36:11 +00:00
sepro bb8bf1db99 [jsinterp] Improve slice implementation (#10664)
Authored by: seproDev
2024-08-05 20:28:24 +00:00
bashonly e7d73bc453 [ie/DiscoveryPlusItaly] Support sport and olympics URLs (#10655)
Closes #10654
Authored by: bashonly
2024-08-04 15:20:45 +00:00
bashonly 919540a964 [ie/olympics] Fix extraction (#10625)
Bugfix for 2f1ddfe12a

Closes #10592
Authored by: bashonly
2024-08-01 20:25:46 +00:00
hugepower 0088c6de23 [ie/youku] Fix extractor (#10626)
Closes #10549
Authored by: hugepower
2024-08-01 16:40:46 +00:00
github-actions[bot] abe10131fc Release 2024.08.01
Created by: bashonly

:ci skip all :ci run dl
2024-08-01 15:11:19 +00:00
bashonly ffd7781d65 [cleanup] Misc (#10623)
Authored by: bashonly
2024-08-01 15:03:49 +00:00
sepro efb42763de [ie/youtube] Change default player clients to ios,tv (#10457)
Closes #10046
Authored by: seproDev
2024-08-01 14:03:03 +00:00
Oğulcan Tokar bb3936ae2b [ie/kick:clips] Add extractor (#10572)
Closes #8115
Authored by: luvyana
2024-08-01 00:00:52 +00:00
bashonly d19fcb9342 [ie/youtube] Fix age-verification workaround (#10610)
Authored by: bashonly, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2024-07-31 21:39:36 +00:00
bashonly 011b4a04db [ie/youtube] Fix n function name extraction for player 20dfca59 (#10611)
Closes #10608
Authored by: bashonly
2024-07-31 21:19:30 +00:00
szantnerb 7e3e4779ad [ie/mediaklikk] Fix extractor (#10605)
Closes #10588
Authored by: szantnerb
2024-07-31 02:22:44 +00:00
vvto33 5260696b1c [ie/tver] Support olympic URLs (#10600)
Closes #10583
Authored by: vvto33
2024-07-31 02:18:43 +00:00
bashonly 2f1ddfe12a [ie/olympics] Fix extractor (#10604)
Closes #10592
Authored by: bashonly
2024-07-31 01:50:20 +00:00
bashonly 4b69e1b53e [ie/mlbtv] Fix makeup game extraction (#10607)
Closes #10606
Authored by: bashonly
2024-07-30 23:17:05 +00:00
bashonly 0e539617a4 [ie/youtube] Player client maintenance (#10573)
- Add clients: android_producer, android_testsuite, android_vr, tv, web_safari
- Remove obsolete clients: android_embedded, ios_embedded, *_embedscreen

Authored by: bashonly
2024-07-30 21:27:06 +00:00
bashonly fe15d3178e [ie/learningonscreen] Add extractor (#10590)
Authored by: bashonly, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2024-07-30 09:09:55 +00:00
trainman261 94a1c5e642 [ie/cbc.ca:player] Fix extractor (#10302)
Closes #10170
Authored by: trainman261, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-07-29 21:58:26 +00:00
bashonly 2b6df93a24 [ie/vimeo:review] Fix password-protected video extraction (#10598)
Closes #10255
Authored by: bashonly
2024-07-29 21:55:06 +00:00
middlingphys ef36d517f9 [ie/abematv] Fix availability extraction (#10569)
Authored by: middlingphys
2024-07-29 05:54:59 +00:00
bashonly 6daf2c27c0 [utils] unified_timestamp: Recognize Sunday (#10589)
Authored by: bashonly
2024-07-29 05:35:46 +00:00
bashonly 28d485714f [ie/tva] Fix extractor (#10567)
Closes #10555
Authored by: bashonly
2024-07-25 22:30:00 +00:00
bashonly 0b77286184 [ie/DiscoveryPlus] Support olympics URLs (#10566)
Closes #10564
Authored by: bashonly
2024-07-25 22:00:58 +00:00
github-actions[bot] 6b1e430d8e Release 2024.07.25
Created by: bashonly

:ci skip all :ci run dl
2024-07-25 03:29:27 +00:00
bashonly f0993391e6 [ie/mlbtv] Fix extractor (#10515)
Closes #10510
Authored by: bashonly
2024-07-24 21:22:55 +00:00
bashonly 1a34a802f4 [ie/facebook] Fix extraction (#10531)
Closes #10532
Authored by: bashonly
2024-07-23 23:08:24 +00:00
bashonly a0a1bc3d8d [ie/vimeo] Fix chapters extraction (#10544)
Closes #5308
Authored by: bashonly
2024-07-23 22:00:57 +00:00
bashonly 2f97779f33 [ie/tiktok] Fix and deprioritize JSON subtitles (#10516)
Fixes regression caused by 5ce582448e

Closes #10514
Authored by: bashonly
2024-07-23 21:49:31 +00:00
bashonly 713b4cd18f [ie/youtube] Fix n function name extraction for player 3400486c (#10542)
Authored by: bashonly
2024-07-23 21:25:49 +00:00
bashonly a3bab4752a [ie/abematv] Adapt key retrieval to request handler framework (#10491)
Fixes a regression caused by a dependence on buggy behavior that was corrected in 150ecc45d9

Closes #10489
Authored by: bashonly
2024-07-18 20:43:31 +00:00
bashonly e046db8a11 [build] Pin setuptools version (#10493)
https://github.com/pypa/setuptools/issues/4480#issuecomment-2236507819
https://github.com/pypa/setuptools/issues/4482

Authored by: bashonly
2024-07-18 20:33:28 +00:00
github-actions[bot] 37c233562d Release 2024.07.16
Created by: bashonly

:ci skip all :ci run dl
2024-07-16 22:08:42 +00:00
bashonly 89a161e8c6 [cleanup] Misc (#10487)
Closes #10483
Authored by: bashonly
2024-07-16 22:01:01 +00:00
bashonly ed1b9ed93d [update] Fix network error handling (#10486)
Authored by: bashonly
2024-07-16 21:10:50 +00:00
Simon Sawicki d9cbced493 [core] Support auto-tty and no_color-tty for --color (#10453)
Authored by: Grub4K
2024-07-16 21:51:56 +02:00
Simon Sawicki 66ce3d76d8 [core] Fix noprogress if test=True with --quiet and --verbose (#10454)
Authored by: Grub4K
2024-07-16 21:48:45 +02:00
bashonly 39e6c4cb44 [ie/dplay] Fix extractors (#10471)
Closes #1623, Closes #2138, Closes #2361, Closes #3841, Closes #8026, Closes #10421
Authored by: bashonly
2024-07-15 22:30:43 +00:00
bashonly e62fa6b0e0 [ie/digitalconcerthall] Extract HEVC and FLAC formats (#10470)
Authored by: bashonly
2024-07-14 22:56:28 -05:00
bashonly cc0070f649 [utils] parse_codecs: Fix parsing of mixed case codec strings
Authored by: bashonly
2024-07-14 22:56:28 -05:00
sepro b85eef0a61 [ie/youtube] Reduce android client priority (#10467)
Authored by: seproDev
2024-07-14 21:10:29 +02:00
DunnesH 22870b81ba [ie/soundcloud:user:permalink] Extract tracks only (#10463)
Closes #10242
Authored by: DunnesH
2024-07-14 19:01:50 +00:00
bashonly b9afb99e7c [ie/generic] Fix direct video link extensions (#10468)
Fixes regression in the generic extractor due in 5ce582448e

Closes #10459
Authored by: bashonly
2024-07-14 18:57:07 +00:00
sepro 16da8ef993 [ie/youtube] Fix initial player response usage (Bugfix for 8b8b442cb0) (#10464)
Authored by: seproDev
2024-07-14 20:42:11 +02:00
Christopher Schreiner 959b7a379b [ie/adn] Adjust for .com domain change (#10399)
Closes #10442
Authored by: infanf
2024-07-14 15:58:05 +00:00
Ian Comaya 8531d2b03b [ie/EpidemicSound] Support sound effects URLs (#10436)
Closes #10435
Authored by: iancmy
2024-07-14 04:52:50 +00:00
bashonly 4cd4146924 [ie/afreecatv] Fix login and use legacy_ssl (#10440)
Fixes regression in e8352ad659 due to cookies bug in curl_cffi < 0.7.1

Closes #10438
Authored by: bashonly
2024-07-14 01:09:00 +00:00
Franklin Lee bacd18b7df [ie/picarto] Fix extractors (#10414)
Closes #10413
Authored by: Frankgoji
2024-07-14 00:16:18 +00:00
coletdjnz 150ecc45d9 [networking] Add legacy_ssl request extension (#10448)
Supported by Urllib, Requests and Websockets request handlers. Ignored by CurlCFFI.

Also added couple cookie-related tests.

Authored by: coletdjnz
2024-07-14 11:22:43 +12:00
sepro 8b8b442cb0 [ie/youtube] Avoid poToken experiment player responses (#10456)
Closes #10397
Authored by: seproDev
2024-07-14 01:19:17 +02:00
bashonly 644d84d778 Revert 4f8448896e
curl-cffi 0.5.10 does not support Windows 32-bit

Authored by: bashonly
2024-07-12 14:34:19 -05:00
bashonly ac30941ae6 [build] Pin curl-cffi to 0.5.10 for Windows
Ref: https://github.com/yifeikong/curl-impersonate/issues/72

Closes #10426
Authored by: bashonly
2024-07-12 14:34:19 -05:00
bashonly cc1a3098c0 [ie/tv5monde] Fix impersonation (Bugfix for 9b95a6765a) (#10430)
Authored by: bashonly
2024-07-11 17:22:37 +00:00
sepro 705f5b84de [ie/box] Support enterprise URLs (#10419)
Closes #10418
Authored by: seproDev
2024-07-10 21:48:50 +02:00
bashonly 9b95a6765a [ie/tv5monde] Support browser impersonation (#10417)
Closes #10153
Authored by: bashonly
2024-07-10 15:13:47 +00:00
bashonly 4f8448896e [build] Include curl_cffi in yt-dlp_x86.exe
Authored by: bashonly
2024-07-09 18:36:15 -05:00
bashonly 4521f30d14 [build] Include curl_cffi in yt-dlp_linux
Authored by: bashonly
2024-07-09 18:36:15 -05:00
coletdjnz 42bfca00a6 [rh:curl_cffi] Support curl_cffi 0.7.X
Authored by: coletdjnz
2024-07-09 18:36:15 -05:00
mokrueger d2189d3d36 [ie/tiktok:live] Fix room ID extraction (#10408)
Closes #10407
Authored by: mokrueger
2024-07-09 23:27:01 +00:00
bashonly 04e17ba20a [ie/youtube] Invalidate nsig cache from < 2024.07.09 (#10401)
Versions after 297b0a3792 and before 7ead7332af may have cached incorrect nsig function data

Authored by: bashonly
2024-07-09 19:04:46 +00:00
github-actions[bot] bbf84bf55e Release 2024.07.09
Created by: seproDev

:ci skip all :ci run dl
2024-07-09 01:51:07 +00:00
sepro 7ead7332af [ie/youtube] Remove broken n function extraction fallback (#10396)
Closes #10391
Authored by: pukkandan, seproDev

Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
2024-07-09 03:45:14 +02:00
sepro 0b570f2a90 [core] Do not alter default format selection when simulated (#9862)
Closes #9843
Authored by: seproDev
2024-07-09 01:51:43 +02:00
github-actions[bot] 1a6ac547ea Release 2024.07.08
Created by: bashonly

:ci skip all :ci run dl
2024-07-08 22:19:18 +00:00
bashonly 4b50b292cc [ie/soundcloud] Fix rate-limit handling (#10389)
Authored by: bashonly
2024-07-08 22:09:08 +00:00
bashonly 297b0a3792 [ie/youtube] Fix JS n function name extraction (#10390)
Fixes nsig decoding for player b22ef6e7

Closes #10391
Authored by: bashonly, seproDev

Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
2024-07-08 22:04:48 +00:00
Simon Sawicki 6c056ea7ae [jsinterp] Implement Function.prototype resolving for call and apply (#10392)
Authored by: Grub4K
2024-07-08 23:46:26 +02:00
github-actions[bot] 39bc699d2e Release 2024.07.07
Created by: bashonly

:ci skip all :ci run dl
2024-07-07 21:35:02 +00:00
bashonly b337d2989c [cleanup] Misc (#10383)
Authored by: bashonly
2024-07-07 21:23:40 +00:00
Hardik Bhimani f0f867f008 [ie/jiosaavn:playlist] Support featured playlists (#10382)
Closes #10369
Authored by: harbhim
2024-07-07 21:08:25 +00:00
DinhHuy2010 987a1f94c2 [ie/vtv] Add extractors (#10173)
Authored by: DinhHuy2010
2024-07-07 21:59:42 +02:00
sepro 4cdc976bd8 [ie/yle_areena] Fix metadata extraction (#10380)
Authored by: seproDev
2024-07-07 21:57:18 +02:00
Simon Sawicki 0d174e8bed [ie/yle_areena] Fix subtitle extraction (#10379)
Authored by: Grub4K
2024-07-07 21:21:00 +02:00
Dong Heon Hee 4862a29854 [ie/chzzk] Extract with API v3 (#10363)
Authored by: hui1601
2024-07-06 03:32:08 +00:00
bashonly 2469119490 [core] Address gaps in allowed extensions (#10362)
Adds some extensions missing in 5ce582448e

Closes #10360, Closes #10365
Authored by: bashonly
2024-07-05 23:17:47 +00:00
Sean Ellingham 00766ece0c [ie/vidyard] Add extractor (#10155)
Closes #4618
Authored by: exterrestris
2024-07-05 23:02:35 +00:00
middlingphys 2a1a1b8e67 [ie/abematv] Extract availability (#10348)
Authored by: middlingphys
2024-07-05 22:31:16 +00:00
bashonly c1c9bb4adb [ie/vimeo] Fix password-protected video extraction (#10341)
Closes #6603
Authored by: bashonly
2024-07-05 18:32:53 +00:00
Thomas Gerbet 6075a029db [ie/douyutv] Do not use dangerous javascript source/URL (#10347)
Ref: https://sansec.io/research/polyfill-supply-chain-attack

Authored by: LeSuisse
2024-07-03 22:35:24 +00:00
bashonly cc767e9490 [core] Fix --ignore-no-formats-error (#10345)
Fixes regression in 5ce582448e

Closes #10344
Authored by: Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2024-07-03 16:46:01 +00:00
github-actions[bot] d28aa87e21 Release 2024.07.02
Created by: bashonly

:ci skip all :ci run dl
2024-07-02 23:13:48 +00:00
bashonly 93d33cb29a [cleanup] Misc (#10330)
Authored by: bashonly
2024-07-02 23:03:08 +00:00
Mozi 7799e51895 [ie/zaiko] Support JWT video URLs (#10130)
Closes #9798
Authored by: pzhlkj6612
2024-07-02 22:22:52 +00:00
Patryk Miś 7509791385 [ie/banbye] Fix extractor (#10332)
Closes #8584
Authored by: PatrykMis, seproDev

Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
2024-07-02 21:51:07 +00:00
DrakoCpp 6403530e2d [ie/murrtube] Fix extractor (#9249)
Closes #7500
Authored by: DrakoCpp
2024-07-02 21:49:09 +00:00
bashonly d502f4c6d9 [pp/embedthumbnail] Fix embedding with mutagen (#10337)
Fixes regression in f2a4ea1794

Closes #10335
Authored by: bashonly
2024-07-02 21:24:17 +00:00
bashonly 773bbb1815 [core] Fix --compat-opt allow-unsafe-ext (#10336)
Fixes bug in 5ce582448e

Authored by: bashonly, rdamas

Co-authored-by: Robert Damas <robert.damas@byom.de>
2024-07-02 21:17:06 +00:00
github-actions[bot] cd68258225 Release 2024.07.01
Created by: Grub4K

:ci skip all :ci run dl
2024-07-01 23:01:05 +00:00
Simon Sawicki 5ce582448e [core] Disallow unsafe extensions (CVE-2024-38519)
Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-79w7-vh3h-8g4j

Authored by: Grub4K
2024-07-02 00:58:40 +02:00
bashonly 6aaf96a3d6 [cleanup] Misc (#10075)
Closes #10303
Authored by: bashonly, seproDev, jucor, c-basalt

Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
Co-authored-by: Julien Cornebise <julien@cornebise.com>
Co-authored-by: c-basalt <117849907+c-basalt@users.noreply.github.com>
2024-07-01 22:51:27 +00:00
bashonly d4b99a2333 [ie/vimeo] Support browser impersonation (#10327)
Closes #10325
Authored by: bashonly
2024-07-01 20:55:18 +00:00
c-basalt 1d6ab17d07 [ie/bilibili] Support legacy formats (#9117)
Adds extractor-arg `prefer_multi_flv`

Closes #6438, Closes #8525, Closes #8553, Closes #10243
Authored by: c-basalt, GD-Slime

Co-authored-by: GD-Slime <82302542+GD-Slime@users.noreply.github.com>
2024-07-01 20:22:49 +00:00
c-basalt 9200bc70c9 [ie/microsoftembed] Add extractors for dev materials (#9177)
Closes #7112
Authored by: c-basalt
2024-07-01 19:11:33 +02:00
DmitryScaletta aefede2556 [ie/nuum] Fix formats extraction (#10316)
Pass referer header to m3u8 requests

Closes #10310
Authored by: DmitryScaletta
2024-07-01 17:01:51 +00:00
c-basalt 4f5d7be3c5 [ie/qqmusic] Fix extractors (#9768)
Closes #9336
Authored by: c-basalt
2024-07-01 16:54:15 +00:00
Thomas R 1d369b4096 [ie/graspop] Add extractor (#10268)
Authored by: Niluge-KiWi
2024-07-01 16:49:19 +00:00
bashonly 55e3e6fd21 Add playlist_channel and playlist_channel_id fields (#10266)
Authored by: bashonly
2024-07-01 16:48:11 +00:00
Alexander Pauls 36e8dd8325 [ie/pokergo] Make metadata extraction non-fatal (#10319)
Authored by: axpauls
2024-07-01 18:30:07 +02:00
sepro e6a22834df [ie/orf:on] Allow downloading of video in segments (#10314)
Closes #10142
Authored by: seproDev
2024-07-01 12:43:52 +02:00
A. Sertaç Akkaya b8da8a98f8 [ie/laracasts] Add extractors (#10055)
Authored by: ASertacAkkaya, seproDev

Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
2024-07-01 12:14:44 +02:00
Marius Gedminas 24f3097ea9 [ie/youtube] Suppress "Unavailable videos are hidden" warning (#10159)
Authored by: mgedmin
2024-06-30 22:17:17 +00:00
Dong Heon Hee 054a3ba7d1 [ie/afreecatv:catchstory] Add extractor (#10235)
Closes #10112
Authored by: hui1601
2024-06-30 22:00:33 +00:00
Dong Heon Hee e8352ad659 [ie/afreecatv] Support browser impersonation (#10174)
Closes #8187
Authored by: hui1601
2024-06-30 21:55:21 +00:00
tippfehlr 2a4f2e82db [ie/digitalconcerthall] Rework extractor (#10152)
Authored by: tippfehlr, seproDev

Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
2024-06-30 22:48:54 +02:00
Varun Chopra 61714f4695 [ie/jiocinema:series] Fix extraction (#10139)
Authored by: varunchopra
2024-06-30 20:29:01 +00:00
bashonly 61edf57f8f [ie/mlbtv] Fix extraction (#10296)
Closes #10275
Authored by: bashonly
2024-06-29 15:43:55 +00:00
sepro 5b1a2aa978 [ie/bitchute] Fix extractors (#10301)
Closes #10293
Authored by: seproDev
2024-06-29 17:32:41 +02:00
sepro 7814c50948 [cleanup] Bump ruff to 0.5.x (#10282)
Authored by: seproDev
2024-06-29 17:30:57 +02:00
bashonly 54a63e80af [test:download] Raise on network errors (#10283)
Authored by: bashonly, seproDev
Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
2024-06-28 00:23:44 +00:00
hafeoz 7a03f88c40 [ie/neteasemusic] Extract more formats from new API (#10258)
Closes #9196, Closes #10239
Authored by: hafeoz
2024-06-27 16:17:32 +00:00
Simon Sawicki f2a4ea1794 [pp/embedthumbnail] Fix postprocessor (#10248)
* [compat] Improve `imghdr.what` detection
* [pp/embedthumbnail] Improve imghdr fail message
* [pp/embedthumbnail] Fix AtomicParsley error handling

Authored by: Grub4K
2024-06-27 16:12:19 +02:00
bashonly 0953209a85 [ie/mediasite] Fix extraction (#10273)
Fix regression in add96eb9f8

Closes #10270
Authored by: bashonly
2024-06-26 23:57:34 +00:00
Cæsim b758877afa [ie/cloudycdn] Fix formats extraction (#10271)
Authored by: Caesim404
2024-06-26 23:56:44 +00:00
megumin f3411af12e [ie/matchtv] Fix extractor (#10190)
Authored by: megumintyan
2024-06-25 00:49:09 +02:00
Peisen Wang a8520244b8 [cookies] Fix --cookies-from-browser DE detection on Linux (#10237)
Align with chromium source by parsing every part of `XDG_CURRENT_DESKTOP`

Authored by: peisenwang
2024-06-22 23:25:16 +00:00
bashonly 8ca1d57ed0 [ie/facebook:reel] Fix extraction (#10232)
Closes #10227
Authored by: bashonly
2024-06-21 23:21:45 +00:00
bashonly 800ec085cc [ie/youtube] Skip formats if nsig decoding fails (#10223)
Ref: https://github.com/ytdl-org/youtube-dl/issues/32815

Authored by: bashonly
2024-06-21 23:19:59 +00:00
bashonly 96472d72f2 [ie/tiktok] Fix API extraction (#10216)
Closes #10213
Authored by: bashonly
2024-06-21 22:57:29 +00:00
bashonly 7aa322c02c [ie/cloudflarestream] Fix _VALID_URL and embed extraction (#10215)
Authored by: bashonly
2024-06-20 22:05:25 +00:00
Haxy 9bd8501993 [ie/youtube] Extract all formats from multi-language m3u8s (#9875)
Authored by: clienthax, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-06-20 21:54:53 +00:00
bashonly 90c3721a32 [ie/brightcove] Upgrade requests to HTTPS (#10202)
Closes #10199
Authored by: bashonly
2024-06-17 16:37:12 +00:00
bashonly d4b52ce3fc [ie/podbayfm] Fix extraction (#10195)
Authored by: bashonly, seproDev

Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
2024-06-17 00:05:46 +00:00
bashonly d6c2c2bc84 [ie/sproutvideo] Add extractors (#10098)
Closes #2933, Closes #8942
Authored by: bashonly, TheZ3ro

Co-authored-by: thezero <io@thezero.org>
2024-06-17 00:01:46 +00:00
bashonly 5dbac313ae [ie/generic] Add key_query extractor-arg
Authored by: bashonly
2024-06-15 18:38:02 -05:00
bashonly ca8885edd9 [fd/hls] Apply extra_param_to_key_url from info dict
Authored by: bashonly
2024-06-15 18:38:02 -05:00
c-basalt 4093eb1fcc [ie/khanacademy] Fix extractors (#9136)
Closes #8775
Authored by: c-basalt
2024-06-15 21:51:27 +02:00
bashonly a0d9967f68 [ie/youtube:tab] Fix channel metadata extraction (#10071)
Closes #9893, Closes #10090
Authored by: bashonly, shoxie007

Co-authored-by: shoxie007 <74592022+shoxie007@users.noreply.github.com>
2024-06-13 23:22:30 +00:00
bashonly ea88129784 [ie/tiktok] Detect and raise when login is required (#10124)
Authored by: bashonly
2024-06-13 23:16:43 +00:00
garret1317 b8e2a5e0e1 [ie/NHKRadiru] Fix extractor (#10106)
Closes #10105
Authored by: garret1317
2024-06-13 23:08:40 +00:00
bashonly e53e56b735 [ie/soundcloud] Fix download format extraction (#10125)
Authored by: bashonly
2024-06-13 23:01:19 +00:00
JSubelj 92a1c4abae [ie/rtvslo.si:show] Add extractor (#8418)
Authored by: JSubelj, seproDev

Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
2024-06-14 00:51:12 +02:00
bashonly 3690c2f598 [ie/francetv] Detect and raise errors for DRM (#10165)
Closes #10163
Authored by: bashonly
2024-06-13 22:44:20 +00:00
bashonly 081708d607 [ie/francetv] Fix extractor (#10177)
Closes #10175
Authored by: bashonly
2024-06-13 22:31:13 +00:00
bashonly d7d861811c [ie/tubitv:series] Fix extractor (#10116)
Closes #8563
Authored by: bashonly
2024-06-13 21:59:17 +00:00
bashonly 46c1b7cfec [build] Cache dependencies for macos job (#10088)
Authored by: bashonly
2024-06-13 21:13:08 +00:00
sepro add96eb9f8 [cleanup] Add more ruff rules (#10149)
Authored by: seproDev

Reviewed-by: bashonly <88596187+bashonly@users.noreply.github.com>
Reviewed-by: Simon Sawicki <contact@grub4k.xyz>
2024-06-12 01:09:58 +02:00
bashonly db50f19d76 [rh:requests] Bump minimum requests version to 2.32.2 (#10079)
Closes #10078
Authored by: bashonly
2024-06-01 18:57:23 +00:00
bashonly 2e5a47da40 [ie/PatreonCampaign] Fix campaign_id extraction (#10070)
Closes #10013
Authored by: bashonly
2024-05-30 23:04:27 +00:00
bashonly 5fdd13006a [build] Bump Pyinstaller to >=6.7.0 for all builds (#10069)
Ref: https://github.com/pyinstaller/pyinstaller/issues/8554

Authored by: bashonly, seproDev

Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
2024-05-30 22:34:02 +00:00
bashonly 03334d639d [build] Use macos-12 image for yt-dlp_macos (#10063)
Ref: https://github.blog/changelog/2024-05-20-actions-upcoming-changes-to-github-hosted-macos-runners/

Authored by: bashonly
2024-05-30 18:53:37 +00:00
sepro 8b46ad4d8b [ie/orf:on] Support segmented episodes (#10053)
Closes #9930
Authored by: seproDev
2024-05-29 23:16:57 +02:00
Ben Galliart bef9a9e536 [ie/TubiTv] Fix extractor (#9975)
Closes #9937
Authored by: chilinux
2024-05-29 04:25:05 +00:00
github-actions[bot] 111b61ddef Release 2024.05.27
Created by: bashonly

:ci skip all :ci run dl
2024-05-27 22:35:55 +00:00
trueauracoral 12b248ce60 [ie/peertube] Support livestreams (#10044)
Closes #2055
Authored by: trueauracoral, bashonly
2024-05-27 22:24:01 +00:00
bashonly 5e3e19c93c [cleanup] Misc (#10043)
Authored by: bashonly
2024-05-27 21:46:07 +00:00
bashonly c53c2e40fd [ie/tiktok:user] Fix extraction loop (#10035)
Closes #10033
Authored by: bashonly
2024-05-27 04:22:46 +00:00
sepro ae2194e1dd [ie/Piksel] Update domain (#9223)
Authored by: seproDev
2024-05-27 01:24:03 +02:00
sepro 26603d0b34 [ie] Fix parsing of base URL in SMIL manifest (#9225)
Authored by: seproDev
2024-05-27 00:06:34 +02:00
github-actions[bot] ed274b60b1 Release 2024.05.26
Created by: bashonly

:ci skip all :ci run dl
2024-05-26 21:55:43 +00:00
bashonly ae2af1104f [cleanup] Misc
Authored by: bashonly, seproDev, Grub4K
2024-05-26 16:52:42 -05:00
Simon Sawicki 5c019f6328 [misc] Cleanup (#9765)
Closes #9763
Authored by: bashonly, seproDev, Grub4K

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
2024-05-26 21:37:49 +00:00
ocococococ 5a2eebc767 [ie/LCI] Fix extractor (#10025)
Authored by: ocococococ
2024-05-26 23:33:15 +02:00
imanoreotwe 119d41f270 [ie/tiktok:collection] Add extractor (#9986)
Closes #9984
Authored by: imanoreotwe, bashonly
2024-05-26 21:26:30 +00:00
bashonly 347f13dd9b [ie/tiktok:user] Fix extractor (#9661)
Closes #3776, Closes #4996
Authored by: bashonly
2024-05-26 21:16:36 +00:00
coletdjnz 96a134dea6 [ie/youtube] Extract upload timestamp if available (#9856)
Closes #4962, Closes #9829
Authored by: coletdjnz
2024-05-26 21:13:12 +00:00
Simon Sawicki a4da9db87b Update to ytdl-commit-a08f2b7 (#10012)
[ie] Rework JWPlayer extraction
- https://github.com/ytdl-org/youtube-dl/commit/f66372403fd9e1661199fea100ba2600fa9697b2
[ie/gbnews] Add extractor
- https://github.com/ytdl-org/youtube-dl/commit/70f230f9cf28e948662599b6257cb7d1262870e3
[ie/caffeinetv] Add extractor
- https://github.com/ytdl-org/youtube-dl/commit/40bd5c18153afe765caa6726302ee1dd8a9a2ce6
[ie/youporn] Improve extraction
- https://github.com/ytdl-org/youtube-dl/commit/0b2ce3685e02ea1a3ccee1026572e081b8f6ac83
[ie/youporn] Add playlist extractors
- https://github.com/ytdl-org/youtube-dl/commit/668332b9733023ca2e927eeb2208725022248af8

Closes #9188, Closes #9523
Authored by: Grub4K, bashonly
2024-05-26 21:09:53 +00:00
Simon Sawicki e897bd8292 [misc] Add hatch, ruff, pre-commit and improve dev docs (#7409)
Authored by: bashonly, seproDev, Grub4K

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
2024-05-26 21:27:21 +02:00
HobbyistDev a2e9031605 [ie/XiaoHongShu] Add extractor (#9646)
Closes #9529
Authored by: HobbyistDev
2024-05-26 01:54:17 +02:00
Finn R. Gärtner 3ba8de62d6 [ie/Piapro] Fix extractor (#9311)
Closes #9884
Authored by: FinnRG, seproDev
2024-05-26 01:40:35 +02:00
bashonly 0d067e77c3 [ie/dangalplay] Add extractors (#10021)
Closes #8258
Authored by: bashonly
2024-05-25 23:16:17 +00:00
bashonly 1463945ae5 [ie/jiocinema] Add extractors (#10026)
Closes #5563, Closes #7759, Closes #8679, Closes #9349
Authored by: bashonly
2024-05-25 23:03:05 +00:00
bashonly c92e4e625e [ie/tele5] Overhaul extractor (#10024)
Closes #3051, Closes #7955, Closes #8501, Closes #9792
Authored by: bashonly
2024-05-25 23:00:33 +00:00
bashonly 90d2da311b [ie/DiscoveryPlus] Fix dmax.de and related extractors (#10020)
Closes #7530
Authored by: bashonly
2024-05-25 15:01:40 +00:00
sepro 3779f2a307 [ie/ORFTVthek] Remove extractor (#10011)
Authored by: seproDev
2024-05-23 22:18:20 +02:00
c-basalt 63b569bc5e [ie/taptap] Add extractors (#9776)
Closes #9643
Authored by: c-basalt
2024-05-23 20:15:56 +02:00
kclauhk 82f4f4444e [ie/reddit] Fix subtitles extraction (#10006)
Authored by: kclauhk
2024-05-23 16:26:24 +00:00
Mozi eead3bbc01 [ie/brilliantpala] Fix login (#9788)
Closes #9771
Authored by: pzhlkj6612
2024-05-23 16:25:16 +00:00
BohwaZ 5bbfdb7c99 [ie/HearThisAt] Improve _VALID_URL (#9949)
Closes #9755
Authored by: bohwaz, seproDev

Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
2024-05-23 06:30:21 +02:00
TuxCoder 0dd53faeca [ie/orf:on] Improve extraction (#9677)
Closes #9652
Authored by: TuxCoder
2024-05-23 06:25:16 +02:00
six be7db1a5a8 [ie/NTSLive] Add extractor (#9641)
Closes #9640
Authored by: lostfictions
2024-05-23 06:13:00 +02:00
HobbyistDev 65e709d235 [ie/GodResource] Add extractor (#9629)
Closes #9551
Authored by: HobbyistDev
2024-05-23 06:09:21 +02:00
Amir Y. Perehodnik 06cb063839 [ie/Instagram] Support /reels/ URLs (#9539)
Closes #6689
Authored by: amir16yp
2024-05-23 06:07:20 +02:00
panatexxa 296df0da1d [ie/Moviepilot] Fix extractor (#9366)
Authored by: panatexxa
2024-05-23 06:03:55 +02:00
vtexier 7b5674949f [ie/ArteTV] Label forced subtitles (#9945)
Authored by: vtexier
2024-05-22 23:09:58 +00:00
bashonly f2816634e3 [ie/crunchyroll] Fix stream extraction (#10005)
Closes #9994
Authored by: bashonly
2024-05-22 22:25:07 +00:00
bashonly beaf832c7a [ie/soundcloud] Add formats extractor-arg (#10004)
Authored by: bashonly
2024-05-22 22:20:29 +00:00
bashonly eef1e9f44f [ie/tiktok] Fix subtitles extraction (#9961)
Authored by: bashonly
2024-05-22 22:17:10 +00:00
bashonly 78c57cc0e0 [build] macos job requires setuptools<70 (#9993)
Authored by: bashonly
2024-05-22 14:30:25 +00:00
Simon Sawicki 3f7999533e [rh:requests] Patch support for requests 2.32.2+ (#9992)
Authored by: Grub4K
2024-05-22 16:22:25 +02:00
bashonly 4ccd73fea0 [ie/tiktok] Extract all web formats (#9960)
Closes #9506
Authored by: bashonly
2024-05-20 23:11:24 +00:00
bashonly 3584b8390b [ie/tiktok] Add device_id extractor-arg (#9951)
Authored by: bashonly
2024-05-20 23:09:28 +00:00
bashonly 6e36d17f40 [build] Exclude requests from py2exe (#9982)
Authored by: bashonly
2024-05-20 23:01:17 +00:00
coletdjnz c36513f1be [rh:requests] Update to requests 2.32.0 (#9980)
Authored by: coletdjnz
2024-05-20 21:44:41 +00:00
bashonly 3e35aa32c7 [ie/twitter] Fix auth for x.com migration (#9952)
Authored by: bashonly
2024-05-18 18:33:30 +00:00
coletdjnz 53b4d44f55 [test] Fix connect timeout test (#9906)
Fixes https://github.com/yt-dlp/yt-dlp/issues/9659

Authored by: coletdjnz
2024-05-18 19:12:21 +12:00
bashonly c999bac02c Bugfix for 61b17437dc
Authored by: bashonly
2024-05-17 23:44:11 -05:00
coletdjnz 12d8ea8246 [ie/youtube] Remove android from default clients (#9553)
Closes #9554
Authored by: coletdjnz, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-05-17 16:03:02 +00:00
Justin Keogh 8e15177b41 [ie/youtube] Fix comments extraction (#9775)
Closes #9358
Authored by: jakeogh, minamotorin, shoxie007, bbilly1

Co-authored-by: minamotorin <76122224+minamotorin@users.noreply.github.com>
Co-authored-by: shoxie007 <74592022+shoxie007@users.noreply.github.com>
Co-authored-by: Simon <35427372+bbilly1@users.noreply.github.com>
2024-05-17 14:37:30 +00:00
Roeniss Moon dd9ad97b1f [cookies] Add --cookies-from-browser support for Whale (#9649)
Closes #9307
Authored by: roeniss
2024-05-17 14:33:12 +00:00
minamotorin 61b17437dc [ie] Add POST data hash to --write-pages filenames (#9879)
Closes #9773
Authored by: minamotorin
2024-05-17 14:28:36 +00:00
kylegustavo 7975ddf245 [ie/bbc] Fix and extend extraction (#9705)
Closes #9701
Authored by: kylegustavo, dirkf, pukkandan
2024-05-17 06:20:13 +00:00
Podiumnoche 6d8a53d870 [ie/cda] Fix age-gated web extraction (#9939)
Closes #5980, Closes #6638
Authored by: Podiumnoche, Szpachlarz, dirkf, emqi
2024-05-16 22:41:34 +00:00
bashonly 4813173e45 [ie/twitter] Support x.com URLs (#9926)
Closes #9923
Authored by: bashonly
2024-05-16 22:36:56 +00:00
bashonly 41ba4a808b [ie/tiktok] Extract via mobile API only if app_info is passed (#9938)
Partially addresses #9506
Authored by: bashonly
2024-05-16 22:27:09 +00:00
Mozi 351dc0bc33 [ie/eplus] Handle URLs without videos (#9855)
Authored by: pzhlkj6612
2024-05-13 23:21:11 +00:00
feederbox826 518c1afc15 [ie/pornhub] Fix login by email address (#9914)
Closes #9717
Authored by: feederbox826
2024-05-13 23:18:14 +00:00
WyohKnott 85ec2a337a [ie/googledrive] Fix formats extraction (#9908)
Closes #8281
Authored by: WyohKnott
2024-05-12 23:05:47 +00:00
Jake Finley b207d26f83 [ie/xvideos:quickies] Fix extractor (#9834)
Closes #6356
Authored by: JakeFinley96
2024-05-12 20:42:33 +00:00
sepro 01395a3434 [cleanup] Remove questionable extractors (#9911)
Closes #6279, Closes #6799
Authored by: seproDev
2024-05-12 22:12:11 +02:00
Haxy cf212d0a33 [ie/youtube] Add mediaconnect client (#9546)
Authored by: clienthax
2024-05-12 16:03:36 +00:00
alard 6db96268c5 [ie/TV5Monde] Fix extractor (#9143)
Closes #9118
Authored by: alard, seproDev

Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
2024-05-11 23:58:15 +02:00
Eric Lam 800a43983e [ie/EuroParlWebstream] Support new URL format (#9647)
Authored by: voidful, seproDev

Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
2024-05-11 23:50:59 +02:00
DaPotato69 7e4259dff0 Better warning when requested subs format not found (#9873)
Closes #9760
Authored by: DaPotato69
2024-05-11 21:11:40 +00:00
Stefan Lobbenmeier f1f158976e [cookies] Get chrome session cookies with --cookies-from-browser (#9747)
Partially addresses #5534
Authored by: StefanLobbenmeier
2024-05-11 17:25:39 +00:00
llamasblade 31b417e1d1 [ie/hytale] Use CloudflareStreamIE explicitly (#9672)
Authored by: llamasblade
2024-05-11 17:01:56 +00:00
Hugo Azevedo fc2879ecb0 [ie/alura] Fix extractor (#9658)
Authored by: hugohaa
2024-05-11 16:54:29 +00:00
rrgomes 0a1a8e3005 [ie/nfb] Fix extractors (#9650)
Authored by: rrgomes
2024-05-11 16:38:41 +00:00
c-basalt 4cc99d7b6c [ie/BilibiliSpaceVideo] Fix extraction (#9905)
Closes #9892
Authored by: c-basalt
2024-05-10 22:34:53 +00:00
coletdjnz 3c7a287e28 [test] Add HTTP proxy tests (#9578)
Also fixes HTTPS proxies for curl_cffi

Authored by: coletdjnz
2024-05-11 10:06:58 +12:00
sepro 98d71d8c5e [ie/commonmistakes] Raise error on blob URLs (#9897)
Authored by: seproDev
2024-05-10 19:20:55 +02:00
kclauhk 00a9f2e1f7 [ie/canalalpha] Fix extractor (#9675)
Authored by: kclauhk
2024-05-10 19:19:57 +02:00
Mozi 73f12119b5 [ie/netease:program] Improve --no-playlist message (#9488)
Authored by: pzhlkj6612
2024-05-10 19:13:35 +02:00
Alexandre Huot 6b54cccdcb [ie/Qub] Fix extractor (#7019)
Closes #4989
Authored by: alexhuot1, dirkf
2024-05-08 22:10:06 +00:00
src-tinkerer c4b87dd885 [ie/ZenYandex] Fix extractor (#9813)
Closes #9803
Authored by: src-tinkerer
2024-05-08 21:27:30 +00:00
fireattack 2338827072 [ie/bilibili] Fix --geo-verification-proxy support (#9817)
Closes #9797
Authored by: fireattack
2024-05-08 21:24:44 +00:00
fireattack 06d52c8731 [ie/BilibiliSpaceVideo] Better error message (#9839)
Closes #9528
Authored by: fireattack
2024-05-08 21:09:38 +00:00
sepro df5c9e733a [ie/vk] Improve format extraction (#9885)
Closes #5675
Authored by: seproDev
2024-05-08 23:02:22 +02:00
Mozi b38018b781 [ie/mixch] Extract comments (#9860)
Authored by: pzhlkj6612
2024-05-08 20:51:16 +00:00
Rasmus Antons 145dc6f656 [ie/boosty] Add cookies support (#9522)
Closes #9401
Authored by: RasmusAntons
2024-05-08 20:16:32 +00:00
bashonly 5904853ae5 [ie/crunchyroll] Support browser impersonation (#9857)
Closes #7442
Authored by: bashonly
2024-05-05 23:15:32 +00:00
Chris Caruso c8bf48f3a8 [ie/cbc.ca:player] Improve _VALID_URL (#9866)
Closes #9825
Authored by: carusocr
2024-05-05 23:02:24 +00:00
The-MAGI 351368cb9a [ie/youporn] Fix extractor (#8827)
Closes #7967
Authored by: The-MAGI
2024-05-05 22:57:38 +00:00
sepro 96da952504 [core] Warn if lack of ffmpeg alters format selection (#9805)
Authored by: seproDev, pukkandan
2024-05-05 00:44:08 +02:00
bashonly bec9a59e8e [networking] Add extensions attribute to Response (#9756)
CurlCFFIRH now provides an `impersonate` field in its responses' extensions

Authored by: bashonly
2024-05-04 22:19:42 +00:00
bashonly 036e0d92c6 [ie/patreon] Extract multiple embeds (#9850)
Closes #9848
Authored by: bashonly
2024-05-04 22:11:11 +00:00
bashonly cb2fb4a643 [ie/crunchyroll] Always make metadata available (#9772)
Closes #9750
Authored by: bashonly
2024-05-04 16:15:44 +00:00
bashonly 231c2eacc4 [ie/soundcloud] Extract genres (#9821)
Authored by: bashonly
2024-05-04 16:14:36 +00:00
bashonly c4853655cb [ie/wrestleuniverse] Avoid partial stream formats (#9800)
Authored by: bashonly
2024-05-04 16:07:15 +00:00
Simon Sawicki ac817bc83e [build] Migrate linux_exe to static musl builds (#9811)
Authored by: Grub4K, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-04-28 22:19:25 +00:00
bashonly 1a366403d9 [build] Run macos_legacy job on macos-12 (#9804)
`macos-latest` has been bumped to `macos-14-arm64` which breaks the builds

Authored by: bashonly
2024-04-28 15:35:17 +00:00
Simon Sawicki 7e26bd53f9 [core/windows] Fix tests for sys.executable with spaces (Fix for 64766459e3)
Authored by: Grub4K
2024-04-28 15:47:55 +02:00
Simon Sawicki 64766459e3 [core/windows] Improve shell quoting and tests (#9802)
Authored by: Grub4K
2024-04-27 10:37:26 +02:00
bashonly 89f535e265 [ci] Fix curl-cffi installation (Bugfix for 02483bea1c)
Authored by: bashonly
2024-04-22 20:36:01 +00:00
bashonly ff38a011d5 [ie/crunchyroll] Fix auth and remove cookies support (#9749)
Closes #9745
Authored by: bashonly
2024-04-21 22:41:40 +00:00
bashonly 8056a3026e [ie/theatercomplextown] Fix extractors (#9754)
Authored by: bashonly
2024-04-21 16:05:42 +00:00
Simon Sawicki 3ee1194288 [ie] Make _search_nextjs_data non fatal (#8937)
Authored by: Grub4K
2024-04-21 13:40:38 +02:00
bashonly e3b42d8b1b [ie/facebook] Fix DASH formats extraction (#9734)
Closes #9720
Authored by: bashonly
2024-04-20 10:23:12 +00:00
bashonly c9ce57d9bf [ie/patreon] Fix Vimeo embed extraction (#9712)
Fixes regression in 36b240f9a7

Closes #9709
Authored by: bashonly
2024-04-18 23:18:56 +00:00
bashonly 02483bea1c [build] Normalize curl_cffi group to curl-cffi (#9698)
Closes #9682
Authored by: bashonly
2024-04-18 23:11:12 +00:00
bashonly 315b354429 [ie/afreecatv:live] Add cdn extractor-arg (#9666)
Closes #6497
Authored by: bashonly
2024-04-13 16:40:53 +00:00
bashonly 0c21c53885 [ie/jiosaavn] Extract via API and fix playlists (#9656)
Closes #9648
Authored by: bashonly
2024-04-13 16:08:25 +00:00
github-actions[bot] 168e72dcd3 Release 2024.04.09
Created by: Grub4K

:ci skip all :ci run dl
2024-04-09 17:03:28 +00:00
Simon Sawicki ff07792676 [core] Prevent RCE when using --exec with %q (CVE-2024-22423)
The shell escape function now properly escapes `%`, `\\` and `\n`. `utils.Popen` as well as `%q` output template expansion have been patched accordingly.

Prior to this fix using `--exec` together with `%q` when on Windows could cause remote code to execute. See https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-hjq6-52gw-2g7p for more details.

Authored by: Grub4K
2024-04-09 18:36:13 +02:00
bashonly 216f6a3cb5 [cleanup] Misc (#9426)
Authored by: bashonly, pukkandan
2024-04-09 16:12:26 +00:00
bashonly b19ae095fd [build] Do not include curl_cffi in macos_legacy (#9653)
Authored by: bashonly
2024-04-08 23:20:58 +00:00
Simon Sawicki 9590cc6b47 Add new option --progress-delta (#9082)
Authored by: Grub4K
2024-04-08 22:47:38 +02:00
luiso1979 79a451e576 [networking] Respect SSLKEYLOGFILE environment variable (#9543)
Authored by: luiso1979
2024-04-08 21:53:30 +02:00
Leo Heitmann Ruiz df0e138fc0 [docs] Various manpage fixes
Authored by: leoheitmannruiz
2024-04-08 21:24:58 +02:00
bashonly 2e94602f24 [ie/jiosaavn] Support playlists (#9622)
Closes #9616
Authored by: bashonly
2024-04-07 20:55:46 +00:00
bashonly 4af9d5c2f6 [ie/nhk] Fix NHK World extractors (#9623)
Closes #9513
Authored by: bashonly
2024-04-07 16:59:38 +00:00
John Victor 36b240f9a7 [ie/patreon] Do not extract dead embed URLs (#9613)
Closes #8702
Authored by: johnvictorfs
2024-04-07 16:26:44 +00:00
bashonly fc53ec13ff [ie/tiktok] Restore carrier_region API parameter (#9637)
Avoids some geo-blocks

Authored by: bashonly
2024-04-07 15:32:11 +00:00
Dmitry Meyer 2ab2651a4a [cookies] Add --cookies-from-browser support for Firefox Flatpak (#9619)
Authored by: un-def
2024-04-07 15:28:59 +00:00
bashonly b15b0c1d21 [ie/vkplay] Fix _VALID_URL (#9636)
Closes #9635
Authored by: bashonly
2024-04-06 20:42:51 +00:00
bashonly c8a61a9100 [ie/kick] Support browser impersonation (#9611)
Closes #6748
Authored by: bashonly
2024-04-06 17:42:32 +00:00
Mozi f2fd449b46 [ie/joqrag] Fix live status detection (#9624)
Authored by: pzhlkj6612
2024-04-06 17:34:51 +00:00
Tomoka1 9415f1a5ef [ie/afreecatv] Overhaul extractor (#9566)
Closes #4592, Closes #8862, Closes #9544
Authored by: bashonly, Tomoka1

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-04-06 17:23:16 +00:00
bashonly a48cc86d6f [ie/dropbox] Fix formats extraction (#9627)
Closes #9533
Authored by: bashonly
2024-04-06 17:19:44 +00:00
bytedream 954e57e405 [ie/crunchyroll] Fix extractor (#9615)
Authored by: bytedream
2024-04-06 12:53:20 +02:00
Dong Heon Hee 9073ae6458 [ie/afreecatv:live] Fix extractor (#9348)
Closes #4466, Closes #9345
Authored by: hui1601
2024-04-04 16:48:05 +00:00
Offert4324 4cd9e251b9 [ie/medici] Fix extractor (#9518)
Closes #8813
Authored by: Offert4324
2024-04-04 16:45:19 +00:00
bashonly 0ae16ceb18 [ie/jiosaavn] Extract artists (#9612)
Closes #9607
Authored by: bashonly
2024-04-03 23:23:04 +00:00
bashonly 443e206ec4 [ie/jiosaavn] Fix format extensions (#9609)
Authored by: bashonly
2024-04-03 23:21:28 +00:00
bashonly 4c3b7a0769 [ie/mixch] Fix extractor (#9608)
Closes #9536
Authored by: bashonly, nipotan
2024-04-03 22:53:42 +00:00
bashonly 16be117729 Add option --no-break-on-existing (#9610)
Authored by: bashonly
2024-04-03 22:51:41 +00:00
trainman261 b49d5ffc53 [ie/cbc.ca:player] Support new URL format (#9561)
Closes #9534
Authored by: trainman261
2024-04-03 19:11:13 +00:00
HobbyistDev 36baaa10e0 [ie/Radio1Be] Add extractor (#9122)
Closes #8707
Authored by: HobbyistDev
2024-04-03 18:51:14 +00:00
Kacper Michajłow 02f93ff51b [ie/twitch] Extract AV1 and HEVC formats (#9158)
Authored by: kasper93
2024-04-03 18:38:51 +00:00
Mozi c59de48e2b [ie/mixch:archive] Fix extractor (#8761)
Closes #2373
Authored by: pzhlkj6612
2024-04-01 22:41:09 +00:00
Mozi 0284f1fee2 [ie/asobistage] Add extractor (#8735)
Authored by: pzhlkj6612
2024-04-01 22:29:14 +00:00
bashonly e8032503b9 [build] Print SHA sums to GHA logs (#9582)
Authored by: bashonly
2024-04-01 17:02:25 +00:00
bashonly 97362712a1 [ie/soundcloud] Support cookies (#9586)
Closes #997
Authored by: bashonly
2024-04-01 16:58:48 +00:00
bashonly 246571ae1d [ie/soundcloud] Support retries for API rate-limit (#9585)
Authored by: bashonly
2024-04-01 16:21:46 +00:00
Simon Sawicki 32abfb00bd [utils] traverse_obj: Convenience improvements (#9577)
Add support for:
- `http.cookies.Morsel`
- Multi type filters (`{type, type}`)

Authored by: Grub4K
2024-04-01 02:12:03 +02:00
pukkandan c305a25c1b [cleanup] Standardize import datetime as dt (#8978) 2024-04-01 05:32:15 +05:30
pukkandan e3a3ed8a98 [ie, cleanup] No from stdlib imports in extractors (#8978) 2024-04-01 05:31:09 +05:30
pukkandan a25a424323 [ie/youtube] Calculate more accurate filesize
YouTube provides slightly different duration for each format.
Calculating file-size based on this duration instead of the
video duration gives more accurate results.

Ref: https://github.com/yt-dlp/yt-dlp/issues/1400#issuecomment-2007441207
2024-04-01 04:56:09 +05:30
sepro 86e3b82261 [core] Fix filesize_approx calculation (#9560)
Reverts 22e4dfacb6

Despite being documented as `Kbit/s`, the extractors/manifests were returning bitrates in SI units of kilobits/sec.

Authored by: seproDev, pukkandan
2024-04-01 04:47:24 +05:30
pukkandan e7b17fce14 [ie/youtube] Update android params
Discovered by LuanRT - https://github.com/LuanRT/YouTube.js/pull/624

Closes #9554
2024-04-01 01:31:53 +05:30
bashonly a2d0840739 [ie/soundcloud] Adjust format sorting (#9584)
- Adapt to 86a972033e

Authored by: bashonly
2024-03-31 20:01:33 +00:00
pukkandan 86a972033e Infer acodec for single-codec containers 2024-03-31 22:50:21 +05:30
bashonly 50c2935231 [ie] Add extractor impersonate API (#9474)
Authored by: bashonly, Grub4K, pukkandan
2024-03-30 23:18:07 +00:00
bashonly 0df63cce69 [ie/thisoldhouse] Support Brightcove embeds (#9576)
Closes #9570
Authored by: bashonly
2024-03-30 23:06:20 +00:00
bashonly 63f685f341 [ie/tiktok] Prefer non-bytevc2 formats (#9575)
Closes #9567
Authored by: bashonly
2024-03-30 22:54:00 +00:00
Simon Sawicki 3699eeb67c [utils] traverse_obj: Allow unbranching using all and any (#9571)
Authored by: Grub4K
2024-03-30 19:54:43 +01:00
Simon Sawicki 979ce2e786 [test] traversal: Separate traversal tests (#9574)
Authored by: Grub4K
2024-03-30 19:32:07 +01:00
bashonly 58dd0f8d1e [build] Optional dependencies cleanup (#9550)
Authored by: bashonly
2024-03-29 23:24:40 +00:00
bashonly cb61e20c26 [ie/tiktok] Fix API extraction (#9548)
Closes #9506
Authored by: bashonly, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2024-03-29 23:20:14 +00:00
bashonly 9c42b7eef5 [fd/ffmpeg] Accept output args from info dict (#9278)
Authored by: bashonly
2024-03-29 23:16:46 +00:00
coletdjnz e5d4f11104 [rh:websockets] Workaround race condition causing issues on PyPy (#9514)
Authored by: coletdjnz
2024-03-23 11:27:10 +13:00
src-tinkerer bc2b8c0596 [ie/fathom] Add extractor (#9495)
Closes #8541
Authored by: src-tinkerer
2024-03-22 14:31:01 +00:00
sta1us aa7e9ae4f4 [ie/xvideos] Support new URL format (#9493) (#9502)
Closes #9493
Authored by: sta1us
2024-03-22 14:28:09 +00:00
Shreyas Minocha 07f5b2f757 [ie/box] Support URLs without file IDs (#9504)
Authored by: shreyasminocha
2024-03-20 23:26:37 +00:00
Daniel Vogt ff349ff94a [ie/sharepoint] Add extractor (#6531)
Authored by: C0D3D3V, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-03-20 23:20:50 +00:00
Hasan Rüzgar f859ed3ba1 [ie/loom] Add extractors (#8686)
Closes #3715
Authored by: bashonly, hruzgar

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-03-20 23:14:37 +00:00
Aron Buzinkay 17d248a587 [ie/youtube:search] Fix params for uncensored results (#9456)
Closes #9156
Authored by: alb, pukkandan
2024-03-19 23:25:04 +00:00
sepro 388c979ac6 [docs] Update yt-dlp tagline (#9481)
Authored by: seproDev, bashonly, coletdjnz, Grub4K, pukkandan
2024-03-19 18:14:04 +01:00
sepro 22e4dfacb6 [ie/youtube] Fix tbr calculation (#9489)
Authored by: pukkandan

Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
2024-03-18 18:07:22 +01:00
Trustin 86d2f4d248 [ie/imgur] Fix extraction (#9471)
Closes #9458
Authored by: trwstin
2024-03-17 05:04:55 +00:00
coletdjnz 52f5be1f1e [rh:curlcffi] Add support for curl_cffi
Authored by: coletdjnz, Grub4K, pukkandan, bashonly

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: bashonly <bashonly@protonmail.com>
2024-03-16 23:15:11 -05:00
coletdjnz 0b81d4d252 Add new options --impersonate and --list-impersonate-targets
Authored by: coletdjnz, Grub4K, pukkandan, bashonly

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: bashonly <bashonly@protonmail.com>
2024-03-16 23:14:13 -05:00
coletdjnz f849d77ab5 [test] Workaround websocket server hanging (#9467)
Authored by: coletdjnz
2024-03-16 16:57:21 +13:00
bashonly f2868b26e9 [ie/SonyLIVSeries] Fix season extraction (#9423)
Authored by: bashonly
2024-03-14 23:21:27 +00:00
bashonly be77923ffe [ie/crunchyroll] Extract vo_adaptive_hls formats by default (#9447)
Closes #9439
Authored by: bashonly
2024-03-14 21:42:35 +00:00
bashonly 8c05b3ebae [ie/tiktok] Update API hostname (#9444)
Closes #9441
Authored by: bashonly
2024-03-14 21:35:46 +00:00
jazz1611 0da66980d3 [ie/gofile] Fix extractor (#9446)
Authored by: jazz1611
2024-03-14 21:34:10 +00:00
bashonly 17b96974a3 [build] Update changelog for tarball and sdist (#9425)
Closes #9417
Authored by: bashonly
2024-03-14 21:10:20 +00:00
1184 changed files with 70138 additions and 41960 deletions
+16 -15
View File
@@ -2,13 +2,11 @@ name: Broken site support
description: Report issue with yt-dlp on a supported site description: Report issue with yt-dlp on a supported site
labels: [triage, site-bug] labels: [triage, site-bug]
body: body:
- type: checkboxes - type: markdown
attributes: attributes:
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE value: |
description: Fill all fields even if you think it is irrelevant for the issue > [!IMPORTANT]
options: > Not providing the required (*) information or removing the template will result in your issue being closed and ignored.
- label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field
required: true
- type: checkboxes - type: checkboxes
id: checklist id: checklist
attributes: attributes:
@@ -24,9 +22,9 @@ body:
required: true required: true
- label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command) - label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
required: true required: true
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates - label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766), [the FAQ](https://github.com/yt-dlp/yt-dlp/wiki/FAQ), and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=is%3Aissue%20-label%3Aspam%20%20) for similar issues **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [policy against AI/LLM contributions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#automated-contributions-ai--llm-policy) and understand I may be blocked from the repository if it is violated
required: true required: true
- label: I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required - label: I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required
- type: input - type: input
@@ -47,6 +45,8 @@ body:
id: verbose id: verbose
attributes: attributes:
label: Provide verbose output that clearly demonstrates the problem label: Provide verbose output that clearly demonstrates the problem
description: |
This is mandatory unless absolutely impossible to provide. If you are unable to provide the output, please explain why.
options: options:
- label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`) - label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
required: true required: true
@@ -63,14 +63,15 @@ body:
placeholder: | placeholder: |
[debug] Command-line config: ['-vU', 'https://www.youtube.com/watch?v=BaW_jenozKc'] [debug] Command-line config: ['-vU', 'https://www.youtube.com/watch?v=BaW_jenozKc']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8 [debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version nightly@... from yt-dlp/yt-dlp [b634ba742] (win_exe) [debug] yt-dlp version nightly@... from yt-dlp/yt-dlp-nightly-builds [1a176d874] (win_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0 [debug] Python 3.10.11 (CPython AMD64 64bit) - Windows-10-10.0.20348-SP0 (OpenSSL 1.1.1t 7 Feb 2023)
[debug] exe versions: ffmpeg N-106550-g072101bd52-20220410 (fdk,setts), ffprobe N-106624-g391ce570c8-20220415, phantomjs 2.1.1 [debug] exe versions: ffmpeg 7.0.2 (setts), ffprobe 7.0.2
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3 [debug] Optional libraries: Cryptodome-3.21.0, brotli-1.1.0, certifi-2024.08.30, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.40.1, urllib3-2.2.3, websockets-13.1
[debug] Proxy map: {} [debug] Proxy map: {}
[debug] Request Handlers: urllib, requests [debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1893 extractors [debug] Loaded 1838 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest [debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: nightly@... from yt-dlp/yt-dlp-nightly-builds
yt-dlp is up to date (nightly@... from yt-dlp/yt-dlp-nightly-builds) yt-dlp is up to date (nightly@... from yt-dlp/yt-dlp-nightly-builds)
[youtube] Extracting URL: https://www.youtube.com/watch?v=BaW_jenozKc [youtube] Extracting URL: https://www.youtube.com/watch?v=BaW_jenozKc
<more lines> <more lines>
@@ -2,13 +2,11 @@ name: Site support request
description: Request support for a new site description: Request support for a new site
labels: [triage, site-request] labels: [triage, site-request]
body: body:
- type: checkboxes - type: markdown
attributes: attributes:
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE value: |
description: Fill all fields even if you think it is irrelevant for the issue > [!IMPORTANT]
options: > Not providing the required (*) information or removing the template will result in your issue being closed and ignored.
- label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field
required: true
- type: checkboxes - type: checkboxes
id: checklist id: checklist
attributes: attributes:
@@ -24,9 +22,9 @@ body:
required: true required: true
- label: I've checked that none of provided URLs [violate any copyrights](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy) or contain any [DRM](https://en.wikipedia.org/wiki/Digital_rights_management) to the best of my knowledge - label: I've checked that none of provided URLs [violate any copyrights](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy) or contain any [DRM](https://en.wikipedia.org/wiki/Digital_rights_management) to the best of my knowledge
required: true required: true
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates - label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=is%3Aissue%20-label%3Aspam%20%20) for similar requests **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [policy against AI/LLM contributions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#automated-contributions-ai--llm-policy) and understand I may be blocked from the repository if it is violated
required: true required: true
- label: I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and am willing to share it if required - label: I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and am willing to share it if required
- type: input - type: input
@@ -59,6 +57,8 @@ body:
id: verbose id: verbose
attributes: attributes:
label: Provide verbose output that clearly demonstrates the problem label: Provide verbose output that clearly demonstrates the problem
description: |
This is mandatory unless absolutely impossible to provide. If you are unable to provide the output, please explain why.
options: options:
- label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`) - label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
required: true required: true
@@ -75,14 +75,15 @@ body:
placeholder: | placeholder: |
[debug] Command-line config: ['-vU', 'https://www.youtube.com/watch?v=BaW_jenozKc'] [debug] Command-line config: ['-vU', 'https://www.youtube.com/watch?v=BaW_jenozKc']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8 [debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version nightly@... from yt-dlp/yt-dlp [b634ba742] (win_exe) [debug] yt-dlp version nightly@... from yt-dlp/yt-dlp-nightly-builds [1a176d874] (win_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0 [debug] Python 3.10.11 (CPython AMD64 64bit) - Windows-10-10.0.20348-SP0 (OpenSSL 1.1.1t 7 Feb 2023)
[debug] exe versions: ffmpeg N-106550-g072101bd52-20220410 (fdk,setts), ffprobe N-106624-g391ce570c8-20220415, phantomjs 2.1.1 [debug] exe versions: ffmpeg 7.0.2 (setts), ffprobe 7.0.2
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3 [debug] Optional libraries: Cryptodome-3.21.0, brotli-1.1.0, certifi-2024.08.30, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.40.1, urllib3-2.2.3, websockets-13.1
[debug] Proxy map: {} [debug] Proxy map: {}
[debug] Request Handlers: urllib, requests [debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1893 extractors [debug] Loaded 1838 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest [debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: nightly@... from yt-dlp/yt-dlp-nightly-builds
yt-dlp is up to date (nightly@... from yt-dlp/yt-dlp-nightly-builds) yt-dlp is up to date (nightly@... from yt-dlp/yt-dlp-nightly-builds)
[youtube] Extracting URL: https://www.youtube.com/watch?v=BaW_jenozKc [youtube] Extracting URL: https://www.youtube.com/watch?v=BaW_jenozKc
<more lines> <more lines>
@@ -1,14 +1,12 @@
name: Site feature request name: Site feature request
description: Request a new functionality for a supported site description: Request new functionality for a site supported by yt-dlp
labels: [triage, site-enhancement] labels: [triage, site-enhancement]
body: body:
- type: checkboxes - type: markdown
attributes: attributes:
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE value: |
description: Fill all fields even if you think it is irrelevant for the issue > [!IMPORTANT]
options: > Not providing the required (*) information or removing the template will result in your issue being closed and ignored.
- label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field
required: true
- type: checkboxes - type: checkboxes
id: checklist id: checklist
attributes: attributes:
@@ -22,9 +20,9 @@ body:
required: true required: true
- label: I've checked that all provided URLs are playable in a browser with the same IP and same login details - label: I've checked that all provided URLs are playable in a browser with the same IP and same login details
required: true required: true
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates - label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=is%3Aissue%20-label%3Aspam%20%20) for similar requests **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [policy against AI/LLM contributions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#automated-contributions-ai--llm-policy) and understand I may be blocked from the repository if it is violated
required: true required: true
- label: I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required - label: I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required
- type: input - type: input
@@ -55,6 +53,8 @@ body:
id: verbose id: verbose
attributes: attributes:
label: Provide verbose output that clearly demonstrates the problem label: Provide verbose output that clearly demonstrates the problem
description: |
This is mandatory unless absolutely impossible to provide. If you are unable to provide the output, please explain why.
options: options:
- label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`) - label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
required: true required: true
@@ -71,14 +71,15 @@ body:
placeholder: | placeholder: |
[debug] Command-line config: ['-vU', 'https://www.youtube.com/watch?v=BaW_jenozKc'] [debug] Command-line config: ['-vU', 'https://www.youtube.com/watch?v=BaW_jenozKc']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8 [debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version nightly@... from yt-dlp/yt-dlp [b634ba742] (win_exe) [debug] yt-dlp version nightly@... from yt-dlp/yt-dlp-nightly-builds [1a176d874] (win_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0 [debug] Python 3.10.11 (CPython AMD64 64bit) - Windows-10-10.0.20348-SP0 (OpenSSL 1.1.1t 7 Feb 2023)
[debug] exe versions: ffmpeg N-106550-g072101bd52-20220410 (fdk,setts), ffprobe N-106624-g391ce570c8-20220415, phantomjs 2.1.1 [debug] exe versions: ffmpeg 7.0.2 (setts), ffprobe 7.0.2
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3 [debug] Optional libraries: Cryptodome-3.21.0, brotli-1.1.0, certifi-2024.08.30, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.40.1, urllib3-2.2.3, websockets-13.1
[debug] Proxy map: {} [debug] Proxy map: {}
[debug] Request Handlers: urllib, requests [debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1893 extractors [debug] Loaded 1838 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest [debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: nightly@... from yt-dlp/yt-dlp-nightly-builds
yt-dlp is up to date (nightly@... from yt-dlp/yt-dlp-nightly-builds) yt-dlp is up to date (nightly@... from yt-dlp/yt-dlp-nightly-builds)
[youtube] Extracting URL: https://www.youtube.com/watch?v=BaW_jenozKc [youtube] Extracting URL: https://www.youtube.com/watch?v=BaW_jenozKc
<more lines> <more lines>
+16 -19
View File
@@ -2,13 +2,11 @@ name: Core bug report
description: Report a bug unrelated to any particular site or extractor description: Report a bug unrelated to any particular site or extractor
labels: [triage, bug] labels: [triage, bug]
body: body:
- type: checkboxes - type: markdown
attributes: attributes:
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE value: |
description: Fill all fields even if you think it is irrelevant for the issue > [!IMPORTANT]
options: > Not providing the required (*) information or removing the template will result in your issue being closed and ignored.
- label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field
required: true
- type: checkboxes - type: checkboxes
id: checklist id: checklist
attributes: attributes:
@@ -20,13 +18,9 @@ body:
required: true required: true
- label: I've verified that I have **updated yt-dlp to nightly or master** ([update instructions](https://github.com/yt-dlp/yt-dlp#update-channels)) - label: I've verified that I have **updated yt-dlp to nightly or master** ([update instructions](https://github.com/yt-dlp/yt-dlp#update-channels))
required: true required: true
- label: I've checked that all provided URLs are playable in a browser with the same IP and same login details - label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766), [the FAQ](https://github.com/yt-dlp/yt-dlp/wiki/FAQ), and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=is%3Aissue%20-label%3Aspam%20%20) for similar issues **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command) - label: I've read the [policy against AI/LLM contributions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#automated-contributions-ai--llm-policy) and understand I may be blocked from the repository if it is violated
required: true
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
required: true required: true
- type: textarea - type: textarea
id: description id: description
@@ -40,6 +34,8 @@ body:
id: verbose id: verbose
attributes: attributes:
label: Provide verbose output that clearly demonstrates the problem label: Provide verbose output that clearly demonstrates the problem
description: |
This is mandatory unless absolutely impossible to provide. If you are unable to provide the output, please explain why.
options: options:
- label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`) - label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
required: true required: true
@@ -56,14 +52,15 @@ body:
placeholder: | placeholder: |
[debug] Command-line config: ['-vU', 'https://www.youtube.com/watch?v=BaW_jenozKc'] [debug] Command-line config: ['-vU', 'https://www.youtube.com/watch?v=BaW_jenozKc']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8 [debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version nightly@... from yt-dlp/yt-dlp [b634ba742] (win_exe) [debug] yt-dlp version nightly@... from yt-dlp/yt-dlp-nightly-builds [1a176d874] (win_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0 [debug] Python 3.10.11 (CPython AMD64 64bit) - Windows-10-10.0.20348-SP0 (OpenSSL 1.1.1t 7 Feb 2023)
[debug] exe versions: ffmpeg N-106550-g072101bd52-20220410 (fdk,setts), ffprobe N-106624-g391ce570c8-20220415, phantomjs 2.1.1 [debug] exe versions: ffmpeg 7.0.2 (setts), ffprobe 7.0.2
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3 [debug] Optional libraries: Cryptodome-3.21.0, brotli-1.1.0, certifi-2024.08.30, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.40.1, urllib3-2.2.3, websockets-13.1
[debug] Proxy map: {} [debug] Proxy map: {}
[debug] Request Handlers: urllib, requests [debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1893 extractors [debug] Loaded 1838 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest [debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: nightly@... from yt-dlp/yt-dlp-nightly-builds
yt-dlp is up to date (nightly@... from yt-dlp/yt-dlp-nightly-builds) yt-dlp is up to date (nightly@... from yt-dlp/yt-dlp-nightly-builds)
[youtube] Extracting URL: https://www.youtube.com/watch?v=BaW_jenozKc [youtube] Extracting URL: https://www.youtube.com/watch?v=BaW_jenozKc
<more lines> <more lines>
+17 -16
View File
@@ -1,14 +1,12 @@
name: Feature request name: Feature request
description: Request a new functionality unrelated to any particular site or extractor description: Request a new feature unrelated to any particular site or extractor
labels: [triage, enhancement] labels: [triage, enhancement]
body: body:
- type: checkboxes - type: markdown
attributes: attributes:
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE value: |
description: Fill all fields even if you think it is irrelevant for the issue > [!IMPORTANT]
options: > Not providing the required (*) information or removing the template will result in your issue being closed and ignored.
- label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field
required: true
- type: checkboxes - type: checkboxes
id: checklist id: checklist
attributes: attributes:
@@ -22,9 +20,9 @@ body:
required: true required: true
- label: I've verified that I have **updated yt-dlp to nightly or master** ([update instructions](https://github.com/yt-dlp/yt-dlp#update-channels)) - label: I've verified that I have **updated yt-dlp to nightly or master** ([update instructions](https://github.com/yt-dlp/yt-dlp#update-channels))
required: true required: true
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates - label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=is%3Aissue%20-label%3Aspam%20%20) for similar requests **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [policy against AI/LLM contributions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#automated-contributions-ai--llm-policy) and understand I may be blocked from the repository if it is violated
required: true required: true
- type: textarea - type: textarea
id: description id: description
@@ -38,6 +36,8 @@ body:
id: verbose id: verbose
attributes: attributes:
label: Provide verbose output that clearly demonstrates the problem label: Provide verbose output that clearly demonstrates the problem
description: |
This is mandatory unless absolutely impossible to provide. If you are unable to provide the output, please explain why.
options: options:
- label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`) - label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- label: "If using API, add `'verbose': True` to `YoutubeDL` params instead" - label: "If using API, add `'verbose': True` to `YoutubeDL` params instead"
@@ -52,14 +52,15 @@ body:
placeholder: | placeholder: |
[debug] Command-line config: ['-vU', 'https://www.youtube.com/watch?v=BaW_jenozKc'] [debug] Command-line config: ['-vU', 'https://www.youtube.com/watch?v=BaW_jenozKc']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8 [debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version nightly@... from yt-dlp/yt-dlp [b634ba742] (win_exe) [debug] yt-dlp version nightly@... from yt-dlp/yt-dlp-nightly-builds [1a176d874] (win_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0 [debug] Python 3.10.11 (CPython AMD64 64bit) - Windows-10-10.0.20348-SP0 (OpenSSL 1.1.1t 7 Feb 2023)
[debug] exe versions: ffmpeg N-106550-g072101bd52-20220410 (fdk,setts), ffprobe N-106624-g391ce570c8-20220415, phantomjs 2.1.1 [debug] exe versions: ffmpeg 7.0.2 (setts), ffprobe 7.0.2
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3 [debug] Optional libraries: Cryptodome-3.21.0, brotli-1.1.0, certifi-2024.08.30, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.40.1, urllib3-2.2.3, websockets-13.1
[debug] Proxy map: {} [debug] Proxy map: {}
[debug] Request Handlers: urllib, requests [debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1893 extractors [debug] Loaded 1838 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest [debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: nightly@... from yt-dlp/yt-dlp-nightly-builds
yt-dlp is up to date (nightly@... from yt-dlp/yt-dlp-nightly-builds) yt-dlp is up to date (nightly@... from yt-dlp/yt-dlp-nightly-builds)
[youtube] Extracting URL: https://www.youtube.com/watch?v=BaW_jenozKc [youtube] Extracting URL: https://www.youtube.com/watch?v=BaW_jenozKc
<more lines> <more lines>
+17 -16
View File
@@ -1,14 +1,12 @@
name: Ask question name: Ask question
description: Ask yt-dlp related question description: Ask a question about using yt-dlp
labels: [question] labels: [question]
body: body:
- type: checkboxes - type: markdown
attributes: attributes:
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE value: |
description: Fill all fields even if you think it is irrelevant for the issue > [!IMPORTANT]
options: > Not providing the required (*) information or removing the template will result in your issue being closed and ignored.
- label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field
required: true
- type: markdown - type: markdown
attributes: attributes:
value: | value: |
@@ -28,9 +26,9 @@ body:
required: true required: true
- label: I've verified that I have **updated yt-dlp to nightly or master** ([update instructions](https://github.com/yt-dlp/yt-dlp#update-channels)) - label: I've verified that I have **updated yt-dlp to nightly or master** ([update instructions](https://github.com/yt-dlp/yt-dlp#update-channels))
required: true required: true
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar questions **including closed ones**. DO NOT post duplicates - label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766), [the FAQ](https://github.com/yt-dlp/yt-dlp/wiki/FAQ), and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=is%3Aissue%20-label%3Aspam%20%20) for similar questions **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [policy against AI/LLM contributions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#automated-contributions-ai--llm-policy) and understand I may be blocked from the repository if it is violated
required: true required: true
- type: textarea - type: textarea
id: question id: question
@@ -44,6 +42,8 @@ body:
id: verbose id: verbose
attributes: attributes:
label: Provide verbose output that clearly demonstrates the problem label: Provide verbose output that clearly demonstrates the problem
description: |
This is mandatory unless absolutely impossible to provide. If you are unable to provide the output, please explain why.
options: options:
- label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`) - label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- label: "If using API, add `'verbose': True` to `YoutubeDL` params instead" - label: "If using API, add `'verbose': True` to `YoutubeDL` params instead"
@@ -58,14 +58,15 @@ body:
placeholder: | placeholder: |
[debug] Command-line config: ['-vU', 'https://www.youtube.com/watch?v=BaW_jenozKc'] [debug] Command-line config: ['-vU', 'https://www.youtube.com/watch?v=BaW_jenozKc']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8 [debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version nightly@... from yt-dlp/yt-dlp [b634ba742] (win_exe) [debug] yt-dlp version nightly@... from yt-dlp/yt-dlp-nightly-builds [1a176d874] (win_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0 [debug] Python 3.10.11 (CPython AMD64 64bit) - Windows-10-10.0.20348-SP0 (OpenSSL 1.1.1t 7 Feb 2023)
[debug] exe versions: ffmpeg N-106550-g072101bd52-20220410 (fdk,setts), ffprobe N-106624-g391ce570c8-20220415, phantomjs 2.1.1 [debug] exe versions: ffmpeg 7.0.2 (setts), ffprobe 7.0.2
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3 [debug] Optional libraries: Cryptodome-3.21.0, brotli-1.1.0, certifi-2024.08.30, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.40.1, urllib3-2.2.3, websockets-13.1
[debug] Proxy map: {} [debug] Proxy map: {}
[debug] Request Handlers: urllib, requests [debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1893 extractors [debug] Loaded 1838 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest [debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: nightly@... from yt-dlp/yt-dlp-nightly-builds
yt-dlp is up to date (nightly@... from yt-dlp/yt-dlp-nightly-builds) yt-dlp is up to date (nightly@... from yt-dlp/yt-dlp-nightly-builds)
[youtube] Extracting URL: https://www.youtube.com/watch?v=BaW_jenozKc [youtube] Extracting URL: https://www.youtube.com/watch?v=BaW_jenozKc
<more lines> <more lines>
+2 -5
View File
@@ -1,8 +1,5 @@
blank_issues_enabled: false blank_issues_enabled: false
contact_links: contact_links:
- name: Get help from the community on Discord - name: Get help on Discord
url: https://discord.gg/H5MNcFW63r url: https://discord.gg/H5MNcFW63r
about: Join the yt-dlp Discord for community-powered support! about: Join the yt-dlp Discord server for support and discussion
- name: Matrix Bridge to the Discord server
url: https://matrix.to/#/#yt-dlp:matrix.org
about: For those who do not want to use Discord
@@ -18,9 +18,9 @@ body:
required: true required: true
- label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command) - label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
required: true required: true
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates - label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766), [the FAQ](https://github.com/yt-dlp/yt-dlp/wiki/FAQ), and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=is%%3Aissue%%20-label%%3Aspam%%20%%20) for similar issues **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [policy against AI/LLM contributions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#automated-contributions-ai--llm-policy) and understand I may be blocked from the repository if it is violated
required: true required: true
- label: I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required - label: I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required
- type: input - type: input
@@ -18,9 +18,9 @@ body:
required: true required: true
- label: I've checked that none of provided URLs [violate any copyrights](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy) or contain any [DRM](https://en.wikipedia.org/wiki/Digital_rights_management) to the best of my knowledge - label: I've checked that none of provided URLs [violate any copyrights](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy) or contain any [DRM](https://en.wikipedia.org/wiki/Digital_rights_management) to the best of my knowledge
required: true required: true
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates - label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=is%%3Aissue%%20-label%%3Aspam%%20%%20) for similar requests **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [policy against AI/LLM contributions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#automated-contributions-ai--llm-policy) and understand I may be blocked from the repository if it is violated
required: true required: true
- label: I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and am willing to share it if required - label: I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and am willing to share it if required
- type: input - type: input
@@ -1,5 +1,5 @@
name: Site feature request name: Site feature request
description: Request a new functionality for a supported site description: Request new functionality for a site supported by yt-dlp
labels: [triage, site-enhancement] labels: [triage, site-enhancement]
body: body:
%(no_skip)s %(no_skip)s
@@ -16,9 +16,9 @@ body:
required: true required: true
- label: I've checked that all provided URLs are playable in a browser with the same IP and same login details - label: I've checked that all provided URLs are playable in a browser with the same IP and same login details
required: true required: true
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates - label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=is%%3Aissue%%20-label%%3Aspam%%20%%20) for similar requests **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [policy against AI/LLM contributions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#automated-contributions-ai--llm-policy) and understand I may be blocked from the repository if it is violated
required: true required: true
- label: I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required - label: I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required
- type: input - type: input
+2 -6
View File
@@ -14,13 +14,9 @@ body:
required: true required: true
- label: I've verified that I have **updated yt-dlp to nightly or master** ([update instructions](https://github.com/yt-dlp/yt-dlp#update-channels)) - label: I've verified that I have **updated yt-dlp to nightly or master** ([update instructions](https://github.com/yt-dlp/yt-dlp#update-channels))
required: true required: true
- label: I've checked that all provided URLs are playable in a browser with the same IP and same login details - label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766), [the FAQ](https://github.com/yt-dlp/yt-dlp/wiki/FAQ), and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=is%%3Aissue%%20-label%%3Aspam%%20%%20) for similar issues **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command) - label: I've read the [policy against AI/LLM contributions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#automated-contributions-ai--llm-policy) and understand I may be blocked from the repository if it is violated
required: true
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
required: true required: true
- type: textarea - type: textarea
id: description id: description
@@ -1,5 +1,5 @@
name: Feature request name: Feature request
description: Request a new functionality unrelated to any particular site or extractor description: Request a new feature unrelated to any particular site or extractor
labels: [triage, enhancement] labels: [triage, enhancement]
body: body:
%(no_skip)s %(no_skip)s
@@ -16,9 +16,9 @@ body:
required: true required: true
- label: I've verified that I have **updated yt-dlp to nightly or master** ([update instructions](https://github.com/yt-dlp/yt-dlp#update-channels)) - label: I've verified that I have **updated yt-dlp to nightly or master** ([update instructions](https://github.com/yt-dlp/yt-dlp#update-channels))
required: true required: true
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates - label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=is%%3Aissue%%20-label%%3Aspam%%20%%20) for similar requests **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [policy against AI/LLM contributions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#automated-contributions-ai--llm-policy) and understand I may be blocked from the repository if it is violated
required: true required: true
- type: textarea - type: textarea
id: description id: description
+3 -3
View File
@@ -1,5 +1,5 @@
name: Ask question name: Ask question
description: Ask yt-dlp related question description: Ask a question about using yt-dlp
labels: [question] labels: [question]
body: body:
%(no_skip)s %(no_skip)s
@@ -22,9 +22,9 @@ body:
required: true required: true
- label: I've verified that I have **updated yt-dlp to nightly or master** ([update instructions](https://github.com/yt-dlp/yt-dlp#update-channels)) - label: I've verified that I have **updated yt-dlp to nightly or master** ([update instructions](https://github.com/yt-dlp/yt-dlp#update-channels))
required: true required: true
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar questions **including closed ones**. DO NOT post duplicates - label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766), [the FAQ](https://github.com/yt-dlp/yt-dlp/wiki/FAQ), and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=is%%3Aissue%%20-label%%3Aspam%%20%%20) for similar questions **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [policy against AI/LLM contributions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#automated-contributions-ai--llm-policy) and understand I may be blocked from the repository if it is violated
required: true required: true
- type: textarea - type: textarea
id: question id: question
+20 -19
View File
@@ -1,14 +1,17 @@
**IMPORTANT**: PRs without the template will be CLOSED <!--
**IMPORTANT**: PRs without the template will be CLOSED
Due to the high volume of pull requests, it may be a while before your PR is reviewed.
Please try to keep your pull request focused on a single bugfix or new feature.
Pull requests with a vast scope and/or very large diff will take much longer to review.
It is recommended for new contributors to stick to smaller pull requests, so you can receive much more immediate feedback as you familiarize yourself with the codebase.
PLEASE AVOID FORCE-PUSHING after opening a PR, as it makes reviewing more difficult.
-->
### Description of your *pull request* and other information ### Description of your *pull request* and other information
<!-- ADD DETAILED DESCRIPTION HERE
Explanation of your *pull request* in arbitrary form goes here. Please **make sure the description explains the purpose and effect** of your *pull request* and is worded well enough to be understood. Provide as much **context and examples** as possible
-->
ADD DESCRIPTION HERE
Fixes # Fixes #
@@ -16,25 +19,23 @@ Fixes #
<details open><summary>Template</summary> <!-- OPEN is intentional --> <details open><summary>Template</summary> <!-- OPEN is intentional -->
<!-- <!--
# PLEASE FOLLOW THE GUIDE BELOW
# PLEASE FOLLOW THE GUIDE BELOW - You will be asked some questions, please read them **carefully** and answer honestly
- Put an `x` into all the boxes `[ ]` relevant to your *pull request* (like [x])
- You will be asked some questions, please read them **carefully** and answer honestly - Use *Preview* tab to see what your *pull request* will actually look like
- Put an `x` into all the boxes `[ ]` relevant to your *pull request* (like [x])
- Use *Preview* tab to see how your *pull request* will actually look like
--> -->
### Before submitting a *pull request* make sure you have: ### Before submitting a *pull request* make sure you have:
- [ ] At least skimmed through [contributing guidelines](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions) including [yt-dlp coding conventions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#yt-dlp-coding-conventions) - [ ] At least skimmed through [contributing guidelines](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions) including [yt-dlp coding conventions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#yt-dlp-coding-conventions)
- [ ] [Searched](https://github.com/yt-dlp/yt-dlp/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests - [ ] [Searched](https://github.com/yt-dlp/yt-dlp/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
- [ ] Checked the code with [flake8](https://pypi.python.org/pypi/flake8) and [ran relevant tests](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions)
### In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check all of the following options that apply: ### In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check those that apply and remove the others:
- [ ] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/) - [ ] I am the original author of the code in this PR, and I am willing to release it under [Unlicense](http://unlicense.org/)
- [ ] I am not the original author of this code but it is in public domain or released under [Unlicense](http://unlicense.org/) (provide reliable evidence) - [ ] I am not the original author of the code in this PR, but it is in the public domain or released under [Unlicense](http://unlicense.org/) (provide reliable evidence)
- [ ] I have read the [policy against AI/LLM contributions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#automated-contributions-ai--llm-policy) and understand I may be blocked from the repository if it is violated
### What is the purpose of your *pull request*? ### What is the purpose of your *pull request*? Check those that apply and remove the others:
- [ ] Fix or improvement to an extractor (Make sure to add/update tests) - [ ] Fix or improvement to an extractor (Make sure to add/update tests)
- [ ] New extractor ([Piracy websites will not be accepted](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy)) - [ ] New extractor ([Piracy websites will not be accepted](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy))
- [ ] Core bug fix/improvement - [ ] Core bug fix/improvement
+28
View File
@@ -0,0 +1,28 @@
self-hosted-runner:
labels:
# Workaround for the outdated runner list in actionlint v1.7.7
# Ref: https://github.com/rhysd/actionlint/issues/533
- windows-11-arm
config-variables:
- KEEP_CACHE_WARM
- PUSH_VERSION_COMMIT
- UPDATE_TO_VERIFICATION
- PYPI_PROJECT
- PYPI_SUFFIX
- NIGHTLY_PYPI_PROJECT
- NIGHTLY_PYPI_SUFFIX
- NIGHTLY_ARCHIVE_REPO
- BUILD_NIGHTLY
- MASTER_PYPI_PROJECT
- MASTER_PYPI_SUFFIX
- MASTER_ARCHIVE_REPO
- BUILD_MASTER
- ISSUE_LOCKDOWN
- SANITIZE_COMMENT
paths:
.github/workflows/build.yml:
ignore:
# SC1090 "Can't follow non-constant source": ignore when using `source` to activate venv
- '.+SC1090.+'
+5 -5
View File
File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 24 KiB

After

Width:  |  Height:  |  Size: 15 KiB

+384 -261
View File
@@ -9,31 +9,27 @@ on:
required: false required: false
default: stable default: stable
type: string type: string
origin:
required: true
type: string
unix: unix:
default: true default: true
type: boolean type: boolean
linux_arm: linux:
default: true
type: boolean
linux_armv7l:
default: true
type: boolean
musllinux:
default: true default: true
type: boolean type: boolean
macos: macos:
default: true default: true
type: boolean type: boolean
macos_legacy:
default: true
type: boolean
windows: windows:
default: true default: true
type: boolean type: boolean
windows32:
default: true
type: boolean
meta_files:
default: true
type: boolean
origin:
required: false
default: ''
type: string
secrets: secrets:
GPG_SIGNING_KEY: GPG_SIGNING_KEY:
required: false required: false
@@ -43,7 +39,9 @@ on:
version: version:
description: | description: |
VERSION: yyyy.mm.dd[.rev] or rev VERSION: yyyy.mm.dd[.rev] or rev
required: true (default: auto-generated)
required: false
default: ''
type: string type: string
channel: channel:
description: | description: |
@@ -52,40 +50,29 @@ on:
default: stable default: stable
type: string type: string
unix: unix:
description: yt-dlp, yt-dlp.tar.gz, yt-dlp_linux, yt-dlp_linux.zip description: yt-dlp, yt-dlp.tar.gz
default: true default: true
type: boolean type: boolean
linux_arm: linux:
description: yt-dlp_linux_aarch64, yt-dlp_linux_armv7l description: yt-dlp_linux, yt-dlp_linux.zip, yt-dlp_linux_aarch64, yt-dlp_linux_aarch64.zip
default: true
type: boolean
linux_armv7l:
description: yt-dlp_linux_armv7l.zip
default: true
type: boolean
musllinux:
description: yt-dlp_musllinux, yt-dlp_musllinux.zip, yt-dlp_musllinux_aarch64, yt-dlp_musllinux_aarch64.zip
default: true default: true
type: boolean type: boolean
macos: macos:
description: yt-dlp_macos, yt-dlp_macos.zip description: yt-dlp_macos, yt-dlp_macos.zip
default: true default: true
type: boolean type: boolean
macos_legacy:
description: yt-dlp_macos_legacy
default: true
type: boolean
windows: windows:
description: yt-dlp.exe, yt-dlp_min.exe, yt-dlp_win.zip description: yt-dlp.exe, yt-dlp_win.zip, yt-dlp_x86.exe, yt-dlp_win_x86.zip, yt-dlp_arm64.exe, yt-dlp_win_arm64.zip
default: true default: true
type: boolean type: boolean
windows32:
description: yt-dlp_x86.exe
default: true
type: boolean
meta_files:
description: SHA2-256SUMS, SHA2-512SUMS, _update_spec
default: true
type: boolean
origin:
description: Origin
required: false
default: 'current repo'
type: choice
options:
- 'current repo'
permissions: permissions:
contents: read contents: read
@@ -94,72 +81,150 @@ jobs:
process: process:
runs-on: ubuntu-latest runs-on: ubuntu-latest
outputs: outputs:
origin: ${{ steps.process_origin.outputs.origin }} origin: ${{ steps.process_inputs.outputs.origin }}
timestamp: ${{ steps.process_inputs.outputs.timestamp }}
version: ${{ steps.process_inputs.outputs.version }}
linux_matrix: ${{ steps.linux_matrix.outputs.matrix }}
steps: steps:
- name: Process origin - name: Process inputs
id: process_origin id: process_inputs
env:
INPUTS: ${{ toJSON(inputs) }}
REPOSITORY: ${{ github.repository }}
shell: python
run: | run: |
echo "origin=${{ inputs.origin == 'current repo' && github.repository || inputs.origin }}" | tee "$GITHUB_OUTPUT" import datetime as dt
import json
import os
import re
INPUTS = json.loads(os.environ['INPUTS'])
timestamp = dt.datetime.now(tz=dt.timezone.utc).strftime('%Y.%m.%d.%H%M%S.%f')
version = INPUTS.get('version')
if version and '.' not in version:
# build.yml was dispatched with only a revision as the version input value
version_parts = [*timestamp.split('.')[:3], version]
elif not version:
# build.yml was dispatched without any version input value, so include .HHMMSS revision
version_parts = timestamp.split('.')[:4]
else:
# build.yml was called or dispatched with a complete version input value
version_parts = version.split('.')
assert all(re.fullmatch(r'[0-9]+', part) for part in version_parts), 'Version must be numeric'
outputs = {
'origin': INPUTS.get('origin') or os.environ['REPOSITORY'],
'timestamp': timestamp,
'version': '.'.join(version_parts),
}
print(json.dumps(outputs, indent=2))
with open(os.environ['GITHUB_OUTPUT'], 'a') as f:
f.write('\n'.join(f'{key}={value}' for key, value in outputs.items()))
- name: Build Linux matrix
id: linux_matrix
env:
INPUTS: ${{ toJSON(inputs) }}
PYTHON_VERSION: '3.13'
UPDATE_TO: yt-dlp/yt-dlp@2025.09.05
shell: python
run: |
import json
import os
EXE_MAP = {
'linux': [{
'os': 'linux',
'arch': 'x86_64',
'runner': 'ubuntu-24.04',
}, {
'os': 'linux',
'arch': 'aarch64',
'runner': 'ubuntu-24.04-arm',
}],
'linux_armv7l': [{
'os': 'linux',
'arch': 'armv7l',
'runner': 'ubuntu-24.04-arm',
'qemu_platform': 'linux/arm/v7',
'onefile': False,
'cache_requirements': True,
'update_to': 'yt-dlp/yt-dlp@2023.03.04',
}],
'musllinux': [{
'os': 'musllinux',
'arch': 'x86_64',
'runner': 'ubuntu-24.04',
}, {
'os': 'musllinux',
'arch': 'aarch64',
'runner': 'ubuntu-24.04-arm',
}],
}
INPUTS = json.loads(os.environ['INPUTS'])
matrix = [exe for key, group in EXE_MAP.items() for exe in group if INPUTS.get(key)]
if not matrix:
# If we send an empty matrix when no linux inputs are given, the entire workflow fails
matrix = [EXE_MAP['linux'][0]]
for exe in matrix:
exe['exe'] = '_'.join(filter(None, (
'yt-dlp',
exe['os'],
exe['arch'] != 'x86_64' and exe['arch'],
)))
exe.setdefault('qemu_platform', None)
exe.setdefault('onefile', True)
exe.setdefault('onedir', True)
exe.setdefault('cache_requirements', False)
exe.setdefault('python_version', os.environ['PYTHON_VERSION'])
exe.setdefault('update_to', os.environ['UPDATE_TO'])
if not any(INPUTS.get(key) for key in EXE_MAP):
print('skipping linux job')
else:
print(json.dumps(matrix, indent=2))
with open(os.environ['GITHUB_OUTPUT'], 'a') as f:
f.write(f'matrix={json.dumps(matrix)}')
unix: unix:
needs: process needs: process
if: inputs.unix if: inputs.unix
runs-on: ubuntu-latest runs-on: ubuntu-latest
env:
CHANNEL: ${{ inputs.channel }}
ORIGIN: ${{ needs.process.outputs.origin }}
VERSION: ${{ needs.process.outputs.version }}
UPDATE_TO: yt-dlp/yt-dlp@2025.09.05
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- uses: actions/setup-python@v5 with:
fetch-depth: 0 # Needed for changelog
- uses: actions/setup-python@v6
with: with:
python-version: "3.10" python-version: "3.10"
- uses: conda-incubator/setup-miniconda@v3
with:
miniforge-variant: Mambaforge
use-mamba: true
channels: conda-forge
auto-update-conda: true
activate-environment: ""
auto-activate-base: false
- name: Install Requirements - name: Install Requirements
run: | run: |
sudo apt -y install zip pandoc man sed sudo apt -y install zip pandoc man sed
cat > ./requirements.txt << EOF
python=3.10.*
brotli-python
EOF
python devscripts/install_deps.py --print \
--exclude brotli --exclude brotlicffi \
--include secretstorage --include pyinstaller >> ./requirements.txt
mamba create -n build --file ./requirements.txt
- name: Prepare - name: Prepare
run: | run: |
python devscripts/update-version.py -c "${{ inputs.channel }}" -r "${{ needs.process.outputs.origin }}" "${{ inputs.version }}" python devscripts/update-version.py -c "${CHANNEL}" -r "${ORIGIN}" "${VERSION}"
python devscripts/update_changelog.py -vv
python devscripts/make_lazy_extractors.py python devscripts/make_lazy_extractors.py
- name: Build Unix platform-independent binary - name: Build Unix platform-independent binary
run: | run: |
make all tar make all tar
- name: Build Unix standalone binary
shell: bash -l {0}
run: |
unset LD_LIBRARY_PATH # Harmful; set by setup-python
conda activate build
python -m bundle.pyinstaller --onedir
(cd ./dist/yt-dlp_linux && zip -r ../yt-dlp_linux.zip .)
python -m bundle.pyinstaller
mv ./dist/yt-dlp_linux ./yt-dlp_linux
mv ./dist/yt-dlp_linux.zip ./yt-dlp_linux.zip
- name: Verify --update-to - name: Verify --update-to
if: vars.UPDATE_TO_VERIFICATION if: vars.UPDATE_TO_VERIFICATION
run: | run: |
binaries=("yt-dlp" "yt-dlp_linux") chmod +x ./yt-dlp
for binary in "${binaries[@]}"; do cp ./yt-dlp ./yt-dlp_downgraded
chmod +x ./${binary} version="$(./yt-dlp --version)"
cp ./${binary} ./${binary}_downgraded ./yt-dlp_downgraded -v --update-to "${UPDATE_TO}"
version="$(./${binary} --version)" downgraded_version="$(./yt-dlp_downgraded --version)"
./${binary}_downgraded -v --update-to yt-dlp/yt-dlp@2023.03.04 [[ "${version}" != "${downgraded_version}" ]]
downgraded_version="$(./${binary}_downgraded --version)"
[[ "$version" != "$downgraded_version" ]]
done
- name: Upload artifacts - name: Upload artifacts
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v4
@@ -168,92 +233,155 @@ jobs:
path: | path: |
yt-dlp yt-dlp
yt-dlp.tar.gz yt-dlp.tar.gz
yt-dlp_linux
yt-dlp_linux.zip
compression-level: 0 compression-level: 0
linux_arm: linux:
name: ${{ matrix.os }} (${{ matrix.arch }})
if: inputs.linux || inputs.linux_armv7l || inputs.musllinux
needs: process needs: process
if: inputs.linux_arm runs-on: ${{ matrix.runner }}
permissions:
contents: read
packages: write # for creating cache
runs-on: ubuntu-latest
strategy: strategy:
fail-fast: false
matrix: matrix:
architecture: include: ${{ fromJSON(needs.process.outputs.linux_matrix) }}
- armv7 env:
- aarch64 CHANNEL: ${{ inputs.channel }}
ORIGIN: ${{ needs.process.outputs.origin }}
VERSION: ${{ needs.process.outputs.version }}
EXE_NAME: ${{ matrix.exe }}
PYTHON_VERSION: ${{ matrix.python_version }}
UPDATE_TO: ${{ (vars.UPDATE_TO_VERIFICATION && matrix.update_to) || '' }}
SKIP_ONEDIR_BUILD: ${{ (!matrix.onedir && '1') || '' }}
SKIP_ONEFILE_BUILD: ${{ (!matrix.onefile && '1') || '' }}
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
with:
path: ./repo
- name: Virtualized Install, Prepare & Build
uses: yt-dlp/run-on-arch-action@v2
with:
# Ref: https://github.com/uraimo/run-on-arch-action/issues/55
env: |
GITHUB_WORKFLOW: build
githubToken: ${{ github.token }} # To cache image
arch: ${{ matrix.architecture }}
distro: ubuntu18.04 # Standalone executable should be built on minimum supported OS
dockerRunArgs: --volume "${PWD}/repo:/repo"
install: | # Installing Python 3.10 from the Deadsnakes repo raises errors
apt update
apt -y install zlib1g-dev libffi-dev python3.8 python3.8-dev python3.8-distutils python3-pip
python3.8 -m pip install -U pip setuptools wheel
# Cannot access any files from the repo directory at this stage
python3.8 -m pip install -U Pyinstaller mutagen pycryptodomex websockets brotli certifi secretstorage cffi
run: | - name: Cache requirements
cd repo if: matrix.cache_requirements
python3.8 devscripts/install_deps.py -o --include build id: cache-venv
python3.8 devscripts/install_deps.py --include pyinstaller --include secretstorage # Cached version may be out of date uses: actions/cache@v4
python3.8 devscripts/update-version.py -c "${{ inputs.channel }}" -r "${{ needs.process.outputs.origin }}" "${{ inputs.version }}" env:
python3.8 devscripts/make_lazy_extractors.py SEGMENT_DOWNLOAD_TIMEOUT_MINS: 1
python3.8 -m bundle.pyinstaller with:
path: |
venv
key: cache-reqs-${{ matrix.os }}_${{ matrix.arch }}-${{ github.ref }}-${{ needs.process.outputs.timestamp }}
restore-keys: |
cache-reqs-${{ matrix.os }}_${{ matrix.arch }}-${{ github.ref }}-
cache-reqs-${{ matrix.os }}_${{ matrix.arch }}-
if ${{ vars.UPDATE_TO_VERIFICATION && 'true' || 'false' }}; then - name: Set up QEMU
arch="${{ (matrix.architecture == 'armv7' && 'armv7l') || matrix.architecture }}" if: matrix.qemu_platform
chmod +x ./dist/yt-dlp_linux_${arch} uses: docker/setup-qemu-action@v3
cp ./dist/yt-dlp_linux_${arch} ./dist/yt-dlp_linux_${arch}_downgraded with:
version="$(./dist/yt-dlp_linux_${arch} --version)" platforms: ${{ matrix.qemu_platform }}
./dist/yt-dlp_linux_${arch}_downgraded -v --update-to yt-dlp/yt-dlp@2023.03.04
downgraded_version="$(./dist/yt-dlp_linux_${arch}_downgraded --version)" - name: Build executable
[[ "$version" != "$downgraded_version" ]] env:
fi SERVICE: ${{ matrix.os }}_${{ matrix.arch }}
run: |
mkdir -p ./venv
mkdir -p ./dist
pushd bundle/docker
docker compose up --build --exit-code-from "${SERVICE}" "${SERVICE}"
popd
if [[ -z "${SKIP_ONEFILE_BUILD}" ]]; then
sudo chown "${USER}:docker" "./dist/${EXE_NAME}"
fi
- name: Verify executable in container
env:
SERVICE: ${{ matrix.os }}_${{ matrix.arch }}_verify
run: |
cd bundle/docker
docker compose up --build --exit-code-from "${SERVICE}" "${SERVICE}"
- name: Upload artifacts - name: Upload artifacts
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v4
with: with:
name: build-bin-linux_${{ matrix.architecture }} name: build-bin-${{ matrix.os }}_${{ matrix.arch }}
path: | # run-on-arch-action designates armv7l as armv7 path: |
repo/dist/yt-dlp_linux_${{ (matrix.architecture == 'armv7' && 'armv7l') || matrix.architecture }} dist/${{ matrix.exe }}*
compression-level: 0 compression-level: 0
macos: macos:
needs: process needs: process
if: inputs.macos if: inputs.macos
runs-on: macos-11 permissions:
contents: read
runs-on: macos-14
env:
CHANNEL: ${{ inputs.channel }}
ORIGIN: ${{ needs.process.outputs.origin }}
VERSION: ${{ needs.process.outputs.version }}
UPDATE_TO: yt-dlp/yt-dlp@2025.09.05
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
# NB: Building universal2 does not work with python from actions/setup-python # NB: Building universal2 does not work with python from actions/setup-python
- name: Cache requirements
id: cache-venv
uses: actions/cache@v4
env:
SEGMENT_DOWNLOAD_TIMEOUT_MINS: 1
with:
path: |
~/yt-dlp-build-venv
key: cache-reqs-${{ github.job }}-${{ github.ref }}-${{ needs.process.outputs.timestamp }}
restore-keys: |
cache-reqs-${{ github.job }}-${{ github.ref }}-
cache-reqs-${{ github.job }}-
- name: Install Requirements - name: Install Requirements
run: | run: |
brew install coreutils brew install coreutils
python3 devscripts/install_deps.py --user -o --include build # We need to use system Python in order to roll our own universal2 curl_cffi wheel
brew uninstall --ignore-dependencies python3
python3 -m venv ~/yt-dlp-build-venv
source ~/yt-dlp-build-venv/bin/activate
python3 devscripts/install_deps.py -o --include build
python3 devscripts/install_deps.py --print --include pyinstaller > requirements.txt python3 devscripts/install_deps.py --print --include pyinstaller > requirements.txt
# We need to ignore wheels otherwise we break universal2 builds # We need to ignore wheels otherwise we break universal2 builds
python3 -m pip install -U --user --no-binary :all: -r requirements.txt python3 -m pip install -U --no-binary :all: -r requirements.txt
# We need to fuse our own universal2 wheels for curl_cffi
python3 -m pip install -U 'delocate==0.11.0'
mkdir curl_cffi_whls curl_cffi_universal2
python3 devscripts/install_deps.py --print -o --include curl-cffi > requirements.txt
for platform in "macosx_11_0_arm64" "macosx_11_0_x86_64"; do
python3 -m pip download \
--only-binary=:all: \
--platform "${platform}" \
-d curl_cffi_whls \
-r requirements.txt
done
( # Overwrite x86_64-only libs with fat/universal2 libs or else Pyinstaller will do the opposite
# See https://github.com/yt-dlp/yt-dlp/pull/10069
cd curl_cffi_whls
mkdir -p curl_cffi/.dylibs
python_libdir=$(python3 -c 'import sys; from pathlib import Path; print(Path(sys.path[1]).parent)')
for dylib in lib{ssl,crypto}.3.dylib; do
cp "${python_libdir}/${dylib}" "curl_cffi/.dylibs/${dylib}"
for wheel in curl_cffi*macos*x86_64.whl; do
zip "${wheel}" "curl_cffi/.dylibs/${dylib}"
done
done
)
python3 -m delocate.cmd.delocate_fuse curl_cffi_whls/curl_cffi*.whl -w curl_cffi_universal2
python3 -m delocate.cmd.delocate_fuse curl_cffi_whls/cffi*.whl -w curl_cffi_universal2
for wheel in curl_cffi_universal2/*cffi*.whl; do
mv -n -- "${wheel}" "${wheel/x86_64/universal2}"
done
python3 -m pip install --force-reinstall -U curl_cffi_universal2/*cffi*.whl
- name: Prepare - name: Prepare
run: | run: |
python3 devscripts/update-version.py -c "${{ inputs.channel }}" -r "${{ needs.process.outputs.origin }}" "${{ inputs.version }}" python3 devscripts/update-version.py -c "${CHANNEL}" -r "${ORIGIN}" "${VERSION}"
python3 devscripts/make_lazy_extractors.py python3 devscripts/make_lazy_extractors.py
- name: Build - name: Build
run: | run: |
source ~/yt-dlp-build-venv/bin/activate
python3 -m bundle.pyinstaller --target-architecture universal2 --onedir python3 -m bundle.pyinstaller --target-architecture universal2 --onedir
(cd ./dist/yt-dlp_macos && zip -r ../yt-dlp_macos.zip .) (cd ./dist/yt-dlp_macos && zip -r ../yt-dlp_macos.zip .)
python3 -m bundle.pyinstaller --target-architecture universal2 python3 -m bundle.pyinstaller --target-architecture universal2
@@ -264,7 +392,7 @@ jobs:
chmod +x ./dist/yt-dlp_macos chmod +x ./dist/yt-dlp_macos
cp ./dist/yt-dlp_macos ./dist/yt-dlp_macos_downgraded cp ./dist/yt-dlp_macos ./dist/yt-dlp_macos_downgraded
version="$(./dist/yt-dlp_macos --version)" version="$(./dist/yt-dlp_macos --version)"
./dist/yt-dlp_macos_downgraded -v --update-to yt-dlp/yt-dlp@2023.03.04 ./dist/yt-dlp_macos_downgraded -v --update-to "${UPDATE_TO}"
downgraded_version="$(./dist/yt-dlp_macos_downgraded --version)" downgraded_version="$(./dist/yt-dlp_macos_downgraded --version)"
[[ "$version" != "$downgraded_version" ]] [[ "$version" != "$downgraded_version" ]]
@@ -277,168 +405,138 @@ jobs:
dist/yt-dlp_macos.zip dist/yt-dlp_macos.zip
compression-level: 0 compression-level: 0
macos_legacy:
needs: process
if: inputs.macos_legacy
runs-on: macos-latest
steps:
- uses: actions/checkout@v4
- name: Install Python
# We need the official Python, because the GA ones only support newer macOS versions
env:
PYTHON_VERSION: 3.10.5
MACOSX_DEPLOYMENT_TARGET: 10.9 # Used up by the Python build tools
run: |
# Hack to get the latest patch version. Uncomment if needed
#brew install python@3.10
#export PYTHON_VERSION=$( $(brew --prefix)/opt/python@3.10/bin/python3 --version | cut -d ' ' -f 2 )
curl https://www.python.org/ftp/python/${PYTHON_VERSION}/python-${PYTHON_VERSION}-macos11.pkg -o "python.pkg"
sudo installer -pkg python.pkg -target /
python3 --version
- name: Install Requirements
run: |
brew install coreutils
python3 devscripts/install_deps.py --user -o --include build
python3 devscripts/install_deps.py --user --include pyinstaller
- name: Prepare
run: |
python3 devscripts/update-version.py -c "${{ inputs.channel }}" -r "${{ needs.process.outputs.origin }}" "${{ inputs.version }}"
python3 devscripts/make_lazy_extractors.py
- name: Build
run: |
python3 -m bundle.pyinstaller
mv dist/yt-dlp_macos dist/yt-dlp_macos_legacy
- name: Verify --update-to
if: vars.UPDATE_TO_VERIFICATION
run: |
chmod +x ./dist/yt-dlp_macos_legacy
cp ./dist/yt-dlp_macos_legacy ./dist/yt-dlp_macos_legacy_downgraded
version="$(./dist/yt-dlp_macos_legacy --version)"
./dist/yt-dlp_macos_legacy_downgraded -v --update-to yt-dlp/yt-dlp@2023.03.04
downgraded_version="$(./dist/yt-dlp_macos_legacy_downgraded --version)"
[[ "$version" != "$downgraded_version" ]]
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: build-bin-${{ github.job }}
path: |
dist/yt-dlp_macos_legacy
compression-level: 0
windows: windows:
name: windows (${{ matrix.arch }})
needs: process needs: process
if: inputs.windows if: inputs.windows
runs-on: windows-latest permissions:
contents: read
runs-on: ${{ matrix.runner }}
strategy:
fail-fast: false
matrix:
include:
- arch: 'x64'
runner: windows-2025
python_version: '3.10'
platform_tag: win_amd64
pyi_version: '6.16.0'
pyi_tag: '2025.09.13.221251'
pyi_hash: b6496c7630c3afe66900cfa824e8234a8c2e2c81704bd7facd79586abc76c0e5
- arch: 'x86'
runner: windows-2025
python_version: '3.10'
platform_tag: win32
pyi_version: '6.16.0'
pyi_tag: '2025.09.13.221251'
pyi_hash: 2d881843580efdc54f3523507fc6d9c5b6051ee49c743a6d9b7003ac5758c226
- arch: 'arm64'
runner: windows-11-arm
python_version: '3.13' # arm64 only has Python >= 3.11 available
platform_tag: win_arm64
pyi_version: '6.16.0'
pyi_tag: '2025.09.13.221251'
pyi_hash: 4250c9085e34a95c898f3ee2f764914fc36ec59f0d97c28e6a75fcf21f7b144f
env:
CHANNEL: ${{ inputs.channel }}
ORIGIN: ${{ needs.process.outputs.origin }}
VERSION: ${{ needs.process.outputs.version }}
SUFFIX: ${{ (matrix.arch != 'x64' && format('_{0}', matrix.arch)) || '' }}
UPDATE_TO: yt-dlp/yt-dlp@2025.09.05
BASE_CACHE_KEY: cache-reqs-${{ github.job }}_${{ matrix.arch }}-${{ matrix.python_version }}
PYI_REPO: https://github.com/yt-dlp/Pyinstaller-Builds
PYI_WHEEL: pyinstaller-${{ matrix.pyi_version }}-py3-none-${{ matrix.platform_tag }}.whl
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- uses: actions/setup-python@v5 - uses: actions/setup-python@v6
with: # 3.8 is used for Win7 support with:
python-version: "3.8" python-version: ${{ matrix.python_version }}
architecture: ${{ matrix.arch }}
- name: Cache requirements
id: cache-venv
if: matrix.arch == 'arm64'
uses: actions/cache@v4
env:
SEGMENT_DOWNLOAD_TIMEOUT_MINS: 1
with:
path: |
/yt-dlp-build-venv
key: ${{ env.BASE_CACHE_KEY }}-${{ github.ref }}-${{ needs.process.outputs.timestamp }}
restore-keys: |
${{ env.BASE_CACHE_KEY }}-${{ github.ref }}-
${{ env.BASE_CACHE_KEY }}-
- name: Install Requirements - name: Install Requirements
run: | # Custom pyinstaller built with https://github.com/yt-dlp/pyinstaller-builds env:
ARCH: ${{ matrix.arch }}
PYI_URL: ${{ env.PYI_REPO }}/releases/download/${{ matrix.pyi_tag }}/${{ env.PYI_WHEEL }}
PYI_HASH: ${{ matrix.pyi_hash }}
shell: pwsh
run: |
python -m venv /yt-dlp-build-venv
/yt-dlp-build-venv/Scripts/Activate.ps1
python -m pip install -U pip
# Install custom PyInstaller build and verify hash
mkdir /pyi-wheels
python -m pip download -d /pyi-wheels --no-deps --require-hashes "pyinstaller@${Env:PYI_URL}#sha256=${Env:PYI_HASH}"
python -m pip install --force-reinstall -U "/pyi-wheels/${Env:PYI_WHEEL}"
python devscripts/install_deps.py -o --include build python devscripts/install_deps.py -o --include build
python devscripts/install_deps.py --include py2exe if ("${Env:ARCH}" -eq "x86") {
python -m pip install -U "https://yt-dlp.github.io/Pyinstaller-Builds/x86_64/pyinstaller-5.8.0-py3-none-any.whl" python devscripts/install_deps.py
} else {
python devscripts/install_deps.py --include curl-cffi
}
- name: Prepare - name: Prepare
shell: pwsh
run: | run: |
python devscripts/update-version.py -c "${{ inputs.channel }}" -r "${{ needs.process.outputs.origin }}" "${{ inputs.version }}" python devscripts/update-version.py -c "${Env:CHANNEL}" -r "${Env:ORIGIN}" "${Env:VERSION}"
python devscripts/make_lazy_extractors.py python devscripts/make_lazy_extractors.py
- name: Build - name: Build
shell: pwsh
run: | run: |
python -m bundle.py2exe /yt-dlp-build-venv/Scripts/Activate.ps1
Move-Item ./dist/yt-dlp.exe ./dist/yt-dlp_min.exe
python -m bundle.pyinstaller python -m bundle.pyinstaller
python -m bundle.pyinstaller --onedir python -m bundle.pyinstaller --onedir
Compress-Archive -Path ./dist/yt-dlp/* -DestinationPath ./dist/yt-dlp_win.zip Compress-Archive -Path ./dist/yt-dlp${Env:SUFFIX}/* -DestinationPath ./dist/yt-dlp_win${Env:SUFFIX}.zip
- name: Verify --update-to - name: Verify --update-to
if: vars.UPDATE_TO_VERIFICATION if: vars.UPDATE_TO_VERIFICATION
shell: pwsh
run: | run: |
foreach ($name in @("yt-dlp","yt-dlp_min")) { $name = "yt-dlp${Env:SUFFIX}"
Copy-Item "./dist/${name}.exe" "./dist/${name}_downgraded.exe" Copy-Item "./dist/${name}.exe" "./dist/${name}_downgraded.exe"
$version = & "./dist/${name}.exe" --version $version = & "./dist/${name}.exe" --version
& "./dist/${name}_downgraded.exe" -v --update-to yt-dlp/yt-dlp@2023.03.04 & "./dist/${name}_downgraded.exe" -v --update-to "${Env:UPDATE_TO}"
$downgraded_version = & "./dist/${name}_downgraded.exe" --version $downgraded_version = & "./dist/${name}_downgraded.exe" --version
if ($version -eq $downgraded_version) { if ($version -eq $downgraded_version) {
exit 1 exit 1
}
} }
- name: Upload artifacts - name: Upload artifacts
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v4
with: with:
name: build-bin-${{ github.job }} name: build-bin-${{ github.job }}-${{ matrix.arch }}
path: | path: |
dist/yt-dlp.exe dist/yt-dlp${{ env.SUFFIX }}.exe
dist/yt-dlp_min.exe dist/yt-dlp_win${{ env.SUFFIX }}.zip
dist/yt-dlp_win.zip
compression-level: 0
windows32:
needs: process
if: inputs.windows32
runs-on: windows-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.8"
architecture: "x86"
- name: Install Requirements
run: |
python devscripts/install_deps.py -o --include build
python devscripts/install_deps.py
python -m pip install -U "https://yt-dlp.github.io/Pyinstaller-Builds/i686/pyinstaller-5.8.0-py3-none-any.whl"
- name: Prepare
run: |
python devscripts/update-version.py -c "${{ inputs.channel }}" -r "${{ needs.process.outputs.origin }}" "${{ inputs.version }}"
python devscripts/make_lazy_extractors.py
- name: Build
run: |
python -m bundle.pyinstaller
- name: Verify --update-to
if: vars.UPDATE_TO_VERIFICATION
run: |
foreach ($name in @("yt-dlp_x86")) {
Copy-Item "./dist/${name}.exe" "./dist/${name}_downgraded.exe"
$version = & "./dist/${name}.exe" --version
& "./dist/${name}_downgraded.exe" -v --update-to yt-dlp/yt-dlp@2023.03.04
$downgraded_version = & "./dist/${name}_downgraded.exe" --version
if ($version -eq $downgraded_version) {
exit 1
}
}
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: build-bin-${{ github.job }}
path: |
dist/yt-dlp_x86.exe
compression-level: 0 compression-level: 0
meta_files: meta_files:
if: inputs.meta_files && always() && !cancelled() if: always() && !cancelled()
needs: needs:
- process - process
- unix - unix
- linux_arm - linux
- macos - macos
- macos_legacy
- windows - windows
- windows32
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/download-artifact@v4 - name: Download artifacts
uses: actions/download-artifact@v4
with: with:
path: artifact path: artifact
pattern: build-bin-* pattern: build-bin-*
@@ -447,8 +545,13 @@ jobs:
- name: Make SHA2-SUMS files - name: Make SHA2-SUMS files
run: | run: |
cd ./artifact/ cd ./artifact/
sha256sum * > ../SHA2-256SUMS # make sure SHA sums are also printed to stdout
sha512sum * > ../SHA2-512SUMS sha256sum -- * | tee ../SHA2-256SUMS
sha512sum -- * | tee ../SHA2-512SUMS
# also print as permanent annotations to the summary page
while read -r shasum; do
echo "::notice title=${shasum##* }::sha256: ${shasum% *}"
done < ../SHA2-256SUMS
- name: Make Update spec - name: Make Update spec
run: | run: |
@@ -457,19 +560,39 @@ jobs:
lock 2022.08.18.36 .+ Python 3\.6 lock 2022.08.18.36 .+ Python 3\.6
lock 2023.11.16 (?!win_x86_exe).+ Python 3\.7 lock 2023.11.16 (?!win_x86_exe).+ Python 3\.7
lock 2023.11.16 win_x86_exe .+ Windows-(?:Vista|2008Server) lock 2023.11.16 win_x86_exe .+ Windows-(?:Vista|2008Server)
lock 2024.10.22 py2exe .+
lock 2024.10.22 zip Python 3\.8
lock 2024.10.22 win(?:_x86)?_exe Python 3\.[78].+ Windows-(?:7-|2008ServerR2)
lock 2025.08.11 darwin_legacy_exe .+
lock 2025.08.27 linux_armv7l_exe .+
lockV2 yt-dlp/yt-dlp 2022.08.18.36 .+ Python 3\.6 lockV2 yt-dlp/yt-dlp 2022.08.18.36 .+ Python 3\.6
lockV2 yt-dlp/yt-dlp 2023.11.16 (?!win_x86_exe).+ Python 3\.7 lockV2 yt-dlp/yt-dlp 2023.11.16 (?!win_x86_exe).+ Python 3\.7
lockV2 yt-dlp/yt-dlp 2023.11.16 win_x86_exe .+ Windows-(?:Vista|2008Server) lockV2 yt-dlp/yt-dlp 2023.11.16 win_x86_exe .+ Windows-(?:Vista|2008Server)
lockV2 yt-dlp/yt-dlp 2024.10.22 py2exe .+
lockV2 yt-dlp/yt-dlp 2024.10.22 zip Python 3\.8
lockV2 yt-dlp/yt-dlp 2024.10.22 win(?:_x86)?_exe Python 3\.[78].+ Windows-(?:7-|2008ServerR2)
lockV2 yt-dlp/yt-dlp 2025.08.11 darwin_legacy_exe .+
lockV2 yt-dlp/yt-dlp 2025.08.27 linux_armv7l_exe .+
lockV2 yt-dlp/yt-dlp-nightly-builds 2023.11.15.232826 (?!win_x86_exe).+ Python 3\.7 lockV2 yt-dlp/yt-dlp-nightly-builds 2023.11.15.232826 (?!win_x86_exe).+ Python 3\.7
lockV2 yt-dlp/yt-dlp-nightly-builds 2023.11.15.232826 win_x86_exe .+ Windows-(?:Vista|2008Server) lockV2 yt-dlp/yt-dlp-nightly-builds 2023.11.15.232826 win_x86_exe .+ Windows-(?:Vista|2008Server)
lockV2 yt-dlp/yt-dlp-nightly-builds 2024.10.22.051025 py2exe .+
lockV2 yt-dlp/yt-dlp-nightly-builds 2024.10.22.051025 zip Python 3\.8
lockV2 yt-dlp/yt-dlp-nightly-builds 2024.10.22.051025 win(?:_x86)?_exe Python 3\.[78].+ Windows-(?:7-|2008ServerR2)
lockV2 yt-dlp/yt-dlp-nightly-builds 2025.08.12.233030 darwin_legacy_exe .+
lockV2 yt-dlp/yt-dlp-nightly-builds 2025.08.30.232839 linux_armv7l_exe .+
lockV2 yt-dlp/yt-dlp-master-builds 2023.11.15.232812 (?!win_x86_exe).+ Python 3\.7 lockV2 yt-dlp/yt-dlp-master-builds 2023.11.15.232812 (?!win_x86_exe).+ Python 3\.7
lockV2 yt-dlp/yt-dlp-master-builds 2023.11.15.232812 win_x86_exe .+ Windows-(?:Vista|2008Server) lockV2 yt-dlp/yt-dlp-master-builds 2023.11.15.232812 win_x86_exe .+ Windows-(?:Vista|2008Server)
lockV2 yt-dlp/yt-dlp-master-builds 2024.10.22.045052 py2exe .+
lockV2 yt-dlp/yt-dlp-master-builds 2024.10.22.060347 zip Python 3\.8
lockV2 yt-dlp/yt-dlp-master-builds 2024.10.22.060347 win(?:_x86)?_exe Python 3\.[78].+ Windows-(?:7-|2008ServerR2)
lockV2 yt-dlp/yt-dlp-master-builds 2025.08.12.232447 darwin_legacy_exe .+
lockV2 yt-dlp/yt-dlp-master-builds 2025.09.05.212910 linux_armv7l_exe .+
EOF EOF
- name: Sign checksum files - name: Sign checksum files
env: env:
GPG_SIGNING_KEY: ${{ secrets.GPG_SIGNING_KEY }} GPG_SIGNING_KEY: ${{ secrets.GPG_SIGNING_KEY }}
if: env.GPG_SIGNING_KEY != '' if: env.GPG_SIGNING_KEY
run: | run: |
gpg --batch --import <<< "${{ secrets.GPG_SIGNING_KEY }}" gpg --batch --import <<< "${{ secrets.GPG_SIGNING_KEY }}"
for signfile in ./SHA*SUMS; do for signfile in ./SHA*SUMS; do
+23
View File
@@ -0,0 +1,23 @@
name: Keep cache warm
on:
workflow_dispatch:
schedule:
- cron: '0 22 1,6,11,16,21,27 * *'
jobs:
build:
if: |
vars.KEEP_CACHE_WARM || github.event_name == 'workflow_dispatch'
uses: ./.github/workflows/build.yml
with:
version: '999999'
channel: stable
origin: ${{ github.repository }}
unix: false
linux: false
linux_armv7l: true
musllinux: false
macos: true
windows: true
permissions:
contents: read
+3 -3
View File
@@ -33,7 +33,7 @@ jobs:
# Initializes the CodeQL tools for scanning. # Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL - name: Initialize CodeQL
uses: github/codeql-action/init@v2 uses: github/codeql-action/init@v3
with: with:
languages: ${{ matrix.language }} languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file. # If you wish to specify custom queries, you can do so here or in a config file.
@@ -47,7 +47,7 @@ jobs:
# Autobuild attempts to build any compiled languages (C/C++, C#, Go, Java, or Swift). # Autobuild attempts to build any compiled languages (C/C++, C#, Go, Java, or Swift).
# If this step fails, then you should remove it and run the build manually (see below) # If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild - name: Autobuild
uses: github/codeql-action/autobuild@v2 uses: github/codeql-action/autobuild@v3
# ️ Command-line programs to run using the OS shell. # ️ Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun # 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
@@ -60,6 +60,6 @@ jobs:
# ./location_of_script_within_repo/buildscript.sh # ./location_of_script_within_repo/buildscript.sh
- name: Perform CodeQL Analysis - name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2 uses: github/codeql-action/analyze@v3
with: with:
category: "/language:${{matrix.language}}" category: "/language:${{matrix.language}}"
+18 -9
View File
@@ -6,7 +6,7 @@ on:
- devscripts/** - devscripts/**
- test/** - test/**
- yt_dlp/**.py - yt_dlp/**.py
- '!yt_dlp/extractor/*.py' - '!yt_dlp/extractor/**.py'
- yt_dlp/extractor/__init__.py - yt_dlp/extractor/__init__.py
- yt_dlp/extractor/common.py - yt_dlp/extractor/common.py
- yt_dlp/extractor/extractors.py - yt_dlp/extractor/extractors.py
@@ -16,7 +16,7 @@ on:
- devscripts/** - devscripts/**
- test/** - test/**
- yt_dlp/**.py - yt_dlp/**.py
- '!yt_dlp/extractor/*.py' - '!yt_dlp/extractor/**.py'
- yt_dlp/extractor/__init__.py - yt_dlp/extractor/__init__.py
- yt_dlp/extractor/common.py - yt_dlp/extractor/common.py
- yt_dlp/extractor/extractors.py - yt_dlp/extractor/extractors.py
@@ -36,26 +36,35 @@ jobs:
fail-fast: false fail-fast: false
matrix: matrix:
os: [ubuntu-latest] os: [ubuntu-latest]
# CPython 3.8 is in quick-test # CPython 3.9 is in quick-test
python-version: ['3.9', '3.10', '3.11', '3.12', pypy-3.8, pypy-3.10] python-version: ['3.10', '3.11', '3.12', '3.13', '3.14-dev', pypy-3.11]
include: include:
# atleast one of each CPython/PyPy tests must be in windows # atleast one of each CPython/PyPy tests must be in windows
- os: windows-latest - os: windows-latest
python-version: '3.8' python-version: '3.9'
- os: windows-latest
python-version: '3.10'
- os: windows-latest
python-version: '3.11'
- os: windows-latest - os: windows-latest
python-version: '3.12' python-version: '3.12'
- os: windows-latest - os: windows-latest
python-version: pypy-3.9 python-version: '3.13'
- os: windows-latest
python-version: '3.14-dev'
- os: windows-latest
python-version: pypy-3.11
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }} - name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5 uses: actions/setup-python@v6
with: with:
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
- name: Install test requirements - name: Install test requirements
run: python3 ./devscripts/install_deps.py --include dev run: python3 ./devscripts/install_deps.py --include test --include curl-cffi
- name: Run tests - name: Run tests
timeout-minutes: 15
continue-on-error: False continue-on-error: False
run: | run: |
python3 -m yt_dlp -v || true # Print debug head python3 -m yt_dlp -v || true # Print debug head
python3 ./devscripts/run_tests.py core python3 ./devscripts/run_tests.py --pytest-args '--reruns 2 --reruns-delay 3.0' core
+5 -5
View File
@@ -11,7 +11,7 @@ jobs:
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- name: Set up Python - name: Set up Python
uses: actions/setup-python@v5 uses: actions/setup-python@v6
with: with:
python-version: 3.9 python-version: 3.9
- name: Install test requirements - name: Install test requirements
@@ -28,17 +28,17 @@ jobs:
fail-fast: true fail-fast: true
matrix: matrix:
os: [ubuntu-latest] os: [ubuntu-latest]
python-version: ['3.10', '3.11', '3.12', pypy-3.8, pypy-3.10] python-version: ['3.10', '3.11', '3.12', '3.13', '3.14-dev', pypy-3.11]
include: include:
# atleast one of each CPython/PyPy tests must be in windows # atleast one of each CPython/PyPy tests must be in windows
- os: windows-latest - os: windows-latest
python-version: '3.8' python-version: '3.9'
- os: windows-latest - os: windows-latest
python-version: pypy-3.9 python-version: pypy-3.11
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }} - name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5 uses: actions/setup-python@v6
with: with:
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
- name: Install test requirements - name: Install test requirements
+21
View File
@@ -0,0 +1,21 @@
name: Issue Lockdown
on:
issues:
types: [opened]
permissions:
issues: write
jobs:
lockdown:
name: Issue Lockdown
if: vars.ISSUE_LOCKDOWN
runs-on: ubuntu-latest
steps:
- name: "Lock new issue"
env:
GH_TOKEN: ${{ github.token }}
ISSUE_NUMBER: ${{ github.event.issue.number }}
REPOSITORY: ${{ github.repository }}
run: |
gh issue lock "${ISSUE_NUMBER}" -R "${REPOSITORY}"
+19 -12
View File
@@ -10,26 +10,33 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- name: Set up Python 3.8 - name: Set up Python 3.9
uses: actions/setup-python@v5 uses: actions/setup-python@v6
with: with:
python-version: '3.8' python-version: '3.9'
- name: Install test requirements - name: Install test requirements
run: python3 ./devscripts/install_deps.py --include dev run: python3 ./devscripts/install_deps.py -o --include test
- name: Run tests - name: Run tests
timeout-minutes: 15
run: | run: |
python3 -m yt_dlp -v || true python3 -m yt_dlp -v || true
python3 ./devscripts/run_tests.py core python3 ./devscripts/run_tests.py --pytest-args '--reruns 2 --reruns-delay 3.0' core
flake8: check:
name: Linter name: Code check
if: "!contains(github.event.head_commit.message, 'ci skip all')" if: "!contains(github.event.head_commit.message, 'ci skip all')"
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- uses: actions/setup-python@v5 - uses: actions/setup-python@v6
- name: Install flake8 with:
run: python3 ./devscripts/install_deps.py -o --include dev python-version: '3.9'
- name: Install dev dependencies
run: python3 ./devscripts/install_deps.py -o --include static-analysis
- name: Make lazy extractors - name: Make lazy extractors
run: python3 ./devscripts/make_lazy_extractors.py run: python3 ./devscripts/make_lazy_extractors.py
- name: Run flake8 - name: Run ruff
run: flake8 . run: ruff check --output-format github .
- name: Run autopep8
run: autopep8 --diff .
- name: Check file mode
run: git ls-files --format="%(objectmode) %(path)" yt_dlp/ | ( ! grep -v "^100644" )
+23 -4
View File
@@ -6,10 +6,12 @@ on:
paths: paths:
- "yt_dlp/**.py" - "yt_dlp/**.py"
- "!yt_dlp/version.py" - "!yt_dlp/version.py"
- "bundle/*.py" - "bundle/**"
- "pyproject.toml" - "pyproject.toml"
- "Makefile" - "Makefile"
- ".github/workflows/build.yml" - ".github/workflows/build.yml"
- ".github/workflows/release.yml"
- ".github/workflows/release-master.yml"
concurrency: concurrency:
group: release-master group: release-master
permissions: permissions:
@@ -17,13 +19,30 @@ permissions:
jobs: jobs:
release: release:
if: vars.BUILD_MASTER != '' if: vars.BUILD_MASTER
uses: ./.github/workflows/release.yml uses: ./.github/workflows/release.yml
with: with:
prerelease: true prerelease: true
source: master source: ${{ (github.repository != 'yt-dlp/yt-dlp' && vars.MASTER_ARCHIVE_REPO) || 'master' }}
target: 'master'
permissions: permissions:
contents: write contents: write
packages: write
id-token: write # mandatory for trusted publishing id-token: write # mandatory for trusted publishing
secrets: inherit secrets: inherit
publish_pypi:
needs: [release]
if: vars.MASTER_PYPI_PROJECT
runs-on: ubuntu-latest
permissions:
id-token: write # mandatory for trusted publishing
steps:
- name: Download artifacts
uses: actions/download-artifact@v4
with:
path: dist
name: build-pypi
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
verbose: true
+24 -3
View File
@@ -7,7 +7,7 @@ permissions:
jobs: jobs:
check_nightly: check_nightly:
if: vars.BUILD_NIGHTLY != '' if: vars.BUILD_NIGHTLY
runs-on: ubuntu-latest runs-on: ubuntu-latest
outputs: outputs:
commit: ${{ steps.check_for_new_commits.outputs.commit }} commit: ${{ steps.check_for_new_commits.outputs.commit }}
@@ -22,9 +22,13 @@ jobs:
"yt_dlp/*.py" "yt_dlp/*.py"
':!yt_dlp/version.py' ':!yt_dlp/version.py'
"bundle/*.py" "bundle/*.py"
"bundle/docker/compose.yml"
"bundle/docker/linux/*"
"pyproject.toml" "pyproject.toml"
"Makefile" "Makefile"
".github/workflows/build.yml" ".github/workflows/build.yml"
".github/workflows/release.yml"
".github/workflows/release-nightly.yml"
) )
echo "commit=$(git log --format=%H -1 --since="24 hours ago" -- "${relevant_files[@]}")" | tee "$GITHUB_OUTPUT" echo "commit=$(git log --format=%H -1 --since="24 hours ago" -- "${relevant_files[@]}")" | tee "$GITHUB_OUTPUT"
@@ -34,9 +38,26 @@ jobs:
uses: ./.github/workflows/release.yml uses: ./.github/workflows/release.yml
with: with:
prerelease: true prerelease: true
source: nightly source: ${{ (github.repository != 'yt-dlp/yt-dlp' && vars.NIGHTLY_ARCHIVE_REPO) || 'nightly' }}
target: 'nightly'
permissions: permissions:
contents: write contents: write
packages: write
id-token: write # mandatory for trusted publishing id-token: write # mandatory for trusted publishing
secrets: inherit secrets: inherit
publish_pypi:
needs: [release]
if: vars.NIGHTLY_PYPI_PROJECT
runs-on: ubuntu-latest
permissions:
id-token: write # mandatory for trusted publishing
steps:
- name: Download artifacts
uses: actions/download-artifact@v4
with:
path: dist
name: build-pypi
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
verbose: true
+121 -183
View File
@@ -2,10 +2,6 @@ name: Release
on: on:
workflow_call: workflow_call:
inputs: inputs:
prerelease:
required: false
default: true
type: boolean
source: source:
required: false required: false
default: '' default: ''
@@ -18,6 +14,14 @@ on:
required: false required: false
default: '' default: ''
type: string type: string
linux_armv7l:
required: false
default: false
type: boolean
prerelease:
required: false
default: true
type: boolean
workflow_dispatch: workflow_dispatch:
inputs: inputs:
source: source:
@@ -43,6 +47,10 @@ on:
required: false required: false
default: '' default: ''
type: string type: string
linux_armv7l:
description: Include linux_armv7l
default: true
type: boolean
prerelease: prerelease:
description: Pre-release description: Pre-release
default: false default: false
@@ -71,146 +79,63 @@ jobs:
with: with:
fetch-depth: 0 fetch-depth: 0
- uses: actions/setup-python@v5 - uses: actions/setup-python@v6
with: with:
python-version: "3.10" python-version: "3.10" # Keep this in sync with test-workflows.yml
- name: Process inputs - name: Process inputs
id: process_inputs id: process_inputs
env:
INPUTS: ${{ toJSON(inputs) }}
run: | run: |
cat << EOF python -m devscripts.setup_variables process_inputs
::group::Inputs
prerelease=${{ inputs.prerelease }}
source=${{ inputs.source }}
target=${{ inputs.target }}
version=${{ inputs.version }}
::endgroup::
EOF
IFS='@' read -r source_repo source_tag <<<"${{ inputs.source }}"
IFS='@' read -r target_repo target_tag <<<"${{ inputs.target }}"
cat << EOF >> "$GITHUB_OUTPUT"
source_repo=${source_repo}
source_tag=${source_tag}
target_repo=${target_repo}
target_tag=${target_tag}
EOF
- name: Setup variables - name: Setup variables
id: setup_variables id: setup_variables
env: env:
source_repo: ${{ steps.process_inputs.outputs.source_repo }} INPUTS: ${{ toJSON(inputs) }}
source_tag: ${{ steps.process_inputs.outputs.source_tag }} PROCESSED: ${{ toJSON(steps.process_inputs.outputs) }}
target_repo: ${{ steps.process_inputs.outputs.target_repo }} REPOSITORY: ${{ github.repository }}
target_tag: ${{ steps.process_inputs.outputs.target_tag }} PUSH_VERSION_COMMIT: ${{ vars.PUSH_VERSION_COMMIT }}
PYPI_PROJECT: ${{ vars.PYPI_PROJECT }}
SOURCE_PYPI_PROJECT: ${{ vars[format('{0}_pypi_project', steps.process_inputs.outputs.source_repo)] }}
SOURCE_PYPI_SUFFIX: ${{ vars[format('{0}_pypi_suffix', steps.process_inputs.outputs.source_repo)] }}
TARGET_PYPI_PROJECT: ${{ vars[format('{0}_pypi_project', steps.process_inputs.outputs.target_repo)] }}
TARGET_PYPI_SUFFIX: ${{ vars[format('{0}_pypi_suffix', steps.process_inputs.outputs.target_repo)] }}
SOURCE_ARCHIVE_REPO: ${{ vars[format('{0}_archive_repo', steps.process_inputs.outputs.source_repo)] }}
TARGET_ARCHIVE_REPO: ${{ vars[format('{0}_archive_repo', steps.process_inputs.outputs.target_repo)] }}
HAS_SOURCE_ARCHIVE_REPO_TOKEN: ${{ !!secrets[format('{0}_archive_repo_token', steps.process_inputs.outputs.source_repo)] }}
HAS_TARGET_ARCHIVE_REPO_TOKEN: ${{ !!secrets[format('{0}_archive_repo_token', steps.process_inputs.outputs.target_repo)] }}
HAS_ARCHIVE_REPO_TOKEN: ${{ !!secrets.ARCHIVE_REPO_TOKEN }}
run: | run: |
# unholy bash monstrosity (sincere apologies) python -m devscripts.setup_variables
fallback_token () {
if ${{ !secrets.ARCHIVE_REPO_TOKEN }}; then
echo "::error::Repository access secret ${target_repo_token^^} not found"
exit 1
fi
target_repo_token=ARCHIVE_REPO_TOKEN
return 0
}
source_is_channel=0 - name: Update version & documentation
[[ "${source_repo}" == 'stable' ]] && source_repo='yt-dlp/yt-dlp'
if [[ -z "${source_repo}" ]]; then
source_repo='${{ github.repository }}'
elif [[ '${{ vars[format('{0}_archive_repo', env.source_repo)] }}' ]]; then
source_is_channel=1
source_channel='${{ vars[format('{0}_archive_repo', env.source_repo)] }}'
elif [[ -z "${source_tag}" && "${source_repo}" != */* ]]; then
source_tag="${source_repo}"
source_repo='${{ github.repository }}'
fi
resolved_source="${source_repo}"
if [[ "${source_tag}" ]]; then
resolved_source="${resolved_source}@${source_tag}"
elif [[ "${source_repo}" == 'yt-dlp/yt-dlp' ]]; then
resolved_source='stable'
fi
revision="${{ (inputs.prerelease || !vars.PUSH_VERSION_COMMIT) && '$(date -u +"%H%M%S")' || '' }}"
version="$(
python devscripts/update-version.py \
-c "${resolved_source}" -r "${{ github.repository }}" ${{ inputs.version || '$revision' }} | \
grep -Po "version=\K\d+\.\d+\.\d+(\.\d+)?")"
if [[ "${target_repo}" ]]; then
if [[ -z "${target_tag}" ]]; then
if [[ '${{ vars[format('{0}_archive_repo', env.target_repo)] }}' ]]; then
target_tag="${source_tag:-${version}}"
else
target_tag="${target_repo}"
target_repo='${{ github.repository }}'
fi
fi
if [[ "${target_repo}" != '${{ github.repository}}' ]]; then
target_repo='${{ vars[format('{0}_archive_repo', env.target_repo)] }}'
target_repo_token='${{ env.target_repo }}_archive_repo_token'
${{ !!secrets[format('{0}_archive_repo_token', env.target_repo)] }} || fallback_token
pypi_project='${{ vars[format('{0}_pypi_project', env.target_repo)] }}'
pypi_suffix='${{ vars[format('{0}_pypi_suffix', env.target_repo)] }}'
fi
else
target_tag="${source_tag:-${version}}"
if ((source_is_channel)); then
target_repo="${source_channel}"
target_repo_token='${{ env.source_repo }}_archive_repo_token'
${{ !!secrets[format('{0}_archive_repo_token', env.source_repo)] }} || fallback_token
pypi_project='${{ vars[format('{0}_pypi_project', env.source_repo)] }}'
pypi_suffix='${{ vars[format('{0}_pypi_suffix', env.source_repo)] }}'
else
target_repo='${{ github.repository }}'
fi
fi
if [[ "${target_repo}" == '${{ github.repository }}' ]] && ${{ !inputs.prerelease }}; then
pypi_project='${{ vars.PYPI_PROJECT }}'
fi
echo "::group::Output variables"
cat << EOF | tee -a "$GITHUB_OUTPUT"
channel=${resolved_source}
version=${version}
target_repo=${target_repo}
target_repo_token=${target_repo_token}
target_tag=${target_tag}
pypi_project=${pypi_project}
pypi_suffix=${pypi_suffix}
EOF
echo "::endgroup::"
- name: Update documentation
env: env:
version: ${{ steps.setup_variables.outputs.version }} CHANNEL: ${{ steps.setup_variables.outputs.channel }}
target_repo: ${{ steps.setup_variables.outputs.target_repo }} # Use base repo since this could be committed; build jobs will call this again with true origin
if: | REPOSITORY: ${{ github.repository }}
!inputs.prerelease && env.target_repo == github.repository VERSION: ${{ steps.setup_variables.outputs.version }}
run: | run: |
python devscripts/update-version.py -c "${CHANNEL}" -r "${REPOSITORY}" "${VERSION}"
python devscripts/update_changelog.py -vv
make doc make doc
sed '/### /Q' Changelog.md >> ./CHANGELOG
echo '### ${{ env.version }}' >> ./CHANGELOG
python ./devscripts/make_changelog.py -vv -c >> ./CHANGELOG
echo >> ./CHANGELOG
grep -Poz '(?s)### \d+\.\d+\.\d+.+' 'Changelog.md' | head -n -1 >> ./CHANGELOG
cat ./CHANGELOG > Changelog.md
- name: Push to release - name: Push to release
id: push_release id: push_release
env: env:
version: ${{ steps.setup_variables.outputs.version }} VERSION: ${{ steps.setup_variables.outputs.version }}
target_repo: ${{ steps.setup_variables.outputs.target_repo }} GITHUB_EVENT_SENDER_LOGIN: ${{ github.event.sender.login }}
GITHUB_EVENT_REF: ${{ github.event.ref }}
if: | if: |
!inputs.prerelease && env.target_repo == github.repository !inputs.prerelease && steps.setup_variables.outputs.target_repo == github.repository
run: | run: |
git config --global user.name "github-actions[bot]" git config --global user.name "github-actions[bot]"
git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add -u git add -u
git commit -m "Release ${{ env.version }}" \ git commit -m "Release ${VERSION}" \
-m "Created by: ${{ github.event.sender.login }}" -m ":ci skip all :ci run dl" -m "Created by: ${GITHUB_EVENT_SENDER_LOGIN}" -m ":ci skip all"
git push origin --force ${{ github.event.ref }}:release git push origin --force "${GITHUB_EVENT_REF}:release"
- name: Get target commitish - name: Get target commitish
id: get_target id: get_target
@@ -219,10 +144,10 @@ jobs:
- name: Update master - name: Update master
env: env:
target_repo: ${{ steps.setup_variables.outputs.target_repo }} GITHUB_EVENT_REF: ${{ github.event.ref }}
if: | if: |
vars.PUSH_VERSION_COMMIT != '' && !inputs.prerelease && env.target_repo == github.repository vars.PUSH_VERSION_COMMIT && !inputs.prerelease && steps.setup_variables.outputs.target_repo == github.repository
run: git push origin ${{ github.event.ref }} run: git push origin "${GITHUB_EVENT_REF}"
build: build:
needs: prepare needs: prepare
@@ -231,9 +156,9 @@ jobs:
version: ${{ needs.prepare.outputs.version }} version: ${{ needs.prepare.outputs.version }}
channel: ${{ needs.prepare.outputs.channel }} channel: ${{ needs.prepare.outputs.channel }}
origin: ${{ needs.prepare.outputs.target_repo }} origin: ${{ needs.prepare.outputs.target_repo }}
linux_armv7l: ${{ inputs.linux_armv7l }}
permissions: permissions:
contents: read contents: read
packages: write # For package cache
secrets: secrets:
GPG_SIGNING_KEY: ${{ secrets.GPG_SIGNING_KEY }} GPG_SIGNING_KEY: ${{ secrets.GPG_SIGNING_KEY }}
@@ -248,7 +173,7 @@ jobs:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
with: with:
fetch-depth: 0 fetch-depth: 0
- uses: actions/setup-python@v5 - uses: actions/setup-python@v6
with: with:
python-version: "3.10" python-version: "3.10"
@@ -259,15 +184,16 @@ jobs:
- name: Prepare - name: Prepare
env: env:
version: ${{ needs.prepare.outputs.version }} VERSION: ${{ needs.prepare.outputs.version }}
suffix: ${{ needs.prepare.outputs.pypi_suffix }} SUFFIX: ${{ needs.prepare.outputs.pypi_suffix }}
channel: ${{ needs.prepare.outputs.channel }} CHANNEL: ${{ needs.prepare.outputs.channel }}
target_repo: ${{ needs.prepare.outputs.target_repo }} TARGET_REPO: ${{ needs.prepare.outputs.target_repo }}
pypi_project: ${{ needs.prepare.outputs.pypi_project }} PYPI_PROJECT: ${{ needs.prepare.outputs.pypi_project }}
run: | run: |
python devscripts/update-version.py -c "${{ env.channel }}" -r "${{ env.target_repo }}" -s "${{ env.suffix }}" "${{ env.version }}" python devscripts/update-version.py -c "${CHANNEL}" -r "${TARGET_REPO}" -s "${SUFFIX}" "${VERSION}"
python devscripts/update_changelog.py -vv
python devscripts/make_lazy_extractors.py python devscripts/make_lazy_extractors.py
sed -i -E '0,/(name = ")[^"]+(")/s//\1${{ env.pypi_project }}\2/' pyproject.toml sed -i -E '0,/(name = ")[^"]+(")/s//\1'"${PYPI_PROJECT}"'\2/' pyproject.toml
- name: Build - name: Build
run: | run: |
@@ -281,7 +207,17 @@ jobs:
make clean-cache make clean-cache
python -m build --no-isolation . python -m build --no-isolation .
- name: Upload artifacts
if: github.event_name != 'workflow_dispatch'
uses: actions/upload-artifact@v4
with:
name: build-pypi
path: |
dist/*
compression-level: 0
- name: Publish to PyPI - name: Publish to PyPI
if: github.event_name == 'workflow_dispatch'
uses: pypa/gh-action-pypi-publish@release/v1 uses: pypa/gh-action-pypi-publish@release/v1
with: with:
verbose: true verbose: true
@@ -291,7 +227,11 @@ jobs:
permissions: permissions:
contents: write contents: write
runs-on: ubuntu-latest runs-on: ubuntu-latest
env:
TARGET_REPO: ${{ needs.prepare.outputs.target_repo }}
TARGET_TAG: ${{ needs.prepare.outputs.target_tag }}
VERSION: ${{ needs.prepare.outputs.version }}
HEAD_SHA: ${{ needs.prepare.outputs.head_sha }}
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
with: with:
@@ -301,87 +241,85 @@ jobs:
path: artifact path: artifact
pattern: build-* pattern: build-*
merge-multiple: true merge-multiple: true
- uses: actions/setup-python@v5 - uses: actions/setup-python@v6
with: with:
python-version: "3.10" python-version: "3.10"
- name: Generate release notes - name: Generate release notes
env: env:
head_sha: ${{ needs.prepare.outputs.head_sha }} REPOSITORY: ${{ github.repository }}
target_repo: ${{ needs.prepare.outputs.target_repo }} BASE_REPO: yt-dlp/yt-dlp
target_tag: ${{ needs.prepare.outputs.target_tag }} NIGHTLY_REPO: yt-dlp/yt-dlp-nightly-builds
MASTER_REPO: yt-dlp/yt-dlp-master-builds
DOCS_PATH: ${{ env.TARGET_REPO == github.repository && format('/tree/{0}', env.TARGET_TAG) || '' }}
run: | run: |
printf '%s' \ printf '%s' \
'[![Installation](https://img.shields.io/badge/-Which%20file%20to%20download%3F-white.svg?style=for-the-badge)]' \ "[![Installation](https://img.shields.io/badge/-Which%20file%20to%20download%3F-white.svg?style=for-the-badge)]" \
'(https://github.com/${{ github.repository }}#installation "Installation instructions") ' \ "(https://github.com/${REPOSITORY}#installation \"Installation instructions\") " \
'[![Discord](https://img.shields.io/discord/807245652072857610?color=blue&labelColor=555555&label=&logo=discord&style=for-the-badge)]' \ "[![Discord](https://img.shields.io/discord/807245652072857610?color=blue&labelColor=555555&label=&logo=discord&style=for-the-badge)]" \
'(https://discord.gg/H5MNcFW63r "Discord") ' \ "(https://discord.gg/H5MNcFW63r \"Discord\") " \
'[![Donate](https://img.shields.io/badge/_-Donate-red.svg?logo=githubsponsors&labelColor=555555&style=for-the-badge)]' \ "[![Donate](https://img.shields.io/badge/_-Donate-red.svg?logo=githubsponsors&labelColor=555555&style=for-the-badge)]" \
'(https://github.com/yt-dlp/yt-dlp/blob/master/Collaborators.md#collaborators "Donate") ' \ "(https://github.com/${BASE_REPO}/blob/master/Collaborators.md#collaborators \"Donate\") " \
'[![Documentation](https://img.shields.io/badge/-Docs-brightgreen.svg?style=for-the-badge&logo=GitBook&labelColor=555555)]' \ "[![Documentation](https://img.shields.io/badge/-Docs-brightgreen.svg?style=for-the-badge&logo=GitBook&labelColor=555555)]" \
'(https://github.com/${{ github.repository }}' \ "(https://github.com/${REPOSITORY}${DOCS_PATH}#readme \"Documentation\") " > ./RELEASE_NOTES
'${{ env.target_repo == github.repository && format('/tree/{0}', env.target_tag) || '' }}#readme "Documentation") ' \ if [[ "${TARGET_REPO}" == "${BASE_REPO}" ]]; then
${{ env.target_repo == 'yt-dlp/yt-dlp' && '\ printf '%s' \
"[![Nightly](https://img.shields.io/badge/Nightly%20builds-purple.svg?style=for-the-badge)]" \ "[![Nightly](https://img.shields.io/badge/Nightly%20builds-purple.svg?style=for-the-badge)]" \
"(https://github.com/yt-dlp/yt-dlp-nightly-builds/releases/latest \"Nightly builds\") " \ "(https://github.com/${NIGHTLY_REPO}/releases/latest \"Nightly builds\") " \
"[![Master](https://img.shields.io/badge/Master%20builds-lightblue.svg?style=for-the-badge)]" \ "[![Master](https://img.shields.io/badge/Master%20builds-lightblue.svg?style=for-the-badge)]" \
"(https://github.com/yt-dlp/yt-dlp-master-builds/releases/latest \"Master builds\")"' || '' }} > ./RELEASE_NOTES "(https://github.com/${MASTER_REPO}/releases/latest \"Master builds\")" >> ./RELEASE_NOTES
printf '\n\n' >> ./RELEASE_NOTES fi
cat >> ./RELEASE_NOTES << EOF printf '\n\n%s\n\n%s%s\n\n---\n' \
#### A description of the various files are in the [README](https://github.com/${{ github.repository }}#release-files) "#### A description of the various files is in the [README](https://github.com/${REPOSITORY}#release-files)" \
--- "The PyInstaller-bundled executables are subject to the licenses described in " \
$(python ./devscripts/make_changelog.py -vv --collapsible) "[THIRD_PARTY_LICENSES.txt](https://github.com/${BASE_REPO}/blob/${HEAD_SHA}/THIRD_PARTY_LICENSES.txt)" >> ./RELEASE_NOTES
EOF python ./devscripts/make_changelog.py -vv --collapsible >> ./RELEASE_NOTES
printf '%s\n\n' '**This is a pre-release build**' >> ./PRERELEASE_NOTES printf '%s\n\n' '**This is a pre-release build**' >> ./PRERELEASE_NOTES
cat ./RELEASE_NOTES >> ./PRERELEASE_NOTES cat ./RELEASE_NOTES >> ./PRERELEASE_NOTES
printf '%s\n\n' 'Generated from: https://github.com/${{ github.repository }}/commit/${{ env.head_sha }}' >> ./ARCHIVE_NOTES printf '%s\n\n' "Generated from: https://github.com/${REPOSITORY}/commit/${HEAD_SHA}" >> ./ARCHIVE_NOTES
cat ./RELEASE_NOTES >> ./ARCHIVE_NOTES cat ./RELEASE_NOTES >> ./ARCHIVE_NOTES
- name: Publish to archive repo - name: Publish to archive repo
env: env:
GH_TOKEN: ${{ secrets[needs.prepare.outputs.target_repo_token] }} GH_TOKEN: ${{ secrets[needs.prepare.outputs.target_repo_token] }}
GH_REPO: ${{ needs.prepare.outputs.target_repo }} GH_REPO: ${{ needs.prepare.outputs.target_repo }}
version: ${{ needs.prepare.outputs.version }} TITLE_PREFIX: ${{ startswith(env.TARGET_REPO, 'yt-dlp/') && 'yt-dlp ' || '' }}
channel: ${{ needs.prepare.outputs.channel }} TITLE: ${{ inputs.target != env.TARGET_REPO && inputs.target || needs.prepare.outputs.channel }}
if: | if: |
inputs.prerelease && env.GH_TOKEN != '' && env.GH_REPO != '' && env.GH_REPO != github.repository inputs.prerelease && env.GH_TOKEN && env.GH_REPO && env.GH_REPO != github.repository
run: | run: |
title="${{ startswith(env.GH_REPO, 'yt-dlp/') && 'yt-dlp ' || '' }}${{ env.channel }}"
gh release create \ gh release create \
--notes-file ARCHIVE_NOTES \ --notes-file ARCHIVE_NOTES \
--title "${title} ${{ env.version }}" \ --title "${TITLE_PREFIX}${TITLE} ${VERSION}" \
${{ env.version }} \ "${VERSION}" \
artifact/* artifact/*
- name: Prune old release - name: Prune old release
env: env:
GH_TOKEN: ${{ github.token }} GH_TOKEN: ${{ github.token }}
version: ${{ needs.prepare.outputs.version }}
target_repo: ${{ needs.prepare.outputs.target_repo }}
target_tag: ${{ needs.prepare.outputs.target_tag }}
if: | if: |
env.target_repo == github.repository && env.target_tag != env.version env.TARGET_REPO == github.repository && env.TARGET_TAG != env.VERSION
run: | run: |
gh release delete --yes --cleanup-tag "${{ env.target_tag }}" || true gh release delete --yes --cleanup-tag "${TARGET_TAG}" || true
git tag --delete "${{ env.target_tag }}" || true git tag --delete "${TARGET_TAG}" || true
sleep 5 # Enough time to cover deletion race condition sleep 5 # Enough time to cover deletion race condition
- name: Publish release - name: Publish release
env: env:
GH_TOKEN: ${{ github.token }} GH_TOKEN: ${{ github.token }}
version: ${{ needs.prepare.outputs.version }} NOTES_FILE: ${{ inputs.prerelease && 'PRERELEASE_NOTES' || 'RELEASE_NOTES' }}
target_repo: ${{ needs.prepare.outputs.target_repo }} TITLE_PREFIX: ${{ github.repository == 'yt-dlp/yt-dlp' && 'yt-dlp ' || '' }}
target_tag: ${{ needs.prepare.outputs.target_tag }} TITLE: ${{ env.TARGET_TAG != env.VERSION && format('{0} ', env.TARGET_TAG) || '' }}
head_sha: ${{ needs.prepare.outputs.head_sha }} PRERELEASE: ${{ inputs.prerelease && '1' || '0' }}
if: | if: |
env.target_repo == github.repository env.TARGET_REPO == github.repository
run: | run: |
title="${{ github.repository == 'yt-dlp/yt-dlp' && 'yt-dlp ' || '' }}" gh_options=(
title+="${{ env.target_tag != env.version && format('{0} ', env.target_tag) || '' }}" --notes-file "${NOTES_FILE}"
gh release create \ --target "${HEAD_SHA}"
--notes-file ${{ inputs.prerelease && 'PRERELEASE_NOTES' || 'RELEASE_NOTES' }} \ --title "${TITLE_PREFIX}${TITLE}${VERSION}"
--target ${{ env.head_sha }} \ )
--title "${title}${{ env.version }}" \ if ((PRERELEASE)); then
${{ inputs.prerelease && '--prerelease' || '' }} \ gh_options+=(--prerelease)
${{ env.target_tag }} \ fi
artifact/* gh release create "${gh_options[@]}" "${TARGET_TAG}" artifact/*
+17
View File
@@ -0,0 +1,17 @@
name: Sanitize comment
on:
issue_comment:
types: [created, edited]
permissions:
issues: write
jobs:
sanitize-comment:
name: Sanitize comment
if: vars.SANITIZE_COMMENT && !github.event.issue.pull_request
runs-on: ubuntu-latest
steps:
- name: Sanitize comment
uses: yt-dlp/sanitize-comment@v1
+41
View File
@@ -0,0 +1,41 @@
name: Signature Tests
on:
push:
paths:
- .github/workflows/signature-tests.yml
- test/test_youtube_signature.py
- yt_dlp/jsinterp.py
pull_request:
paths:
- .github/workflows/signature-tests.yml
- test/test_youtube_signature.py
- yt_dlp/jsinterp.py
permissions:
contents: read
concurrency:
group: signature-tests-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
tests:
name: Signature Tests
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest]
python-version: ['3.9', '3.10', '3.11', '3.12', '3.13', '3.14-dev', pypy-3.11]
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}
- name: Install test requirements
run: python3 ./devscripts/install_deps.py --only-optional --include test
- name: Run tests
timeout-minutes: 15
run: |
python3 -m yt_dlp -v || true # Print debug head
python3 ./devscripts/run_tests.py test/test_youtube_signature.py
+52
View File
@@ -0,0 +1,52 @@
name: Test and lint workflows
on:
push:
paths:
- .github/workflows/*
- bundle/docker/linux/*.sh
- devscripts/setup_variables.py
- devscripts/setup_variables_tests.py
- devscripts/utils.py
pull_request:
paths:
- .github/workflows/*
- bundle/docker/linux/*.sh
- devscripts/setup_variables.py
- devscripts/setup_variables_tests.py
- devscripts/utils.py
permissions:
contents: read
env:
ACTIONLINT_VERSION: "1.7.7"
ACTIONLINT_SHA256SUM: 023070a287cd8cccd71515fedc843f1985bf96c436b7effaecce67290e7e0757
ACTIONLINT_REPO: https://github.com/rhysd/actionlint
jobs:
check:
name: Check workflows
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v6
with:
python-version: "3.10" # Keep this in sync with release.yml's prepare job
- name: Install requirements
env:
ACTIONLINT_TARBALL: ${{ format('actionlint_{0}_linux_amd64.tar.gz', env.ACTIONLINT_VERSION) }}
run: |
python -m devscripts.install_deps -o --include test
sudo apt -y install shellcheck
python -m pip install -U pyflakes
curl -LO "${ACTIONLINT_REPO}/releases/download/v${ACTIONLINT_VERSION}/${ACTIONLINT_TARBALL}"
printf '%s %s' "${ACTIONLINT_SHA256SUM}" "${ACTIONLINT_TARBALL}" | sha256sum -c -
tar xvzf "${ACTIONLINT_TARBALL}" actionlint
chmod +x actionlint
- name: Run actionlint
run: |
./actionlint -color
- name: Check Docker shell scripts
run: |
shellcheck bundle/docker/linux/*.sh
- name: Test GHA devscripts
run: |
pytest -Werror --tb=short --color=yes devscripts/setup_variables_tests.py
+5 -2
View File
@@ -51,7 +51,6 @@ cookies
*.srt *.srt
*.ssa *.ssa
*.swf *.swf
*.swp
*.tt *.tt
*.ttml *.ttml
*.url *.url
@@ -67,7 +66,7 @@ cookies
# Python # Python
*.pyc *.pyc
*.pyo *.pyo
.pytest_cache .*_cache
wine-py2exe/ wine-py2exe/
py2exe.log py2exe.log
build/ build/
@@ -93,6 +92,7 @@ updates_key.pem
*.class *.class
*.isorted *.isorted
*.stackdump *.stackdump
uv.lock
# Generated # Generated
AUTHORS AUTHORS
@@ -105,6 +105,8 @@ README.txt
*.zsh *.zsh
*.spec *.spec
test/testdata/sigs/player-*.js test/testdata/sigs/player-*.js
test/testdata/thumbnails/empty.webp
test/testdata/thumbnails/foo\ %d\ bar/foo_%d.*
# Binary # Binary
/youtube-dl /youtube-dl
@@ -119,6 +121,7 @@ yt-dlp.zip
.vscode .vscode
*.sublime-* *.sublime-*
*.code-workspace *.code-workspace
*.swp
# Lazy extractors # Lazy extractors
*/extractor/lazy_extractors.py */extractor/lazy_extractors.py
+14
View File
@@ -0,0 +1,14 @@
repos:
- repo: local
hooks:
- id: linter
name: Apply linter fixes
entry: ruff check --fix .
language: system
types: [python]
require_serial: true
- id: format
name: Apply formatting fixes
entry: autopep8 --in-place .
language: system
types: [python]
+9
View File
@@ -0,0 +1,9 @@
repos:
- repo: local
hooks:
- id: fix
name: Apply code fixes
entry: hatch fmt
language: system
types: [python]
require_serial: true
+95 -37
View File
@@ -12,6 +12,7 @@
- [Is your question about yt-dlp?](#is-your-question-about-yt-dlp) - [Is your question about yt-dlp?](#is-your-question-about-yt-dlp)
- [Are you willing to share account details if needed?](#are-you-willing-to-share-account-details-if-needed) - [Are you willing to share account details if needed?](#are-you-willing-to-share-account-details-if-needed)
- [Is the website primarily used for piracy](#is-the-website-primarily-used-for-piracy) - [Is the website primarily used for piracy](#is-the-website-primarily-used-for-piracy)
- [AUTOMATED CONTRIBUTIONS (AI / LLM) POLICY](#automated-contributions-ai--llm-policy)
- [DEVELOPER INSTRUCTIONS](#developer-instructions) - [DEVELOPER INSTRUCTIONS](#developer-instructions)
- [Adding new feature or making overarching changes](#adding-new-feature-or-making-overarching-changes) - [Adding new feature or making overarching changes](#adding-new-feature-or-making-overarching-changes)
- [Adding support for a new site](#adding-support-for-a-new-site) - [Adding support for a new site](#adding-support-for-a-new-site)
@@ -37,14 +38,18 @@ Bugs and suggestions should be reported at: [yt-dlp/yt-dlp/issues](https://githu
**Please include the full output of yt-dlp when run with `-vU`**, i.e. **add** `-vU` flag to **your command line**, copy the **whole** output and post it in the issue body wrapped in \`\`\` for better formatting. It should look similar to this: **Please include the full output of yt-dlp when run with `-vU`**, i.e. **add** `-vU` flag to **your command line**, copy the **whole** output and post it in the issue body wrapped in \`\`\` for better formatting. It should look similar to this:
``` ```
$ yt-dlp -vU <your command line> $ yt-dlp -vU <your command line>
[debug] Command-line config: ['-v', 'demo.com'] [debug] Command-line config: ['-vU', 'https://www.example.com/']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8 [debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2021.09.25 (zip) [debug] yt-dlp version nightly@... from yt-dlp/yt-dlp-nightly-builds [1a176d874] (win_exe)
[debug] Python version 3.8.10 (CPython 64bit) - Linux-5.4.0-74-generic-x86_64-with-glibc2.29 [debug] Python 3.10.11 (CPython AMD64 64bit) - Windows-10-10.0.20348-SP0 (OpenSSL 1.1.1t 7 Feb 2023)
[debug] exe versions: ffmpeg 4.2.4, ffprobe 4.2.4 [debug] exe versions: ffmpeg 7.0.2 (setts), ffprobe 7.0.2
[debug] Optional libraries: Cryptodome-3.21.0, brotli-1.1.0, certifi-2024.08.30, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.40.1, urllib3-2.2.3, websockets-13.1
[debug] Proxy map: {} [debug] Proxy map: {}
Current Build Hash 25cc412d1d3c0725a1f2f5b7e4682f6fb40e6d15f7024e96f7afd572e9919535 [debug] Request Handlers: urllib, requests, websockets, curl_cffi
yt-dlp is up to date (2021.09.25) [debug] Loaded 1838 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: nightly@... from yt-dlp/yt-dlp-nightly-builds
yt-dlp is up to date (nightly@... from yt-dlp/yt-dlp-nightly-builds)
... ...
``` ```
**Do not post screenshots of verbose logs; only plain text is acceptable.** **Do not post screenshots of verbose logs; only plain text is acceptable.**
@@ -122,30 +127,76 @@ By sharing an account with anyone, you agree to bear all risks associated with i
While these steps won't necessarily ensure that no misuse of the account takes place, these are still some good practices to follow. While these steps won't necessarily ensure that no misuse of the account takes place, these are still some good practices to follow.
- Look for people with `Member` (maintainers of the project) or `Contributor` (people who have previously contributed code) tag on their messages. - Look for people with `Member` (maintainers of the project) or `Contributor` (people who have previously contributed code) tag on their messages.
- Change the password before sharing the account to something random (use [this](https://passwordsgenerator.net/) if you don't have a random password generator). - Change the password before sharing the account to something random.
- Change the password after receiving the account back. - Change the password after receiving the account back.
### Is the website primarily used for piracy? ### Is the website primarily used for piracy?
We follow [youtube-dl's policy](https://github.com/ytdl-org/youtube-dl#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free) to not support services that is primarily used for infringing copyright. Additionally, it has been decided to not to support porn sites that specialize in fakes. We also cannot support any service that serves only [DRM protected content](https://en.wikipedia.org/wiki/Digital_rights_management). We follow [youtube-dl's policy](https://github.com/ytdl-org/youtube-dl#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free) to not support services that is primarily used for infringing copyright. Additionally, it has been decided to not to support porn sites that specialize in fakes. We also cannot support any service that serves only [DRM protected content](https://en.wikipedia.org/wiki/Digital_rights_management).
# AUTOMATED CONTRIBUTIONS (AI / LLM) POLICY
Please refrain from submitting issues or pull requests that have been generated by an LLM or other fully-automated tools. Any submission that is in violation of this policy will be closed, and the submitter may be blocked from this repository without warning.
If you submit an issue, you need to understand what your issue description is saying. You need to be able to answer questions about your bug report or feature request. Using an AI tool to *proofread* your issue/comment text is acceptable. Using an AI tool to *write* your issue/comment text is unacceptable.
If you submit a pull request, you need to understand what every line of code you've changed does. If you can't explain why your PR is doing something, then do not submit it. Using an AI tool to generate entire lines of code is unacceptable.
The rationale behind this policy is that automated contributions are a waste of the maintainers' time. Humans spend their time and brainpower reviewing every submission. Issues or pull requests generated by automation tools create an imbalance of effort between the submitter and the reviewer. Nobody learns anything when a maintainer reviews code written by an LLM.
Additionally, AI-generated code conflicts with this project's license (Unlicense), since you cannot truly release code into the public domain if you didn't author it yourself.
# DEVELOPER INSTRUCTIONS # DEVELOPER INSTRUCTIONS
Most users do not need to build yt-dlp and can [download the builds](https://github.com/yt-dlp/yt-dlp/releases) or get them via [the other installation methods](README.md#installation). Most users do not need to build yt-dlp and can [download the builds](https://github.com/yt-dlp/yt-dlp/releases), get them via [the other installation methods](README.md#installation) or directly run it using `python -m yt_dlp`.
To run yt-dlp as a developer, you don't need to build anything either. Simply execute `yt-dlp` uses [`hatch`](<https://hatch.pypa.io>) as a project management tool.
You can easily install it using [`pipx`](<https://pipx.pypa.io>) via `pipx install hatch`, or else via `pip` or your package manager of choice. Make sure you are using at least version `1.10.0`, otherwise some functionality might not work as expected.
python3 -m yt_dlp If you plan on contributing to `yt-dlp`, best practice is to start by running the following command:
To run all the available core tests, use: ```shell
$ hatch run setup
```
python3 devscripts/run_tests.py The above command will install a `pre-commit` hook so that required checks/fixes (linting, formatting) will run automatically before each commit. If any code needs to be linted or formatted, then the commit will be blocked and the necessary changes will be made; you should review all edits and re-commit the fixed version.
After this you can use `hatch shell` to enable a virtual environment that has `yt-dlp` and its development dependencies installed.
In addition, the following script commands can be used to run simple tasks such as linting or testing (without having to run `hatch shell` first):
* `hatch fmt`: Automatically fix linter violations and apply required code formatting changes
* See `hatch fmt --help` for more info
* `hatch test`: Run extractor or core tests
* See `hatch test --help` for more info
See item 6 of [new extractor tutorial](#adding-support-for-a-new-site) for how to run extractor specific test cases. See item 6 of [new extractor tutorial](#adding-support-for-a-new-site) for how to run extractor specific test cases.
While it is strongly recommended to use `hatch` for yt-dlp development, if you are unable to do so, alternatively you can manually create a virtual environment and use the following commands:
```shell
# To only install development dependencies:
$ python -m devscripts.install_deps --include dev
# Or, for an editable install plus dev dependencies:
$ python -m pip install -e ".[default,dev]"
# To setup the pre-commit hook:
$ pre-commit install
# To be used in place of `hatch test`:
$ python -m devscripts.run_tests
# To be used in place of `hatch fmt`:
$ ruff check --fix .
$ autopep8 --in-place .
# To only check code instead of applying fixes:
$ ruff check .
$ autopep8 --diff .
```
If you want to create a build of yt-dlp yourself, you can follow the instructions [here](README.md#compile). If you want to create a build of yt-dlp yourself, you can follow the instructions [here](README.md#compile).
@@ -165,19 +216,23 @@ After you have ensured this site is distributing its content legally, you can fo
1. [Fork this repository](https://github.com/yt-dlp/yt-dlp/fork) 1. [Fork this repository](https://github.com/yt-dlp/yt-dlp/fork)
1. Check out the source code with: 1. Check out the source code with:
git clone git@github.com:YOUR_GITHUB_USERNAME/yt-dlp.git ```shell
$ git clone git@github.com:YOUR_GITHUB_USERNAME/yt-dlp.git
```
1. Start a new git branch with 1. Start a new git branch with
cd yt-dlp ```shell
git checkout -b yourextractor $ cd yt-dlp
$ git checkout -b yourextractor
```
1. Start with this simple template and save it to `yt_dlp/extractor/yourextractor.py`: 1. Start with this simple template and save it to `yt_dlp/extractor/yourextractor.py`:
```python ```python
from .common import InfoExtractor from .common import InfoExtractor
class YourExtractorIE(InfoExtractor): class YourExtractorIE(InfoExtractor):
_VALID_URL = r'https?://(?:www\.)?yourextractor\.com/watch/(?P<id>[0-9]+)' _VALID_URL = r'https?://(?:www\.)?yourextractor\.com/watch/(?P<id>[0-9]+)'
_TESTS = [{ _TESTS = [{
@@ -194,7 +249,7 @@ After you have ensured this site is distributing its content legally, you can fo
# * MD5 checksum; start the string with 'md5:', e.g. # * MD5 checksum; start the string with 'md5:', e.g.
# 'description': 'md5:098f6bcd4621d373cade4e832627b4f6', # 'description': 'md5:098f6bcd4621d373cade4e832627b4f6',
# * A regular expression; start the string with 're:', e.g. # * A regular expression; start the string with 're:', e.g.
# 'thumbnail': r're:^https?://.*\.jpg$', # 'thumbnail': r're:https?://.*\.jpg$',
# * A count of elements in a list; start the string with 'count:', e.g. # * A count of elements in a list; start the string with 'count:', e.g.
# 'tags': 'count:10', # 'tags': 'count:10',
# * Any Python type, e.g. # * Any Python type, e.g.
@@ -205,7 +260,7 @@ After you have ensured this site is distributing its content legally, you can fo
def _real_extract(self, url): def _real_extract(self, url):
video_id = self._match_id(url) video_id = self._match_id(url)
webpage = self._download_webpage(url, video_id) webpage = self._download_webpage(url, video_id)
# TODO more code goes here, for example ... # TODO more code goes here, for example ...
title = self._html_search_regex(r'<h1>(.+?)</h1>', webpage, 'title') title = self._html_search_regex(r'<h1>(.+?)</h1>', webpage, 'title')
@@ -217,21 +272,27 @@ After you have ensured this site is distributing its content legally, you can fo
# TODO more properties (see yt_dlp/extractor/common.py) # TODO more properties (see yt_dlp/extractor/common.py)
} }
``` ```
1. Add an import in [`yt_dlp/extractor/_extractors.py`](yt_dlp/extractor/_extractors.py). Note that the class name must end with `IE`. 1. Add an import in [`yt_dlp/extractor/_extractors.py`](yt_dlp/extractor/_extractors.py). Note that the class name must end with `IE`. Also note that when adding a parenthesized import group, the last import in the group must have a trailing comma in order for this formatting to be respected by our code formatter.
1. Run `python3 devscripts/run_tests.py YourExtractor`. This *may fail* at first, but you can continually re-run it until you're done. Upon failure, it will output the missing fields and/or correct values which you can copy. If you decide to add more than one test, the tests will then be named `YourExtractor`, `YourExtractor_1`, `YourExtractor_2`, etc. Note that tests with an `only_matching` key in the test's dict are not included in the count. You can also run all the tests in one go with `YourExtractor_all` 1. Run `hatch test YourExtractor`. This *may fail* at first, but you can continually re-run it until you're done. Upon failure, it will output the missing fields and/or correct values which you can copy. If you decide to add more than one test, the tests will then be named `YourExtractor`, `YourExtractor_1`, `YourExtractor_2`, etc. Note that tests with an `only_matching` key in the test's dict are not included in the count. You can also run all the tests in one go with `YourExtractor_all`
1. Make sure you have at least one test for your extractor. Even if all videos covered by the extractor are expected to be inaccessible for automated testing, tests should still be added with a `skip` parameter indicating why the particular test is disabled from running. 1. Make sure you have at least one test for your extractor. Even if all videos covered by the extractor are expected to be inaccessible for automated testing, tests should still be added with a `skip` parameter indicating why the particular test is disabled from running.
1. Have a look at [`yt_dlp/extractor/common.py`](yt_dlp/extractor/common.py) for possible helper methods and a [detailed description of what your extractor should and may return](yt_dlp/extractor/common.py#L119-L440). Add tests and code for as many as you want. 1. Have a look at [`yt_dlp/extractor/common.py`](yt_dlp/extractor/common.py) for possible helper methods and a [detailed description of what your extractor should and may return](yt_dlp/extractor/common.py#L119-L440). Add tests and code for as many as you want.
1. Make sure your code follows [yt-dlp coding conventions](#yt-dlp-coding-conventions) and check the code with [flake8](https://flake8.pycqa.org/en/latest/index.html#quickstart): 1. Make sure your code follows [yt-dlp coding conventions](#yt-dlp-coding-conventions), passes [ruff](https://docs.astral.sh/ruff/tutorial/#getting-started) code checks and is properly formatted:
$ flake8 yt_dlp/extractor/yourextractor.py ```shell
$ hatch fmt --check
```
1. Make sure your code works under all [Python](https://www.python.org/) versions supported by yt-dlp, namely CPython and PyPy for Python 3.8 and above. Backward compatibility is not required for even older versions of Python. You can use `hatch fmt` to automatically fix problems. Rules that the linter/formatter enforces should not be disabled with `# noqa` unless a maintainer requests it. The only exception allowed is for old/printf-style string formatting in GraphQL query templates (use `# noqa: UP031`).
1. Make sure your code works under all [Python](https://www.python.org/) versions supported by yt-dlp, namely CPython >=3.9 and PyPy >=3.11. Backward compatibility is not required for even older versions of Python.
1. When the tests pass, [add](https://git-scm.com/docs/git-add) the new files, [commit](https://git-scm.com/docs/git-commit) them and [push](https://git-scm.com/docs/git-push) the result, like this: 1. When the tests pass, [add](https://git-scm.com/docs/git-add) the new files, [commit](https://git-scm.com/docs/git-commit) them and [push](https://git-scm.com/docs/git-push) the result, like this:
$ git add yt_dlp/extractor/_extractors.py ```shell
$ git add yt_dlp/extractor/yourextractor.py $ git add yt_dlp/extractor/_extractors.py
$ git commit -m '[yourextractor] Add extractor' $ git add yt_dlp/extractor/yourextractor.py
$ git push origin yourextractor $ git commit -m '[yourextractor] Add extractor'
$ git push origin yourextractor
```
1. Finally, [create a pull request](https://help.github.com/articles/creating-a-pull-request). We'll then review and merge it. 1. Finally, [create a pull request](https://help.github.com/articles/creating-a-pull-request). We'll then review and merge it.
@@ -257,10 +318,9 @@ Extractors are very fragile by nature since they depend on the layout of the sou
For extraction to work yt-dlp relies on metadata your extractor extracts and provides to yt-dlp expressed by an [information dictionary](yt_dlp/extractor/common.py#L119-L440) or simply *info dict*. Only the following meta fields in the *info dict* are considered mandatory for a successful extraction process by yt-dlp: For extraction to work yt-dlp relies on metadata your extractor extracts and provides to yt-dlp expressed by an [information dictionary](yt_dlp/extractor/common.py#L119-L440) or simply *info dict*. Only the following meta fields in the *info dict* are considered mandatory for a successful extraction process by yt-dlp:
- `id` (media identifier) - `id` (media identifier)
- `title` (media title)
- `url` (media download URL) or `formats` - `url` (media download URL) or `formats`
The aforementioned metafields are the critical data that the extraction does not make any sense without and if any of them fail to be extracted then the extractor is considered completely broken. While all extractors must return a `title`, they must also allow it's extraction to be non-fatal. The aforementioned metadata fields are the critical data without which extraction does not make any sense. If any of them fail to be extracted, then the extractor is considered broken. All other metadata extraction should be completely non-fatal.
For pornographic sites, appropriate `age_limit` must also be returned. For pornographic sites, appropriate `age_limit` must also be returned.
@@ -275,7 +335,7 @@ Say you have some source dictionary `meta` that you've fetched as JSON with HTTP
```python ```python
meta = self._download_json(url, video_id) meta = self._download_json(url, video_id)
``` ```
Assume at this point `meta`'s layout is: Assume at this point `meta`'s layout is:
```python ```python
@@ -705,7 +765,7 @@ Use `url_or_none` for safe URL processing.
Use `traverse_obj` and `try_call` (superseeds `dict_get` and `try_get`) for safe metadata extraction from parsed JSON. Use `traverse_obj` and `try_call` (superseeds `dict_get` and `try_get`) for safe metadata extraction from parsed JSON.
Use `unified_strdate` for uniform `upload_date` or any `YYYYMMDD` meta field extraction, `unified_timestamp` for uniform `timestamp` extraction, `parse_filesize` for `filesize` extraction, `parse_count` for count meta fields extraction, `parse_resolution`, `parse_duration` for `duration` extraction, `parse_age_limit` for `age_limit` extraction. Use `unified_strdate` for uniform `upload_date` or any `YYYYMMDD` meta field extraction, `unified_timestamp` for uniform `timestamp` extraction, `parse_filesize` for `filesize` extraction, `parse_count` for count meta fields extraction, `parse_resolution`, `parse_duration` for `duration` extraction, `parse_age_limit` for `age_limit` extraction.
Explore [`yt_dlp/utils/`](yt_dlp/utils/) for more useful convenience functions. Explore [`yt_dlp/utils/`](yt_dlp/utils/) for more useful convenience functions.
@@ -720,12 +780,10 @@ view_count = int_or_none(video.get('views'))
``` ```
# My pull request is labeled pending-fixes ## My pull request is labeled pending-fixes
The `pending-fixes` label is added when there are changes requested to a PR. When the necessary changes are made, the label should be removed. However, despite our best efforts, it may sometimes happen that the maintainer did not see the changes or forgot to remove the label. If your PR is still marked as `pending-fixes` a few days after all requested changes have been made, feel free to ping the maintainer who labeled your issue and ask them to re-review and remove the label. The `pending-fixes` label is added when there are changes requested to a PR. When the necessary changes are made, the label should be removed. However, despite our best efforts, it may sometimes happen that the maintainer did not see the changes or forgot to remove the label. If your PR is still marked as `pending-fixes` a few days after all requested changes have been made, feel free to ping the maintainer who labeled your issue and ask them to re-review and remove the label.
# EMBEDDING YT-DLP # EMBEDDING YT-DLP
See [README.md#embedding-yt-dlp](README.md#embedding-yt-dlp) for instructions on how to embed yt-dlp in another Python program See [README.md#embedding-yt-dlp](README.md#embedding-yt-dlp) for instructions on how to embed yt-dlp in another Python program
+212 -1
View File
@@ -4,6 +4,7 @@ coletdjnz/colethedj (collaborator)
Ashish0804 (collaborator) Ashish0804 (collaborator)
bashonly (collaborator) bashonly (collaborator)
Grub4K (collaborator) Grub4K (collaborator)
seproDev (collaborator)
h-h-h-h h-h-h-h
pauldubois98 pauldubois98
nixxo nixxo
@@ -403,7 +404,6 @@ rebane2001
road-master road-master
rohieb rohieb
sdht0 sdht0
seproDev
Hill-98 Hill-98
LXYan2333 LXYan2333
mushbite mushbite
@@ -600,3 +600,214 @@ xpadev-net
Xpl0itU Xpl0itU
YoshichikaAAA YoshichikaAAA
zhijinwuu zhijinwuu
alb
hruzgar
kasper93
leoheitmannruiz
luiso1979
nipotan
Offert4324
sta1us
Tomoka1
trwstin
alexhuot1
clienthax
DaPotato69
emqi
hugohaa
imanoreotwe
JakeFinley96
lostfictions
minamotorin
ocococococ
Podiumnoche
RasmusAntons
roeniss
shoxie007
Szpachlarz
The-MAGI
TuxCoder
voidful
vtexier
WyohKnott
trueauracoral
ASertacAkkaya
axpauls
chilinux
hafeoz
JSubelj
jucor
megumintyan
mgedmin
Niluge-KiWi
peisenwang
TheZ3ro
tippfehlr
varunchopra
DrakoCpp
PatrykMis
DinhHuy2010
exterrestris
harbhim
LeSuisse
DunnesH
iancmy
mokrueger
luvyana
szantnerb
hugepower
scribblemaniac
Codenade
Demon000
Deukhoofd
grqz
hibes
Khaoklong51
kieraneglin
lengzuo
naglis
ndyanx
otovalek
quad
rakslice
sahilsinghss73
tony-hn
xingchensong
BallzCrasher
coreywright
eric321
poyhen
tetra-fox
444995
63427083
allendema
DarkZeros
DTrombett
imranh2
KarboniteKream
mikkovedru
pktiuk
rubyevadestaxes
avagordon01
CounterPillow
JoseAngelB
KBelmin
kesor
MellowKyler
Wesley107772
a13ssandr0
ChocoLZS
doe1080
hugovdev
jshumphrey
julionc
manavchaudhary1
powergold1
Sakura286
SamDecrock
stratus-ss
subrat-lima
gitninja1234
jkruse
xiaomac
wesson09
Crypto90
MutantPiggieGolem1
Sanceilaks
Strkmn
0x9fff00
4ft35t
7x11x13
b5i
cotko
d3d9
Dioarya
finch71
hexahigh
InvalidUsernameException
jixunmoe
knackku
krandor
kvk-2015
lonble
msm595
n10dollar
NecroRomnt
pjrobertson
subsense
test20140
arantius
entourage8
lfavole
mp3butcher
slipinthedove
YoshiTabletopGamer
Arc8ne
benfaerber
chrisellsworth
fries1234
Kenshin9977
MichaelDeBoey
msikma
pedro
pferreir
red-acid
refack
rysson
somini
thedenv
vallovic
arabcoders
mireq
mlabeeb03
1271
CasperMcFadden95
Kicer86
Kiritomo
leeblackc
meGAmeS1
NeonMan
pj47x
troex
WouterGordts
baierjan
GeoffreyFrogeye
Pawka
v3DJG6GL
yozel
brian6932
iednod55
maxbin123
nullpos
anlar
eason1478
ceandreasen
chauhantirth
helpimnotdrowning
adamralph
averageFOSSenjoyer
bubo
flanter21
Georift
moonshinerd
R0hanW
ShockedPlot7560
swayll
atsushi2965
barryvan
injust
iribeirocampos
rolandcrosby
Sojiroh
tchebb
AzartX47
e2dk4r
junyilou
PierreMesure
Randalix
runarmod
gitchasing
zakaryan2004
cdce8p
nicolaasjan
willsmillie
+1597
View File
File diff suppressed because it is too large Load Diff
+7
View File
@@ -61,3 +61,10 @@ You can also find lists of all [contributors of yt-dlp](CONTRIBUTORS) and [autho
* Reworked internals like `traverse_obj`, various core refactors and bugs fixes * Reworked internals like `traverse_obj`, various core refactors and bugs fixes
* Implemented proper progress reporting for parallel downloads * Implemented proper progress reporting for parallel downloads
* Improved/fixed/added Bundestag, crunchyroll, pr0gramm, Twitter, WrestleUniverse etc * Improved/fixed/added Bundestag, crunchyroll, pr0gramm, Twitter, WrestleUniverse etc
## [sepro](https://github.com/seproDev)
* UX improvements: Warn when ffmpeg is missing, warn when double-clicking exe
* Code cleanup: Remove dead extractors, mark extractors as broken, enable/apply ruff rules
* Improved/fixed/added ArdMediathek, DRTV, Floatplane, MagentaMusik, Naver, Nebula, OnDemandKorea, Vbox7 etc
+27 -13
View File
@@ -2,7 +2,7 @@ all: lazy-extractors yt-dlp doc pypi-files
clean: clean-test clean-dist clean: clean-test clean-dist
clean-all: clean clean-cache clean-all: clean clean-cache
completions: completion-bash completion-fish completion-zsh completions: completion-bash completion-fish completion-zsh
doc: README.md CONTRIBUTING.md issuetemplates supportedsites doc: README.md CONTRIBUTING.md CONTRIBUTORS issuetemplates supportedsites
ot: offlinetest ot: offlinetest
tar: yt-dlp.tar.gz tar: yt-dlp.tar.gz
@@ -10,21 +10,25 @@ tar: yt-dlp.tar.gz
# intended use: when building a source distribution, # intended use: when building a source distribution,
# make pypi-files && python3 -m build -sn . # make pypi-files && python3 -m build -sn .
pypi-files: AUTHORS Changelog.md LICENSE README.md README.txt supportedsites \ pypi-files: AUTHORS Changelog.md LICENSE README.md README.txt supportedsites \
completions yt-dlp.1 pyproject.toml setup.cfg devscripts/* test/* completions yt-dlp.1 pyproject.toml devscripts/* test/*
.PHONY: all clean install test tar pypi-files completions ot offlinetest codetest supportedsites .PHONY: all clean clean-all clean-test clean-dist clean-cache \
completions completion-bash completion-fish completion-zsh \
doc issuetemplates supportedsites ot offlinetest codetest test \
tar pypi-files lazy-extractors install uninstall
clean-test: clean-test:
rm -rf test/testdata/sigs/player-*.js tmp/ *.annotations.xml *.aria2 *.description *.dump *.frag \ rm -rf tmp/ *.annotations.xml *.aria2 *.description *.dump *.frag \
*.frag.aria2 *.frag.urls *.info.json *.live_chat.json *.meta *.part* *.tmp *.temp *.unknown_video *.ytdl \ *.frag.aria2 *.frag.urls *.info.json *.live_chat.json *.meta *.part* *.tmp *.temp *.unknown_video *.ytdl \
*.3gp *.ape *.ass *.avi *.desktop *.f4v *.flac *.flv *.gif *.jpeg *.jpg *.lrc *.m4a *.m4v *.mhtml *.mkv *.mov *.mp3 *.mp4 \ *.3gp *.ape *.ass *.avi *.desktop *.f4v *.flac *.flv *.gif *.jpeg *.jpg *.lrc *.m4a *.m4v *.mhtml *.mkv *.mov *.mp3 *.mp4 \
*.mpg *.mpga *.oga *.ogg *.opus *.png *.sbv *.srt *.ssa *.swf *.swp *.tt *.ttml *.url *.vtt *.wav *.webloc *.webm *.webp *.mpg *.mpga *.oga *.ogg *.opus *.png *.sbv *.srt *.ssa *.swf *.tt *.ttml *.url *.vtt *.wav *.webloc *.webm *.webp \
test/testdata/sigs/player-*.js test/testdata/thumbnails/empty.webp "test/testdata/thumbnails/foo %d bar/foo_%d."*
clean-dist: clean-dist:
rm -rf yt-dlp.1.temp.md yt-dlp.1 README.txt MANIFEST build/ dist/ .coverage cover/ yt-dlp.tar.gz completions/ \ rm -rf yt-dlp.1.temp.md yt-dlp.1 README.txt MANIFEST build/ dist/ .coverage cover/ yt-dlp.tar.gz completions/ \
yt_dlp/extractor/lazy_extractors.py *.spec CONTRIBUTING.md.tmp yt-dlp yt-dlp.exe yt_dlp.egg-info/ AUTHORS yt_dlp/extractor/lazy_extractors.py *.spec CONTRIBUTING.md.tmp yt-dlp yt-dlp.exe yt_dlp.egg-info/ AUTHORS
clean-cache: clean-cache:
find . \( \ find . \( \
-type d -name .pytest_cache -o -type d -name __pycache__ -o -name "*.pyc" -o -name "*.class" \ -type d -name ".*_cache" -o -type d -name __pycache__ -o -name "*.pyc" -o -name "*.class" \
\) -prune -exec rm -rf {} \; \) -prune -exec rm -rf {} \;
completion-bash: completions/bash/yt-dlp completion-bash: completions/bash/yt-dlp
@@ -67,14 +71,15 @@ uninstall:
rm -f $(DESTDIR)$(SHAREDIR)/fish/vendor_completions.d/yt-dlp.fish rm -f $(DESTDIR)$(SHAREDIR)/fish/vendor_completions.d/yt-dlp.fish
codetest: codetest:
flake8 . ruff check .
autopep8 --diff .
test: test:
$(PYTHON) -m pytest $(PYTHON) -m pytest -Werror
$(MAKE) codetest $(MAKE) codetest
offlinetest: codetest offlinetest: codetest
$(PYTHON) -m pytest -k "not download" $(PYTHON) -m pytest -Werror -m "not download"
CODE_FOLDERS_CMD = find yt_dlp -type f -name '__init__.py' | sed 's,/__init__.py,,' | grep -v '/__' | sort CODE_FOLDERS_CMD = find yt_dlp -type f -name '__init__.py' | sed 's,/__init__.py,,' | grep -v '/__' | sort
CODE_FOLDERS != $(CODE_FOLDERS_CMD) CODE_FOLDERS != $(CODE_FOLDERS_CMD)
@@ -148,13 +153,22 @@ yt-dlp.tar.gz: all
--exclude '*.pyo' \ --exclude '*.pyo' \
--exclude '*~' \ --exclude '*~' \
--exclude '__pycache__' \ --exclude '__pycache__' \
--exclude '.pytest_cache' \ --exclude '.*_cache' \
--exclude '.git' \ --exclude '.git' \
-- \ -- \
README.md supportedsites.md Changelog.md LICENSE \ README.md supportedsites.md Changelog.md LICENSE \
CONTRIBUTING.md Collaborators.md CONTRIBUTORS AUTHORS \ CONTRIBUTING.md Collaborators.md CONTRIBUTORS AUTHORS \
Makefile yt-dlp.1 README.txt completions .gitignore \ Makefile yt-dlp.1 README.txt completions .gitignore \
setup.cfg yt-dlp yt_dlp pyproject.toml devscripts test yt-dlp yt_dlp pyproject.toml devscripts test
AUTHORS: AUTHORS: Changelog.md
git shortlog -s -n HEAD | cut -f2 | sort > AUTHORS @if [ -d '.git' ] && command -v git > /dev/null ; then \
echo 'Generating $@ from git commit history' ; \
git shortlog -s -n HEAD | cut -f2 | sort > $@ ; \
fi
CONTRIBUTORS: Changelog.md
@if [ -d '.git' ] && command -v git > /dev/null ; then \
echo 'Updating $@ from git commit history' ; \
$(PYTHON) devscripts/make_changelog.py -v -c > /dev/null ; \
fi
+335 -214
View File
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+178
View File
@@ -0,0 +1,178 @@
services:
linux_x86_64:
build:
context: linux
target: build
platforms:
- "linux/amd64"
args:
BUILDIMAGE: ghcr.io/yt-dlp/manylinux2014_x86_64-shared:latest
environment:
EXE_NAME: ${EXE_NAME:?}
CHANNEL: ${CHANNEL:?}
ORIGIN: ${ORIGIN:?}
VERSION:
PYTHON_VERSION:
SKIP_ONEDIR_BUILD:
SKIP_ONEFILE_BUILD:
volumes:
- ../..:/yt-dlp
linux_x86_64_verify:
build:
context: linux
target: verify
platforms:
- "linux/amd64"
args:
VERIFYIMAGE: quay.io/pypa/manylinux2014_x86_64:latest
environment:
EXE_NAME: ${EXE_NAME:?}
UPDATE_TO:
SKIP_ONEDIR_BUILD:
SKIP_ONEFILE_BUILD:
volumes:
- ../../dist:/build
linux_aarch64:
build:
context: linux
target: build
platforms:
- "linux/arm64"
args:
BUILDIMAGE: ghcr.io/yt-dlp/manylinux2014_aarch64-shared:latest
environment:
EXE_NAME: ${EXE_NAME:?}
CHANNEL: ${CHANNEL:?}
ORIGIN: ${ORIGIN:?}
VERSION:
PYTHON_VERSION:
SKIP_ONEDIR_BUILD:
SKIP_ONEFILE_BUILD:
volumes:
- ../..:/yt-dlp
linux_aarch64_verify:
build:
context: linux
target: verify
platforms:
- "linux/arm64"
args:
VERIFYIMAGE: quay.io/pypa/manylinux2014_aarch64:latest
environment:
EXE_NAME: ${EXE_NAME:?}
UPDATE_TO:
SKIP_ONEDIR_BUILD:
SKIP_ONEFILE_BUILD:
volumes:
- ../../dist:/build
linux_armv7l:
build:
context: linux
target: build
platforms:
- "linux/arm/v7"
args:
BUILDIMAGE: ghcr.io/yt-dlp/manylinux_2_31_armv7l-shared:latest
environment:
EXE_NAME: ${EXE_NAME:?}
CHANNEL: ${CHANNEL:?}
ORIGIN: ${ORIGIN:?}
VERSION:
PYTHON_VERSION:
SKIP_ONEDIR_BUILD:
SKIP_ONEFILE_BUILD:
volumes:
- ../..:/yt-dlp
- ../../venv:/yt-dlp-build-venv
linux_armv7l_verify:
build:
context: linux
target: verify
platforms:
- "linux/arm/v7"
args:
VERIFYIMAGE: arm32v7/debian:bullseye
environment:
EXE_NAME: ${EXE_NAME:?}
UPDATE_TO:
SKIP_ONEDIR_BUILD:
SKIP_ONEFILE_BUILD:
volumes:
- ../../dist:/build
musllinux_x86_64:
build:
context: linux
target: build
platforms:
- "linux/amd64"
args:
BUILDIMAGE: ghcr.io/yt-dlp/musllinux_1_2_x86_64-shared:latest
environment:
EXE_NAME: ${EXE_NAME:?}
CHANNEL: ${CHANNEL:?}
ORIGIN: ${ORIGIN:?}
VERSION:
PYTHON_VERSION:
SKIP_ONEDIR_BUILD:
SKIP_ONEFILE_BUILD:
volumes:
- ../..:/yt-dlp
musllinux_x86_64_verify:
build:
context: linux
target: verify
platforms:
- "linux/amd64"
args:
VERIFYIMAGE: alpine:3.22
environment:
EXE_NAME: ${EXE_NAME:?}
UPDATE_TO:
SKIP_ONEDIR_BUILD:
SKIP_ONEFILE_BUILD:
volumes:
- ../../dist:/build
musllinux_aarch64:
build:
context: linux
target: build
platforms:
- "linux/arm64"
args:
BUILDIMAGE: ghcr.io/yt-dlp/musllinux_1_2_aarch64-shared:latest
environment:
EXE_NAME: ${EXE_NAME:?}
CHANNEL: ${CHANNEL:?}
ORIGIN: ${ORIGIN:?}
VERSION:
PYTHON_VERSION:
SKIP_ONEDIR_BUILD:
SKIP_ONEFILE_BUILD:
EXCLUDE_CURL_CFFI: "1"
volumes:
- ../..:/yt-dlp
musllinux_aarch64_verify:
build:
context: linux
target: verify
platforms:
- "linux/arm64"
args:
VERIFYIMAGE: alpine:3.22
environment:
EXE_NAME: ${EXE_NAME:?}
UPDATE_TO:
SKIP_ONEDIR_BUILD:
SKIP_ONEFILE_BUILD:
volumes:
- ../../dist:/build
+16
View File
@@ -0,0 +1,16 @@
ARG BUILDIMAGE=ghcr.io/yt-dlp/manylinux2014_x86_64-shared:latest
ARG VERIFYIMAGE=alpine:3.22
FROM $BUILDIMAGE AS build
WORKDIR /yt-dlp
COPY build.sh /build.sh
ENTRYPOINT ["/build.sh"]
FROM $VERIFYIMAGE AS verify
WORKDIR /testing
COPY verify.sh /verify.sh
ENTRYPOINT ["/verify.sh"]
+48
View File
@@ -0,0 +1,48 @@
#!/bin/bash
set -exuo pipefail
if [[ -z "${PYTHON_VERSION:-}" ]]; then
PYTHON_VERSION="3.13"
echo "Defaulting to using Python ${PYTHON_VERSION}"
fi
function runpy {
"/opt/shared-cpython-${PYTHON_VERSION}/bin/python${PYTHON_VERSION}" "$@"
}
function venvpy {
"python${PYTHON_VERSION}" "$@"
}
INCLUDES=(
--include pyinstaller
--include secretstorage
)
if [[ -z "${EXCLUDE_CURL_CFFI:-}" ]]; then
INCLUDES+=(--include curl-cffi)
fi
runpy -m venv /yt-dlp-build-venv
# shellcheck disable=SC1091
source /yt-dlp-build-venv/bin/activate
# Inside the venv we use venvpy instead of runpy
venvpy -m ensurepip --upgrade --default-pip
venvpy -m devscripts.install_deps -o --include build
venvpy -m devscripts.install_deps "${INCLUDES[@]}"
venvpy -m devscripts.make_lazy_extractors
venvpy devscripts/update-version.py -c "${CHANNEL}" -r "${ORIGIN}" "${VERSION}"
if [[ -z "${SKIP_ONEDIR_BUILD:-}" ]]; then
mkdir -p /build
venvpy -m bundle.pyinstaller --onedir --distpath=/build
pushd "/build/${EXE_NAME}"
chmod +x "${EXE_NAME}"
venvpy -m zipfile -c "/yt-dlp/dist/${EXE_NAME}.zip" ./
popd
fi
if [[ -z "${SKIP_ONEFILE_BUILD:-}" ]]; then
venvpy -m bundle.pyinstaller
chmod +x "./dist/${EXE_NAME}"
fi
+51
View File
@@ -0,0 +1,51 @@
#!/bin/sh
set -eu
if [ -n "${SKIP_ONEFILE_BUILD:-}" ]; then
if [ -n "${SKIP_ONEDIR_BUILD:-}" ]; then
echo "All executable builds were skipped"
exit 1
fi
echo "Extracting zip to verify onedir build"
if command -v python3 >/dev/null 2>&1; then
python3 -m zipfile -e "/build/${EXE_NAME}.zip" ./
else
echo "Attempting to install unzip"
if command -v dnf >/dev/null 2>&1; then
dnf -y install --allowerasing unzip
elif command -v yum >/dev/null 2>&1; then
yum -y install unzip
elif command -v apt-get >/dev/null 2>&1; then
DEBIAN_FRONTEND=noninteractive apt-get update -qq
DEBIAN_FRONTEND=noninteractive apt-get install -qq -y --no-install-recommends unzip
elif command -v apk >/dev/null 2>&1; then
apk add --no-cache unzip
else
echo "Unsupported image"
exit 1
fi
unzip "/build/${EXE_NAME}.zip" -d ./
fi
chmod +x "./${EXE_NAME}"
"./${EXE_NAME}" -v || true
"./${EXE_NAME}" --version
exit 0
fi
echo "Verifying onefile build"
cp "/build/${EXE_NAME}" ./
chmod +x "./${EXE_NAME}"
if [ -z "${UPDATE_TO:-}" ]; then
"./${EXE_NAME}" -v || true
"./${EXE_NAME}" --version
exit 0
fi
cp "./${EXE_NAME}" "./${EXE_NAME}_downgraded"
version="$("./${EXE_NAME}" --version)"
"./${EXE_NAME}_downgraded" -v --update-to "${UPDATE_TO}"
downgraded_version="$("./${EXE_NAME}_downgraded" --version)"
if [ "${version}" = "${downgraded_version}" ]; then
exit 1
fi
-59
View File
@@ -1,59 +0,0 @@
#!/usr/bin/env python3
# Allow execution from anywhere
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import warnings
from py2exe import freeze
from devscripts.utils import read_version
VERSION = read_version()
def main():
warnings.warn(
'py2exe builds do not support pycryptodomex and needs VC++14 to run. '
'It is recommended to run "pyinst.py" to build using pyinstaller instead')
freeze(
console=[{
'script': './yt_dlp/__main__.py',
'dest_base': 'yt-dlp',
'icon_resources': [(1, 'devscripts/logo.ico')],
}],
version_info={
'version': VERSION,
'description': 'A youtube-dl fork with additional features and patches',
'comments': 'Official repository: <https://github.com/yt-dlp/yt-dlp>',
'product_name': 'yt-dlp',
'product_version': VERSION,
},
options={
'bundle_files': 0,
'compressed': 1,
'optimize': 2,
'dist_dir': './dist',
'excludes': [
# py2exe cannot import Crypto
'Crypto',
'Cryptodome',
# py2exe appears to confuse this with our socks library.
# We don't use pysocks and urllib3.contrib.socks would fail to import if tried.
'urllib3.contrib.socks'
],
'dll_excludes': ['w9xpopen.exe', 'crypt32.dll'],
# Modules that are only imported dynamically must be added here
'includes': ['yt_dlp.compat._legacy', 'yt_dlp.compat._deprecated',
'yt_dlp.utils._legacy', 'yt_dlp.utils._deprecated'],
},
zipfile=None,
)
if __name__ == '__main__':
main()
+17 -7
View File
@@ -13,6 +13,8 @@ from PyInstaller.__main__ import run as run_pyinstaller
from devscripts.utils import read_version from devscripts.utils import read_version
OS_NAME, MACHINE, ARCH = sys.platform, platform.machine().lower(), platform.architecture()[0][:2] OS_NAME, MACHINE, ARCH = sys.platform, platform.machine().lower(), platform.architecture()[0][:2]
if OS_NAME == 'linux' and platform.libc_ver()[0] != 'glibc':
OS_NAME = 'musllinux'
if MACHINE in ('x86', 'x86_64', 'amd64', 'i386', 'i686'): if MACHINE in ('x86', 'x86_64', 'amd64', 'i386', 'i686'):
MACHINE = 'x86' if ARCH == '32' else '' MACHINE = 'x86' if ARCH == '32' else ''
@@ -36,6 +38,9 @@ def main():
f'--name={name}', f'--name={name}',
'--icon=devscripts/logo.ico', '--icon=devscripts/logo.ico',
'--upx-exclude=vcruntime140.dll', '--upx-exclude=vcruntime140.dll',
# Ref: https://github.com/yt-dlp/yt-dlp/issues/13311
# https://github.com/pyinstaller/pyinstaller/issues/9149
'--exclude-module=pkg_resources',
'--noconfirm', '--noconfirm',
'--additional-hooks-dir=yt_dlp/__pyinstaller', '--additional-hooks-dir=yt_dlp/__pyinstaller',
*opts, *opts,
@@ -59,16 +64,22 @@ def parse_options():
def exe(onedir): def exe(onedir):
"""@returns (name, path)""" """@returns (name, path)"""
platform_name, machine, extension = {
'win32': (None, MACHINE, '.exe'),
'darwin': ('macos', None, None),
}.get(OS_NAME, (OS_NAME, MACHINE, None))
name = '_'.join(filter(None, ( name = '_'.join(filter(None, (
'yt-dlp', 'yt-dlp',
{'win32': '', 'darwin': 'macos'}.get(OS_NAME, OS_NAME), platform_name,
MACHINE, machine,
))) )))
return name, ''.join(filter(None, ( return name, ''.join(filter(None, (
'dist/', 'dist/',
onedir and f'{name}/', onedir and f'{name}/',
name, name,
OS_NAME == 'win32' and '.exe' extension,
))) )))
@@ -113,18 +124,17 @@ def windows_set_version(exe, version):
), ),
kids=[ kids=[
StringFileInfo([StringTable('040904B0', [ StringFileInfo([StringTable('040904B0', [
StringStruct('Comments', 'yt-dlp%s Command Line Interface' % suffix), StringStruct('Comments', f'yt-dlp{suffix} Command Line Interface'),
StringStruct('CompanyName', 'https://github.com/yt-dlp'), StringStruct('CompanyName', 'https://github.com/yt-dlp'),
StringStruct('FileDescription', 'yt-dlp%s' % (MACHINE and f' ({MACHINE})')), StringStruct('FileDescription', 'yt-dlp%s' % (MACHINE and f' ({MACHINE})')),
StringStruct('FileVersion', version), StringStruct('FileVersion', version),
StringStruct('InternalName', f'yt-dlp{suffix}'), StringStruct('InternalName', f'yt-dlp{suffix}'),
StringStruct('LegalCopyright', 'pukkandan.ytdlp@gmail.com | UNLICENSE'),
StringStruct('OriginalFilename', f'yt-dlp{suffix}.exe'), StringStruct('OriginalFilename', f'yt-dlp{suffix}.exe'),
StringStruct('ProductName', f'yt-dlp{suffix}'), StringStruct('ProductName', f'yt-dlp{suffix}'),
StringStruct( StringStruct(
'ProductVersion', f'{version}{suffix} on Python {platform.python_version()}'), 'ProductVersion', f'{version}{suffix} on Python {platform.python_version()}'),
])]), VarFileInfo([VarStruct('Translation', [0, 1200])]) ])]), VarFileInfo([VarStruct('Translation', [0, 1200])]),
] ],
)) ))
+5 -1
View File
@@ -6,13 +6,17 @@ __yt_dlp()
prev="${COMP_WORDS[COMP_CWORD-1]}" prev="${COMP_WORDS[COMP_CWORD-1]}"
opts="{{flags}}" opts="{{flags}}"
keywords=":ytfavorites :ytrecommended :ytsubscriptions :ytwatchlater :ythistory" keywords=":ytfavorites :ytrecommended :ytsubscriptions :ytwatchlater :ythistory"
fileopts="-a|--batch-file|--download-archive|--cookies|--load-info" fileopts="-a|--batch-file|--download-archive|--cookies|--load-info-json"
diropts="--cache-dir" diropts="--cache-dir"
if [[ ${prev} =~ ${fileopts} ]]; then if [[ ${prev} =~ ${fileopts} ]]; then
local IFS=$'\n'
type compopt &>/dev/null && compopt -o filenames
COMPREPLY=( $(compgen -f -- ${cur}) ) COMPREPLY=( $(compgen -f -- ${cur}) )
return 0 return 0
elif [[ ${prev} =~ ${diropts} ]]; then elif [[ ${prev} =~ ${diropts} ]]; then
local IFS=$'\n'
type compopt &>/dev/null && compopt -o dirnames
COMPREPLY=( $(compgen -d -- ${cur}) ) COMPREPLY=( $(compgen -d -- ${cur}) )
return 0 return 0
fi fi
+4 -4
View File
@@ -9,8 +9,8 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import yt_dlp import yt_dlp
BASH_COMPLETION_FILE = "completions/bash/yt-dlp" BASH_COMPLETION_FILE = 'completions/bash/yt-dlp'
BASH_COMPLETION_TEMPLATE = "devscripts/bash-completion.in" BASH_COMPLETION_TEMPLATE = 'devscripts/bash-completion.in'
def build_completion(opt_parser): def build_completion(opt_parser):
@@ -21,9 +21,9 @@ def build_completion(opt_parser):
opts_flag.append(option.get_opt_string()) opts_flag.append(option.get_opt_string())
with open(BASH_COMPLETION_TEMPLATE) as f: with open(BASH_COMPLETION_TEMPLATE) as f:
template = f.read() template = f.read()
with open(BASH_COMPLETION_FILE, "w") as f: with open(BASH_COMPLETION_FILE, 'w') as f:
# just using the special char # just using the special char
filled_template = template.replace("{{flags}}", " ".join(opts_flag)) filled_template = template.replace('{{flags}}', ' '.join(opts_flag))
f.write(filled_template) f.write(filled_template)
+172
View File
@@ -126,5 +126,177 @@
"when": "4ce57d3b873c2887814cbec03d029533e82f7db5", "when": "4ce57d3b873c2887814cbec03d029533e82f7db5",
"short": "[ie] Support multi-period MPD streams (#6654)", "short": "[ie] Support multi-period MPD streams (#6654)",
"authors": ["alard", "pukkandan"] "authors": ["alard", "pukkandan"]
},
{
"action": "change",
"when": "aa7e9ae4f48276bd5d0173966c77db9484f65a0a",
"short": "[ie/xvideos] Support new URL format (#9502)",
"authors": ["sta1us"]
},
{
"action": "remove",
"when": "22e4dfacb61f62dfbb3eb41b31c7b69ba1059b80"
},
{
"action": "change",
"when": "e3a3ed8a981d9395c4859b6ef56cd02bc3148db2",
"short": "[cleanup:ie] No `from` stdlib imports in extractors",
"authors": ["pukkandan"]
},
{
"action": "add",
"when": "9590cc6b4768e190183d7d071a6c78170889116a",
"short": "[priority] Security: [[CVE-2024-22423](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-22423)] [Prevent RCE when using `--exec` with `%q` on Windows](https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-hjq6-52gw-2g7p)\n - The shell escape function now properly escapes `%`, `\\` and `\\n`.\n - `utils.Popen` has been patched accordingly."
},
{
"action": "change",
"when": "41ba4a808b597a3afed78c89675a30deb6844450",
"short": "[ie/tiktok] Extract via mobile API only if extractor-arg is passed (#9938)",
"authors": ["bashonly"]
},
{
"action": "remove",
"when": "6e36d17f404556f0e3a43f441c477a71a91877d9"
},
{
"action": "change",
"when": "beaf832c7a9d57833f365ce18f6115b88071b296",
"short": "[ie/soundcloud] Add `formats` extractor-arg (#10004)",
"authors": ["bashonly", "Grub4K"]
},
{
"action": "change",
"when": "5c019f6328ad40d66561eac3c4de0b3cd070d0f6",
"short": "[cleanup] Misc (#9765)",
"authors": ["bashonly", "Grub4K", "seproDev"]
},
{
"action": "change",
"when": "e6a22834df1776ec4e486526f6df2bf53cb7e06f",
"short": "[ie/orf:on] Add `prefer_segments_playlist` extractor-arg (#10314)",
"authors": ["seproDev"]
},
{
"action": "add",
"when": "6aaf96a3d6e7d0d426e97e11a2fcf52fda00e733",
"short": "[priority] Security: [[CVE-2024-38519](https://nvd.nist.gov/vuln/detail/CVE-2024-38519)] [Properly sanitize file-extension to prevent file system modification and RCE](https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-79w7-vh3h-8g4j)\n - Unsafe extensions are now blocked from being downloaded"
},
{
"action": "add",
"when": "6075a029dba70a89675ae1250e7cdfd91f0eba41",
"short": "[priority] Security: [[ie/douyutv] Do not use dangerous javascript source/URL](https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-3v33-3wmw-3785)\n - A dependency on potentially malicious third-party JavaScript code has been removed from the Douyu extractors"
},
{
"action": "add",
"when": "fb8b7f226d251e521a89b23c415e249e5b788e5c",
"short": "[priority] **The minimum *recommended* Python version has been raised to 3.9**\nSince Python 3.8 will reach end-of-life in October 2024, support for it will be dropped soon. [Read more](https://github.com/yt-dlp/yt-dlp/issues/10086)"
},
{
"action": "change",
"when": "b31b81d85f00601710d4fac590c3e4efb4133283",
"short": "[ci] Rerun failed tests (#11143)",
"authors": ["Grub4K"]
},
{
"action": "add",
"when": "a886cf3e900f4a2ec00af705f883539269545609",
"short": "[priority] **py2exe is no longer supported**\nThis release's `yt-dlp_min.exe` will be the last, and it's actually a PyInstaller-bundled executable so that yt-dlp users updating their py2exe build with `-U` will be automatically migrated. [Read more](https://github.com/yt-dlp/yt-dlp/issues/10087)"
},
{
"action": "add",
"when": "a886cf3e900f4a2ec00af705f883539269545609",
"short": "[priority] **Following this release, yt-dlp's Python dependencies *must* be installed using the `default` group**\nIf you're installing yt-dlp with pip/pipx or requiring yt-dlp in your own Python project, you'll need to specify `yt-dlp[default]` if you want to also install yt-dlp's optional dependencies (which were previously included by default). [Read more](https://github.com/yt-dlp/yt-dlp/pull/11255)"
},
{
"action": "add",
"when": "87884f15580910e4e0fe0e1db73508debc657471",
"short": "[priority] **Beginning with this release, yt-dlp's Python dependencies *must* be installed using the `default` group**\nIf you're installing yt-dlp with pip/pipx or requiring yt-dlp in your own Python project, you'll need to specify `yt-dlp[default]` if you want to also install yt-dlp's optional dependencies (which were previously included by default). [Read more](https://github.com/yt-dlp/yt-dlp/pull/11255)"
},
{
"action": "add",
"when": "d784464399b600ba9516bbcec6286f11d68974dd",
"short": "[priority] **The minimum *required* Python version has been raised to 3.9**\nPython 3.8 reached its end-of-life on 2024.10.07, and yt-dlp has now removed support for it. As an unfortunate side effect, the official `yt-dlp.exe` and `yt-dlp_x86.exe` binaries are no longer supported on Windows 7. [Read more](https://github.com/yt-dlp/yt-dlp/issues/10086)"
},
{
"action": "change",
"when": "914af9a0cf51c9a3f74aa88d952bee8334c67511",
"short": "Expand paths in `--plugin-dirs` (#11334)",
"authors": ["bashonly"]
},
{
"action": "change",
"when": "c29f5a7fae93a08f3cfbb6127b2faa75145b06a0",
"short": "[ie/generic] Do not impersonate by default (#11336)",
"authors": ["bashonly"]
},
{
"action": "change",
"when": "57212a5f97ce367590aaa5c3e9a135eead8f81f7",
"short": "[ie/vimeo] Fix API retries (#11351)",
"authors": ["bashonly"]
},
{
"action": "add",
"when": "52c0ffe40ad6e8404d93296f575007b05b04c686",
"short": "[priority] **Login with OAuth is no longer supported for YouTube**\nDue to a change made by the site, yt-dlp is no longer able to support OAuth login for YouTube. [Read more](https://github.com/yt-dlp/yt-dlp/issues/11462#issuecomment-2471703090)"
},
{
"action": "change",
"when": "76ac023ff02f06e8c003d104f02a03deeddebdcd",
"short": "[ie/youtube:tab] Improve shorts title extraction (#11997)",
"authors": ["bashonly", "d3d9"]
},
{
"action": "add",
"when": "88eb1e7a9a2720ac89d653c0d0e40292388823bb",
"short": "[priority] **New option `--preset-alias`/`-t` has been added**\nThis provides convenient predefined aliases for common use cases. Available presets include `mp4`, `mp3`, `mkv`, `aac`, and `sleep`. See [the README](https://github.com/yt-dlp/yt-dlp/blob/master/README.md#preset-aliases) for more details."
},
{
"action": "remove",
"when": "d596824c2f8428362c072518856065070616e348"
},
{
"action": "remove",
"when": "7b81634fb1d15999757e7a9883daa6ef09ea785b"
},
{
"action": "remove",
"when": "500761e41acb96953a5064e951d41d190c287e46"
},
{
"action": "add",
"when": "f3008bc5f89d2691f2f8dfc51b406ef4e25281c3",
"short": "[priority] **Default behaviour changed from `--mtime` to `--no-mtime`**\nyt-dlp no longer applies the server modified time to downloaded files by default. [Read more](https://github.com/yt-dlp/yt-dlp/issues/12780)"
},
{
"action": "add",
"when": "959ac99e98c3215437e573c22d64be42d361e863",
"short": "[priority] Security: [[CVE-2025-54072](https://nvd.nist.gov/vuln/detail/CVE-2025-54072)] [Fix `--exec` placeholder expansion on Windows](https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-45hg-7f49-5h56)\n - When `--exec` is used on Windows, the filepath expanded from `{}` (or the default placeholder) is now properly escaped"
},
{
"action": "change",
"when": "b831406a1d3be34c159835079d12bae624c43610",
"short": "[ie/rtve.es:program] Add extractor (#12955)",
"authors": ["meGAmeS1", "seproDev"]
},
{
"action": "add",
"when": "23c658b9cbe34a151f8f921ab1320bb5d4e40a4d",
"short": "[priority] **The minimum *recommended* Python version has been raised to 3.10**\nSince Python 3.9 will reach end-of-life in October 2025, support for it will be dropped soon. [Read more](https://github.com/yt-dlp/yt-dlp/issues/13858)"
},
{
"action": "add",
"when": "cc5a5caac5fbc0d605b52bde0778d6fd5f97b5ab",
"short": "[priority] **darwin_legacy_exe builds are being discontinued**\nThis release's `yt-dlp_macos_legacy` binary will likely be the last one. [Read more](https://github.com/yt-dlp/yt-dlp/issues/13856)"
},
{
"action": "add",
"when": "c76ce28e06c816eb5b261dfb6aff6e69dd9b7382",
"short": "[priority] **linux_armv7l_exe builds are being discontinued**\nThis release's `yt-dlp_linux_armv7l` binary could be the last one. [Read more](https://github.com/yt-dlp/yt-dlp/issues/13976)"
},
{
"action": "add",
"when": "08d78996831bd8e1e3c2592d740c3def00bbf548",
"short": "[priority] **Several options have been deprecated**\nIn order to simplify the codebase and reduce maintenance burden, various options have been deprecated. Please remove them from your commands/configurations. [Read more](https://github.com/yt-dlp/yt-dlp/issues/14198)"
} }
] ]
Regular → Executable
+3
View File
@@ -1,3 +1,5 @@
#!/usr/bin/env python3
# Allow direct execution # Allow direct execution
import os import os
import sys import sys
@@ -18,6 +20,7 @@ def parse_patched_options(opts):
'fragment_retries': 0, 'fragment_retries': 0,
'extract_flat': False, 'extract_flat': False,
'concat_playlist': 'never', 'concat_playlist': 'never',
'update_self': False,
}) })
yt_dlp.options.create_parser = lambda: patched_parser yt_dlp.options.create_parser = lambda: patched_parser
try: try:
+1 -2
View File
@@ -11,13 +11,12 @@ import codecs
import subprocess import subprocess
from yt_dlp.aes import aes_encrypt, key_expansion from yt_dlp.aes import aes_encrypt, key_expansion
from yt_dlp.utils import intlist_to_bytes
secret_msg = b'Secret message goes here' secret_msg = b'Secret message goes here'
def hex_str(int_list): def hex_str(int_list):
return codecs.encode(intlist_to_bytes(int_list), 'hex') return codecs.encode(bytes(int_list), 'hex')
def openssl_encode(algo, key, iv): def openssl_encode(algo, key, iv):
+316
View File
@@ -0,0 +1,316 @@
import requests
from dataclasses import dataclass
from pathlib import Path
import hashlib
DEFAULT_OUTPUT = 'THIRD_PARTY_LICENSES.txt'
CACHE_LOCATION = '.license_cache'
HEADER = '''THIRD-PARTY LICENSES
This file aggregates license texts of third-party components included with the yt-dlp PyInstaller-bundled executables.
yt-dlp itself is licensed under the Unlicense (see LICENSE file).
Source code for bundled third-party components is available from the original projects.
If you cannot obtain it, the maintainers will provide it as per license obligation; maintainer emails are listed in pyproject.toml.'''
@dataclass(frozen=True)
class Dependency:
name: str
license_url: str
project_url: str = ''
license: str = ''
comment: str = ''
DEPENDENCIES: list[Dependency] = [
# Core runtime environment components
Dependency(
name='Python',
license='PSF-2.0',
license_url='https://raw.githubusercontent.com/python/cpython/refs/heads/main/LICENSE',
project_url='https://www.python.org/',
),
Dependency(
name='Microsoft Distributable Code',
license_url='https://raw.githubusercontent.com/python/cpython/refs/heads/main/PC/crtlicense.txt',
comment='Only included in Windows builds',
),
Dependency(
name='bzip2',
license='bzip2-1.0.6',
license_url='https://gitlab.com/federicomenaquintero/bzip2/-/raw/master/COPYING',
project_url='https://sourceware.org/bzip2/',
),
Dependency(
name='libffi',
license='MIT',
license_url='https://raw.githubusercontent.com/libffi/libffi/refs/heads/master/LICENSE',
project_url='https://sourceware.org/libffi/',
),
Dependency(
name='OpenSSL 3.0+',
license='Apache-2.0',
license_url='https://raw.githubusercontent.com/openssl/openssl/refs/heads/master/LICENSE.txt',
project_url='https://www.openssl.org/',
),
Dependency(
name='SQLite',
license='Public Domain', # Technically does not need to be included
license_url='https://sqlite.org/src/raw/e108e1e69ae8e8a59e93c455654b8ac9356a11720d3345df2a4743e9590fb20d?at=LICENSE.md',
project_url='https://www.sqlite.org/',
),
Dependency(
name='liblzma',
license='0BSD', # Technically does not need to be included
license_url='https://raw.githubusercontent.com/tukaani-project/xz/refs/heads/master/COPYING',
project_url='https://tukaani.org/xz/',
),
Dependency(
name='mpdecimal',
license='BSD-2-Clause',
# No official repo URL
license_url='https://gist.githubusercontent.com/seproDev/9e5dbfc08af35c3f2463e64eb9b27161/raw/61f5a98bc1a4ad7d48b1c793fc3314d4d43c2ab1/mpdecimal_COPYRIGHT.txt',
project_url='https://www.bytereef.org/mpdecimal/',
),
Dependency(
name='zlib',
license='zlib',
license_url='https://raw.githubusercontent.com/madler/zlib/refs/heads/develop/LICENSE',
project_url='https://zlib.net/',
),
Dependency(
name='Expat',
license='MIT',
license_url='https://raw.githubusercontent.com/libexpat/libexpat/refs/heads/master/COPYING',
project_url='https://libexpat.github.io/',
),
Dependency(
name='ncurses',
license='X11-distribute-modifications-variant',
license_url='https://raw.githubusercontent.com/mirror/ncurses/refs/heads/master/COPYING',
comment='Only included in Linux/macOS builds',
project_url='https://invisible-island.net/ncurses/',
),
Dependency(
name='GNU Readline',
license='GPL-3.0-or-later',
license_url='https://tiswww.case.edu/php/chet/readline/COPYING',
comment='Only included in Linux builds',
project_url='https://www.gnu.org/software/readline/',
),
Dependency(
name='libstdc++',
license='GPL-3.0-with-GCC-exception',
license_url='https://raw.githubusercontent.com/gcc-mirror/gcc/refs/heads/master/COPYING.RUNTIME',
comment='Only included in Linux builds',
project_url='https://gcc.gnu.org/onlinedocs/libstdc++/',
),
Dependency(
name='libgcc',
license='GPL-3.0-with-GCC-exception',
license_url='https://raw.githubusercontent.com/gcc-mirror/gcc/refs/heads/master/COPYING.RUNTIME',
comment='Only included in Linux builds',
project_url='https://gcc.gnu.org/',
),
Dependency(
name='libuuid',
license='BSD-3-Clause',
license_url='https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git/plain/lib/uuid/COPYING',
comment='Only included in Linux builds',
project_url='https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git/tree/lib/uuid',
),
Dependency(
name='libintl',
license='LGPL-2.1-or-later',
license_url='https://raw.githubusercontent.com/autotools-mirror/gettext/refs/heads/master/gettext-runtime/intl/COPYING.LIB',
comment='Only included in macOS builds',
project_url='https://www.gnu.org/software/gettext/',
),
Dependency(
name='libidn2',
license='LGPL-3.0-or-later',
license_url='https://gitlab.com/libidn/libidn2/-/raw/master/COPYING.LESSERv3',
comment='Only included in macOS builds',
project_url='https://www.gnu.org/software/libidn/',
),
Dependency(
name='libidn2 (Unicode character data files)',
license='Unicode-TOU AND Unicode-DFS-2016',
license_url='https://gitlab.com/libidn/libidn2/-/raw/master/COPYING.unicode',
comment='Only included in macOS builds',
project_url='https://www.gnu.org/software/libidn/',
),
Dependency(
name='libunistring',
license='LGPL-3.0-or-later',
license_url='https://gitweb.git.savannah.gnu.org/gitweb/?p=libunistring.git;a=blob_plain;f=COPYING.LIB;hb=HEAD',
comment='Only included in macOS builds',
project_url='https://www.gnu.org/software/libunistring/',
),
Dependency(
name='librtmp',
license='LGPL-2.1-or-later',
# No official repo URL
license_url='https://gist.githubusercontent.com/seproDev/31d8c691ccddebe37b8b379307cb232d/raw/053408e98547ea8c7d9ba3a80c965f33e163b881/librtmp_COPYING.txt',
comment='Only included in macOS builds',
project_url='https://rtmpdump.mplayerhq.hu/',
),
Dependency(
name='zstd',
license='BSD-3-Clause',
license_url='https://raw.githubusercontent.com/facebook/zstd/refs/heads/dev/LICENSE',
comment='Only included in macOS builds',
project_url='https://facebook.github.io/zstd/',
),
# Python packages
Dependency(
name='brotli',
license='MIT',
license_url='https://raw.githubusercontent.com/google/brotli/refs/heads/master/LICENSE',
project_url='https://brotli.org/',
),
Dependency(
name='curl_cffi',
license='MIT',
license_url='https://raw.githubusercontent.com/lexiforest/curl_cffi/refs/heads/main/LICENSE',
comment='Not included in `yt-dlp_x86` and `yt-dlp_musllinux_aarch64` builds',
project_url='https://curl-cffi.readthedocs.io/',
),
# Dependency of curl_cffi
Dependency(
name='curl-impersonate',
license='MIT',
license_url='https://raw.githubusercontent.com/lexiforest/curl-impersonate/refs/heads/main/LICENSE',
comment='Not included in `yt-dlp_x86` and `yt-dlp_musllinux_aarch64` builds',
project_url='https://github.com/lexiforest/curl-impersonate',
),
Dependency(
name='cffi',
license='MIT-0', # Technically does not need to be included
license_url='https://raw.githubusercontent.com/python-cffi/cffi/refs/heads/main/LICENSE',
project_url='https://cffi.readthedocs.io/',
),
# Dependecy of cffi
Dependency(
name='pycparser',
license='BSD-3-Clause',
license_url='https://raw.githubusercontent.com/eliben/pycparser/refs/heads/main/LICENSE',
project_url='https://github.com/eliben/pycparser',
),
Dependency(
name='mutagen',
license='GPL-2.0-or-later',
license_url='https://raw.githubusercontent.com/quodlibet/mutagen/refs/heads/main/COPYING',
project_url='https://mutagen.readthedocs.io/',
),
Dependency(
name='PyCryptodome',
license='Public Domain and BSD-2-Clause',
license_url='https://raw.githubusercontent.com/Legrandin/pycryptodome/refs/heads/master/LICENSE.rst',
project_url='https://www.pycryptodome.org/',
),
Dependency(
name='certifi',
license='MPL-2.0',
license_url='https://raw.githubusercontent.com/certifi/python-certifi/refs/heads/master/LICENSE',
project_url='https://github.com/certifi/python-certifi',
),
Dependency(
name='requests',
license='Apache-2.0',
license_url='https://raw.githubusercontent.com/psf/requests/refs/heads/main/LICENSE',
project_url='https://requests.readthedocs.io/',
),
# Dependency of requests
Dependency(
name='charset-normalizer',
license='MIT',
license_url='https://raw.githubusercontent.com/jawah/charset_normalizer/refs/heads/master/LICENSE',
project_url='https://charset-normalizer.readthedocs.io/',
),
# Dependency of requests
Dependency(
name='idna',
license='BSD-3-Clause',
license_url='https://raw.githubusercontent.com/kjd/idna/refs/heads/master/LICENSE.md',
project_url='https://github.com/kjd/idna',
),
Dependency(
name='urllib3',
license='MIT',
license_url='https://raw.githubusercontent.com/urllib3/urllib3/refs/heads/main/LICENSE.txt',
project_url='https://urllib3.readthedocs.io/',
),
Dependency(
name='SecretStorage',
license='BSD-3-Clause',
license_url='https://raw.githubusercontent.com/mitya57/secretstorage/refs/heads/master/LICENSE',
comment='Only included in Linux builds',
project_url='https://secretstorage.readthedocs.io/',
),
# Dependency of SecretStorage
Dependency(
name='cryptography',
license='Apache-2.0', # Also available as BSD-3-Clause
license_url='https://raw.githubusercontent.com/pyca/cryptography/refs/heads/main/LICENSE.APACHE',
comment='Only included in Linux builds',
project_url='https://cryptography.io/',
),
# Dependency of SecretStorage
Dependency(
name='Jeepney',
license='MIT',
license_url='https://gitlab.com/takluyver/jeepney/-/raw/master/LICENSE',
comment='Only included in Linux builds',
project_url='https://jeepney.readthedocs.io/',
),
Dependency(
name='websockets',
license='BSD-3-Clause',
license_url='https://raw.githubusercontent.com/python-websockets/websockets/refs/heads/main/LICENSE',
project_url='https://websockets.readthedocs.io/',
),
]
def fetch_text(dep: Dependency) -> str:
cache_dir = Path(CACHE_LOCATION)
cache_dir.mkdir(exist_ok=True)
url_hash = hashlib.sha256(dep.license_url.encode('utf-8')).hexdigest()
cache_file = cache_dir / f'{url_hash}.txt'
if cache_file.exists():
return cache_file.read_text()
# UA needed since some domains block requests default UA
req = requests.get(dep.license_url, headers={'User-Agent': 'yt-dlp license fetcher'})
req.raise_for_status()
text = req.text
cache_file.write_text(text)
return text
def build_output() -> str:
lines = [HEADER]
for d in DEPENDENCIES:
lines.append('\n')
lines.append('-' * 80)
header = f'{d.name}'
if d.license:
header += f' | {d.license}'
if d.comment:
header += f'\nNote: {d.comment}'
if d.project_url:
header += f'\nURL: {d.project_url}'
lines.append(header)
lines.append('-' * 80)
text = fetch_text(d)
lines.append(text.strip('\n') + '\n')
return '\n'.join(lines)
if __name__ == '__main__':
content = build_output()
Path(DEFAULT_OUTPUT).write_text(content)
+27 -19
View File
@@ -10,6 +10,8 @@ import argparse
import re import re
import subprocess import subprocess
from pathlib import Path
from devscripts.tomlparse import parse_toml from devscripts.tomlparse import parse_toml
from devscripts.utils import read_file from devscripts.utils import read_file
@@ -17,44 +19,50 @@ from devscripts.utils import read_file
def parse_args(): def parse_args():
parser = argparse.ArgumentParser(description='Install dependencies for yt-dlp') parser = argparse.ArgumentParser(description='Install dependencies for yt-dlp')
parser.add_argument( parser.add_argument(
'input', nargs='?', metavar='TOMLFILE', default='pyproject.toml', help='Input file (default: %(default)s)') 'input', nargs='?', metavar='TOMLFILE', default=Path(__file__).parent.parent / 'pyproject.toml',
help='input file (default: %(default)s)')
parser.add_argument( parser.add_argument(
'-e', '--exclude', metavar='DEPENDENCY', action='append', help='Exclude a dependency') '-e', '--exclude', metavar='DEPENDENCY', action='append',
help='exclude a dependency')
parser.add_argument( parser.add_argument(
'-i', '--include', metavar='GROUP', action='append', help='Include an optional dependency group') '-i', '--include', metavar='GROUP', action='append',
help='include an optional dependency group')
parser.add_argument( parser.add_argument(
'-o', '--only-optional', action='store_true', help='Only install optional dependencies') '-o', '--only-optional', action='store_true',
help='only install optional dependencies')
parser.add_argument( parser.add_argument(
'-p', '--print', action='store_true', help='Only print a requirements.txt to stdout') '-p', '--print', action='store_true',
help='only print requirements to stdout')
parser.add_argument( parser.add_argument(
'-u', '--user', action='store_true', help='Install with pip as --user') '-u', '--user', action='store_true',
help='install with pip as --user')
return parser.parse_args() return parser.parse_args()
def main(): def main():
args = parse_args() args = parse_args()
project_table = parse_toml(read_file(args.input))['project'] project_table = parse_toml(read_file(args.input))['project']
recursive_pattern = re.compile(rf'{project_table["name"]}\[(?P<group_name>[\w-]+)\]')
optional_groups = project_table['optional-dependencies'] optional_groups = project_table['optional-dependencies']
excludes = args.exclude or [] excludes = args.exclude or []
deps = [] def yield_deps(group):
for dep in group:
if mobj := recursive_pattern.fullmatch(dep):
yield from optional_groups.get(mobj.group('group_name'), [])
else:
yield dep
targets = []
if not args.only_optional: # `-o` should exclude 'dependencies' and the 'default' group if not args.only_optional: # `-o` should exclude 'dependencies' and the 'default' group
deps.extend(project_table['dependencies']) targets.extend(project_table['dependencies'])
if 'default' not in excludes: # `--exclude default` should exclude entire 'default' group if 'default' not in excludes: # `--exclude default` should exclude entire 'default' group
deps.extend(optional_groups['default']) targets.extend(yield_deps(optional_groups['default']))
def name(dependency):
return re.match(r'[\w-]+', dependency)[0].lower()
target_map = {name(dep): dep for dep in deps}
for include in filter(None, map(optional_groups.get, args.include or [])): for include in filter(None, map(optional_groups.get, args.include or [])):
target_map.update(zip(map(name, include), include)) targets.extend(yield_deps(include))
for exclude in map(name, excludes): targets = [t for t in targets if re.match(r'[\w-]+', t).group(0).lower() not in excludes]
target_map.pop(exclude, None)
targets = list(target_map.values())
if args.print: if args.print:
for target in targets: for target in targets:
+53 -48
View File
@@ -71,14 +71,13 @@ class CommitGroup(enum.Enum):
def get(cls, value: str) -> tuple[CommitGroup | None, str | None]: def get(cls, value: str) -> tuple[CommitGroup | None, str | None]:
group, _, subgroup = (group.strip().lower() for group in value.partition('/')) group, _, subgroup = (group.strip().lower() for group in value.partition('/'))
result = cls.group_lookup().get(group) if result := cls.group_lookup().get(group):
if not result: return result, subgroup or None
if subgroup:
return None, value
subgroup = group
result = cls.subgroup_lookup().get(subgroup)
return result, subgroup or None if subgroup:
return None, value
return cls.subgroup_lookup().get(group), group or None
@dataclass @dataclass
@@ -136,8 +135,7 @@ class Changelog:
first = False first = False
yield '\n<details><summary><h3>Changelog</h3></summary>\n' yield '\n<details><summary><h3>Changelog</h3></summary>\n'
group = groups[item] if group := groups[item]:
if group:
yield self.format_module(item.value, group) yield self.format_module(item.value, group)
if self._collapsible: if self._collapsible:
@@ -223,10 +221,10 @@ class Changelog:
return message if not sep else f'{message}{sep}{rest}' return message if not sep else f'{message}{sep}{rest}'
def _format_message_link(self, message, hash): def _format_message_link(self, message, commit_hash):
assert message or hash, 'Improperly defined commit message or override' assert message or commit_hash, 'Improperly defined commit message or override'
message = message if message else hash[:HASH_LENGTH] message = message if message else commit_hash[:HASH_LENGTH]
return f'[{message}]({self.repo_url}/commit/{hash})' if hash else message return f'[{message}]({self.repo_url}/commit/{commit_hash})' if commit_hash else message
def _format_issues(self, issues): def _format_issues(self, issues):
return ', '.join(f'[#{issue}]({self.repo_url}/issues/{issue})' for issue in issues) return ', '.join(f'[#{issue}]({self.repo_url}/issues/{issue})' for issue in issues)
@@ -253,7 +251,7 @@ class CommitRange:
''', re.VERBOSE | re.DOTALL) ''', re.VERBOSE | re.DOTALL)
EXTRACTOR_INDICATOR_RE = re.compile(r'(?:Fix|Add)\s+Extractors?', re.IGNORECASE) EXTRACTOR_INDICATOR_RE = re.compile(r'(?:Fix|Add)\s+Extractors?', re.IGNORECASE)
REVERT_RE = re.compile(r'(?:\[[^\]]+\]\s+)?(?i:Revert)\s+([\da-f]{40})') REVERT_RE = re.compile(r'(?:\[[^\]]+\]\s+)?(?i:Revert)\s+([\da-f]{40})')
FIXES_RE = re.compile(r'(?i:Fix(?:es)?(?:\s+bugs?)?(?:\s+in|\s+for)?|Revert|Improve)\s+([\da-f]{40})') FIXES_RE = re.compile(r'(?i:(?:bug\s*)?fix(?:es)?(?:\s+bugs?)?(?:\s+in|\s+for)?|Improve)\s+([\da-f]{40})')
UPSTREAM_MERGE_RE = re.compile(r'Update to ytdl-commit-([\da-f]+)') UPSTREAM_MERGE_RE = re.compile(r'Update to ytdl-commit-([\da-f]+)')
def __init__(self, start, end, default_author=None): def __init__(self, start, end, default_author=None):
@@ -287,11 +285,16 @@ class CommitRange:
short = next(lines) short = next(lines)
skip = short.startswith('Release ') or short == '[version] update' skip = short.startswith('Release ') or short == '[version] update'
fix_commitish = None
if match := self.FIXES_RE.search(short):
fix_commitish = match.group(1)
authors = [default_author] if default_author else [] authors = [default_author] if default_author else []
for line in iter(lambda: next(lines), self.COMMIT_SEPARATOR): for line in iter(lambda: next(lines), self.COMMIT_SEPARATOR):
match = self.AUTHOR_INDICATOR_RE.match(line) if match := self.AUTHOR_INDICATOR_RE.match(line):
if match:
authors = sorted(map(str.strip, line[match.end():].split(',')), key=str.casefold) authors = sorted(map(str.strip, line[match.end():].split(',')), key=str.casefold)
if not fix_commitish and (match := self.FIXES_RE.fullmatch(line)):
fix_commitish = match.group(1)
commit = Commit(commit_hash, short, authors) commit = Commit(commit_hash, short, authors)
if skip and (self._start or not i): if skip and (self._start or not i):
@@ -301,21 +304,17 @@ class CommitRange:
logger.debug(f'Reached Release commit, breaking: {commit}') logger.debug(f'Reached Release commit, breaking: {commit}')
break break
revert_match = self.REVERT_RE.fullmatch(commit.short) if match := self.REVERT_RE.fullmatch(commit.short):
if revert_match: reverts[match.group(1)] = commit
reverts[revert_match.group(1)] = commit
continue continue
fix_match = self.FIXES_RE.search(commit.short) if fix_commitish:
if fix_match: fixes[fix_commitish].append(commit)
commitish = fix_match.group(1)
fixes[commitish].append(commit)
commits[commit.hash] = commit commits[commit.hash] = commit
for commitish, revert_commit in reverts.items(): for commitish, revert_commit in reverts.items():
reverted = commits.pop(commitish, None) if reverted := commits.pop(commitish, None):
if reverted:
logger.debug(f'{commitish} fully reverted {reverted}') logger.debug(f'{commitish} fully reverted {reverted}')
else: else:
commits[revert_commit.hash] = revert_commit commits[revert_commit.hash] = revert_commit
@@ -356,7 +355,7 @@ class CommitRange:
logger.info(f'CHANGE {self._commits[commit.hash]} -> {commit}') logger.info(f'CHANGE {self._commits[commit.hash]} -> {commit}')
self._commits[commit.hash] = commit self._commits[commit.hash] = commit
self._commits = {key: value for key, value in reversed(self._commits.items())} self._commits = dict(reversed(self._commits.items()))
def groups(self): def groups(self):
group_dict = defaultdict(list) group_dict = defaultdict(list)
@@ -445,7 +444,31 @@ def get_new_contributors(contributors_path, commits):
return sorted(new_contributors, key=str.casefold) return sorted(new_contributors, key=str.casefold)
if __name__ == '__main__': def create_changelog(args):
logging.basicConfig(
datefmt='%Y-%m-%d %H-%M-%S', format='{asctime} | {levelname:<8} | {message}',
level=logging.WARNING - 10 * args.verbosity, style='{', stream=sys.stderr)
commits = CommitRange(None, args.commitish, args.default_author)
if not args.no_override:
if args.override_path.exists():
overrides = json.loads(read_file(args.override_path))
commits.apply_overrides(overrides)
else:
logger.warning(f'File {args.override_path.as_posix()} does not exist')
logger.info(f'Loaded {len(commits)} commits')
if new_contributors := get_new_contributors(args.contributors_path, commits):
if args.contributors:
write_file(args.contributors_path, '\n'.join(new_contributors) + '\n', mode='a')
logger.info(f'New contributors: {", ".join(new_contributors)}')
return Changelog(commits.groups(), args.repo, args.collapsible)
def create_parser():
import argparse import argparse
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(
@@ -477,27 +500,9 @@ if __name__ == '__main__':
parser.add_argument( parser.add_argument(
'--collapsible', action='store_true', '--collapsible', action='store_true',
help='make changelog collapsible (default: %(default)s)') help='make changelog collapsible (default: %(default)s)')
args = parser.parse_args()
logging.basicConfig( return parser
datefmt='%Y-%m-%d %H-%M-%S', format='{asctime} | {levelname:<8} | {message}',
level=logging.WARNING - 10 * args.verbosity, style='{', stream=sys.stderr)
commits = CommitRange(None, args.commitish, args.default_author)
if not args.no_override: if __name__ == '__main__':
if args.override_path.exists(): print(create_changelog(create_parser().parse_args()))
overrides = json.loads(read_file(args.override_path))
commits.apply_overrides(overrides)
else:
logger.warning(f'File {args.override_path.as_posix()} does not exist')
logger.info(f'Loaded {len(commits)} commits')
new_contributors = get_new_contributors(args.contributors_path, commits)
if new_contributors:
if args.contributors:
write_file(args.contributors_path, '\n'.join(new_contributors) + '\n', mode='a')
logger.info(f'New contributors: {", ".join(new_contributors)}')
print(Changelog(commits.groups(), args.repo, args.collapsible))
+1 -1
View File
@@ -8,7 +8,7 @@ def main():
return # This is unused in yt-dlp return # This is unused in yt-dlp
parser = optparse.OptionParser(usage='%prog INFILE OUTFILE') parser = optparse.OptionParser(usage='%prog INFILE OUTFILE')
options, args = parser.parse_args() _, args = parser.parse_args()
if len(args) != 2: if len(args) != 2:
parser.error('Expected an input and an output filename') parser.error('Expected an input and an output filename')
+20 -17
View File
@@ -11,11 +11,13 @@ import re
from devscripts.utils import get_filename_args, read_file, write_file from devscripts.utils import get_filename_args, read_file, write_file
VERBOSE_TMPL = ''' VERBOSE = '''
- type: checkboxes - type: checkboxes
id: verbose id: verbose
attributes: attributes:
label: Provide verbose output that clearly demonstrates the problem label: Provide verbose output that clearly demonstrates the problem
description: |
This is mandatory unless absolutely impossible to provide. If you are unable to provide the output, please explain why.
options: options:
- label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`) - label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
required: true required: true
@@ -32,14 +34,15 @@ VERBOSE_TMPL = '''
placeholder: | placeholder: |
[debug] Command-line config: ['-vU', 'https://www.youtube.com/watch?v=BaW_jenozKc'] [debug] Command-line config: ['-vU', 'https://www.youtube.com/watch?v=BaW_jenozKc']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8 [debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version nightly@... from yt-dlp/yt-dlp [b634ba742] (win_exe) [debug] yt-dlp version nightly@... from yt-dlp/yt-dlp-nightly-builds [1a176d874] (win_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0 [debug] Python 3.10.11 (CPython AMD64 64bit) - Windows-10-10.0.20348-SP0 (OpenSSL 1.1.1t 7 Feb 2023)
[debug] exe versions: ffmpeg N-106550-g072101bd52-20220410 (fdk,setts), ffprobe N-106624-g391ce570c8-20220415, phantomjs 2.1.1 [debug] exe versions: ffmpeg 7.0.2 (setts), ffprobe 7.0.2
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3 [debug] Optional libraries: Cryptodome-3.21.0, brotli-1.1.0, certifi-2024.08.30, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.40.1, urllib3-2.2.3, websockets-13.1
[debug] Proxy map: {} [debug] Proxy map: {}
[debug] Request Handlers: urllib, requests [debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1893 extractors [debug] Loaded 1838 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest [debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: nightly@... from yt-dlp/yt-dlp-nightly-builds
yt-dlp is up to date (nightly@... from yt-dlp/yt-dlp-nightly-builds) yt-dlp is up to date (nightly@... from yt-dlp/yt-dlp-nightly-builds)
[youtube] Extracting URL: https://www.youtube.com/watch?v=BaW_jenozKc [youtube] Extracting URL: https://www.youtube.com/watch?v=BaW_jenozKc
<more lines> <more lines>
@@ -49,20 +52,20 @@ VERBOSE_TMPL = '''
'''.strip() '''.strip()
NO_SKIP = ''' NO_SKIP = '''
- type: checkboxes - type: markdown
attributes: attributes:
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE value: |
description: Fill all fields even if you think it is irrelevant for the issue > [!IMPORTANT]
options: > Not providing the required (*) information or removing the template will result in your issue being closed and ignored.
- label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\\* field
required: true
'''.strip() '''.strip()
def main(): def main():
fields = {'no_skip': NO_SKIP} fields = {
fields['verbose'] = VERBOSE_TMPL % fields 'no_skip': NO_SKIP,
fields['verbose_optional'] = re.sub(r'(\n\s+validations:)?\n\s+required: true', '', fields['verbose']) 'verbose': VERBOSE,
'verbose_optional': re.sub(r'(\n\s+validations:)?\n\s+required: true', '', VERBOSE),
}
infile, outfile = get_filename_args(has_infile=True) infile, outfile = get_filename_args(has_infile=True)
write_file(outfile, read_file(infile) % fields) write_file(outfile, read_file(infile) % fields)
+9 -26
View File
@@ -2,7 +2,6 @@
# Allow direct execution # Allow direct execution
import os import os
import shutil
import sys import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
@@ -11,6 +10,9 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from inspect import getsource from inspect import getsource
from devscripts.utils import get_filename_args, read_file, write_file from devscripts.utils import get_filename_args, read_file, write_file
from yt_dlp.extractor import import_extractors
from yt_dlp.extractor.common import InfoExtractor, SearchInfoExtractor
from yt_dlp.globals import extractors
NO_ATTR = object() NO_ATTR = object()
STATIC_CLASS_PROPERTIES = [ STATIC_CLASS_PROPERTIES = [
@@ -34,17 +36,12 @@ MODULE_TEMPLATE = read_file('devscripts/lazy_load_template.py')
def main(): def main():
os.environ['YTDLP_NO_PLUGINS'] = 'true'
os.environ['YTDLP_NO_LAZY_EXTRACTORS'] = 'true'
lazy_extractors_filename = get_filename_args(default_outfile='yt_dlp/extractor/lazy_extractors.py') lazy_extractors_filename = get_filename_args(default_outfile='yt_dlp/extractor/lazy_extractors.py')
if os.path.exists(lazy_extractors_filename):
os.remove(lazy_extractors_filename)
_ALL_CLASSES = get_all_ies() # Must be before import import_extractors()
import yt_dlp.plugins
from yt_dlp.extractor.common import InfoExtractor, SearchInfoExtractor
# Filter out plugins
_ALL_CLASSES = [cls for cls in _ALL_CLASSES if not cls.__module__.startswith(f'{yt_dlp.plugins.PACKAGE_NAME}.')]
DummyInfoExtractor = type('InfoExtractor', (InfoExtractor,), {'IE_NAME': NO_ATTR}) DummyInfoExtractor = type('InfoExtractor', (InfoExtractor,), {'IE_NAME': NO_ATTR})
module_src = '\n'.join(( module_src = '\n'.join((
@@ -52,26 +49,12 @@ def main():
' _module = None', ' _module = None',
*extra_ie_code(DummyInfoExtractor), *extra_ie_code(DummyInfoExtractor),
'\nclass LazyLoadSearchExtractor(LazyLoadExtractor):\n pass\n', '\nclass LazyLoadSearchExtractor(LazyLoadExtractor):\n pass\n',
*build_ies(_ALL_CLASSES, (InfoExtractor, SearchInfoExtractor), DummyInfoExtractor), *build_ies(list(extractors.value.values()), (InfoExtractor, SearchInfoExtractor), DummyInfoExtractor),
)) ))
write_file(lazy_extractors_filename, f'{module_src}\n') write_file(lazy_extractors_filename, f'{module_src}\n')
def get_all_ies():
PLUGINS_DIRNAME = 'ytdlp_plugins'
BLOCKED_DIRNAME = f'{PLUGINS_DIRNAME}_blocked'
if os.path.exists(PLUGINS_DIRNAME):
# os.rename cannot be used, e.g. in Docker. See https://github.com/yt-dlp/yt-dlp/pull/4958
shutil.move(PLUGINS_DIRNAME, BLOCKED_DIRNAME)
try:
from yt_dlp.extractor.extractors import _ALL_CLASSES
finally:
if os.path.exists(BLOCKED_DIRNAME):
shutil.move(BLOCKED_DIRNAME, PLUGINS_DIRNAME)
return _ALL_CLASSES
def extra_ie_code(ie, base=None): def extra_ie_code(ie, base=None):
for var in STATIC_CLASS_PROPERTIES: for var in STATIC_CLASS_PROPERTIES:
val = getattr(ie, var) val = getattr(ie, var)
@@ -92,7 +75,7 @@ def build_ies(ies, bases, attr_base):
if ie in ies: if ie in ies:
names.append(ie.__name__) names.append(ie.__name__)
yield f'\n_ALL_CLASSES = [{", ".join(names)}]' yield '\n_CLASS_LOOKUP = {%s}' % ', '.join(f'{name!r}: {name}' for name in names)
def sort_ies(ies, ignored_bases): def sort_ies(ies, ignored_bases):
+6 -6
View File
@@ -51,7 +51,7 @@ PATCHES = (
), ),
( # Headings ( # Headings
r'(?m)^ (\w.+\n)( (?=\w))?', r'(?m)^ (\w.+\n)( (?=\w))?',
r'## \1' r'## \1',
), ),
( # Fixup `--date` formatting ( # Fixup `--date` formatting
rf'(?m)( --date DATE.+({delim}[^\[]+)*)\[.+({delim}.+)*$', rf'(?m)( --date DATE.+({delim}[^\[]+)*)\[.+({delim}.+)*$',
@@ -61,26 +61,26 @@ PATCHES = (
), ),
( # Do not split URLs ( # Do not split URLs
rf'({delim[:-1]})? (?P<label>\[\S+\] )?(?P<url>https?({delim})?:({delim})?/({delim})?/(({delim})?\S+)+)\s', rf'({delim[:-1]})? (?P<label>\[\S+\] )?(?P<url>https?({delim})?:({delim})?/({delim})?/(({delim})?\S+)+)\s',
lambda mobj: ''.join((delim, mobj.group('label') or '', re.sub(r'\s+', '', mobj.group('url')), '\n')) lambda mobj: ''.join((delim, mobj.group('label') or '', re.sub(r'\s+', '', mobj.group('url')), '\n')),
), ),
( # Do not split "words" ( # Do not split "words"
rf'(?m)({delim}\S+)+$', rf'(?m)({delim}\S+)+$',
lambda mobj: ''.join((delim, mobj.group(0).replace(delim, ''))) lambda mobj: ''.join((delim, mobj.group(0).replace(delim, ''))),
), ),
( # Allow overshooting last line ( # Allow overshooting last line
rf'(?m)^(?P<prev>.+)${delim}(?P<current>.+)$(?!{delim})', rf'(?m)^(?P<prev>.+)${delim}(?P<current>.+)$(?!{delim})',
lambda mobj: (mobj.group().replace(delim, ' ') lambda mobj: (mobj.group().replace(delim, ' ')
if len(mobj.group()) - len(delim) + 1 <= max_width + ALLOWED_OVERSHOOT if len(mobj.group()) - len(delim) + 1 <= max_width + ALLOWED_OVERSHOOT
else mobj.group()) else mobj.group()),
), ),
( # Avoid newline when a space is available b/w switch and description ( # Avoid newline when a space is available b/w switch and description
DISABLE_PATCH, # This creates issues with prepare_manpage DISABLE_PATCH, # This creates issues with prepare_manpage
r'(?m)^(\s{4}-.{%d})(%s)' % (switch_col_width - 6, delim), r'(?m)^(\s{4}-.{%d})(%s)' % (switch_col_width - 6, delim),
r'\1 ' r'\1 ',
), ),
( # Replace brackets with a Markdown link ( # Replace brackets with a Markdown link
r'SponsorBlock API \((http.+)\)', r'SponsorBlock API \((http.+)\)',
r'[SponsorBlock API](\1)' r'[SponsorBlock API](\1)',
), ),
) )
+12 -1
View File
@@ -10,10 +10,21 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from devscripts.utils import get_filename_args, write_file from devscripts.utils import get_filename_args, write_file
from yt_dlp.extractor import list_extractor_classes from yt_dlp.extractor import list_extractor_classes
TEMPLATE = '''\
# Supported sites
Below is a list of all extractors that are currently included with yt-dlp.
If a site is not listed here, it might still be supported by yt-dlp's embed extraction or generic extractor.
Not all sites listed here are guaranteed to work; websites are constantly changing and sometimes this breaks yt-dlp's support for them.
The only reliable way to check if a site is supported is to try it.
{ie_list}
'''
def main(): def main():
out = '\n'.join(ie.description() for ie in list_extractor_classes() if ie.IE_DESC is not False) out = '\n'.join(ie.description() for ie in list_extractor_classes() if ie.IE_DESC is not False)
write_file(get_filename_args(), f'# Supported sites\n{out}\n') write_file(get_filename_args(), TEMPLATE.format(ie_list=out))
if __name__ == '__main__': if __name__ == '__main__':
+26 -3
View File
@@ -24,7 +24,7 @@ PREFIX = r'''%yt-dlp(1)
# NAME # NAME
yt\-dlp \- A youtube-dl fork with additional features and patches yt\-dlp \- A feature\-rich command\-line audio/video downloader
# SYNOPSIS # SYNOPSIS
@@ -43,6 +43,27 @@ def filter_excluded_sections(readme):
'', readme) '', readme)
def _convert_code_blocks(readme):
current_code_block = None
for line in readme.splitlines(True):
if current_code_block:
if line == current_code_block:
current_code_block = None
yield '\n'
else:
yield f' {line}'
elif line.startswith('```'):
current_code_block = line.count('`') * '`' + '\n'
yield '\n'
else:
yield line
def convert_code_blocks(readme):
return ''.join(_convert_code_blocks(readme))
def move_sections(readme): def move_sections(readme):
MOVE_TAG_TEMPLATE = '<!-- MANPAGE: MOVE "%s" SECTION HERE -->' MOVE_TAG_TEMPLATE = '<!-- MANPAGE: MOVE "%s" SECTION HERE -->'
sections = re.findall(r'(?m)^%s$' % ( sections = re.findall(r'(?m)^%s$' % (
@@ -65,8 +86,10 @@ def move_sections(readme):
def filter_options(readme): def filter_options(readme):
section = re.search(r'(?sm)^# USAGE AND OPTIONS\n.+?(?=^# )', readme).group(0) section = re.search(r'(?sm)^# USAGE AND OPTIONS\n.+?(?=^# )', readme).group(0)
section_new = section.replace('*', R'\*')
options = '# OPTIONS\n' options = '# OPTIONS\n'
for line in section.split('\n')[1:]: for line in section_new.split('\n')[1:]:
mobj = re.fullmatch(r'''(?x) mobj = re.fullmatch(r'''(?x)
\s{4}(?P<opt>-(?:,\s|[^\s])+) \s{4}(?P<opt>-(?:,\s|[^\s])+)
(?:\s(?P<meta>(?:[^\s]|\s(?!\s))+))? (?:\s(?P<meta>(?:[^\s]|\s(?!\s))+))?
@@ -86,7 +109,7 @@ def filter_options(readme):
return readme.replace(section, options, 1) return readme.replace(section, options, 1)
TRANSFORM = compose_functions(filter_excluded_sections, move_sections, filter_options) TRANSFORM = compose_functions(filter_excluded_sections, convert_code_blocks, move_sections, filter_options)
def main(): def main():
-4
View File
@@ -1,4 +0,0 @@
@echo off
>&2 echo run_tests.bat is deprecated. Please use `devscripts/run_tests.py` instead
python %~dp0run_tests.py %~1
+15 -9
View File
@@ -4,6 +4,7 @@ import argparse
import functools import functools
import os import os
import re import re
import shlex
import subprocess import subprocess
import sys import sys
from pathlib import Path from pathlib import Path
@@ -15,29 +16,34 @@ fix_test_name = functools.partial(re.compile(r'IE(_all|_\d+)?$').sub, r'\1')
def parse_args(): def parse_args():
parser = argparse.ArgumentParser(description='Run selected yt-dlp tests') parser = argparse.ArgumentParser(description='Run selected yt-dlp tests')
parser.add_argument( parser.add_argument(
'test', help='a extractor tests, or one of "core" or "download"', nargs='*') 'test', help='an extractor test, test path, or one of "core" or "download"', nargs='*')
parser.add_argument( parser.add_argument(
'-k', help='run a test matching EXPRESSION. Same as "pytest -k"', metavar='EXPRESSION') '-k', help='run a test matching EXPRESSION. Same as "pytest -k"', metavar='EXPRESSION')
parser.add_argument(
'--pytest-args', help='arguments to passthrough to pytest')
return parser.parse_args() return parser.parse_args()
def run_tests(*tests, pattern=None, ci=False): def run_tests(*tests, pattern=None, ci=False):
run_core = 'core' in tests or (not pattern and not tests) # XXX: hatch uses `tests` if no arguments are passed
run_core = 'core' in tests or 'tests' in tests or (not pattern and not tests)
run_download = 'download' in tests run_download = 'download' in tests
tests = list(map(fix_test_name, tests))
arguments = ['pytest', '-Werror', '--tb=short'] pytest_args = args.pytest_args or os.getenv('HATCH_TEST_ARGS', '')
arguments = ['pytest', '-Werror', '--tb=short', *shlex.split(pytest_args)]
if ci: if ci:
arguments.append('--color=yes') arguments.append('--color=yes')
if pattern:
arguments.extend(['-k', pattern])
if run_core: if run_core:
arguments.extend(['-m', 'not download']) arguments.extend(['-m', 'not download'])
elif run_download: elif run_download:
arguments.extend(['-m', 'download']) arguments.extend(['-m', 'download'])
elif pattern:
arguments.extend(['-k', pattern])
else: else:
arguments.extend( arguments.extend(
f'test/test_download.py::TestDownload::test_{test}' for test in tests) test if '/' in test
else f'test/test_download.py::TestDownload::test_{fix_test_name(test)}'
for test in tests)
print(f'Running {arguments}', flush=True) print(f'Running {arguments}', flush=True)
try: try:
@@ -46,13 +52,13 @@ def run_tests(*tests, pattern=None, ci=False):
pass pass
arguments = [sys.executable, '-Werror', '-m', 'unittest'] arguments = [sys.executable, '-Werror', '-m', 'unittest']
if pattern:
arguments.extend(['-k', pattern])
if run_core: if run_core:
print('"pytest" needs to be installed to run core tests', file=sys.stderr, flush=True) print('"pytest" needs to be installed to run core tests', file=sys.stderr, flush=True)
return 1 return 1
elif run_download: elif run_download:
arguments.append('test.test_download') arguments.append('test.test_download')
elif pattern:
arguments.extend(['-k', pattern])
else: else:
arguments.extend( arguments.extend(
f'test.test_download.TestDownload.test_{test}' for test in tests) f'test.test_download.TestDownload.test_{test}' for test in tests)
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env sh
>&2 echo 'run_tests.sh is deprecated. Please use `devscripts/run_tests.py` instead'
python3 devscripts/run_tests.py "$1"
+1 -1
View File
@@ -30,7 +30,7 @@ def property_setter(name, value):
opts = parse_options() opts = parse_options()
transform = compose_functions( transform = compose_functions(
property_setter('VARIANT', opts.variant), property_setter('VARIANT', opts.variant),
property_setter('UPDATE_HINT', opts.update_message) property_setter('UPDATE_HINT', opts.update_message),
) )
write_file(VERSION_FILE, transform(read_file(VERSION_FILE))) write_file(VERSION_FILE, transform(read_file(VERSION_FILE)))
+157
View File
@@ -0,0 +1,157 @@
# Allow direct execution
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import datetime as dt
import json
from devscripts.utils import calculate_version
STABLE_REPOSITORY = 'yt-dlp/yt-dlp'
def setup_variables(environment):
"""
`environment` must contain these keys:
REPOSITORY, INPUTS, PROCESSED,
PUSH_VERSION_COMMIT, PYPI_PROJECT,
SOURCE_PYPI_PROJECT, SOURCE_PYPI_SUFFIX,
TARGET_PYPI_PROJECT, TARGET_PYPI_SUFFIX,
SOURCE_ARCHIVE_REPO, TARGET_ARCHIVE_REPO,
HAS_SOURCE_ARCHIVE_REPO_TOKEN,
HAS_TARGET_ARCHIVE_REPO_TOKEN,
HAS_ARCHIVE_REPO_TOKEN
`INPUTS` must contain these keys:
prerelease
`PROCESSED` must contain these keys:
source_repo, source_tag,
target_repo, target_tag
"""
REPOSITORY = environment['REPOSITORY']
INPUTS = json.loads(environment['INPUTS'])
PROCESSED = json.loads(environment['PROCESSED'])
source_channel = None
does_not_have_needed_token = False
target_repo_token = None
pypi_project = None
pypi_suffix = None
source_repo = PROCESSED['source_repo']
source_tag = PROCESSED['source_tag']
if source_repo == 'stable':
source_repo = STABLE_REPOSITORY
if not source_repo:
source_repo = REPOSITORY
elif environment['SOURCE_ARCHIVE_REPO']:
source_channel = environment['SOURCE_ARCHIVE_REPO']
elif not source_tag and '/' not in source_repo:
source_tag = source_repo
source_repo = REPOSITORY
resolved_source = source_repo
if source_tag:
resolved_source = f'{resolved_source}@{source_tag}'
elif source_repo == STABLE_REPOSITORY:
resolved_source = 'stable'
revision = None
if INPUTS['prerelease'] or not environment['PUSH_VERSION_COMMIT']:
revision = dt.datetime.now(tz=dt.timezone.utc).strftime('%H%M%S')
version = calculate_version(INPUTS.get('version') or revision)
target_repo = PROCESSED['target_repo']
target_tag = PROCESSED['target_tag']
if target_repo:
if target_repo == 'stable':
target_repo = STABLE_REPOSITORY
if not target_tag:
if target_repo == STABLE_REPOSITORY:
target_tag = version
elif environment['TARGET_ARCHIVE_REPO']:
target_tag = source_tag or version
else:
target_tag = target_repo
target_repo = REPOSITORY
if target_repo != REPOSITORY:
target_repo = environment['TARGET_ARCHIVE_REPO']
target_repo_token = f'{PROCESSED["target_repo"].upper()}_ARCHIVE_REPO_TOKEN'
if not json.loads(environment['HAS_TARGET_ARCHIVE_REPO_TOKEN']):
does_not_have_needed_token = True
pypi_project = environment['TARGET_PYPI_PROJECT'] or None
pypi_suffix = environment['TARGET_PYPI_SUFFIX'] or None
else:
target_tag = source_tag or version
if source_channel:
target_repo = source_channel
target_repo_token = f'{PROCESSED["source_repo"].upper()}_ARCHIVE_REPO_TOKEN'
if not json.loads(environment['HAS_SOURCE_ARCHIVE_REPO_TOKEN']):
does_not_have_needed_token = True
pypi_project = environment['SOURCE_PYPI_PROJECT'] or None
pypi_suffix = environment['SOURCE_PYPI_SUFFIX'] or None
else:
target_repo = REPOSITORY
if does_not_have_needed_token:
if not json.loads(environment['HAS_ARCHIVE_REPO_TOKEN']):
print(f'::error::Repository access secret {target_repo_token} not found')
return None
target_repo_token = 'ARCHIVE_REPO_TOKEN'
if target_repo == REPOSITORY and not INPUTS['prerelease']:
pypi_project = environment['PYPI_PROJECT'] or None
return {
'channel': resolved_source,
'version': version,
'target_repo': target_repo,
'target_repo_token': target_repo_token,
'target_tag': target_tag,
'pypi_project': pypi_project,
'pypi_suffix': pypi_suffix,
}
def process_inputs(inputs):
outputs = {}
for key in ('source', 'target'):
repo, _, tag = inputs.get(key, '').partition('@')
outputs[f'{key}_repo'] = repo
outputs[f'{key}_tag'] = tag
return outputs
if __name__ == '__main__':
if not os.getenv('GITHUB_OUTPUT'):
print('This script is only intended for use with GitHub Actions', file=sys.stderr)
sys.exit(1)
if 'process_inputs' in sys.argv:
inputs = json.loads(os.environ['INPUTS'])
print('::group::Inputs')
print(json.dumps(inputs, indent=2))
print('::endgroup::')
outputs = process_inputs(inputs)
print('::group::Processed')
print(json.dumps(outputs, indent=2))
print('::endgroup::')
with open(os.environ['GITHUB_OUTPUT'], 'a') as f:
f.write('\n'.join(f'{key}={value}' for key, value in outputs.items()))
sys.exit(0)
outputs = setup_variables(dict(os.environ))
if not outputs:
sys.exit(1)
print('::group::Output variables')
print(json.dumps(outputs, indent=2))
print('::endgroup::')
with open(os.environ['GITHUB_OUTPUT'], 'a') as f:
f.write('\n'.join(f'{key}={value or ""}' for key, value in outputs.items()))
+324
View File
@@ -0,0 +1,324 @@
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import datetime as dt
import json
from devscripts.setup_variables import STABLE_REPOSITORY, process_inputs, setup_variables
from devscripts.utils import calculate_version
def _test(github_repository, note, repo_vars, repo_secrets, inputs, expected=None, ignore_revision=False):
inp = inputs.copy()
inp.setdefault('linux_armv7l', True)
inp.setdefault('prerelease', False)
processed = process_inputs(inp)
source_repo = processed['source_repo'].upper()
target_repo = processed['target_repo'].upper()
variables = {k.upper(): v for k, v in repo_vars.items()}
secrets = {k.upper(): v for k, v in repo_secrets.items()}
env = {
# Keep this in sync with prepare.setup_variables in release.yml
'INPUTS': json.dumps(inp),
'PROCESSED': json.dumps(processed),
'REPOSITORY': github_repository,
'PUSH_VERSION_COMMIT': variables.get('PUSH_VERSION_COMMIT') or '',
'PYPI_PROJECT': variables.get('PYPI_PROJECT') or '',
'SOURCE_PYPI_PROJECT': variables.get(f'{source_repo}_PYPI_PROJECT') or '',
'SOURCE_PYPI_SUFFIX': variables.get(f'{source_repo}_PYPI_SUFFIX') or '',
'TARGET_PYPI_PROJECT': variables.get(f'{target_repo}_PYPI_PROJECT') or '',
'TARGET_PYPI_SUFFIX': variables.get(f'{target_repo}_PYPI_SUFFIX') or '',
'SOURCE_ARCHIVE_REPO': variables.get(f'{source_repo}_ARCHIVE_REPO') or '',
'TARGET_ARCHIVE_REPO': variables.get(f'{target_repo}_ARCHIVE_REPO') or '',
'HAS_SOURCE_ARCHIVE_REPO_TOKEN': json.dumps(bool(secrets.get(f'{source_repo}_ARCHIVE_REPO_TOKEN'))),
'HAS_TARGET_ARCHIVE_REPO_TOKEN': json.dumps(bool(secrets.get(f'{target_repo}_ARCHIVE_REPO_TOKEN'))),
'HAS_ARCHIVE_REPO_TOKEN': json.dumps(bool(secrets.get('ARCHIVE_REPO_TOKEN'))),
}
result = setup_variables(env)
if not expected:
print(' {\n' + '\n'.join(f' {k!r}: {v!r},' for k, v in result.items()) + '\n }')
return
exp = expected.copy()
if ignore_revision:
assert len(result['version']) == len(exp['version']), f'revision missing: {github_repository} {note}'
version_is_tag = result['version'] == result['target_tag']
for dct in (result, exp):
dct['version'] = '.'.join(dct['version'].split('.')[:3])
if version_is_tag:
dct['target_tag'] = dct['version']
assert result == exp, f'unexpected result: {github_repository} {note}'
def test_setup_variables():
DEFAULT_VERSION_WITH_REVISION = dt.datetime.now(tz=dt.timezone.utc).strftime('%Y.%m.%d.%H%M%S')
DEFAULT_VERSION = calculate_version()
BASE_REPO_VARS = {
'MASTER_ARCHIVE_REPO': 'yt-dlp/yt-dlp-master-builds',
'NIGHTLY_ARCHIVE_REPO': 'yt-dlp/yt-dlp-nightly-builds',
'NIGHTLY_PYPI_PROJECT': 'yt-dlp',
'NIGHTLY_PYPI_SUFFIX': 'dev',
'PUSH_VERSION_COMMIT': '1',
'PYPI_PROJECT': 'yt-dlp',
}
BASE_REPO_SECRETS = {
'ARCHIVE_REPO_TOKEN': '1',
}
FORK_REPOSITORY = 'fork/yt-dlp'
FORK_ORG = FORK_REPOSITORY.partition('/')[0]
_test(
STABLE_REPOSITORY, 'official vars/secrets, stable',
BASE_REPO_VARS, BASE_REPO_SECRETS, {}, {
'channel': 'stable',
'version': DEFAULT_VERSION,
'target_repo': STABLE_REPOSITORY,
'target_repo_token': None,
'target_tag': DEFAULT_VERSION,
'pypi_project': 'yt-dlp',
'pypi_suffix': None,
})
_test(
STABLE_REPOSITORY, 'official vars/secrets, nightly (w/o target)',
BASE_REPO_VARS, BASE_REPO_SECRETS, {
'source': 'nightly',
'prerelease': True,
}, {
'channel': 'nightly',
'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': 'yt-dlp/yt-dlp-nightly-builds',
'target_repo_token': 'ARCHIVE_REPO_TOKEN',
'target_tag': DEFAULT_VERSION_WITH_REVISION,
'pypi_project': 'yt-dlp',
'pypi_suffix': 'dev',
}, ignore_revision=True)
_test(
STABLE_REPOSITORY, 'official vars/secrets, nightly',
BASE_REPO_VARS, BASE_REPO_SECRETS, {
'source': 'nightly',
'target': 'nightly',
'prerelease': True,
}, {
'channel': 'nightly',
'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': 'yt-dlp/yt-dlp-nightly-builds',
'target_repo_token': 'ARCHIVE_REPO_TOKEN',
'target_tag': DEFAULT_VERSION_WITH_REVISION,
'pypi_project': 'yt-dlp',
'pypi_suffix': 'dev',
}, ignore_revision=True)
_test(
STABLE_REPOSITORY, 'official vars/secrets, master (w/o target)',
BASE_REPO_VARS, BASE_REPO_SECRETS, {
'source': 'master',
'prerelease': True,
}, {
'channel': 'master',
'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': 'yt-dlp/yt-dlp-master-builds',
'target_repo_token': 'ARCHIVE_REPO_TOKEN',
'target_tag': DEFAULT_VERSION_WITH_REVISION,
'pypi_project': None,
'pypi_suffix': None,
}, ignore_revision=True)
_test(
STABLE_REPOSITORY, 'official vars/secrets, master',
BASE_REPO_VARS, BASE_REPO_SECRETS, {
'source': 'master',
'target': 'master',
'prerelease': True,
}, {
'channel': 'master',
'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': 'yt-dlp/yt-dlp-master-builds',
'target_repo_token': 'ARCHIVE_REPO_TOKEN',
'target_tag': DEFAULT_VERSION_WITH_REVISION,
'pypi_project': None,
'pypi_suffix': None,
}, ignore_revision=True)
_test(
STABLE_REPOSITORY, 'official vars/secrets, special tag, updates to stable',
BASE_REPO_VARS, BASE_REPO_SECRETS, {
'target': f'{STABLE_REPOSITORY}@experimental',
'prerelease': True,
}, {
'channel': 'stable',
'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': STABLE_REPOSITORY,
'target_repo_token': None,
'target_tag': 'experimental',
'pypi_project': None,
'pypi_suffix': None,
}, ignore_revision=True)
_test(
STABLE_REPOSITORY, 'official vars/secrets, special tag, "stable" as target repo',
BASE_REPO_VARS, BASE_REPO_SECRETS, {
'target': 'stable@experimental',
'prerelease': True,
}, {
'channel': 'stable',
'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': STABLE_REPOSITORY,
'target_repo_token': None,
'target_tag': 'experimental',
'pypi_project': None,
'pypi_suffix': None,
}, ignore_revision=True)
_test(
FORK_REPOSITORY, 'fork w/o vars/secrets, stable',
{}, {}, {}, {
'channel': FORK_REPOSITORY,
'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': FORK_REPOSITORY,
'target_repo_token': None,
'target_tag': DEFAULT_VERSION_WITH_REVISION,
'pypi_project': None,
'pypi_suffix': None,
}, ignore_revision=True)
_test(
FORK_REPOSITORY, 'fork w/o vars/secrets, prerelease',
{}, {}, {'prerelease': True}, {
'channel': FORK_REPOSITORY,
'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': FORK_REPOSITORY,
'target_repo_token': None,
'target_tag': DEFAULT_VERSION_WITH_REVISION,
'pypi_project': None,
'pypi_suffix': None,
}, ignore_revision=True)
_test(
FORK_REPOSITORY, 'fork w/o vars/secrets, nightly',
{}, {}, {
'prerelease': True,
'source': 'nightly',
'target': 'nightly',
}, {
'channel': f'{FORK_REPOSITORY}@nightly',
'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': FORK_REPOSITORY,
'target_repo_token': None,
'target_tag': 'nightly',
'pypi_project': None,
'pypi_suffix': None,
}, ignore_revision=True)
_test(
FORK_REPOSITORY, 'fork w/o vars/secrets, master',
{}, {}, {
'prerelease': True,
'source': 'master',
'target': 'master',
}, {
'channel': f'{FORK_REPOSITORY}@master',
'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': FORK_REPOSITORY,
'target_repo_token': None,
'target_tag': 'master',
'pypi_project': None,
'pypi_suffix': None,
}, ignore_revision=True)
_test(
FORK_REPOSITORY, 'fork w/o vars/secrets, revision',
{}, {}, {'version': '123'}, {
'channel': FORK_REPOSITORY,
'version': f'{DEFAULT_VERSION[:10]}.123',
'target_repo': FORK_REPOSITORY,
'target_repo_token': None,
'target_tag': f'{DEFAULT_VERSION[:10]}.123',
'pypi_project': None,
'pypi_suffix': None,
})
_test(
FORK_REPOSITORY, 'fork w/ PUSH_VERSION_COMMIT, stable',
{'PUSH_VERSION_COMMIT': '1'}, {}, {}, {
'channel': FORK_REPOSITORY,
'version': DEFAULT_VERSION,
'target_repo': FORK_REPOSITORY,
'target_repo_token': None,
'target_tag': DEFAULT_VERSION,
'pypi_project': None,
'pypi_suffix': None,
})
_test(
FORK_REPOSITORY, 'fork w/ PUSH_VERSION_COMMIT, prerelease',
{'PUSH_VERSION_COMMIT': '1'}, {}, {'prerelease': True}, {
'channel': FORK_REPOSITORY,
'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': FORK_REPOSITORY,
'target_repo_token': None,
'target_tag': DEFAULT_VERSION_WITH_REVISION,
'pypi_project': None,
'pypi_suffix': None,
}, ignore_revision=True)
_test(
FORK_REPOSITORY, 'fork w/NIGHTLY_ARCHIVE_REPO_TOKEN, nightly', {
'NIGHTLY_ARCHIVE_REPO': f'{FORK_ORG}/yt-dlp-nightly-builds',
'PYPI_PROJECT': 'yt-dlp-test',
}, {
'NIGHTLY_ARCHIVE_REPO_TOKEN': '1',
}, {
'source': f'{FORK_ORG}/yt-dlp-nightly-builds',
'target': 'nightly',
'prerelease': True,
}, {
'channel': f'{FORK_ORG}/yt-dlp-nightly-builds',
'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': f'{FORK_ORG}/yt-dlp-nightly-builds',
'target_repo_token': 'NIGHTLY_ARCHIVE_REPO_TOKEN',
'target_tag': DEFAULT_VERSION_WITH_REVISION,
'pypi_project': None,
'pypi_suffix': None,
}, ignore_revision=True)
_test(
FORK_REPOSITORY, 'fork w/MASTER_ARCHIVE_REPO_TOKEN, master', {
'MASTER_ARCHIVE_REPO': f'{FORK_ORG}/yt-dlp-master-builds',
'MASTER_PYPI_PROJECT': 'yt-dlp-test',
'MASTER_PYPI_SUFFIX': 'dev',
}, {
'MASTER_ARCHIVE_REPO_TOKEN': '1',
}, {
'source': f'{FORK_ORG}/yt-dlp-master-builds',
'target': 'master',
'prerelease': True,
}, {
'channel': f'{FORK_ORG}/yt-dlp-master-builds',
'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': f'{FORK_ORG}/yt-dlp-master-builds',
'target_repo_token': 'MASTER_ARCHIVE_REPO_TOKEN',
'target_tag': DEFAULT_VERSION_WITH_REVISION,
'pypi_project': 'yt-dlp-test',
'pypi_suffix': 'dev',
}, ignore_revision=True)
_test(
FORK_REPOSITORY, 'fork, non-numeric tag',
{}, {}, {'source': 'experimental'}, {
'channel': f'{FORK_REPOSITORY}@experimental',
'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': FORK_REPOSITORY,
'target_repo_token': None,
'target_tag': 'experimental',
'pypi_project': None,
'pypi_suffix': None,
}, ignore_revision=True)
_test(
FORK_REPOSITORY, 'fork, non-numeric tag, updates to stable',
{}, {}, {
'prerelease': True,
'source': 'stable',
'target': 'experimental',
}, {
'channel': 'stable',
'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': FORK_REPOSITORY,
'target_repo_token': None,
'target_tag': 'experimental',
'pypi_project': None,
'pypi_suffix': None,
}, ignore_revision=True)
+5 -5
View File
@@ -11,7 +11,7 @@ IMPORTANT: INVALID FILES OR MULTILINE STRINGS ARE NOT SUPPORTED!
from __future__ import annotations from __future__ import annotations
import datetime import datetime as dt
import json import json
import re import re
@@ -115,9 +115,9 @@ def parse_value(data: str, index: int):
for func in [ for func in [
int, int,
float, float,
datetime.time.fromisoformat, dt.time.fromisoformat,
datetime.date.fromisoformat, dt.date.fromisoformat,
datetime.datetime.fromisoformat, dt.datetime.fromisoformat,
{'true': True, 'false': False}.get, {'true': True, 'false': False}.get,
]: ]:
try: try:
@@ -179,7 +179,7 @@ def main():
data = file.read() data = file.read()
def default(obj): def default(obj):
if isinstance(obj, (datetime.date, datetime.time, datetime.datetime)): if isinstance(obj, (dt.date, dt.time, dt.datetime)):
return obj.isoformat() return obj.isoformat()
print(json.dumps(parse_toml(data), default=default)) print(json.dumps(parse_toml(data), default=default))
+2 -19
View File
@@ -10,23 +10,8 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import argparse import argparse
import contextlib import contextlib
import sys import sys
from datetime import datetime, timezone
from devscripts.utils import read_version, run_process, write_file from devscripts.utils import calculate_version, run_process, write_file
def get_new_version(version, revision):
if not version:
version = datetime.now(timezone.utc).strftime('%Y.%m.%d')
if revision:
assert revision.isdecimal(), 'Revision must be a number'
else:
old_version = read_version().split('.')
if version.split('.') == old_version[:3]:
revision = str(int((old_version + [0])[3]) + 1)
return f'{version}.{revision}' if revision else version
def get_git_head(): def get_git_head():
@@ -72,9 +57,7 @@ if __name__ == '__main__':
args = parser.parse_args() args = parser.parse_args()
git_head = get_git_head() git_head = get_git_head()
version = ( version = calculate_version(args.version)
args.version if args.version and '.' in args.version
else get_new_version(None, args.version))
write_file(args.output, VERSION_TEMPLATE.format( write_file(args.output, VERSION_TEMPLATE.format(
version=version, git_head=git_head, channel=args.channel, origin=args.origin, version=version, git_head=git_head, channel=args.channel, origin=args.origin,
package_version=f'{version}{args.suffix}')) package_version=f'{version}{args.suffix}'))
+28
View File
@@ -0,0 +1,28 @@
#!/usr/bin/env python3
# Allow direct execution
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from pathlib import Path
from devscripts.make_changelog import create_changelog, create_parser
from devscripts.utils import read_file, read_version, write_file
# Always run after devscripts/update-version.py, and run before `make doc|pypi-files|tar|all`
if __name__ == '__main__':
parser = create_parser()
parser.description = 'Update an existing changelog file with an entry for a new release'
parser.add_argument(
'--changelog-path', type=Path, default=Path(__file__).parent.parent / 'Changelog.md',
help='path to the Changelog file')
args = parser.parse_args()
header, sep, changelog = read_file(args.changelog_path).partition('\n### ')
current_version = read_version()
if current_version != changelog.splitlines()[0]:
new_entry = create_changelog(args)
write_file(args.changelog_path, f'{header}{sep}{current_version}\n{new_entry}\n{sep}{changelog}')
+19
View File
@@ -1,5 +1,7 @@
import argparse import argparse
import datetime as dt
import functools import functools
import re
import subprocess import subprocess
@@ -20,6 +22,23 @@ def read_version(fname='yt_dlp/version.py', varname='__version__'):
return items[varname] return items[varname]
def calculate_version(version=None, fname='yt_dlp/version.py'):
if version and '.' in version:
return version
revision = version
version = dt.datetime.now(dt.timezone.utc).strftime('%Y.%m.%d')
if revision:
assert re.fullmatch(r'[0-9]+', revision), 'Revision must be numeric'
else:
old_version = read_version(fname=fname).split('.')
if version.split('.') == old_version[:3]:
revision = str(int(([*old_version, 0])[3]) + 1)
return f'{version}.{revision}' if revision else version
def get_filename_args(has_infile=False, default_outfile=None): def get_filename_args(has_infile=False, default_outfile=None):
parser = argparse.ArgumentParser() parser = argparse.ArgumentParser()
if has_infile: if has_infile:
+8 -8
View File
@@ -9,15 +9,15 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import yt_dlp import yt_dlp
ZSH_COMPLETION_FILE = "completions/zsh/_yt-dlp" ZSH_COMPLETION_FILE = 'completions/zsh/_yt-dlp'
ZSH_COMPLETION_TEMPLATE = "devscripts/zsh-completion.in" ZSH_COMPLETION_TEMPLATE = 'devscripts/zsh-completion.in'
def build_completion(opt_parser): def build_completion(opt_parser):
opts = [opt for group in opt_parser.option_groups opts = [opt for group in opt_parser.option_groups
for opt in group.option_list] for opt in group.option_list]
opts_file = [opt for opt in opts if opt.metavar == "FILE"] opts_file = [opt for opt in opts if opt.metavar == 'FILE']
opts_dir = [opt for opt in opts if opt.metavar == "DIR"] opts_dir = [opt for opt in opts if opt.metavar == 'DIR']
fileopts = [] fileopts = []
for opt in opts_file: for opt in opts_file:
@@ -38,11 +38,11 @@ def build_completion(opt_parser):
with open(ZSH_COMPLETION_TEMPLATE) as f: with open(ZSH_COMPLETION_TEMPLATE) as f:
template = f.read() template = f.read()
template = template.replace("{{fileopts}}", "|".join(fileopts)) template = template.replace('{{fileopts}}', '|'.join(fileopts))
template = template.replace("{{diropts}}", "|".join(diropts)) template = template.replace('{{diropts}}', '|'.join(diropts))
template = template.replace("{{flags}}", " ".join(flags)) template = template.replace('{{flags}}', ' '.join(flags))
with open(ZSH_COMPLETION_FILE, "w") as f: with open(ZSH_COMPLETION_FILE, 'w') as f:
f.write(template) f.write(template)
-17
View File
@@ -1,17 +0,0 @@
#!/usr/bin/env python3
# Allow execution from anywhere
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import warnings
from bundle.pyinstaller import main
warnings.warn(DeprecationWarning('`pyinst.py` is deprecated and will be removed in a future version. '
'Use `bundle.pyinstaller` instead'))
if __name__ == '__main__':
main()
+302 -22
View File
@@ -1,5 +1,5 @@
[build-system] [build-system]
requires = ["hatchling"] requires = ["hatchling>=1.27.0"]
build-backend = "hatchling.build" build-backend = "hatchling.build"
[project] [project]
@@ -9,67 +9,82 @@ maintainers = [
{name = "Grub4K", email = "contact@grub4k.xyz"}, {name = "Grub4K", email = "contact@grub4k.xyz"},
{name = "bashonly", email = "bashonly@protonmail.com"}, {name = "bashonly", email = "bashonly@protonmail.com"},
{name = "coletdjnz", email = "coletdjnz@protonmail.com"}, {name = "coletdjnz", email = "coletdjnz@protonmail.com"},
{name = "sepro", email = "sepro@sepr0.com"},
] ]
description = "A youtube-dl fork with additional features and patches" description = "A feature-rich command-line audio/video downloader"
readme = "README.md" readme = "README.md"
requires-python = ">=3.8" requires-python = ">=3.9"
keywords = [ keywords = [
"cli",
"downloader",
"youtube-dl", "youtube-dl",
"video-downloader",
"youtube-downloader", "youtube-downloader",
"sponsorblock", "sponsorblock",
"youtube-dlc",
"yt-dlp", "yt-dlp",
] ]
license = {file = "LICENSE"} license = "Unlicense"
license-files = ["LICENSE"]
classifiers = [ classifiers = [
"Topic :: Multimedia :: Video", "Topic :: Multimedia :: Video",
"Development Status :: 5 - Production/Stable", "Development Status :: 5 - Production/Stable",
"Environment :: Console", "Environment :: Console",
"Programming Language :: Python", "Programming Language :: Python",
"Programming Language :: Python :: 3 :: Only", "Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9", "Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
"Programming Language :: Python :: Implementation", "Programming Language :: Python :: Implementation",
"Programming Language :: Python :: Implementation :: CPython", "Programming Language :: Python :: Implementation :: CPython",
"Programming Language :: Python :: Implementation :: PyPy", "Programming Language :: Python :: Implementation :: PyPy",
"License :: OSI Approved :: The Unlicense (Unlicense)",
"Operating System :: OS Independent", "Operating System :: OS Independent",
] ]
dynamic = ["version"] dynamic = ["version"]
dependencies = [ dependencies = []
[project.optional-dependencies]
default = [
"brotli; implementation_name=='cpython'", "brotli; implementation_name=='cpython'",
"brotlicffi; implementation_name!='cpython'", "brotlicffi; implementation_name!='cpython'",
"certifi", "certifi",
"mutagen", "mutagen",
"pycryptodomex", "pycryptodomex",
"requests>=2.31.0,<3", "requests>=2.32.2,<3",
"urllib3>=1.26.17,<3", "urllib3>=2.0.2,<3",
"websockets>=12.0", "websockets>=13.0",
]
curl-cffi = [
"curl-cffi>=0.5.10,!=0.6.*,!=0.7.*,!=0.8.*,!=0.9.*,<0.14; implementation_name=='cpython'",
] ]
[project.optional-dependencies]
default = []
secretstorage = [ secretstorage = [
"cffi", "cffi",
"secretstorage", "secretstorage",
] ]
build = [ build = [
"build", "build",
"hatchling", "hatchling>=1.27.0",
"pip", "pip",
"setuptools>=71.0.2,<81", # See https://github.com/pyinstaller/pyinstaller/issues/9149
"wheel", "wheel",
] ]
dev = [ dev = [
"flake8", "pre-commit",
"isort", "yt-dlp[static-analysis]",
"pytest", "yt-dlp[test]",
]
static-analysis = [
"autopep8~=2.0",
"ruff~=0.13.0",
]
test = [
"pytest~=8.1",
"pytest-rerunfailures~=14.0",
]
pyinstaller = [
"pyinstaller>=6.13.0", # Windows temp cleanup fixed in 6.13.0
] ]
pyinstaller = ["pyinstaller>=6.3"]
py2exe = ["py2exe>=0.12"]
[project.urls] [project.urls]
Documentation = "https://github.com/yt-dlp/yt-dlp#readme" Documentation = "https://github.com/yt-dlp/yt-dlp#readme"
@@ -93,7 +108,6 @@ include = [
"/LICENSE", # included as license "/LICENSE", # included as license
"/pyproject.toml", # included by default "/pyproject.toml", # included by default
"/README.md", # included as readme "/README.md", # included as readme
"/setup.cfg",
"/supportedsites.md", "/supportedsites.md",
] ]
artifacts = [ artifacts = [
@@ -118,3 +132,269 @@ artifacts = ["/yt_dlp/extractor/lazy_extractors.py"]
[tool.hatch.version] [tool.hatch.version]
path = "yt_dlp/version.py" path = "yt_dlp/version.py"
pattern = "_pkg_version = '(?P<version>[^']+)'" pattern = "_pkg_version = '(?P<version>[^']+)'"
[tool.hatch.envs.default]
features = ["curl-cffi", "default"]
dependencies = ["pre-commit"]
path = ".venv"
installer = "uv"
[tool.hatch.envs.default.scripts]
setup = "pre-commit install --config .pre-commit-hatch.yaml"
yt-dlp = "python -Werror -Xdev -m yt_dlp {args}"
[tool.hatch.envs.hatch-static-analysis]
detached = true
features = ["static-analysis"]
dependencies = [] # override hatch ruff version
config-path = "pyproject.toml"
[tool.hatch.envs.hatch-static-analysis.scripts]
format-check = "autopep8 --diff {args:.}"
format-fix = "autopep8 --in-place {args:.}"
lint-check = "ruff check {args:.}"
lint-fix = "ruff check --fix {args:.}"
[tool.hatch.envs.hatch-test]
features = ["test"]
dependencies = [
"pytest-randomly~=3.15",
"pytest-xdist[psutil]~=3.5",
]
[tool.hatch.envs.hatch-test.scripts]
run = "python -m devscripts.run_tests {args}"
run-cov = "echo Code coverage not implemented && exit 1"
[[tool.hatch.envs.hatch-test.matrix]]
python = [
"3.9",
"3.10",
"3.11",
"3.12",
"3.13",
"3.14",
"pypy3.11",
]
[tool.ruff]
line-length = 120
[tool.ruff.lint]
ignore = [
"E402", # module-import-not-at-top-of-file
"E501", # line-too-long
"E731", # lambda-assignment
"E741", # ambiguous-variable-name
"UP031", # printf-string-formatting
"UP036", # outdated-version-block
"B006", # mutable-argument-default
"B008", # function-call-in-default-argument
"B011", # assert-false
"B017", # assert-raises-exception
"B023", # function-uses-loop-variable (false positives)
"B028", # no-explicit-stacklevel
"B904", # raise-without-from-inside-except
"A005", # stdlib-module-shadowing
"C401", # unnecessary-generator-set
"C402", # unnecessary-generator-dict
"PIE790", # unnecessary-placeholder
"SIM102", # collapsible-if
"SIM108", # if-else-block-instead-of-if-exp
"SIM112", # uncapitalized-environment-variables
"SIM113", # enumerate-for-loop
"SIM114", # if-with-same-arms
"SIM115", # open-file-with-context-handler
"SIM117", # multiple-with-statements
"SIM223", # expr-and-false
"SIM300", # yoda-conditions
"TD001", # invalid-todo-tag
"TD002", # missing-todo-author
"TD003", # missing-todo-link
"PLC0415", # import-outside-top-level
"PLE0604", # invalid-all-object (false positives)
"PLE0643", # potential-index-error (false positives)
"PLW0603", # global-statement
"PLW1510", # subprocess-run-without-check
"PLW1641", # eq-without-hash
"PLW2901", # redefined-loop-name
"RUF001", # ambiguous-unicode-character-string
"RUF012", # mutable-class-default
"RUF100", # unused-noqa (flake8 has slightly different behavior)
]
select = [
"E", # pycodestyle Error
"W", # pycodestyle Warning
"F", # Pyflakes
"I", # isort
"Q", # flake8-quotes
"N803", # invalid-argument-name
"N804", # invalid-first-argument-name-for-class-method
"UP", # pyupgrade
"B", # flake8-bugbear
"A", # flake8-builtins
"COM", # flake8-commas
"C4", # flake8-comprehensions
"FA", # flake8-future-annotations
"ISC", # flake8-implicit-str-concat
"ICN003", # banned-import-from
"PIE", # flake8-pie
"T20", # flake8-print
"RSE", # flake8-raise
"RET504", # unnecessary-assign
"SIM", # flake8-simplify
"TID251", # banned-api
"TD", # flake8-todos
"PLC", # Pylint Convention
"PLE", # Pylint Error
"PLW", # Pylint Warning
"RUF", # Ruff-specific rules
]
[tool.ruff.lint.per-file-ignores]
"devscripts/lazy_load_template.py" = [
"F401", # unused-import
]
"!yt_dlp/extractor/**.py" = [
"I", # isort
"ICN003", # banned-import-from
"T20", # flake8-print
"A002", # builtin-argument-shadowing
"C408", # unnecessary-collection-call
]
[tool.ruff.lint.isort]
known-first-party = [
"bundle",
"devscripts",
"test",
]
relative-imports-order = "closest-to-furthest"
[tool.ruff.lint.flake8-quotes]
docstring-quotes = "double"
multiline-quotes = "single"
inline-quotes = "single"
avoid-escape = false
[tool.ruff.lint.pep8-naming]
classmethod-decorators = [
"yt_dlp.utils.classproperty",
]
[tool.ruff.lint.flake8-import-conventions]
banned-from = [
"base64",
"datetime",
"functools",
"glob",
"hashlib",
"itertools",
"json",
"math",
"os",
"pathlib",
"random",
"re",
"string",
"sys",
"time",
"urllib.parse",
"uuid",
"xml",
]
[tool.ruff.lint.flake8-tidy-imports.banned-api]
"yt_dlp.compat.compat_str".msg = "Use `str` instead."
"yt_dlp.compat.compat_b64decode".msg = "Use `base64.b64decode` instead."
"yt_dlp.compat.compat_urlparse".msg = "Use `urllib.parse` instead."
"yt_dlp.compat.compat_parse_qs".msg = "Use `urllib.parse.parse_qs` instead."
"yt_dlp.compat.compat_urllib_parse_unquote".msg = "Use `urllib.parse.unquote` instead."
"yt_dlp.compat.compat_urllib_parse_urlencode".msg = "Use `urllib.parse.urlencode` instead."
"yt_dlp.compat.compat_urllib_parse_urlparse".msg = "Use `urllib.parse.urlparse` instead."
"yt_dlp.compat.compat_shlex_quote".msg = "Use `yt_dlp.utils.shell_quote` instead."
"yt_dlp.utils.error_to_compat_str".msg = "Use `str` instead."
"yt_dlp.utils.bytes_to_intlist".msg = "Use `list` instead."
"yt_dlp.utils.intlist_to_bytes".msg = "Use `bytes` instead."
"yt_dlp.utils.jwt_encode_hs256".msg = "Use `yt_dlp.utils.jwt_encode` instead."
"yt_dlp.utils.decodeArgument".msg = "Do not use"
"yt_dlp.utils.decodeFilename".msg = "Do not use"
"yt_dlp.utils.encodeFilename".msg = "Do not use"
"yt_dlp.compat.compat_os_name".msg = "Use `os.name` instead."
"yt_dlp.compat.compat_realpath".msg = "Use `os.path.realpath` instead."
"yt_dlp.compat.functools".msg = "Use `functools` instead."
"yt_dlp.utils.decodeOption".msg = "Do not use"
"yt_dlp.utils.compiled_regex_type".msg = "Use `re.Pattern` instead."
[tool.autopep8]
max_line_length = 120
recursive = true
exit-code = true
jobs = 0
select = [
"E101",
"E112",
"E113",
"E115",
"E116",
"E117",
"E121",
"E122",
"E123",
"E124",
"E125",
"E126",
"E127",
"E128",
"E129",
"E131",
"E201",
"E202",
"E203",
"E211",
"E221",
"E222",
"E223",
"E224",
"E225",
"E226",
"E227",
"E228",
"E231",
"E241",
"E242",
"E251",
"E252",
"E261",
"E262",
"E265",
"E266",
"E271",
"E272",
"E273",
"E274",
"E275",
"E301",
"E302",
"E303",
"E304",
"E305",
"E306",
"E502",
"E701",
"E702",
"E704",
"W391",
"W504",
]
exclude = "*/extractor/lazy_extractors.py,*venv*,*/test/testdata/sigs/player-*.js,.idea,.vscode"
[tool.pytest.ini_options]
addopts = [
"-ra", # summary: all except passed
"--verbose",
"--strict-markers",
]
markers = [
"download",
]
-45
View File
@@ -1,45 +0,0 @@
[flake8]
exclude = build,venv,.tox,.git,.pytest_cache
ignore = E402,E501,E731,E741,W503
max_line_length = 120
per_file_ignores =
devscripts/lazy_load_template.py: F401
[autoflake]
ignore-init-module-imports = true
ignore-pass-after-docstring = true
remove-all-unused-imports = true
remove-duplicate-keys = true
remove-unused-variables = true
[tool:pytest]
addopts = -ra -v --strict-markers
markers =
download
[tox:tox]
skipsdist = true
envlist = py{38,39,310,311,312},pypy{38,39,310}
skip_missing_interpreters = true
[testenv] # tox
deps =
pytest
commands = pytest {posargs:"-m not download"}
passenv = HOME # For test_compat_expanduser
setenv =
# PYTHONWARNINGS = error # Catches PIP's warnings too
[isort]
py_version = 38
multi_line_output = VERTICAL_HANGING_INDENT
line_length = 80
reverse_relative = true
ensure_newline_before_comments = true
include_trailing_comma = true
known_first_party =
test
-36
View File
@@ -1,36 +0,0 @@
#!/usr/bin/env python3
# Allow execution from anywhere
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import warnings
if sys.argv[1:2] == ['py2exe']:
warnings.warn(DeprecationWarning('`setup.py py2exe` is deprecated and will be removed in a future version. '
'Use `bundle.py2exe` instead'))
import bundle.py2exe
bundle.py2exe.main()
elif 'build_lazy_extractors' in sys.argv:
warnings.warn(DeprecationWarning('`setup.py build_lazy_extractors` is deprecated and will be removed in a future version. '
'Use `devscripts.make_lazy_extractors` instead'))
import subprocess
os.chdir(sys.path[0])
print('running build_lazy_extractors')
subprocess.run([sys.executable, 'devscripts/make_lazy_extractors.py'])
else:
print(
'ERROR: Building by calling `setup.py` is deprecated. '
'Use a build frontend like `build` instead. ',
'Refer to https://build.pypa.io for more info', file=sys.stderr)
sys.exit(1)
+244 -209
View File
File diff suppressed because it is too large Load Diff
+44 -6
View File
@@ -1,4 +1,3 @@
import functools
import inspect import inspect
import pytest import pytest
@@ -10,7 +9,9 @@ from yt_dlp.utils._utils import _YDLLogger as FakeLogger
@pytest.fixture @pytest.fixture
def handler(request): def handler(request):
RH_KEY = request.param RH_KEY = getattr(request, 'param', None)
if not RH_KEY:
return
if inspect.isclass(RH_KEY) and issubclass(RH_KEY, RequestHandler): if inspect.isclass(RH_KEY) and issubclass(RH_KEY, RequestHandler):
handler = RH_KEY handler = RH_KEY
elif RH_KEY in _REQUEST_HANDLERS: elif RH_KEY in _REQUEST_HANDLERS:
@@ -18,9 +19,46 @@ def handler(request):
else: else:
pytest.skip(f'{RH_KEY} request handler is not available') pytest.skip(f'{RH_KEY} request handler is not available')
return functools.partial(handler, logger=FakeLogger) class HandlerWrapper(handler):
RH_KEY = handler.RH_KEY
def __init__(self, **kwargs):
super().__init__(logger=FakeLogger, **kwargs)
return HandlerWrapper
def validate_and_send(rh, req): @pytest.fixture(autouse=True)
rh.validate(req) def skip_handler(request, handler):
return rh.send(req) """usage: pytest.mark.skip_handler('my_handler', 'reason')"""
for marker in request.node.iter_markers('skip_handler'):
if marker.args[0] == handler.RH_KEY:
pytest.skip(marker.args[1] if len(marker.args) > 1 else '')
@pytest.fixture(autouse=True)
def skip_handler_if(request, handler):
"""usage: pytest.mark.skip_handler_if('my_handler', lambda request: True, 'reason')"""
for marker in request.node.iter_markers('skip_handler_if'):
if marker.args[0] == handler.RH_KEY and marker.args[1](request):
pytest.skip(marker.args[2] if len(marker.args) > 2 else '')
@pytest.fixture(autouse=True)
def skip_handlers_if(request, handler):
"""usage: pytest.mark.skip_handlers_if(lambda request, handler: True, 'reason')"""
for marker in request.node.iter_markers('skip_handlers_if'):
if handler and marker.args[0](request, handler):
pytest.skip(marker.args[1] if len(marker.args) > 1 else '')
def pytest_configure(config):
config.addinivalue_line(
'markers', 'skip_handler(handler): skip test for the given handler',
)
config.addinivalue_line(
'markers', 'skip_handler_if(handler): skip test for the given handler if condition is true',
)
config.addinivalue_line(
'markers', 'skip_handlers_if(handler): skip test for handlers when the condition is true',
)
+136 -92
View File
@@ -9,15 +9,14 @@ import types
import yt_dlp.extractor import yt_dlp.extractor
from yt_dlp import YoutubeDL from yt_dlp import YoutubeDL
from yt_dlp.compat import compat_os_name
from yt_dlp.utils import preferredencoding, try_call, write_string, find_available_port from yt_dlp.utils import preferredencoding, try_call, write_string, find_available_port
if 'pytest' in sys.modules: if 'pytest' in sys.modules:
import pytest import pytest
is_download_test = pytest.mark.download is_download_test = pytest.mark.download
else: else:
def is_download_test(testClass): def is_download_test(test_class):
return testClass return test_class
def get_params(override=None): def get_params(override=None):
@@ -45,11 +44,11 @@ def try_rm(filename):
def report_warning(message, *args, **kwargs): def report_warning(message, *args, **kwargs):
''' """
Print the message to stderr, it will be prefixed with 'WARNING:' Print the message to stderr, it will be prefixed with 'WARNING:'
If stderr is a tty file the 'WARNING:' will be colored If stderr is a tty file the 'WARNING:' will be colored
''' """
if sys.stderr.isatty() and compat_os_name != 'nt': if sys.stderr.isatty() and os.name != 'nt':
_msg_header = '\033[0;33mWARNING:\033[0m' _msg_header = '\033[0;33mWARNING:\033[0m'
else: else:
_msg_header = 'WARNING:' _msg_header = 'WARNING:'
@@ -102,88 +101,109 @@ def getwebpagetestcases():
md5 = lambda s: hashlib.md5(s.encode()).hexdigest() md5 = lambda s: hashlib.md5(s.encode()).hexdigest()
def expect_value(self, got, expected, field): def _iter_differences(got, expected, field):
if isinstance(expected, str) and expected.startswith('re:'): if isinstance(expected, str):
match_str = expected[len('re:'):] op, _, val = expected.partition(':')
match_rex = re.compile(match_str) if op in ('mincount', 'maxcount', 'count'):
if not isinstance(got, (list, dict)):
yield field, f'expected either {list.__name__} or {dict.__name__}, got {type(got).__name__}'
return
self.assertTrue( expected_num = int(val)
isinstance(got, str), got_num = len(got)
f'Expected a {str.__name__} object, but got {type(got).__name__} for field {field}')
self.assertTrue(
match_rex.match(got),
f'field {field} (value: {got!r}) should match {match_str!r}')
elif isinstance(expected, str) and expected.startswith('startswith:'):
start_str = expected[len('startswith:'):]
self.assertTrue(
isinstance(got, str),
f'Expected a {str.__name__} object, but got {type(got).__name__} for field {field}')
self.assertTrue(
got.startswith(start_str),
f'field {field} (value: {got!r}) should start with {start_str!r}')
elif isinstance(expected, str) and expected.startswith('contains:'):
contains_str = expected[len('contains:'):]
self.assertTrue(
isinstance(got, str),
f'Expected a {str.__name__} object, but got {type(got).__name__} for field {field}')
self.assertTrue(
contains_str in got,
f'field {field} (value: {got!r}) should contain {contains_str!r}')
elif isinstance(expected, type):
self.assertTrue(
isinstance(got, expected),
f'Expected type {expected!r} for field {field}, but got value {got!r} of type {type(got)!r}')
elif isinstance(expected, dict) and isinstance(got, dict):
expect_dict(self, got, expected)
elif isinstance(expected, list) and isinstance(got, list):
self.assertEqual(
len(expected), len(got),
'Expect a list of length %d, but got a list of length %d for field %s' % (
len(expected), len(got), field))
for index, (item_got, item_expected) in enumerate(zip(got, expected)):
type_got = type(item_got)
type_expected = type(item_expected)
self.assertEqual(
type_expected, type_got,
'Type mismatch for list item at index %d for field %s, expected %r, got %r' % (
index, field, type_expected, type_got))
expect_value(self, item_got, item_expected, field)
else:
if isinstance(expected, str) and expected.startswith('md5:'):
self.assertTrue(
isinstance(got, str),
f'Expected field {field} to be a unicode object, but got value {got!r} of type {type(got)!r}')
got = 'md5:' + md5(got)
elif isinstance(expected, str) and re.match(r'^(?:min|max)?count:\d+', expected):
self.assertTrue(
isinstance(got, (list, dict)),
f'Expected field {field} to be a list or a dict, but it is of type {type(got).__name__}')
op, _, expected_num = expected.partition(':')
expected_num = int(expected_num)
if op == 'mincount': if op == 'mincount':
assert_func = assertGreaterEqual if got_num < expected_num:
msg_tmpl = 'Expected %d items in field %s, but only got %d' yield field, f'expected at least {val} items, got {got_num}'
elif op == 'maxcount': return
assert_func = assertLessEqual
msg_tmpl = 'Expected maximum %d items in field %s, but got %d' if op == 'maxcount':
elif op == 'count': if got_num > expected_num:
assert_func = assertEqual yield field, f'expected at most {val} items, got {got_num}'
msg_tmpl = 'Expected exactly %d items in field %s, but got %d' return
else:
assert False assert op == 'count'
assert_func( if got_num != expected_num:
self, len(got), expected_num, yield field, f'expected exactly {val} items, got {got_num}'
msg_tmpl % (expected_num, field, len(got)))
return return
self.assertEqual(
expected, got, if not isinstance(got, str):
f'Invalid value for field {field}, expected {expected!r}, got {got!r}') yield field, f'expected {str.__name__}, got {type(got).__name__}'
return
if op == 're':
if not re.match(val, got):
yield field, f'should match {val!r}, got {got!r}'
return
if op == 'startswith':
if not got.startswith(val):
yield field, f'should start with {val!r}, got {got!r}'
return
if op == 'contains':
if not val.startswith(got):
yield field, f'should contain {val!r}, got {got!r}'
return
if op == 'md5':
hash_val = md5(got)
if hash_val != val:
yield field, f'expected hash {val}, got {hash_val}'
return
if got != expected:
yield field, f'expected {expected!r}, got {got!r}'
return
if isinstance(expected, dict) and isinstance(got, dict):
for key, expected_val in expected.items():
if key not in got:
yield field, f'missing key: {key!r}'
continue
field_name = key if field is None else f'{field}.{key}'
yield from _iter_differences(got[key], expected_val, field_name)
return
if isinstance(expected, type):
if not isinstance(got, expected):
yield field, f'expected {expected.__name__}, got {type(got).__name__}'
return
if isinstance(expected, list) and isinstance(got, list):
# TODO: clever diffing algorithm lmao
if len(expected) != len(got):
yield field, f'expected length of {len(expected)}, got {len(got)}'
return
for index, (got_val, expected_val) in enumerate(zip(got, expected)):
field_name = str(index) if field is None else f'{field}.{index}'
yield from _iter_differences(got_val, expected_val, field_name)
return
if got != expected:
yield field, f'expected {expected!r}, got {got!r}'
def _expect_value(message, got, expected, field):
mismatches = list(_iter_differences(got, expected, field))
if not mismatches:
return
fields = [field for field, _ in mismatches if field is not None]
return ''.join((
message, f' ({", ".join(fields)})' if fields else '',
*(f'\n\t{field}: {message}' for field, message in mismatches)))
def expect_value(self, got, expected, field):
if message := _expect_value('values differ', got, expected, field):
self.fail(message)
def expect_dict(self, got_dict, expected_dict): def expect_dict(self, got_dict, expected_dict):
for info_field, expected in expected_dict.items(): if message := _expect_value('dictionaries differ', got_dict, expected_dict, None):
got = got_dict.get(info_field) self.fail(message)
expect_value(self, got, expected, info_field)
def sanitize_got_info_dict(got_dict): def sanitize_got_info_dict(got_dict):
@@ -224,7 +244,7 @@ def sanitize_got_info_dict(got_dict):
test_info_dict.pop('display_id') test_info_dict.pop('display_id')
# Remove deprecated fields # Remove deprecated fields
for old in YoutubeDL._deprecated_multivalue_fields.keys(): for old in YoutubeDL._deprecated_multivalue_fields:
test_info_dict.pop(old, None) test_info_dict.pop(old, None)
# release_year may be generated from release_date # release_year may be generated from release_date
@@ -239,6 +259,20 @@ def sanitize_got_info_dict(got_dict):
def expect_info_dict(self, got_dict, expected_dict): def expect_info_dict(self, got_dict, expected_dict):
ALLOWED_KEYS_SORT_ORDER = (
# NB: Keep in sync with the docstring of extractor/common.py
'id', 'ext', 'direct', 'display_id', 'title', 'alt_title', 'description', 'media_type',
'uploader', 'uploader_id', 'uploader_url', 'channel', 'channel_id', 'channel_url', 'channel_is_verified',
'channel_follower_count', 'comment_count', 'view_count', 'concurrent_view_count',
'like_count', 'dislike_count', 'repost_count', 'average_rating', 'age_limit', 'duration', 'thumbnail', 'heatmap',
'chapters', 'chapter', 'chapter_number', 'chapter_id', 'start_time', 'end_time', 'section_start', 'section_end',
'categories', 'tags', 'cast', 'composers', 'artists', 'album_artists', 'creators', 'genres',
'track', 'track_number', 'track_id', 'album', 'album_type', 'disc_number',
'series', 'series_id', 'season', 'season_number', 'season_id', 'episode', 'episode_number', 'episode_id',
'timestamp', 'upload_date', 'release_timestamp', 'release_date', 'release_year', 'modified_timestamp', 'modified_date',
'playable_in_embed', 'availability', 'live_status', 'location', 'license', '_old_archive_ids',
)
expect_dict(self, got_dict, expected_dict) expect_dict(self, got_dict, expected_dict)
# Check for the presence of mandatory fields # Check for the presence of mandatory fields
if got_dict.get('_type') not in ('playlist', 'multi_video'): if got_dict.get('_type') not in ('playlist', 'multi_video'):
@@ -246,19 +280,25 @@ def expect_info_dict(self, got_dict, expected_dict):
if expected_dict.get('ext'): if expected_dict.get('ext'):
mandatory_fields.extend(('url', 'ext')) mandatory_fields.extend(('url', 'ext'))
for key in mandatory_fields: for key in mandatory_fields:
self.assertTrue(got_dict.get(key), 'Missing mandatory field %s' % key) self.assertTrue(got_dict.get(key), f'Missing mandatory field {key}')
# Check for mandatory fields that are automatically set by YoutubeDL # Check for mandatory fields that are automatically set by YoutubeDL
if got_dict.get('_type', 'video') == 'video': if got_dict.get('_type', 'video') == 'video':
for key in ['webpage_url', 'extractor', 'extractor_key']: for key in ['webpage_url', 'extractor', 'extractor_key']:
self.assertTrue(got_dict.get(key), 'Missing field: %s' % key) self.assertTrue(got_dict.get(key), f'Missing field: {key}')
test_info_dict = sanitize_got_info_dict(got_dict) test_info_dict = sanitize_got_info_dict(got_dict)
missing_keys = set(test_info_dict.keys()) - set(expected_dict.keys()) # Check for invalid/misspelled field names being returned by the extractor
invalid_keys = sorted(test_info_dict.keys() - ALLOWED_KEYS_SORT_ORDER)
self.assertFalse(invalid_keys, f'Invalid fields returned by the extractor: {", ".join(invalid_keys)}')
missing_keys = sorted(
test_info_dict.keys() - expected_dict.keys(),
key=lambda x: ALLOWED_KEYS_SORT_ORDER.index(x))
if missing_keys: if missing_keys:
def _repr(v): def _repr(v):
if isinstance(v, str): if isinstance(v, str):
return "'%s'" % v.replace('\\', '\\\\').replace("'", "\\'").replace('\n', '\\n') return "'{}'".format(v.replace('\\', '\\\\').replace("'", "\\'").replace('\n', '\\n'))
elif isinstance(v, type): elif isinstance(v, type):
return v.__name__ return v.__name__
else: else:
@@ -275,8 +315,7 @@ def expect_info_dict(self, got_dict, expected_dict):
write_string(info_dict_str.replace('\n', '\n '), out=sys.stderr) write_string(info_dict_str.replace('\n', '\n '), out=sys.stderr)
self.assertFalse( self.assertFalse(
missing_keys, missing_keys,
'Missing keys in test definition: %s' % ( 'Missing keys in test definition: {}'.format(', '.join(sorted(missing_keys))))
', '.join(sorted(missing_keys))))
def assertRegexpMatches(self, text, regexp, msg=None): def assertRegexpMatches(self, text, regexp, msg=None):
@@ -285,9 +324,9 @@ def assertRegexpMatches(self, text, regexp, msg=None):
else: else:
m = re.match(regexp, text) m = re.match(regexp, text)
if not m: if not m:
note = 'Regexp didn\'t match: %r not found' % (regexp) note = f'Regexp didn\'t match: {regexp!r} not found'
if len(text) < 1000: if len(text) < 1000:
note += ' in %r' % text note += f' in {text!r}'
if msg is None: if msg is None:
msg = note msg = note
else: else:
@@ -310,7 +349,7 @@ def assertLessEqual(self, got, expected, msg=None):
def assertEqual(self, got, expected, msg=None): def assertEqual(self, got, expected, msg=None):
if not (got == expected): if got != expected:
if msg is None: if msg is None:
msg = f'{got!r} not equal to {expected!r}' msg = f'{got!r} not equal to {expected!r}'
self.assertTrue(got == expected, msg) self.assertTrue(got == expected, msg)
@@ -338,3 +377,8 @@ def http_server_port(httpd):
def verify_address_availability(address): def verify_address_availability(address):
if find_available_port(address) is None: if find_available_port(address) is None:
pytest.skip(f'Unable to bind to source address {address} (address may not exist)') pytest.skip(f'Unable to bind to source address {address} (address may not exist)')
def validate_and_send(rh, req):
rh.validate(req)
return rh.send(req)
-1
View File
@@ -36,7 +36,6 @@
"verbose": true, "verbose": true,
"writedescription": false, "writedescription": false,
"writeinfojson": true, "writeinfojson": true,
"writeannotations": false,
"writelink": false, "writelink": false,
"writeurllink": false, "writeurllink": false,
"writewebloclink": false, "writewebloclink": false,
+333 -78
View File
@@ -36,6 +36,18 @@ class InfoExtractorTestRequestHandler(http.server.BaseHTTPRequestHandler):
self.send_header('Content-Type', 'text/html; charset=utf-8') self.send_header('Content-Type', 'text/html; charset=utf-8')
self.end_headers() self.end_headers()
self.wfile.write(TEAPOT_RESPONSE_BODY.encode()) self.wfile.write(TEAPOT_RESPONSE_BODY.encode())
elif self.path == '/fake.m3u8':
self.send_response(200)
self.send_header('Content-Length', '1024')
self.end_headers()
self.wfile.write(1024 * b'\x00')
elif self.path == '/bipbop.m3u8':
with open('test/testdata/m3u8/bipbop_16x9.m3u8', 'rb') as f:
data = f.read()
self.send_response(200)
self.send_header('Content-Length', str(len(data)))
self.end_headers()
self.wfile.write(data)
else: else:
assert False assert False
@@ -53,6 +65,18 @@ class TestInfoExtractor(unittest.TestCase):
def test_ie_key(self): def test_ie_key(self):
self.assertEqual(get_info_extractor(YoutubeIE.ie_key()), YoutubeIE) self.assertEqual(get_info_extractor(YoutubeIE.ie_key()), YoutubeIE)
def test_get_netrc_login_info(self):
for params in [
{'usenetrc': True, 'netrc_location': './test/testdata/netrc/netrc'},
{'netrc_cmd': f'{sys.executable} ./test/testdata/netrc/print_netrc.py'},
]:
ie = DummyIE(FakeYDL(params))
self.assertEqual(ie._get_netrc_login_info(netrc_machine='normal_use'), ('user', 'pass'))
self.assertEqual(ie._get_netrc_login_info(netrc_machine='empty_user'), ('', 'pass'))
self.assertEqual(ie._get_netrc_login_info(netrc_machine='empty_pass'), ('user', ''))
self.assertEqual(ie._get_netrc_login_info(netrc_machine='both_empty'), ('', ''))
self.assertEqual(ie._get_netrc_login_info(netrc_machine='nonexistent'), (None, None))
def test_html_search_regex(self): def test_html_search_regex(self):
html = '<p id="foo">Watch this <a href="http://www.youtube.com/watch?v=BaW_jenozKc">video</a></p>' html = '<p id="foo">Watch this <a href="http://www.youtube.com/watch?v=BaW_jenozKc">video</a></p>'
search = lambda re, *args: self.ie._html_search_regex(re, html, *args) search = lambda re, *args: self.ie._html_search_regex(re, html, *args)
@@ -262,19 +286,19 @@ class TestInfoExtractor(unittest.TestCase):
''', ''',
{ {
'chapters': [ 'chapters': [
{"title": "Explosie Turnhout", "start_time": 70, "end_time": 440}, {'title': 'Explosie Turnhout', 'start_time': 70, 'end_time': 440},
{"title": "Jaarwisseling", "start_time": 440, "end_time": 1179}, {'title': 'Jaarwisseling', 'start_time': 440, 'end_time': 1179},
{"title": "Natuurbranden Colorado", "start_time": 1179, "end_time": 1263}, {'title': 'Natuurbranden Colorado', 'start_time': 1179, 'end_time': 1263},
{"title": "Klimaatverandering", "start_time": 1263, "end_time": 1367}, {'title': 'Klimaatverandering', 'start_time': 1263, 'end_time': 1367},
{"title": "Zacht weer", "start_time": 1367, "end_time": 1383}, {'title': 'Zacht weer', 'start_time': 1367, 'end_time': 1383},
{"title": "Financiële balans", "start_time": 1383, "end_time": 1484}, {'title': 'Financiële balans', 'start_time': 1383, 'end_time': 1484},
{"title": "Club Brugge", "start_time": 1484, "end_time": 1575}, {'title': 'Club Brugge', 'start_time': 1484, 'end_time': 1575},
{"title": "Mentale gezondheid bij topsporters", "start_time": 1575, "end_time": 1728}, {'title': 'Mentale gezondheid bij topsporters', 'start_time': 1575, 'end_time': 1728},
{"title": "Olympische Winterspelen", "start_time": 1728, "end_time": 1873}, {'title': 'Olympische Winterspelen', 'start_time': 1728, 'end_time': 1873},
{"title": "Sober oudjaar in Nederland", "start_time": 1873, "end_time": 2079.23} {'title': 'Sober oudjaar in Nederland', 'start_time': 1873, 'end_time': 2079.23},
], ],
'title': 'Het journaal - Aflevering 365 (Seizoen 2021)' 'title': 'Het journaal - Aflevering 365 (Seizoen 2021)',
}, {} }, {},
), ),
( (
# test multiple thumbnails in a list # test multiple thumbnails in a list
@@ -301,13 +325,27 @@ class TestInfoExtractor(unittest.TestCase):
'thumbnails': [{'url': 'https://www.rainews.it/cropgd/640x360/dl/img/2021/12/30/1640886376927_GettyImages.jpg'}], 'thumbnails': [{'url': 'https://www.rainews.it/cropgd/640x360/dl/img/2021/12/30/1640886376927_GettyImages.jpg'}],
}, },
{}, {},
) ),
(
# test thumbnail_url key without URL scheme
r'''
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "VideoObject",
"thumbnail_url": "//www.nobelprize.org/images/12693-landscape-medium-gallery.jpg"
}</script>''',
{
'thumbnails': [{'url': 'https://www.nobelprize.org/images/12693-landscape-medium-gallery.jpg'}],
},
{},
),
] ]
for html, expected_dict, search_json_ld_kwargs in _TESTS: for html, expected_dict, search_json_ld_kwargs in _TESTS:
expect_dict( expect_dict(
self, self,
self.ie._search_json_ld(html, None, **search_json_ld_kwargs), self.ie._search_json_ld(html, None, **search_json_ld_kwargs),
expected_dict expected_dict,
) )
def test_download_json(self): def test_download_json(self):
@@ -366,7 +404,7 @@ class TestInfoExtractor(unittest.TestCase):
'height': 740, 'height': 740,
'tbr': 1500, 'tbr': 1500,
}], }],
'thumbnail': '//pics.r18.com/digital/amateur/mgmr105/mgmr105jp.jpg' 'thumbnail': '//pics.r18.com/digital/amateur/mgmr105/mgmr105jp.jpg',
}) })
# from https://www.csfd.cz/ # from https://www.csfd.cz/
@@ -419,9 +457,9 @@ class TestInfoExtractor(unittest.TestCase):
'height': 1080, 'height': 1080,
}], }],
'subtitles': { 'subtitles': {
'cs': [{'url': 'https://video.csfd.cz/files/subtitles/163/344/163344115_4c388b.srt'}] 'cs': [{'url': 'https://video.csfd.cz/files/subtitles/163/344/163344115_4c388b.srt'}],
}, },
'thumbnail': 'https://img.csfd.cz/files/images/film/video/preview/163/344/163344118_748d20.png?h360' 'thumbnail': 'https://img.csfd.cz/files/images/film/video/preview/163/344/163344118_748d20.png?h360',
}) })
# from https://tamasha.com/v/Kkdjw # from https://tamasha.com/v/Kkdjw
@@ -452,7 +490,7 @@ class TestInfoExtractor(unittest.TestCase):
'ext': 'mp4', 'ext': 'mp4',
'format_id': '144p', 'format_id': '144p',
'height': 144, 'height': 144,
}] }],
}) })
# from https://www.directvnow.com # from https://www.directvnow.com
@@ -470,7 +508,7 @@ class TestInfoExtractor(unittest.TestCase):
'formats': [{ 'formats': [{
'ext': 'mp4', 'ext': 'mp4',
'url': 'https://cdn.directv.com/content/dam/dtv/prod/website_directvnow-international/videos/DTVN_hdr_HBO_v3.mp4', 'url': 'https://cdn.directv.com/content/dam/dtv/prod/website_directvnow-international/videos/DTVN_hdr_HBO_v3.mp4',
}] }],
}) })
# from https://www.directvnow.com # from https://www.directvnow.com
@@ -488,7 +526,7 @@ class TestInfoExtractor(unittest.TestCase):
'formats': [{ 'formats': [{
'url': 'https://cdn.directv.com/content/dam/dtv/prod/website_directvnow-international/videos/DTVN_hdr_HBO_v3.mp4', 'url': 'https://cdn.directv.com/content/dam/dtv/prod/website_directvnow-international/videos/DTVN_hdr_HBO_v3.mp4',
'ext': 'mp4', 'ext': 'mp4',
}] }],
}) })
# from https://www.klarna.com/uk/ # from https://www.klarna.com/uk/
@@ -547,8 +585,8 @@ class TestInfoExtractor(unittest.TestCase):
'id': 'XEgvuql4', 'id': 'XEgvuql4',
'formats': [{ 'formats': [{
'url': 'rtmp://192.138.214.154/live/sjclive', 'url': 'rtmp://192.138.214.154/live/sjclive',
'ext': 'flv' 'ext': 'flv',
}] }],
}) })
# from https://www.pornoxo.com/videos/7564/striptease-from-sexy-secretary/ # from https://www.pornoxo.com/videos/7564/striptease-from-sexy-secretary/
@@ -588,8 +626,8 @@ class TestInfoExtractor(unittest.TestCase):
'thumbnail': 'https://t03.vipstreamservice.com/thumbs/pxo-full/2009-12/14/a4b2157147afe5efa93ce1978e0265289c193874e02597.flv-full-13.jpg', 'thumbnail': 'https://t03.vipstreamservice.com/thumbs/pxo-full/2009-12/14/a4b2157147afe5efa93ce1978e0265289c193874e02597.flv-full-13.jpg',
'formats': [{ 'formats': [{
'url': 'https://cdn.pornoxo.com/key=MF+oEbaxqTKb50P-w9G3nA,end=1489689259,ip=104.199.146.27/ip=104.199.146.27/speed=6573765/buffer=3.0/2009-12/4b2157147afe5efa93ce1978e0265289c193874e02597.flv', 'url': 'https://cdn.pornoxo.com/key=MF+oEbaxqTKb50P-w9G3nA,end=1489689259,ip=104.199.146.27/ip=104.199.146.27/speed=6573765/buffer=3.0/2009-12/4b2157147afe5efa93ce1978e0265289c193874e02597.flv',
'ext': 'flv' 'ext': 'flv',
}] }],
}) })
# from http://www.indiedb.com/games/king-machine/videos # from http://www.indiedb.com/games/king-machine/videos
@@ -610,12 +648,12 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'formats': [{ 'formats': [{
'url': 'http://cdn.dbolical.com/cache/videos/games/1/50/49678/encode_mp4/king-machine-trailer.mp4', 'url': 'http://cdn.dbolical.com/cache/videos/games/1/50/49678/encode_mp4/king-machine-trailer.mp4',
'height': 360, 'height': 360,
'ext': 'mp4' 'ext': 'mp4',
}, { }, {
'url': 'http://cdn.dbolical.com/cache/videos/games/1/50/49678/encode720p_mp4/king-machine-trailer.mp4', 'url': 'http://cdn.dbolical.com/cache/videos/games/1/50/49678/encode720p_mp4/king-machine-trailer.mp4',
'height': 720, 'height': 720,
'ext': 'mp4' 'ext': 'mp4',
}] }],
}) })
def test_parse_m3u8_formats(self): def test_parse_m3u8_formats(self):
@@ -626,6 +664,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'img_bipbop_adv_example_fmp4', 'img_bipbop_adv_example_fmp4',
'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8', 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
[{ [{
# 60kbps (bitrate not provided in m3u8); sorted as worst because it's grouped with lowest bitrate video track
'format_id': 'aud1-English', 'format_id': 'aud1-English',
'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/a1/prog_index.m3u8', 'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/a1/prog_index.m3u8',
'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8', 'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
@@ -633,15 +672,9 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'ext': 'mp4', 'ext': 'mp4',
'protocol': 'm3u8_native', 'protocol': 'm3u8_native',
'audio_ext': 'mp4', 'audio_ext': 'mp4',
'source_preference': 0,
}, { }, {
'format_id': 'aud2-English', # 192kbps (bitrate not provided in m3u8)
'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/a2/prog_index.m3u8',
'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
'language': 'en',
'ext': 'mp4',
'protocol': 'm3u8_native',
'audio_ext': 'mp4',
}, {
'format_id': 'aud3-English', 'format_id': 'aud3-English',
'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/a3/prog_index.m3u8', 'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/a3/prog_index.m3u8',
'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8', 'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
@@ -649,6 +682,17 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'ext': 'mp4', 'ext': 'mp4',
'protocol': 'm3u8_native', 'protocol': 'm3u8_native',
'audio_ext': 'mp4', 'audio_ext': 'mp4',
'source_preference': 1,
}, {
# 384kbps (bitrate not provided in m3u8); sorted as best because it's grouped with the highest bitrate video track
'format_id': 'aud2-English',
'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/a2/prog_index.m3u8',
'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
'language': 'en',
'ext': 'mp4',
'protocol': 'm3u8_native',
'audio_ext': 'mp4',
'source_preference': 2,
}, { }, {
'format_id': '530', 'format_id': '530',
'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v2/prog_index.m3u8', 'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v2/prog_index.m3u8',
@@ -866,7 +910,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'height': 1080, 'height': 1080,
'vcodec': 'avc1.64002a', 'vcodec': 'avc1.64002a',
}], }],
{} {},
), ),
( (
'bipbop_16x9', 'bipbop_16x9',
@@ -990,45 +1034,45 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'en': [{ 'en': [{
'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/eng/prog_index.m3u8', 'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/eng/prog_index.m3u8',
'ext': 'vtt', 'ext': 'vtt',
'protocol': 'm3u8_native' 'protocol': 'm3u8_native',
}, { }, {
'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/eng_forced/prog_index.m3u8', 'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/eng_forced/prog_index.m3u8',
'ext': 'vtt', 'ext': 'vtt',
'protocol': 'm3u8_native' 'protocol': 'm3u8_native',
}], }],
'fr': [{ 'fr': [{
'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/fra/prog_index.m3u8', 'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/fra/prog_index.m3u8',
'ext': 'vtt', 'ext': 'vtt',
'protocol': 'm3u8_native' 'protocol': 'm3u8_native',
}, { }, {
'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/fra_forced/prog_index.m3u8', 'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/fra_forced/prog_index.m3u8',
'ext': 'vtt', 'ext': 'vtt',
'protocol': 'm3u8_native' 'protocol': 'm3u8_native',
}], }],
'es': [{ 'es': [{
'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/spa/prog_index.m3u8', 'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/spa/prog_index.m3u8',
'ext': 'vtt', 'ext': 'vtt',
'protocol': 'm3u8_native' 'protocol': 'm3u8_native',
}, { }, {
'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/spa_forced/prog_index.m3u8', 'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/spa_forced/prog_index.m3u8',
'ext': 'vtt', 'ext': 'vtt',
'protocol': 'm3u8_native' 'protocol': 'm3u8_native',
}], }],
'ja': [{ 'ja': [{
'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/jpn/prog_index.m3u8', 'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/jpn/prog_index.m3u8',
'ext': 'vtt', 'ext': 'vtt',
'protocol': 'm3u8_native' 'protocol': 'm3u8_native',
}, { }, {
'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/jpn_forced/prog_index.m3u8', 'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/jpn_forced/prog_index.m3u8',
'ext': 'vtt', 'ext': 'vtt',
'protocol': 'm3u8_native' 'protocol': 'm3u8_native',
}], }],
} },
), ),
] ]
for m3u8_file, m3u8_url, expected_formats, expected_subs in _TEST_CASES: for m3u8_file, m3u8_url, expected_formats, expected_subs in _TEST_CASES:
with open('./test/testdata/m3u8/%s.m3u8' % m3u8_file, encoding='utf-8') as f: with open(f'./test/testdata/m3u8/{m3u8_file}.m3u8', encoding='utf-8') as f:
formats, subs = self.ie._parse_m3u8_formats_and_subtitles( formats, subs = self.ie._parse_m3u8_formats_and_subtitles(
f.read(), m3u8_url, ext='mp4') f.read(), m3u8_url, ext='mp4')
self.ie._sort_formats(formats) self.ie._sort_formats(formats)
@@ -1366,14 +1410,14 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd', 'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/', 'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
'protocol': 'http_dash_segments', 'protocol': 'http_dash_segments',
} },
] ],
}, },
) ),
] ]
for mpd_file, mpd_url, mpd_base_url, expected_formats, expected_subtitles in _TEST_CASES: for mpd_file, mpd_url, mpd_base_url, expected_formats, expected_subtitles in _TEST_CASES:
with open('./test/testdata/mpd/%s.mpd' % mpd_file, encoding='utf-8') as f: with open(f'./test/testdata/mpd/{mpd_file}.mpd', encoding='utf-8') as f:
formats, subtitles = self.ie._parse_mpd_formats_and_subtitles( formats, subtitles = self.ie._parse_mpd_formats_and_subtitles(
compat_etree_fromstring(f.read().encode()), compat_etree_fromstring(f.read().encode()),
mpd_base_url=mpd_base_url, mpd_url=mpd_url) mpd_base_url=mpd_base_url, mpd_url=mpd_url)
@@ -1408,7 +1452,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'sampling_rate': 48000, 'sampling_rate': 48000,
'channels': 2, 'channels': 2,
'bits_per_sample': 16, 'bits_per_sample': 16,
'nal_unit_length_field': 4 'nal_unit_length_field': 4,
}, },
}, { }, {
'format_id': 'video-100', 'format_id': 'video-100',
@@ -1431,7 +1475,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'codec_private_data': '00000001674D401FDA0544EFFC2D002CBC40000003004000000C03C60CA80000000168EF32C8', 'codec_private_data': '00000001674D401FDA0544EFFC2D002CBC40000003004000000C03C60CA80000000168EF32C8',
'channels': 2, 'channels': 2,
'bits_per_sample': 16, 'bits_per_sample': 16,
'nal_unit_length_field': 4 'nal_unit_length_field': 4,
}, },
}, { }, {
'format_id': 'video-326', 'format_id': 'video-326',
@@ -1454,7 +1498,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'codec_private_data': '00000001674D401FDA0241FE23FFC3BC83BA44000003000400000300C03C60CA800000000168EF32C8', 'codec_private_data': '00000001674D401FDA0241FE23FFC3BC83BA44000003000400000300C03C60CA800000000168EF32C8',
'channels': 2, 'channels': 2,
'bits_per_sample': 16, 'bits_per_sample': 16,
'nal_unit_length_field': 4 'nal_unit_length_field': 4,
}, },
}, { }, {
'format_id': 'video-698', 'format_id': 'video-698',
@@ -1477,7 +1521,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'codec_private_data': '00000001674D401FDA0350BFB97FF06AF06AD1000003000100000300300F1832A00000000168EF32C8', 'codec_private_data': '00000001674D401FDA0350BFB97FF06AF06AD1000003000100000300300F1832A00000000168EF32C8',
'channels': 2, 'channels': 2,
'bits_per_sample': 16, 'bits_per_sample': 16,
'nal_unit_length_field': 4 'nal_unit_length_field': 4,
}, },
}, { }, {
'format_id': 'video-1493', 'format_id': 'video-1493',
@@ -1500,7 +1544,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'codec_private_data': '00000001674D401FDA011C3DE6FFF0D890D871000003000100000300300F1832A00000000168EF32C8', 'codec_private_data': '00000001674D401FDA011C3DE6FFF0D890D871000003000100000300300F1832A00000000168EF32C8',
'channels': 2, 'channels': 2,
'bits_per_sample': 16, 'bits_per_sample': 16,
'nal_unit_length_field': 4 'nal_unit_length_field': 4,
}, },
}, { }, {
'format_id': 'video-4482', 'format_id': 'video-4482',
@@ -1523,7 +1567,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'codec_private_data': '00000001674D401FDA01A816F97FFC1ABC1AB440000003004000000C03C60CA80000000168EF32C8', 'codec_private_data': '00000001674D401FDA01A816F97FFC1ABC1AB440000003004000000C03C60CA80000000168EF32C8',
'channels': 2, 'channels': 2,
'bits_per_sample': 16, 'bits_per_sample': 16,
'nal_unit_length_field': 4 'nal_unit_length_field': 4,
}, },
}], }],
{ {
@@ -1538,10 +1582,10 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'duration': 8880746666, 'duration': 8880746666,
'timescale': 10000000, 'timescale': 10000000,
'fourcc': 'TTML', 'fourcc': 'TTML',
'codec_private_data': '' 'codec_private_data': '',
} },
} },
] ],
}, },
), ),
( (
@@ -1571,7 +1615,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'sampling_rate': 48000, 'sampling_rate': 48000,
'channels': 2, 'channels': 2,
'bits_per_sample': 16, 'bits_per_sample': 16,
'nal_unit_length_field': 4 'nal_unit_length_field': 4,
}, },
}, { }, {
'format_id': 'audio_deu_1-224', 'format_id': 'audio_deu_1-224',
@@ -1597,7 +1641,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'sampling_rate': 48000, 'sampling_rate': 48000,
'channels': 6, 'channels': 6,
'bits_per_sample': 16, 'bits_per_sample': 16,
'nal_unit_length_field': 4 'nal_unit_length_field': 4,
}, },
}, { }, {
'format_id': 'video_deu-23', 'format_id': 'video_deu-23',
@@ -1622,7 +1666,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'codec_private_data': '000000016742C00CDB06077E5C05A808080A00000300020000030009C0C02EE0177CC6300F142AE00000000168CA8DC8', 'codec_private_data': '000000016742C00CDB06077E5C05A808080A00000300020000030009C0C02EE0177CC6300F142AE00000000168CA8DC8',
'channels': 2, 'channels': 2,
'bits_per_sample': 16, 'bits_per_sample': 16,
'nal_unit_length_field': 4 'nal_unit_length_field': 4,
}, },
}, { }, {
'format_id': 'video_deu-403', 'format_id': 'video_deu-403',
@@ -1647,7 +1691,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'codec_private_data': '00000001674D4014E98323B602D4040405000003000100000300320F1429380000000168EAECF2', 'codec_private_data': '00000001674D4014E98323B602D4040405000003000100000300320F1429380000000168EAECF2',
'channels': 2, 'channels': 2,
'bits_per_sample': 16, 'bits_per_sample': 16,
'nal_unit_length_field': 4 'nal_unit_length_field': 4,
}, },
}, { }, {
'format_id': 'video_deu-680', 'format_id': 'video_deu-680',
@@ -1672,7 +1716,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'codec_private_data': '00000001674D401EE981405FF2E02D4040405000000300100000030320F162D3800000000168EAECF2', 'codec_private_data': '00000001674D401EE981405FF2E02D4040405000000300100000030320F162D3800000000168EAECF2',
'channels': 2, 'channels': 2,
'bits_per_sample': 16, 'bits_per_sample': 16,
'nal_unit_length_field': 4 'nal_unit_length_field': 4,
}, },
}, { }, {
'format_id': 'video_deu-1253', 'format_id': 'video_deu-1253',
@@ -1698,7 +1742,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'codec_private_data': '00000001674D401EE981405FF2E02D4040405000000300100000030320F162D3800000000168EAECF2', 'codec_private_data': '00000001674D401EE981405FF2E02D4040405000000300100000030320F162D3800000000168EAECF2',
'channels': 2, 'channels': 2,
'bits_per_sample': 16, 'bits_per_sample': 16,
'nal_unit_length_field': 4 'nal_unit_length_field': 4,
}, },
}, { }, {
'format_id': 'video_deu-2121', 'format_id': 'video_deu-2121',
@@ -1723,7 +1767,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'codec_private_data': '00000001674D401EECA0601BD80B50101014000003000400000300C83C58B6580000000168E93B3C80', 'codec_private_data': '00000001674D401EECA0601BD80B50101014000003000400000300C83C58B6580000000168E93B3C80',
'channels': 2, 'channels': 2,
'bits_per_sample': 16, 'bits_per_sample': 16,
'nal_unit_length_field': 4 'nal_unit_length_field': 4,
}, },
}, { }, {
'format_id': 'video_deu-3275', 'format_id': 'video_deu-3275',
@@ -1748,7 +1792,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'codec_private_data': '00000001674D4020ECA02802DD80B501010140000003004000000C83C60C65800000000168E93B3C80', 'codec_private_data': '00000001674D4020ECA02802DD80B501010140000003004000000C83C60C65800000000168E93B3C80',
'channels': 2, 'channels': 2,
'bits_per_sample': 16, 'bits_per_sample': 16,
'nal_unit_length_field': 4 'nal_unit_length_field': 4,
}, },
}, { }, {
'format_id': 'video_deu-5300', 'format_id': 'video_deu-5300',
@@ -1773,7 +1817,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'codec_private_data': '00000001674D4028ECA03C0113F2E02D4040405000000300100000030320F18319600000000168E93B3C80', 'codec_private_data': '00000001674D4028ECA03C0113F2E02D4040405000000300100000030320F18319600000000168E93B3C80',
'channels': 2, 'channels': 2,
'bits_per_sample': 16, 'bits_per_sample': 16,
'nal_unit_length_field': 4 'nal_unit_length_field': 4,
}, },
}, { }, {
'format_id': 'video_deu-8079', 'format_id': 'video_deu-8079',
@@ -1798,7 +1842,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'codec_private_data': '00000001674D4028ECA03C0113F2E02D4040405000000300100000030320F18319600000000168E93B3C80', 'codec_private_data': '00000001674D4028ECA03C0113F2E02D4040405000000300100000030320F18319600000000168E93B3C80',
'channels': 2, 'channels': 2,
'bits_per_sample': 16, 'bits_per_sample': 16,
'nal_unit_length_field': 4 'nal_unit_length_field': 4,
}, },
}], }],
{}, {},
@@ -1806,7 +1850,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
] ]
for ism_file, ism_url, expected_formats, expected_subtitles in _TEST_CASES: for ism_file, ism_url, expected_formats, expected_subtitles in _TEST_CASES:
with open('./test/testdata/ism/%s.Manifest' % ism_file, encoding='utf-8') as f: with open(f'./test/testdata/ism/{ism_file}.Manifest', encoding='utf-8') as f:
formats, subtitles = self.ie._parse_ism_formats_and_subtitles( formats, subtitles = self.ie._parse_ism_formats_and_subtitles(
compat_etree_fromstring(f.read().encode()), ism_url=ism_url) compat_etree_fromstring(f.read().encode()), ism_url=ism_url)
self.ie._sort_formats(formats) self.ie._sort_formats(formats)
@@ -1827,12 +1871,12 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
'tbr': 2148, 'tbr': 2148,
'width': 1280, 'width': 1280,
'height': 720, 'height': 720,
}] }],
), ),
] ]
for f4m_file, f4m_url, expected_formats in _TEST_CASES: for f4m_file, f4m_url, expected_formats in _TEST_CASES:
with open('./test/testdata/f4m/%s.f4m' % f4m_file, encoding='utf-8') as f: with open(f'./test/testdata/f4m/{f4m_file}.f4m', encoding='utf-8') as f:
formats = self.ie._parse_f4m_formats( formats = self.ie._parse_f4m_formats(
compat_etree_fromstring(f.read().encode()), compat_etree_fromstring(f.read().encode()),
f4m_url, None) f4m_url, None)
@@ -1873,13 +1917,13 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
}, { }, {
'manifest_url': 'https://example.org/src/foo_xspf.xspf', 'manifest_url': 'https://example.org/src/foo_xspf.xspf',
'url': 'https://example.com/track3.mp3', 'url': 'https://example.com/track3.mp3',
}] }],
}] }],
), ),
] ]
for xspf_file, xspf_url, expected_entries in _TEST_CASES: for xspf_file, xspf_url, expected_entries in _TEST_CASES:
with open('./test/testdata/xspf/%s.xspf' % xspf_file, encoding='utf-8') as f: with open(f'./test/testdata/xspf/{xspf_file}.xspf', encoding='utf-8') as f:
entries = self.ie._parse_xspf( entries = self.ie._parse_xspf(
compat_etree_fromstring(f.read().encode()), compat_etree_fromstring(f.read().encode()),
xspf_file, xspf_url=xspf_url, xspf_base_url=xspf_url) xspf_file, xspf_url=xspf_url, xspf_base_url=xspf_url)
@@ -1901,11 +1945,222 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
server_thread.daemon = True server_thread.daemon = True
server_thread.start() server_thread.start()
(content, urlh) = self.ie._download_webpage_handle( content, _ = self.ie._download_webpage_handle(
'http://127.0.0.1:%d/teapot' % port, None, f'http://127.0.0.1:{port}/teapot', None,
expected_status=TEAPOT_RESPONSE_STATUS) expected_status=TEAPOT_RESPONSE_STATUS)
self.assertEqual(content, TEAPOT_RESPONSE_BODY) self.assertEqual(content, TEAPOT_RESPONSE_BODY)
def test_search_nextjs_data(self):
data = '<script id="__NEXT_DATA__" type="application/json">{"props":{}}</script>'
self.assertEqual(self.ie._search_nextjs_data(data, None), {'props': {}})
self.assertEqual(self.ie._search_nextjs_data('', None, fatal=False), {})
self.assertEqual(self.ie._search_nextjs_data('', None, default=None), None)
self.assertEqual(self.ie._search_nextjs_data('', None, default={}), {})
with self.assertWarns(DeprecationWarning):
self.assertEqual(self.ie._search_nextjs_data('', None, default='{}'), {})
def test_search_nextjs_v13_data(self):
HTML = R'''
<script>(self.__next_f=self.__next_f||[]).push([0])</script>
<script>self.__next_f.push([2,"0:[\"$\",\"$L0\",null,{\"do_not_add_this\":\"fail\"}]\n"])</script>
<script>self.__next_f.push([1,"1:I[46975,[],\"HTTPAccessFallbackBoundary\"]\n2:I[32630,[\"8183\",\"static/chunks/8183-768193f6a9e33cdd.js\"]]\n"])</script>
<script nonce="abc123">self.__next_f.push([1,"e:[false,[\"$\",\"div\",null,{\"children\":[\"$\",\"$L18\",null,{\"foo\":\"bar\"}]}],false]\n "])</script>
<script>self.__next_f.push([1,"2a:[[\"$\",\"div\",null,{\"className\":\"flex flex-col\",\"children\":[]}],[\"$\",\"$L16\",null,{\"meta\":{\"dateCreated\":1730489700,\"uuid\":\"40cac41d-8d29-4ef5-aa11-75047b9f0907\"}}]]\n"])</script>
<script>self.__next_f.push([1,"df:[\"$undefined\",[\"$\",\"div\",null,{\"children\":[\"$\",\"$L17\",null,{}],\"do_not_include_this_field\":\"fail\"}],[\"$\",\"div\",null,{\"children\":[[\"$\",\"$L19\",null,{\"duplicated_field_name\":{\"x\":1}}],[\"$\",\"$L20\",null,{\"duplicated_field_name\":{\"y\":2}}]]}],\"$undefined\"]\n"])</script>
<script>self.__next_f.push([3,"MzM6WyIkIiwiJEwzMiIsbnVsbCx7ImRlY29kZWQiOiJzdWNjZXNzIn1d"])</script>
'''
EXPECTED = {
'18': {
'foo': 'bar',
},
'16': {
'meta': {
'dateCreated': 1730489700,
'uuid': '40cac41d-8d29-4ef5-aa11-75047b9f0907',
},
},
'19': {
'duplicated_field_name': {'x': 1},
},
'20': {
'duplicated_field_name': {'y': 2},
},
}
self.assertEqual(self.ie._search_nextjs_v13_data(HTML, None), EXPECTED)
self.assertEqual(self.ie._search_nextjs_v13_data('', None, fatal=False), {})
self.assertEqual(self.ie._search_nextjs_v13_data(None, None, fatal=False), {})
def test_search_nuxt_json(self):
HTML_TMPL = '<script data-ssr="true" id="__NUXT_DATA__" type="application/json">[{}]</script>'
VALID_DATA = '''
["ShallowReactive",1],
{"data":2,"state":21,"once":25,"_errors":28,"_server_errors":30},
["ShallowReactive",3],
{"$abcdef123456":4},
{"podcast":5,"activeEpisodeData":7},
{"podcast":6,"seasons":14},
{"title":10,"id":11},
["Reactive",8],
{"episode":9,"creators":18,"empty_list":20},
{"title":12,"id":13,"refs":34,"empty_refs":35},
"Series Title",
"podcast-id-01",
"Episode Title",
"episode-id-99",
[15,16,17],
1,
2,
3,
[19],
"Podcast Creator",
[],
{"$ssite-config":22},
{"env":23,"name":24,"map":26,"numbers":14},
"production",
"podcast-website",
["Set"],
["Reactive",27],
["Map"],
["ShallowReactive",29],
{},
["NuxtError",31],
{"status":32,"message":33},
503,
"Service Unavailable",
[36,37],
[38,39],
["Ref",40],
["ShallowRef",41],
["EmptyRef",42],
["EmptyShallowRef",43],
"ref",
"shallow_ref",
"{\\"ref\\":1}",
"{\\"shallow_ref\\":2}"
'''
PAYLOAD = {
'data': {
'$abcdef123456': {
'podcast': {
'podcast': {
'title': 'Series Title',
'id': 'podcast-id-01',
},
'seasons': [1, 2, 3],
},
'activeEpisodeData': {
'episode': {
'title': 'Episode Title',
'id': 'episode-id-99',
'refs': ['ref', 'shallow_ref'],
'empty_refs': [{'ref': 1}, {'shallow_ref': 2}],
},
'creators': ['Podcast Creator'],
'empty_list': [],
},
},
},
'state': {
'$ssite-config': {
'env': 'production',
'name': 'podcast-website',
'map': [],
'numbers': [1, 2, 3],
},
},
'once': [],
'_errors': {},
'_server_errors': {
'status': 503,
'message': 'Service Unavailable',
},
}
PARTIALLY_INVALID = [(
'''
{"data":1},
{"invalid_raw_list":2},
[15,16,17]
''',
{'data': {'invalid_raw_list': [None, None, None]}},
), (
'''
{"data":1},
["EmptyRef",2],
"not valid JSON"
''',
{'data': None},
), (
'''
{"data":1},
["EmptyShallowRef",2],
"not valid JSON"
''',
{'data': None},
)]
INVALID = [
'''
[]
''',
'''
["unsupported",1],
{"data":2},
{}
''',
]
DEFAULT = object()
self.assertEqual(self.ie._search_nuxt_json(HTML_TMPL.format(VALID_DATA), None), PAYLOAD)
self.assertEqual(self.ie._search_nuxt_json('', None, fatal=False), {})
self.assertIs(self.ie._search_nuxt_json('', None, default=DEFAULT), DEFAULT)
for data, expected in PARTIALLY_INVALID:
self.assertEqual(
self.ie._search_nuxt_json(HTML_TMPL.format(data), None, fatal=False), expected)
for data in INVALID:
self.assertIs(
self.ie._search_nuxt_json(HTML_TMPL.format(data), None, default=DEFAULT), DEFAULT)
class TestInfoExtractorNetwork(unittest.TestCase):
def setUp(self, /):
self.httpd = http.server.HTTPServer(
('127.0.0.1', 0), InfoExtractorTestRequestHandler)
self.port = http_server_port(self.httpd)
self.server_thread = threading.Thread(target=self.httpd.serve_forever)
self.server_thread.daemon = True
self.server_thread.start()
self.called = False
def require_warning(*args, **kwargs):
self.called = True
self.ydl = FakeYDL()
self.ydl.report_warning = require_warning
self.ie = DummyIE(self.ydl)
def tearDown(self, /):
self.ydl.close()
self.httpd.shutdown()
self.httpd.server_close()
self.server_thread.join(1)
def test_extract_m3u8_formats(self):
formats, subtitles = self.ie._extract_m3u8_formats_and_subtitles(
f'http://127.0.0.1:{self.port}/bipbop.m3u8', None, fatal=False)
self.assertFalse(self.called)
self.assertTrue(formats)
self.assertTrue(subtitles)
def test_extract_m3u8_formats_warning(self):
formats, subtitles = self.ie._extract_m3u8_formats_and_subtitles(
f'http://127.0.0.1:{self.port}/fake.m3u8', None, fatal=False)
self.assertTrue(self.called, 'Warning was not issued for binary m3u8 file')
self.assertFalse(formats)
self.assertFalse(subtitles)
if __name__ == '__main__': if __name__ == '__main__':
unittest.main() unittest.main()
+153 -38
View File
@@ -4,16 +4,19 @@
import os import os
import sys import sys
import unittest import unittest
from unittest.mock import patch
from yt_dlp.globals import all_plugins_loaded
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import contextlib
import copy import copy
import json import json
from test.helper import FakeYDL, assertRegexpMatches, try_rm from test.helper import FakeYDL, assertRegexpMatches, try_rm
from yt_dlp import YoutubeDL from yt_dlp import YoutubeDL
from yt_dlp.compat import compat_os_name
from yt_dlp.extractor import YoutubeIE from yt_dlp.extractor import YoutubeIE
from yt_dlp.extractor.common import InfoExtractor from yt_dlp.extractor.common import InfoExtractor
from yt_dlp.postprocessor.common import PostProcessor from yt_dlp.postprocessor.common import PostProcessor
@@ -129,8 +132,8 @@ class TestFormatSelection(unittest.TestCase):
'allow_multiple_audio_streams': multi, 'allow_multiple_audio_streams': multi,
}) })
ydl.process_ie_result(info_dict.copy()) ydl.process_ie_result(info_dict.copy())
downloaded = map(lambda x: x['format_id'], ydl.downloaded_info_dicts) downloaded = [x['format_id'] for x in ydl.downloaded_info_dicts]
self.assertEqual(list(downloaded), list(expected)) self.assertEqual(downloaded, list(expected))
test('20/47', '47') test('20/47', '47')
test('20/71/worst', '35') test('20/71/worst', '35')
@@ -183,7 +186,7 @@ class TestFormatSelection(unittest.TestCase):
] ]
info_dict = _make_result(formats) info_dict = _make_result(formats)
ydl = YDL({'format': 'best'}) ydl = YDL({'format': 'best', 'format_sort': ['abr', 'ext']})
ydl.sort_formats(info_dict) ydl.sort_formats(info_dict)
ydl.process_ie_result(copy.deepcopy(info_dict)) ydl.process_ie_result(copy.deepcopy(info_dict))
downloaded = ydl.downloaded_info_dicts[0] downloaded = ydl.downloaded_info_dicts[0]
@@ -195,7 +198,7 @@ class TestFormatSelection(unittest.TestCase):
downloaded = ydl.downloaded_info_dicts[0] downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'mp3-64') self.assertEqual(downloaded['format_id'], 'mp3-64')
ydl = YDL({'prefer_free_formats': True}) ydl = YDL({'prefer_free_formats': True, 'format_sort': ['abr', 'ext']})
ydl.sort_formats(info_dict) ydl.sort_formats(info_dict)
ydl.process_ie_result(copy.deepcopy(info_dict)) ydl.process_ie_result(copy.deepcopy(info_dict))
downloaded = ydl.downloaded_info_dicts[0] downloaded = ydl.downloaded_info_dicts[0]
@@ -234,6 +237,35 @@ class TestFormatSelection(unittest.TestCase):
downloaded = ydl.downloaded_info_dicts[0] downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'vid-vcodec-dot') self.assertEqual(downloaded['format_id'], 'vid-vcodec-dot')
def test_format_selection_by_vcodec_sort(self):
formats = [
{'format_id': 'av1-format', 'ext': 'mp4', 'vcodec': 'av1', 'acodec': 'none', 'url': TEST_URL},
{'format_id': 'vp9-hdr-format', 'ext': 'mp4', 'vcodec': 'vp09.02.50.10.01.09.18.09.00', 'acodec': 'none', 'url': TEST_URL},
{'format_id': 'vp9-sdr-format', 'ext': 'mp4', 'vcodec': 'vp09.00.50.08', 'acodec': 'none', 'url': TEST_URL},
{'format_id': 'h265-format', 'ext': 'mp4', 'vcodec': 'h265', 'acodec': 'none', 'url': TEST_URL},
]
info_dict = _make_result(formats)
ydl = YDL({'format': 'bestvideo', 'format_sort': ['vcodec:vp9.2']})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'vp9-hdr-format')
ydl = YDL({'format': 'bestvideo', 'format_sort': ['vcodec:vp9']})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'vp9-sdr-format')
ydl = YDL({'format': 'bestvideo', 'format_sort': ['+vcodec:vp9.2']})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'vp9-hdr-format')
ydl = YDL({'format': 'bestvideo', 'format_sort': ['+vcodec:vp9']})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'vp9-sdr-format')
def test_format_selection_string_ops(self): def test_format_selection_string_ops(self):
formats = [ formats = [
{'format_id': 'abc-cba', 'ext': 'mp4', 'url': TEST_URL}, {'format_id': 'abc-cba', 'ext': 'mp4', 'url': TEST_URL},
@@ -456,11 +488,11 @@ class TestFormatSelection(unittest.TestCase):
def test_format_filtering(self): def test_format_filtering(self):
formats = [ formats = [
{'format_id': 'A', 'filesize': 500, 'width': 1000}, {'format_id': 'A', 'filesize': 500, 'width': 1000, 'aspect_ratio': 1.0},
{'format_id': 'B', 'filesize': 1000, 'width': 500}, {'format_id': 'B', 'filesize': 1000, 'width': 500, 'aspect_ratio': 1.33},
{'format_id': 'C', 'filesize': 1000, 'width': 400}, {'format_id': 'C', 'filesize': 1000, 'width': 400, 'aspect_ratio': 1.5},
{'format_id': 'D', 'filesize': 2000, 'width': 600}, {'format_id': 'D', 'filesize': 2000, 'width': 600, 'aspect_ratio': 1.78},
{'format_id': 'E', 'filesize': 3000}, {'format_id': 'E', 'filesize': 3000, 'aspect_ratio': 0.56},
{'format_id': 'F'}, {'format_id': 'F'},
{'format_id': 'G', 'filesize': 1000000}, {'format_id': 'G', 'filesize': 1000000},
] ]
@@ -515,13 +547,62 @@ class TestFormatSelection(unittest.TestCase):
self.assertEqual(downloaded_ids, ['D', 'C', 'B']) self.assertEqual(downloaded_ids, ['D', 'C', 'B'])
ydl = YDL({'format': 'best[height<40]'}) ydl = YDL({'format': 'best[height<40]'})
try: with contextlib.suppress(ExtractorError):
ydl.process_ie_result(info_dict) ydl.process_ie_result(info_dict)
except ExtractorError:
pass
self.assertEqual(ydl.downloaded_info_dicts, []) self.assertEqual(ydl.downloaded_info_dicts, [])
def test_default_format_spec(self): ydl = YDL({'format': 'best[aspect_ratio=1]'})
ydl.process_ie_result(info_dict)
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'A')
ydl = YDL({'format': 'all[aspect_ratio > 1.00]'})
ydl.process_ie_result(info_dict)
downloaded_ids = [info['format_id'] for info in ydl.downloaded_info_dicts]
self.assertEqual(downloaded_ids, ['D', 'C', 'B'])
ydl = YDL({'format': 'all[aspect_ratio < 1.00]'})
ydl.process_ie_result(info_dict)
downloaded_ids = [info['format_id'] for info in ydl.downloaded_info_dicts]
self.assertEqual(downloaded_ids, ['E'])
ydl = YDL({'format': 'best[aspect_ratio=1.5]'})
ydl.process_ie_result(info_dict)
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'C')
ydl = YDL({'format': 'all[aspect_ratio!=1]'})
ydl.process_ie_result(info_dict)
downloaded_ids = [info['format_id'] for info in ydl.downloaded_info_dicts]
self.assertEqual(downloaded_ids, ['E', 'D', 'C', 'B'])
@patch('yt_dlp.postprocessor.ffmpeg.FFmpegMergerPP.available', False)
def test_default_format_spec_without_ffmpeg(self):
ydl = YDL({})
self.assertEqual(ydl._default_format_spec({}), 'best/bestvideo+bestaudio')
ydl = YDL({'simulate': True})
self.assertEqual(ydl._default_format_spec({}), 'best/bestvideo+bestaudio')
ydl = YDL({})
self.assertEqual(ydl._default_format_spec({'is_live': True}), 'best/bestvideo+bestaudio')
ydl = YDL({'simulate': True})
self.assertEqual(ydl._default_format_spec({'is_live': True}), 'best/bestvideo+bestaudio')
ydl = YDL({'outtmpl': '-'})
self.assertEqual(ydl._default_format_spec({}), 'best/bestvideo+bestaudio')
ydl = YDL({})
self.assertEqual(ydl._default_format_spec({}), 'best/bestvideo+bestaudio')
self.assertEqual(ydl._default_format_spec({'is_live': True}), 'best/bestvideo+bestaudio')
@patch('yt_dlp.postprocessor.ffmpeg.FFmpegMergerPP.available', True)
@patch('yt_dlp.postprocessor.ffmpeg.FFmpegMergerPP.can_merge', lambda _: True)
def test_default_format_spec_with_ffmpeg(self):
ydl = YDL({})
self.assertEqual(ydl._default_format_spec({}), 'bestvideo*+bestaudio/best')
ydl = YDL({'simulate': True}) ydl = YDL({'simulate': True})
self.assertEqual(ydl._default_format_spec({}), 'bestvideo*+bestaudio/best') self.assertEqual(ydl._default_format_spec({}), 'bestvideo*+bestaudio/best')
@@ -529,13 +610,13 @@ class TestFormatSelection(unittest.TestCase):
self.assertEqual(ydl._default_format_spec({'is_live': True}), 'best/bestvideo+bestaudio') self.assertEqual(ydl._default_format_spec({'is_live': True}), 'best/bestvideo+bestaudio')
ydl = YDL({'simulate': True}) ydl = YDL({'simulate': True})
self.assertEqual(ydl._default_format_spec({'is_live': True}), 'bestvideo*+bestaudio/best') self.assertEqual(ydl._default_format_spec({'is_live': True}), 'best/bestvideo+bestaudio')
ydl = YDL({'outtmpl': '-'}) ydl = YDL({'outtmpl': '-'})
self.assertEqual(ydl._default_format_spec({}), 'best/bestvideo+bestaudio') self.assertEqual(ydl._default_format_spec({}), 'best/bestvideo+bestaudio')
ydl = YDL({}) ydl = YDL({})
self.assertEqual(ydl._default_format_spec({}, download=False), 'bestvideo*+bestaudio/best') self.assertEqual(ydl._default_format_spec({}), 'bestvideo*+bestaudio/best')
self.assertEqual(ydl._default_format_spec({'is_live': True}), 'best/bestvideo+bestaudio') self.assertEqual(ydl._default_format_spec({'is_live': True}), 'best/bestvideo+bestaudio')
@@ -652,8 +733,8 @@ class TestYoutubeDL(unittest.TestCase):
'formats': [ 'formats': [
{'id': 'id 1', 'height': 1080, 'width': 1920}, {'id': 'id 1', 'height': 1080, 'width': 1920},
{'id': 'id 2', 'height': 720}, {'id': 'id 2', 'height': 720},
{'id': 'id 3'} {'id': 'id 3'},
] ],
} }
def test_prepare_outtmpl_and_filename(self): def test_prepare_outtmpl_and_filename(self):
@@ -707,6 +788,13 @@ class TestYoutubeDL(unittest.TestCase):
test('%(width)06d.%%(ext)s', 'NA.%(ext)s') test('%(width)06d.%%(ext)s', 'NA.%(ext)s')
test('%%(width)06d.%(ext)s', '%(width)06d.mp4') test('%%(width)06d.%(ext)s', '%(width)06d.mp4')
# Sanitization options
test('%(title3)s', (None, 'foobartest'))
test('%(title5)s', (None, 'aei_A'), restrictfilenames=True)
test('%(title3)s', (None, 'foo_bar_test'), windowsfilenames=False, restrictfilenames=True)
if sys.platform != 'win32':
test('%(title3)s', (None, 'foobar\\test'), windowsfilenames=False)
# ID sanitization # ID sanitization
test('%(id)s', '_abcd', info={'id': '_abcd'}) test('%(id)s', '_abcd', info={'id': '_abcd'})
test('%(some_id)s', '_abcd', info={'some_id': '_abcd'}) test('%(some_id)s', '_abcd', info={'some_id': '_abcd'})
@@ -773,7 +861,7 @@ class TestYoutubeDL(unittest.TestCase):
test('%(formats)j', (json.dumps(FORMATS), None)) test('%(formats)j', (json.dumps(FORMATS), None))
test('%(formats)#j', ( test('%(formats)#j', (
json.dumps(FORMATS, indent=4), json.dumps(FORMATS, indent=4),
json.dumps(FORMATS, indent=4).replace(':', '').replace('"', "").replace('\n', ' ') json.dumps(FORMATS, indent=4).replace(':', '').replace('"', '').replace('\n', ' '),
)) ))
test('%(title5).3B', 'á') test('%(title5).3B', 'á')
test('%(title5)U', 'áéí 𝐀') test('%(title5)U', 'áéí 𝐀')
@@ -784,8 +872,8 @@ class TestYoutubeDL(unittest.TestCase):
test('%(filesize)#D', '1Ki') test('%(filesize)#D', '1Ki')
test('%(height)5.2D', ' 1.08k') test('%(height)5.2D', ' 1.08k')
test('%(title4)#S', 'foo_bar_test') test('%(title4)#S', 'foo_bar_test')
test('%(title4).10S', ('foo bar ', 'foo bar' + ('#' if compat_os_name == 'nt' else ' '))) test('%(title4).10S', ('foo bar ', 'foo bar' + ('#' if os.name == 'nt' else ' ')))
if compat_os_name == 'nt': if os.name == 'nt':
test('%(title4)q', ('"foo ""bar"" test"', None)) test('%(title4)q', ('"foo ""bar"" test"', None))
test('%(formats.:.id)#q', ('"id 1" "id 2" "id 3"', None)) test('%(formats.:.id)#q', ('"id 1" "id 2" "id 3"', None))
test('%(formats.0.id)#q', ('"id 1"', None)) test('%(formats.0.id)#q', ('"id 1"', None))
@@ -843,14 +931,14 @@ class TestYoutubeDL(unittest.TestCase):
# Empty filename # Empty filename
test('%(foo|)s-%(bar|)s.%(ext)s', '-.mp4') test('%(foo|)s-%(bar|)s.%(ext)s', '-.mp4')
# test('%(foo|)s.%(ext)s', ('.mp4', '_.mp4')) # fixme # test('%(foo|)s.%(ext)s', ('.mp4', '_.mp4')) # FIXME: ?
# test('%(foo|)s', ('', '_')) # fixme # test('%(foo|)s', ('', '_')) # FIXME: ?
# Environment variable expansion for prepare_filename # Environment variable expansion for prepare_filename
os.environ['__yt_dlp_var'] = 'expanded' os.environ['__yt_dlp_var'] = 'expanded'
envvar = '%__yt_dlp_var%' if compat_os_name == 'nt' else '$__yt_dlp_var' envvar = '%__yt_dlp_var%' if os.name == 'nt' else '$__yt_dlp_var'
test(envvar, (envvar, 'expanded')) test(envvar, (envvar, 'expanded'))
if compat_os_name == 'nt': if os.name == 'nt':
test('%s%', ('%s%', '%s%')) test('%s%', ('%s%', '%s%'))
os.environ['s'] = 'expanded' os.environ['s'] = 'expanded'
test('%s%', ('%s%', 'expanded')) # %s% should be expanded before escaping %s test('%s%', ('%s%', 'expanded')) # %s% should be expanded before escaping %s
@@ -861,7 +949,7 @@ class TestYoutubeDL(unittest.TestCase):
test('Hello %(title1)s', 'Hello $PATH') test('Hello %(title1)s', 'Hello $PATH')
test('Hello %(title2)s', 'Hello %PATH%') test('Hello %(title2)s', 'Hello %PATH%')
test('%(title3)s', ('foo/bar\\test', 'foobartest')) test('%(title3)s', ('foo/bar\\test', 'foobartest'))
test('folder/%(title3)s', ('folder/foo/bar\\test', 'folder%sfoobartest' % os.path.sep)) test('folder/%(title3)s', ('folder/foo/bar\\test', f'folder{os.path.sep}foobartest'))
def test_format_note(self): def test_format_note(self):
ydl = YoutubeDL() ydl = YoutubeDL()
@@ -883,22 +971,22 @@ class TestYoutubeDL(unittest.TestCase):
f.write('EXAMPLE') f.write('EXAMPLE')
return [info['filepath']], info return [info['filepath']], info
def run_pp(params, PP): def run_pp(params, pp):
with open(filename, 'w') as f: with open(filename, 'w') as f:
f.write('EXAMPLE') f.write('EXAMPLE')
ydl = YoutubeDL(params) ydl = YoutubeDL(params)
ydl.add_post_processor(PP()) ydl.add_post_processor(pp())
ydl.post_process(filename, {'filepath': filename}) ydl.post_process(filename, {'filepath': filename})
run_pp({'keepvideo': True}, SimplePP) run_pp({'keepvideo': True}, SimplePP)
self.assertTrue(os.path.exists(filename), '%s doesn\'t exist' % filename) self.assertTrue(os.path.exists(filename), f'{filename} doesn\'t exist')
self.assertTrue(os.path.exists(audiofile), '%s doesn\'t exist' % audiofile) self.assertTrue(os.path.exists(audiofile), f'{audiofile} doesn\'t exist')
os.unlink(filename) os.unlink(filename)
os.unlink(audiofile) os.unlink(audiofile)
run_pp({'keepvideo': False}, SimplePP) run_pp({'keepvideo': False}, SimplePP)
self.assertFalse(os.path.exists(filename), '%s exists' % filename) self.assertFalse(os.path.exists(filename), f'{filename} exists')
self.assertTrue(os.path.exists(audiofile), '%s doesn\'t exist' % audiofile) self.assertTrue(os.path.exists(audiofile), f'{audiofile} doesn\'t exist')
os.unlink(audiofile) os.unlink(audiofile)
class ModifierPP(PostProcessor): class ModifierPP(PostProcessor):
@@ -908,7 +996,7 @@ class TestYoutubeDL(unittest.TestCase):
return [], info return [], info
run_pp({'keepvideo': False}, ModifierPP) run_pp({'keepvideo': False}, ModifierPP)
self.assertTrue(os.path.exists(filename), '%s doesn\'t exist' % filename) self.assertTrue(os.path.exists(filename), f'{filename} doesn\'t exist')
os.unlink(filename) os.unlink(filename)
def test_match_filter(self): def test_match_filter(self):
@@ -920,7 +1008,7 @@ class TestYoutubeDL(unittest.TestCase):
'duration': 30, 'duration': 30,
'filesize': 10 * 1024, 'filesize': 10 * 1024,
'playlist_id': '42', 'playlist_id': '42',
'uploader': "變態妍字幕版 太妍 тест", 'uploader': '變態妍字幕版 太妍 тест',
'creator': "тест ' 123 ' тест--", 'creator': "тест ' 123 ' тест--",
'webpage_url': 'http://example.com/watch?v=shenanigans', 'webpage_url': 'http://example.com/watch?v=shenanigans',
} }
@@ -933,7 +1021,7 @@ class TestYoutubeDL(unittest.TestCase):
'description': 'foo', 'description': 'foo',
'filesize': 5 * 1024, 'filesize': 5 * 1024,
'playlist_id': '43', 'playlist_id': '43',
'uploader': "тест 123", 'uploader': 'тест 123',
'webpage_url': 'http://example.com/watch?v=SHENANIGANS', 'webpage_url': 'http://example.com/watch?v=SHENANIGANS',
} }
videos = [first, second] videos = [first, second]
@@ -1180,7 +1268,7 @@ class TestYoutubeDL(unittest.TestCase):
}) })
return { return {
'id': video_id, 'id': video_id,
'title': 'Video %s' % video_id, 'title': f'Video {video_id}',
'formats': formats, 'formats': formats,
} }
@@ -1194,8 +1282,8 @@ class TestYoutubeDL(unittest.TestCase):
'_type': 'url_transparent', '_type': 'url_transparent',
'ie_key': VideoIE.ie_key(), 'ie_key': VideoIE.ie_key(),
'id': video_id, 'id': video_id,
'url': 'video:%s' % video_id, 'url': f'video:{video_id}',
'title': 'Video Transparent %s' % video_id, 'title': f'Video Transparent {video_id}',
} }
def _real_extract(self, url): def _real_extract(self, url):
@@ -1341,6 +1429,33 @@ class TestYoutubeDL(unittest.TestCase):
self.assertFalse(result.get('cookies'), msg='Cookies set in cookies field for wrong domain') self.assertFalse(result.get('cookies'), msg='Cookies set in cookies field for wrong domain')
self.assertFalse(ydl.cookiejar.get_cookie_header(fmt['url']), msg='Cookies set in cookiejar for wrong domain') self.assertFalse(ydl.cookiejar.get_cookie_header(fmt['url']), msg='Cookies set in cookiejar for wrong domain')
def test_load_plugins_compat(self):
# Should try to reload plugins if they haven't already been loaded
all_plugins_loaded.value = False
FakeYDL().close()
assert all_plugins_loaded.value
def test_close_hooks(self):
# Should call all registered close hooks on close
close_hook_called = False
close_hook_two_called = False
def close_hook():
nonlocal close_hook_called
close_hook_called = True
def close_hook_two():
nonlocal close_hook_two_called
close_hook_two_called = True
ydl = FakeYDL()
ydl.add_close_hook(close_hook)
ydl.add_close_hook(close_hook_two)
ydl.close()
self.assertTrue(close_hook_called, 'Close hook was not called')
self.assertTrue(close_hook_two_called, 'Close hook two was not called')
if __name__ == '__main__': if __name__ == '__main__':
unittest.main() unittest.main()
+40 -29
View File
@@ -27,7 +27,6 @@ from yt_dlp.aes import (
pad_block, pad_block,
) )
from yt_dlp.dependencies import Cryptodome from yt_dlp.dependencies import Cryptodome
from yt_dlp.utils import bytes_to_intlist, intlist_to_bytes
# the encrypted data can be generate with 'devscripts/generate_aes_testdata.py' # the encrypted data can be generate with 'devscripts/generate_aes_testdata.py'
@@ -40,33 +39,33 @@ class TestAES(unittest.TestCase):
def test_encrypt(self): def test_encrypt(self):
msg = b'message' msg = b'message'
key = list(range(16)) key = list(range(16))
encrypted = aes_encrypt(bytes_to_intlist(msg), key) encrypted = aes_encrypt(list(msg), key)
decrypted = intlist_to_bytes(aes_decrypt(encrypted, key)) decrypted = bytes(aes_decrypt(encrypted, key))
self.assertEqual(decrypted, msg) self.assertEqual(decrypted, msg)
def test_cbc_decrypt(self): def test_cbc_decrypt(self):
data = b'\x97\x92+\xe5\x0b\xc3\x18\x91ky9m&\xb3\xb5@\xe6\x27\xc2\x96.\xc8u\x88\xab9-[\x9e|\xf1\xcd' data = b'\x97\x92+\xe5\x0b\xc3\x18\x91ky9m&\xb3\xb5@\xe6\x27\xc2\x96.\xc8u\x88\xab9-[\x9e|\xf1\xcd'
decrypted = intlist_to_bytes(aes_cbc_decrypt(bytes_to_intlist(data), self.key, self.iv)) decrypted = bytes(aes_cbc_decrypt(list(data), self.key, self.iv))
self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg) self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg)
if Cryptodome.AES: if Cryptodome.AES:
decrypted = aes_cbc_decrypt_bytes(data, intlist_to_bytes(self.key), intlist_to_bytes(self.iv)) decrypted = aes_cbc_decrypt_bytes(data, bytes(self.key), bytes(self.iv))
self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg) self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg)
def test_cbc_encrypt(self): def test_cbc_encrypt(self):
data = bytes_to_intlist(self.secret_msg) data = list(self.secret_msg)
encrypted = intlist_to_bytes(aes_cbc_encrypt(data, self.key, self.iv)) encrypted = bytes(aes_cbc_encrypt(data, self.key, self.iv))
self.assertEqual( self.assertEqual(
encrypted, encrypted,
b'\x97\x92+\xe5\x0b\xc3\x18\x91ky9m&\xb3\xb5@\xe6\'\xc2\x96.\xc8u\x88\xab9-[\x9e|\xf1\xcd') b'\x97\x92+\xe5\x0b\xc3\x18\x91ky9m&\xb3\xb5@\xe6\'\xc2\x96.\xc8u\x88\xab9-[\x9e|\xf1\xcd')
def test_ctr_decrypt(self): def test_ctr_decrypt(self):
data = bytes_to_intlist(b'\x03\xc7\xdd\xd4\x8e\xb3\xbc\x1a*O\xdc1\x12+8Aio\xd1z\xb5#\xaf\x08') data = list(b'\x03\xc7\xdd\xd4\x8e\xb3\xbc\x1a*O\xdc1\x12+8Aio\xd1z\xb5#\xaf\x08')
decrypted = intlist_to_bytes(aes_ctr_decrypt(data, self.key, self.iv)) decrypted = bytes(aes_ctr_decrypt(data, self.key, self.iv))
self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg) self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg)
def test_ctr_encrypt(self): def test_ctr_encrypt(self):
data = bytes_to_intlist(self.secret_msg) data = list(self.secret_msg)
encrypted = intlist_to_bytes(aes_ctr_encrypt(data, self.key, self.iv)) encrypted = bytes(aes_ctr_encrypt(data, self.key, self.iv))
self.assertEqual( self.assertEqual(
encrypted, encrypted,
b'\x03\xc7\xdd\xd4\x8e\xb3\xbc\x1a*O\xdc1\x12+8Aio\xd1z\xb5#\xaf\x08') b'\x03\xc7\xdd\xd4\x8e\xb3\xbc\x1a*O\xdc1\x12+8Aio\xd1z\xb5#\xaf\x08')
@@ -75,47 +74,59 @@ class TestAES(unittest.TestCase):
data = b'\x159Y\xcf5eud\x90\x9c\x85&]\x14\x1d\x0f.\x08\xb4T\xe4/\x17\xbd' data = b'\x159Y\xcf5eud\x90\x9c\x85&]\x14\x1d\x0f.\x08\xb4T\xe4/\x17\xbd'
authentication_tag = b'\xe8&I\x80rI\x07\x9d}YWuU@:e' authentication_tag = b'\xe8&I\x80rI\x07\x9d}YWuU@:e'
decrypted = intlist_to_bytes(aes_gcm_decrypt_and_verify( decrypted = bytes(aes_gcm_decrypt_and_verify(
bytes_to_intlist(data), self.key, bytes_to_intlist(authentication_tag), self.iv[:12])) list(data), self.key, list(authentication_tag), self.iv[:12]))
self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg) self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg)
if Cryptodome.AES: if Cryptodome.AES:
decrypted = aes_gcm_decrypt_and_verify_bytes( decrypted = aes_gcm_decrypt_and_verify_bytes(
data, intlist_to_bytes(self.key), authentication_tag, intlist_to_bytes(self.iv[:12])) data, bytes(self.key), authentication_tag, bytes(self.iv[:12]))
self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg) self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg)
def test_gcm_aligned_decrypt(self):
data = b'\x159Y\xcf5eud\x90\x9c\x85&]\x14\x1d\x0f'
authentication_tag = b'\x08\xb1\x9d!&\x98\xd0\xeaRq\x90\xe6;\xb5]\xd8'
decrypted = bytes(aes_gcm_decrypt_and_verify(
list(data), self.key, list(authentication_tag), self.iv[:12]))
self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg[:16])
if Cryptodome.AES:
decrypted = aes_gcm_decrypt_and_verify_bytes(
data, bytes(self.key), authentication_tag, bytes(self.iv[:12]))
self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg[:16])
def test_decrypt_text(self): def test_decrypt_text(self):
password = intlist_to_bytes(self.key).decode() password = bytes(self.key).decode()
encrypted = base64.b64encode( encrypted = base64.b64encode(
intlist_to_bytes(self.iv[:8]) bytes(self.iv[:8])
+ b'\x17\x15\x93\xab\x8d\x80V\xcdV\xe0\t\xcdo\xc2\xa5\xd8ksM\r\xe27N\xae' + b'\x17\x15\x93\xab\x8d\x80V\xcdV\xe0\t\xcdo\xc2\xa5\xd8ksM\r\xe27N\xae',
).decode() ).decode()
decrypted = (aes_decrypt_text(encrypted, password, 16)) decrypted = (aes_decrypt_text(encrypted, password, 16))
self.assertEqual(decrypted, self.secret_msg) self.assertEqual(decrypted, self.secret_msg)
password = intlist_to_bytes(self.key).decode() password = bytes(self.key).decode()
encrypted = base64.b64encode( encrypted = base64.b64encode(
intlist_to_bytes(self.iv[:8]) bytes(self.iv[:8])
+ b'\x0b\xe6\xa4\xd9z\x0e\xb8\xb9\xd0\xd4i_\x85\x1d\x99\x98_\xe5\x80\xe7.\xbf\xa5\x83' + b'\x0b\xe6\xa4\xd9z\x0e\xb8\xb9\xd0\xd4i_\x85\x1d\x99\x98_\xe5\x80\xe7.\xbf\xa5\x83',
).decode() ).decode()
decrypted = (aes_decrypt_text(encrypted, password, 32)) decrypted = (aes_decrypt_text(encrypted, password, 32))
self.assertEqual(decrypted, self.secret_msg) self.assertEqual(decrypted, self.secret_msg)
def test_ecb_encrypt(self): def test_ecb_encrypt(self):
data = bytes_to_intlist(self.secret_msg) data = list(self.secret_msg)
encrypted = intlist_to_bytes(aes_ecb_encrypt(data, self.key)) encrypted = bytes(aes_ecb_encrypt(data, self.key))
self.assertEqual( self.assertEqual(
encrypted, encrypted,
b'\xaa\x86]\x81\x97>\x02\x92\x9d\x1bR[[L/u\xd3&\xd1(h\xde{\x81\x94\xba\x02\xae\xbd\xa6\xd0:') b'\xaa\x86]\x81\x97>\x02\x92\x9d\x1bR[[L/u\xd3&\xd1(h\xde{\x81\x94\xba\x02\xae\xbd\xa6\xd0:')
def test_ecb_decrypt(self): def test_ecb_decrypt(self):
data = bytes_to_intlist(b'\xaa\x86]\x81\x97>\x02\x92\x9d\x1bR[[L/u\xd3&\xd1(h\xde{\x81\x94\xba\x02\xae\xbd\xa6\xd0:') data = list(b'\xaa\x86]\x81\x97>\x02\x92\x9d\x1bR[[L/u\xd3&\xd1(h\xde{\x81\x94\xba\x02\xae\xbd\xa6\xd0:')
decrypted = intlist_to_bytes(aes_ecb_decrypt(data, self.key, self.iv)) decrypted = bytes(aes_ecb_decrypt(data, self.key, self.iv))
self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg) self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg)
def test_key_expansion(self): def test_key_expansion(self):
key = '4f6bdaa39e2f8cb07f5e722d9edef314' key = '4f6bdaa39e2f8cb07f5e722d9edef314'
self.assertEqual(key_expansion(bytes_to_intlist(bytearray.fromhex(key))), [ self.assertEqual(key_expansion(list(bytearray.fromhex(key))), [
0x4F, 0x6B, 0xDA, 0xA3, 0x9E, 0x2F, 0x8C, 0xB0, 0x7F, 0x5E, 0x72, 0x2D, 0x9E, 0xDE, 0xF3, 0x14, 0x4F, 0x6B, 0xDA, 0xA3, 0x9E, 0x2F, 0x8C, 0xB0, 0x7F, 0x5E, 0x72, 0x2D, 0x9E, 0xDE, 0xF3, 0x14,
0x53, 0x66, 0x20, 0xA8, 0xCD, 0x49, 0xAC, 0x18, 0xB2, 0x17, 0xDE, 0x35, 0x2C, 0xC9, 0x2D, 0x21, 0x53, 0x66, 0x20, 0xA8, 0xCD, 0x49, 0xAC, 0x18, 0xB2, 0x17, 0xDE, 0x35, 0x2C, 0xC9, 0x2D, 0x21,
0x8C, 0xBE, 0xDD, 0xD9, 0x41, 0xF7, 0x71, 0xC1, 0xF3, 0xE0, 0xAF, 0xF4, 0xDF, 0x29, 0x82, 0xD5, 0x8C, 0xBE, 0xDD, 0xD9, 0x41, 0xF7, 0x71, 0xC1, 0xF3, 0xE0, 0xAF, 0xF4, 0xDF, 0x29, 0x82, 0xD5,
@@ -132,16 +143,16 @@ class TestAES(unittest.TestCase):
block = [0x21, 0xA0, 0x43, 0xFF] block = [0x21, 0xA0, 0x43, 0xFF]
self.assertEqual(pad_block(block, 'pkcs7'), self.assertEqual(pad_block(block, 'pkcs7'),
block + [0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C]) [*block, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C])
self.assertEqual(pad_block(block, 'iso7816'), self.assertEqual(pad_block(block, 'iso7816'),
block + [0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]) [*block, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00])
self.assertEqual(pad_block(block, 'whitespace'), self.assertEqual(pad_block(block, 'whitespace'),
block + [0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20]) [*block, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20])
self.assertEqual(pad_block(block, 'zero'), self.assertEqual(pad_block(block, 'zero'),
block + [0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]) [*block, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00])
block = list(range(16)) block = list(range(16))
for mode in ('pkcs7', 'iso7816', 'whitespace', 'zero'): for mode in ('pkcs7', 'iso7816', 'whitespace', 'zero'):
+43 -44
View File
@@ -1,6 +1,7 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# Allow direct execution # Allow direct execution
import datetime as dt
import os import os
import sys import sys
import unittest import unittest
@@ -12,27 +13,19 @@ import struct
from yt_dlp import compat from yt_dlp import compat
from yt_dlp.compat import urllib # isort: split from yt_dlp.compat import urllib # isort: split
from yt_dlp.compat import ( from yt_dlp.compat import compat_etree_fromstring, compat_expanduser, compat_datetime_from_timestamp
compat_etree_fromstring,
compat_expanduser,
compat_urllib_parse_unquote,
compat_urllib_parse_urlencode,
)
from yt_dlp.compat.urllib.request import getproxies from yt_dlp.compat.urllib.request import getproxies
class TestCompat(unittest.TestCase): class TestCompat(unittest.TestCase):
def test_compat_passthrough(self): def test_compat_passthrough(self):
with self.assertWarns(DeprecationWarning): with self.assertWarns(DeprecationWarning):
compat.compat_basestring _ = compat.compat_basestring
with self.assertWarns(DeprecationWarning):
compat.WINDOWS_VT_MODE
self.assertEqual(urllib.request.getproxies, getproxies) self.assertEqual(urllib.request.getproxies, getproxies)
with self.assertWarns(DeprecationWarning): with self.assertWarns(DeprecationWarning):
compat.compat_pycrypto_AES # Must not raise error _ = compat.compat_pycrypto_AES # Must not raise error
def test_compat_expanduser(self): def test_compat_expanduser(self):
old_home = os.environ.get('HOME') old_home = os.environ.get('HOME')
@@ -43,39 +36,6 @@ class TestCompat(unittest.TestCase):
finally: finally:
os.environ['HOME'] = old_home or '' os.environ['HOME'] = old_home or ''
def test_compat_urllib_parse_unquote(self):
self.assertEqual(compat_urllib_parse_unquote('abc%20def'), 'abc def')
self.assertEqual(compat_urllib_parse_unquote('%7e/abc+def'), '~/abc+def')
self.assertEqual(compat_urllib_parse_unquote(''), '')
self.assertEqual(compat_urllib_parse_unquote('%'), '%')
self.assertEqual(compat_urllib_parse_unquote('%%'), '%%')
self.assertEqual(compat_urllib_parse_unquote('%%%'), '%%%')
self.assertEqual(compat_urllib_parse_unquote('%2F'), '/')
self.assertEqual(compat_urllib_parse_unquote('%2f'), '/')
self.assertEqual(compat_urllib_parse_unquote('%E6%B4%A5%E6%B3%A2'), '津波')
self.assertEqual(
compat_urllib_parse_unquote('''<meta property="og:description" content="%E2%96%81%E2%96%82%E2%96%83%E2%96%84%25%E2%96%85%E2%96%86%E2%96%87%E2%96%88" />
%<a href="https://ar.wikipedia.org/wiki/%D8%AA%D8%B3%D9%88%D9%86%D8%A7%D9%85%D9%8A">%a'''),
'''<meta property="og:description" content="▁▂▃▄%▅▆▇█" />
%<a href="https://ar.wikipedia.org/wiki/تسونامي">%a''')
self.assertEqual(
compat_urllib_parse_unquote('''%28%5E%E2%97%A3_%E2%97%A2%5E%29%E3%81%A3%EF%B8%BB%E3%83%87%E2%95%90%E4%B8%80 %E2%87%80 %E2%87%80 %E2%87%80 %E2%87%80 %E2%87%80 %E2%86%B6%I%Break%25Things%'''),
'''(^◣_◢^)っ︻デ═一 ⇀ ⇀ ⇀ ⇀ ⇀ ↶%I%Break%Things%''')
def test_compat_urllib_parse_unquote_plus(self):
self.assertEqual(urllib.parse.unquote_plus('abc%20def'), 'abc def')
self.assertEqual(urllib.parse.unquote_plus('%7e/abc+def'), '~/abc def')
def test_compat_urllib_parse_urlencode(self):
self.assertEqual(compat_urllib_parse_urlencode({'abc': 'def'}), 'abc=def')
self.assertEqual(compat_urllib_parse_urlencode({'abc': b'def'}), 'abc=def')
self.assertEqual(compat_urllib_parse_urlencode({b'abc': 'def'}), 'abc=def')
self.assertEqual(compat_urllib_parse_urlencode({b'abc': b'def'}), 'abc=def')
self.assertEqual(compat_urllib_parse_urlencode([('abc', 'def')]), 'abc=def')
self.assertEqual(compat_urllib_parse_urlencode([('abc', b'def')]), 'abc=def')
self.assertEqual(compat_urllib_parse_urlencode([(b'abc', 'def')]), 'abc=def')
self.assertEqual(compat_urllib_parse_urlencode([(b'abc', b'def')]), 'abc=def')
def test_compat_etree_fromstring(self): def test_compat_etree_fromstring(self):
xml = ''' xml = '''
<root foo="bar" spam="中文"> <root foo="bar" spam="中文">
@@ -100,6 +60,45 @@ class TestCompat(unittest.TestCase):
def test_struct_unpack(self): def test_struct_unpack(self):
self.assertEqual(struct.unpack('!B', b'\x00'), (0,)) self.assertEqual(struct.unpack('!B', b'\x00'), (0,))
def test_compat_datetime_from_timestamp(self):
self.assertEqual(
compat_datetime_from_timestamp(0),
dt.datetime(1970, 1, 1, 0, 0, 0, tzinfo=dt.timezone.utc))
self.assertEqual(
compat_datetime_from_timestamp(1),
dt.datetime(1970, 1, 1, 0, 0, 1, tzinfo=dt.timezone.utc))
self.assertEqual(
compat_datetime_from_timestamp(3600),
dt.datetime(1970, 1, 1, 1, 0, 0, tzinfo=dt.timezone.utc))
self.assertEqual(
compat_datetime_from_timestamp(-1),
dt.datetime(1969, 12, 31, 23, 59, 59, tzinfo=dt.timezone.utc))
self.assertEqual(
compat_datetime_from_timestamp(-86400),
dt.datetime(1969, 12, 31, 0, 0, 0, tzinfo=dt.timezone.utc))
self.assertEqual(
compat_datetime_from_timestamp(0.5),
dt.datetime(1970, 1, 1, 0, 0, 0, 500000, tzinfo=dt.timezone.utc))
self.assertEqual(
compat_datetime_from_timestamp(1.000001),
dt.datetime(1970, 1, 1, 0, 0, 1, 1, tzinfo=dt.timezone.utc))
self.assertEqual(
compat_datetime_from_timestamp(-1.25),
dt.datetime(1969, 12, 31, 23, 59, 58, 750000, tzinfo=dt.timezone.utc))
self.assertEqual(
compat_datetime_from_timestamp(-1577923200),
dt.datetime(1920, 1, 1, 0, 0, 0, tzinfo=dt.timezone.utc))
self.assertEqual(
compat_datetime_from_timestamp(4102444800),
dt.datetime(2100, 1, 1, 0, 0, 0, tzinfo=dt.timezone.utc))
self.assertEqual(
compat_datetime_from_timestamp(173568960000),
dt.datetime(7470, 3, 8, 0, 0, 0, tzinfo=dt.timezone.utc))
if __name__ == '__main__': if __name__ == '__main__':
unittest.main() unittest.main()
+1 -1
View File
@@ -71,7 +71,7 @@ def _generate_expected_groups():
Path('/etc/yt-dlp.conf'), Path('/etc/yt-dlp.conf'),
Path('/etc/yt-dlp/config'), Path('/etc/yt-dlp/config'),
Path('/etc/yt-dlp/config.txt'), Path('/etc/yt-dlp/config.txt'),
] ],
} }
+93 -68
View File
@@ -1,5 +1,5 @@
import datetime as dt
import unittest import unittest
from datetime import datetime, timezone
from yt_dlp import cookies from yt_dlp import cookies
from yt_dlp.cookies import ( from yt_dlp.cookies import (
@@ -58,6 +58,14 @@ class TestCookies(unittest.TestCase):
({'DESKTOP_SESSION': 'kde'}, _LinuxDesktopEnvironment.KDE3), ({'DESKTOP_SESSION': 'kde'}, _LinuxDesktopEnvironment.KDE3),
({'DESKTOP_SESSION': 'xfce'}, _LinuxDesktopEnvironment.XFCE), ({'DESKTOP_SESSION': 'xfce'}, _LinuxDesktopEnvironment.XFCE),
({'XDG_CURRENT_DESKTOP': 'my_custom_de', 'DESKTOP_SESSION': 'gnome'}, _LinuxDesktopEnvironment.GNOME),
({'XDG_CURRENT_DESKTOP': 'my_custom_de', 'DESKTOP_SESSION': 'mate'}, _LinuxDesktopEnvironment.GNOME),
({'XDG_CURRENT_DESKTOP': 'my_custom_de', 'DESKTOP_SESSION': 'kde4'}, _LinuxDesktopEnvironment.KDE4),
({'XDG_CURRENT_DESKTOP': 'my_custom_de', 'DESKTOP_SESSION': 'kde'}, _LinuxDesktopEnvironment.KDE3),
({'XDG_CURRENT_DESKTOP': 'my_custom_de', 'DESKTOP_SESSION': 'xfce'}, _LinuxDesktopEnvironment.XFCE),
({'XDG_CURRENT_DESKTOP': 'my_custom_de', 'DESKTOP_SESSION': 'my_custom_de', 'GNOME_DESKTOP_SESSION_ID': 1}, _LinuxDesktopEnvironment.GNOME),
({'GNOME_DESKTOP_SESSION_ID': 1}, _LinuxDesktopEnvironment.GNOME), ({'GNOME_DESKTOP_SESSION_ID': 1}, _LinuxDesktopEnvironment.GNOME),
({'KDE_FULL_SESSION': 1}, _LinuxDesktopEnvironment.KDE3), ({'KDE_FULL_SESSION': 1}, _LinuxDesktopEnvironment.KDE3),
({'KDE_FULL_SESSION': 1, 'DESKTOP_SESSION': 'kde4'}, _LinuxDesktopEnvironment.KDE4), ({'KDE_FULL_SESSION': 1, 'DESKTOP_SESSION': 'kde4'}, _LinuxDesktopEnvironment.KDE4),
@@ -67,6 +75,7 @@ class TestCookies(unittest.TestCase):
({'XDG_CURRENT_DESKTOP': 'GNOME'}, _LinuxDesktopEnvironment.GNOME), ({'XDG_CURRENT_DESKTOP': 'GNOME'}, _LinuxDesktopEnvironment.GNOME),
({'XDG_CURRENT_DESKTOP': 'GNOME:GNOME-Classic'}, _LinuxDesktopEnvironment.GNOME), ({'XDG_CURRENT_DESKTOP': 'GNOME:GNOME-Classic'}, _LinuxDesktopEnvironment.GNOME),
({'XDG_CURRENT_DESKTOP': 'GNOME : GNOME-Classic'}, _LinuxDesktopEnvironment.GNOME), ({'XDG_CURRENT_DESKTOP': 'GNOME : GNOME-Classic'}, _LinuxDesktopEnvironment.GNOME),
({'XDG_CURRENT_DESKTOP': 'ubuntu:GNOME'}, _LinuxDesktopEnvironment.GNOME),
({'XDG_CURRENT_DESKTOP': 'Unity', 'DESKTOP_SESSION': 'gnome-fallback'}, _LinuxDesktopEnvironment.GNOME), ({'XDG_CURRENT_DESKTOP': 'Unity', 'DESKTOP_SESSION': 'gnome-fallback'}, _LinuxDesktopEnvironment.GNOME),
({'XDG_CURRENT_DESKTOP': 'KDE', 'KDE_SESSION_VERSION': '5'}, _LinuxDesktopEnvironment.KDE5), ({'XDG_CURRENT_DESKTOP': 'KDE', 'KDE_SESSION_VERSION': '5'}, _LinuxDesktopEnvironment.KDE5),
@@ -104,15 +113,31 @@ class TestCookies(unittest.TestCase):
decryptor = LinuxChromeCookieDecryptor('Chrome', Logger()) decryptor = LinuxChromeCookieDecryptor('Chrome', Logger())
self.assertEqual(decryptor.decrypt(encrypted_value), value) self.assertEqual(decryptor.decrypt(encrypted_value), value)
def test_chrome_cookie_decryptor_linux_v10_meta24(self):
with MonkeyPatch(cookies, {'_get_linux_keyring_password': lambda *args, **kwargs: b''}):
encrypted_value = b'v10\x1f\xe4\x0e[\x83\x0c\xcc*kPi \xce\x8d\x1d\xbb\x80\r\x11\t\xbb\x9e^Hy\x94\xf4\x963\x9f\x82\xba\xfe\xa1\xed\xb9\xf1)\x00710\x92\xc8/<\x96B'
value = 'DE'
decryptor = LinuxChromeCookieDecryptor('Chrome', Logger(), meta_version=24)
self.assertEqual(decryptor.decrypt(encrypted_value), value)
def test_chrome_cookie_decryptor_windows_v10(self): def test_chrome_cookie_decryptor_windows_v10(self):
with MonkeyPatch(cookies, { with MonkeyPatch(cookies, {
'_get_windows_v10_key': lambda *args, **kwargs: b'Y\xef\xad\xad\xeerp\xf0Y\xe6\x9b\x12\xc2<z\x16]\n\xbb\xb8\xcb\xd7\x9bA\xc3\x14e\x99{\xd6\xf4&' '_get_windows_v10_key': lambda *args, **kwargs: b'Y\xef\xad\xad\xeerp\xf0Y\xe6\x9b\x12\xc2<z\x16]\n\xbb\xb8\xcb\xd7\x9bA\xc3\x14e\x99{\xd6\xf4&',
}): }):
encrypted_value = b'v10T\xb8\xf3\xb8\x01\xa7TtcV\xfc\x88\xb8\xb8\xef\x05\xb5\xfd\x18\xc90\x009\xab\xb1\x893\x85)\x87\xe1\xa9-\xa3\xad=' encrypted_value = b'v10T\xb8\xf3\xb8\x01\xa7TtcV\xfc\x88\xb8\xb8\xef\x05\xb5\xfd\x18\xc90\x009\xab\xb1\x893\x85)\x87\xe1\xa9-\xa3\xad='
value = '32101439' value = '32101439'
decryptor = WindowsChromeCookieDecryptor('', Logger()) decryptor = WindowsChromeCookieDecryptor('', Logger())
self.assertEqual(decryptor.decrypt(encrypted_value), value) self.assertEqual(decryptor.decrypt(encrypted_value), value)
def test_chrome_cookie_decryptor_windows_v10_meta24(self):
with MonkeyPatch(cookies, {
'_get_windows_v10_key': lambda *args, **kwargs: b'\xea\x8b\x02\xc3\xc6\xc5\x99\xc3\xa3[ j\xfa\xf6\xfcU\xac\x13u\xdc\x0c\x0e\xf1\x03\x90\xb6\xdf\xbb\x8fL\xb1\xb2',
}):
encrypted_value = b'v10dN\xe1\xacy\x84^\xe1I\xact\x03r\xfb\xe2\xce{^\x0e<(\xb0y\xeb\x01\xfb@"\x9e\x8c\xa53~\xdb*\x8f\xac\x8b\xe3\xfd3\x06\xe5\x93\x19OyOG\xb2\xfb\x1d$\xc0\xda\x13j\x9e\xfe\xc5\xa3\xa8\xfe\xd9'
value = '1234'
decryptor = WindowsChromeCookieDecryptor('', Logger(), meta_version=24)
self.assertEqual(decryptor.decrypt(encrypted_value), value)
def test_chrome_cookie_decryptor_mac_v10(self): def test_chrome_cookie_decryptor_mac_v10(self):
with MonkeyPatch(cookies, {'_get_mac_keyring_password': lambda *args, **kwargs: b'6eIDUdtKAacvlHwBVwvg/Q=='}): with MonkeyPatch(cookies, {'_get_mac_keyring_password': lambda *args, **kwargs: b'6eIDUdtKAacvlHwBVwvg/Q=='}):
encrypted_value = b'v10\xb3\xbe\xad\xa1[\x9fC\xa1\x98\xe0\x9a\x01\xd9\xcf\xbfc' encrypted_value = b'v10\xb3\xbe\xad\xa1[\x9fC\xa1\x98\xe0\x9a\x01\xd9\xcf\xbfc'
@@ -121,24 +146,24 @@ class TestCookies(unittest.TestCase):
self.assertEqual(decryptor.decrypt(encrypted_value), value) self.assertEqual(decryptor.decrypt(encrypted_value), value)
def test_safari_cookie_parsing(self): def test_safari_cookie_parsing(self):
cookies = \ cookies = (
b'cook\x00\x00\x00\x01\x00\x00\x00i\x00\x00\x01\x00\x01\x00\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00Y' \ b'cook\x00\x00\x00\x01\x00\x00\x00i\x00\x00\x01\x00\x01\x00\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00Y'
b'\x00\x00\x00\x00\x00\x00\x00 \x00\x00\x00\x00\x00\x00\x008\x00\x00\x00B\x00\x00\x00F\x00\x00\x00H' \ b'\x00\x00\x00\x00\x00\x00\x00 \x00\x00\x00\x00\x00\x00\x008\x00\x00\x00B\x00\x00\x00F\x00\x00\x00H'
b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x80\x03\xa5>\xc3A\x00\x00\x80\xc3\x07:\xc3A' \ b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x80\x03\xa5>\xc3A\x00\x00\x80\xc3\x07:\xc3A'
b'localhost\x00foo\x00/\x00test%20%3Bcookie\x00\x00\x00\x054\x07\x17 \x05\x00\x00\x00Kbplist00\xd1\x01' \ b'localhost\x00foo\x00/\x00test%20%3Bcookie\x00\x00\x00\x054\x07\x17 \x05\x00\x00\x00Kbplist00\xd1\x01'
b'\x02_\x10\x18NSHTTPCookieAcceptPolicy\x10\x02\x08\x0b&\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00' \ b'\x02_\x10\x18NSHTTPCookieAcceptPolicy\x10\x02\x08\x0b&\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00'
b'\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00(' b'\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00(')
jar = parse_safari_cookies(cookies) jar = parse_safari_cookies(cookies)
self.assertEqual(len(jar), 1) self.assertEqual(len(jar), 1)
cookie = list(jar)[0] cookie = next(iter(jar))
self.assertEqual(cookie.domain, 'localhost') self.assertEqual(cookie.domain, 'localhost')
self.assertEqual(cookie.port, None) self.assertEqual(cookie.port, None)
self.assertEqual(cookie.path, '/') self.assertEqual(cookie.path, '/')
self.assertEqual(cookie.name, 'foo') self.assertEqual(cookie.name, 'foo')
self.assertEqual(cookie.value, 'test%20%3Bcookie') self.assertEqual(cookie.value, 'test%20%3Bcookie')
self.assertFalse(cookie.secure) self.assertFalse(cookie.secure)
expected_expiration = datetime(2021, 6, 18, 21, 39, 19, tzinfo=timezone.utc) expected_expiration = dt.datetime(2021, 6, 18, 21, 39, 19, tzinfo=dt.timezone.utc)
self.assertEqual(cookie.expires, int(expected_expiration.timestamp())) self.assertEqual(cookie.expires, int(expected_expiration.timestamp()))
def test_pbkdf2_sha1(self): def test_pbkdf2_sha1(self):
@@ -164,7 +189,7 @@ class TestLenientSimpleCookie(unittest.TestCase):
attributes = { attributes = {
key: value key: value
for key, value in dict(morsel).items() for key, value in dict(morsel).items()
if value != "" if value != ''
} }
self.assertEqual(attributes, expected_attributes, message) self.assertEqual(attributes, expected_attributes, message)
@@ -174,133 +199,133 @@ class TestLenientSimpleCookie(unittest.TestCase):
self._run_tests( self._run_tests(
# Copied from https://github.com/python/cpython/blob/v3.10.7/Lib/test/test_http_cookies.py # Copied from https://github.com/python/cpython/blob/v3.10.7/Lib/test/test_http_cookies.py
( (
"Test basic cookie", 'Test basic cookie',
"chips=ahoy; vienna=finger", 'chips=ahoy; vienna=finger',
{"chips": "ahoy", "vienna": "finger"}, {'chips': 'ahoy', 'vienna': 'finger'},
), ),
( (
"Test quoted cookie", 'Test quoted cookie',
'keebler="E=mc2; L=\\"Loves\\"; fudge=\\012;"', 'keebler="E=mc2; L=\\"Loves\\"; fudge=\\012;"',
{"keebler": 'E=mc2; L="Loves"; fudge=\012;'}, {'keebler': 'E=mc2; L="Loves"; fudge=\012;'},
), ),
( (
"Allow '=' in an unquoted value", "Allow '=' in an unquoted value",
"keebler=E=mc2", 'keebler=E=mc2',
{"keebler": "E=mc2"}, {'keebler': 'E=mc2'},
), ),
( (
"Allow cookies with ':' in their name", "Allow cookies with ':' in their name",
"key:term=value:term", 'key:term=value:term',
{"key:term": "value:term"}, {'key:term': 'value:term'},
), ),
( (
"Allow '[' and ']' in cookie values", "Allow '[' and ']' in cookie values",
"a=b; c=[; d=r; f=h", 'a=b; c=[; d=r; f=h',
{"a": "b", "c": "[", "d": "r", "f": "h"}, {'a': 'b', 'c': '[', 'd': 'r', 'f': 'h'},
), ),
( (
"Test basic cookie attributes", 'Test basic cookie attributes',
'Customer="WILE_E_COYOTE"; Version=1; Path=/acme', 'Customer="WILE_E_COYOTE"; Version=1; Path=/acme',
{"Customer": ("WILE_E_COYOTE", {"version": "1", "path": "/acme"})}, {'Customer': ('WILE_E_COYOTE', {'version': '1', 'path': '/acme'})},
), ),
( (
"Test flag only cookie attributes", 'Test flag only cookie attributes',
'Customer="WILE_E_COYOTE"; HttpOnly; Secure', 'Customer="WILE_E_COYOTE"; HttpOnly; Secure',
{"Customer": ("WILE_E_COYOTE", {"httponly": True, "secure": True})}, {'Customer': ('WILE_E_COYOTE', {'httponly': True, 'secure': True})},
), ),
( (
"Test flag only attribute with values", 'Test flag only attribute with values',
"eggs=scrambled; httponly=foo; secure=bar; Path=/bacon", 'eggs=scrambled; httponly=foo; secure=bar; Path=/bacon',
{"eggs": ("scrambled", {"httponly": "foo", "secure": "bar", "path": "/bacon"})}, {'eggs': ('scrambled', {'httponly': 'foo', 'secure': 'bar', 'path': '/bacon'})},
), ),
( (
"Test special case for 'expires' attribute, 4 digit year", "Test special case for 'expires' attribute, 4 digit year",
'Customer="W"; expires=Wed, 01 Jan 2010 00:00:00 GMT', 'Customer="W"; expires=Wed, 01 Jan 2010 00:00:00 GMT',
{"Customer": ("W", {"expires": "Wed, 01 Jan 2010 00:00:00 GMT"})}, {'Customer': ('W', {'expires': 'Wed, 01 Jan 2010 00:00:00 GMT'})},
), ),
( (
"Test special case for 'expires' attribute, 2 digit year", "Test special case for 'expires' attribute, 2 digit year",
'Customer="W"; expires=Wed, 01 Jan 98 00:00:00 GMT', 'Customer="W"; expires=Wed, 01 Jan 98 00:00:00 GMT',
{"Customer": ("W", {"expires": "Wed, 01 Jan 98 00:00:00 GMT"})}, {'Customer': ('W', {'expires': 'Wed, 01 Jan 98 00:00:00 GMT'})},
), ),
( (
"Test extra spaces in keys and values", 'Test extra spaces in keys and values',
"eggs = scrambled ; secure ; path = bar ; foo=foo ", 'eggs = scrambled ; secure ; path = bar ; foo=foo ',
{"eggs": ("scrambled", {"secure": True, "path": "bar"}), "foo": "foo"}, {'eggs': ('scrambled', {'secure': True, 'path': 'bar'}), 'foo': 'foo'},
), ),
( (
"Test quoted attributes", 'Test quoted attributes',
'Customer="WILE_E_COYOTE"; Version="1"; Path="/acme"', 'Customer="WILE_E_COYOTE"; Version="1"; Path="/acme"',
{"Customer": ("WILE_E_COYOTE", {"version": "1", "path": "/acme"})} {'Customer': ('WILE_E_COYOTE', {'version': '1', 'path': '/acme'})},
), ),
# Our own tests that CPython passes # Our own tests that CPython passes
( (
"Allow ';' in quoted value", "Allow ';' in quoted value",
'chips="a;hoy"; vienna=finger', 'chips="a;hoy"; vienna=finger',
{"chips": "a;hoy", "vienna": "finger"}, {'chips': 'a;hoy', 'vienna': 'finger'},
), ),
( (
"Keep only the last set value", 'Keep only the last set value',
"a=c; a=b", 'a=c; a=b',
{"a": "b"}, {'a': 'b'},
), ),
) )
def test_lenient_parsing(self): def test_lenient_parsing(self):
self._run_tests( self._run_tests(
( (
"Ignore and try to skip invalid cookies", 'Ignore and try to skip invalid cookies',
'chips={"ahoy;": 1}; vienna="finger;"', 'chips={"ahoy;": 1}; vienna="finger;"',
{"vienna": "finger;"}, {'vienna': 'finger;'},
), ),
( (
"Ignore cookies without a name", 'Ignore cookies without a name',
"a=b; unnamed; c=d", 'a=b; unnamed; c=d',
{"a": "b", "c": "d"}, {'a': 'b', 'c': 'd'},
), ),
( (
"Ignore '\"' cookie without name", "Ignore '\"' cookie without name",
'a=b; "; c=d', 'a=b; "; c=d',
{"a": "b", "c": "d"}, {'a': 'b', 'c': 'd'},
), ),
( (
"Skip all space separated values", 'Skip all space separated values',
"x a=b c=d x; e=f", 'x a=b c=d x; e=f',
{"a": "b", "c": "d", "e": "f"}, {'a': 'b', 'c': 'd', 'e': 'f'},
), ),
( (
"Skip all space separated values", 'Skip all space separated values',
'x a=b; data={"complex": "json", "with": "key=value"}; x c=d x', 'x a=b; data={"complex": "json", "with": "key=value"}; x c=d x',
{"a": "b", "c": "d"}, {'a': 'b', 'c': 'd'},
), ),
( (
"Expect quote mending", 'Expect quote mending',
'a=b; invalid="; c=d', 'a=b; invalid="; c=d',
{"a": "b", "c": "d"}, {'a': 'b', 'c': 'd'},
), ),
( (
"Reset morsel after invalid to not capture attributes", 'Reset morsel after invalid to not capture attributes',
"a=b; invalid; Version=1; c=d", 'a=b; invalid; Version=1; c=d',
{"a": "b", "c": "d"}, {'a': 'b', 'c': 'd'},
), ),
( (
"Reset morsel after invalid to not capture attributes", 'Reset morsel after invalid to not capture attributes',
"a=b; $invalid; $Version=1; c=d", 'a=b; $invalid; $Version=1; c=d',
{"a": "b", "c": "d"}, {'a': 'b', 'c': 'd'},
), ),
( (
"Continue after non-flag attribute without value", 'Continue after non-flag attribute without value',
"a=b; path; Version=1; c=d", 'a=b; path; Version=1; c=d',
{"a": "b", "c": "d"}, {'a': 'b', 'c': 'd'},
), ),
( (
"Allow cookie attributes with `$` prefix", 'Allow cookie attributes with `$` prefix',
'Customer="WILE_E_COYOTE"; $Version=1; $Secure; $Path=/acme', 'Customer="WILE_E_COYOTE"; $Version=1; $Secure; $Path=/acme',
{"Customer": ("WILE_E_COYOTE", {"version": "1", "secure": True, "path": "/acme"})}, {'Customer': ('WILE_E_COYOTE', {'version': '1', 'secure': True, 'path': '/acme'})},
), ),
( (
"Invalid Morsel keys should not result in an error", 'Invalid Morsel keys should not result in an error',
"Key=Value; [Invalid]=Value; Another=Value", 'Key=Value; [Invalid]=Value; Another=Value',
{"Key": "Value", "Another": "Value"}, {'Key': 'Value', 'Another': 'Value'},
), ),
) )
+235
View File
@@ -0,0 +1,235 @@
#!/usr/bin/env python3
# Allow direct execution
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import datetime as dt
import json
import math
import re
import unittest
from yt_dlp.utils.jslib import devalue
TEST_CASES_EQUALS = [{
'name': 'int',
'unparsed': [-42],
'parsed': -42,
}, {
'name': 'str',
'unparsed': ['woo!!!'],
'parsed': 'woo!!!',
}, {
'name': 'Number',
'unparsed': [['Object', 42]],
'parsed': 42,
}, {
'name': 'String',
'unparsed': [['Object', 'yar']],
'parsed': 'yar',
}, {
'name': 'Infinity',
'unparsed': -4,
'parsed': math.inf,
}, {
'name': 'negative Infinity',
'unparsed': -5,
'parsed': -math.inf,
}, {
'name': 'negative zero',
'unparsed': -6,
'parsed': -0.0,
}, {
'name': 'RegExp',
'unparsed': [['RegExp', 'regexp', 'gim']], # XXX: flags are ignored
'parsed': re.compile('regexp'),
}, {
'name': 'Date',
'unparsed': [['Date', '2001-09-09T01:46:40.000Z']],
'parsed': dt.datetime.fromtimestamp(1e9, tz=dt.timezone.utc),
}, {
'name': 'Array',
'unparsed': [[1, 2, 3], 'a', 'b', 'c'],
'parsed': ['a', 'b', 'c'],
}, {
'name': 'Array (empty)',
'unparsed': [[]],
'parsed': [],
}, {
'name': 'Array (sparse)',
'unparsed': [[-2, 1, -2], 'b'],
'parsed': [None, 'b', None],
}, {
'name': 'Object',
'unparsed': [{'foo': 1, 'x-y': 2}, 'bar', 'z'],
'parsed': {'foo': 'bar', 'x-y': 'z'},
}, {
'name': 'Set',
'unparsed': [['Set', 1, 2, 3], 1, 2, 3],
'parsed': [1, 2, 3],
}, {
'name': 'Map',
'unparsed': [['Map', 1, 2], 'a', 'b'],
'parsed': [['a', 'b']],
}, {
'name': 'BigInt',
'unparsed': [['BigInt', '1']],
'parsed': 1,
}, {
'name': 'Uint8Array',
'unparsed': [['Uint8Array', 'AQID']],
'parsed': [1, 2, 3],
}, {
'name': 'ArrayBuffer',
'unparsed': [['ArrayBuffer', 'AQID']],
'parsed': [1, 2, 3],
}, {
'name': 'str (repetition)',
'unparsed': [[1, 1], 'a string'],
'parsed': ['a string', 'a string'],
}, {
'name': 'None (repetition)',
'unparsed': [[1, 1], None],
'parsed': [None, None],
}, {
'name': 'dict (repetition)',
'unparsed': [[1, 1], {}],
'parsed': [{}, {}],
}, {
'name': 'Object without prototype',
'unparsed': [['null']],
'parsed': {},
}, {
'name': 'cross-realm POJO',
'unparsed': [{}],
'parsed': {},
}]
TEST_CASES_IS = [{
'name': 'bool',
'unparsed': [True],
'parsed': True,
}, {
'name': 'Boolean',
'unparsed': [['Object', False]],
'parsed': False,
}, {
'name': 'undefined',
'unparsed': -1,
'parsed': None,
}, {
'name': 'null',
'unparsed': [None],
'parsed': None,
}, {
'name': 'NaN',
'unparsed': -3,
'parsed': math.nan,
}]
TEST_CASES_INVALID = [{
'name': 'empty string',
'unparsed': '',
'error': ValueError,
'pattern': r'expected int or list as input',
}, {
'name': 'hole',
'unparsed': -2,
'error': ValueError,
'pattern': r'invalid integer input',
}, {
'name': 'string',
'unparsed': 'hello',
'error': ValueError,
'pattern': r'expected int or list as input',
}, {
'name': 'number',
'unparsed': 42,
'error': ValueError,
'pattern': r'invalid integer input',
}, {
'name': 'boolean',
'unparsed': True,
'error': ValueError,
'pattern': r'expected int or list as input',
}, {
'name': 'null',
'unparsed': None,
'error': ValueError,
'pattern': r'expected int or list as input',
}, {
'name': 'object',
'unparsed': {},
'error': ValueError,
'pattern': r'expected int or list as input',
}, {
'name': 'empty array',
'unparsed': [],
'error': ValueError,
'pattern': r'expected a non-empty list as input',
}, {
'name': 'Python negative indexing',
'unparsed': [[1, 2, 3, 4, 5, 6, 7, -7], 1, 2, 3, 4, 5, 6, 7],
'error': IndexError,
'pattern': r'invalid index: -7',
}]
class TestDevalue(unittest.TestCase):
def test_devalue_parse_equals(self):
for tc in TEST_CASES_EQUALS:
self.assertEqual(devalue.parse(tc['unparsed']), tc['parsed'], tc['name'])
def test_devalue_parse_is(self):
for tc in TEST_CASES_IS:
self.assertIs(devalue.parse(tc['unparsed']), tc['parsed'], tc['name'])
def test_devalue_parse_invalid(self):
for tc in TEST_CASES_INVALID:
with self.assertRaisesRegex(tc['error'], tc['pattern'], msg=tc['name']):
devalue.parse(tc['unparsed'])
def test_devalue_parse_cyclical(self):
name = 'Map (cyclical)'
result = devalue.parse([['Map', 1, 0], 'self'])
self.assertEqual(result[0][0], 'self', name)
self.assertIs(result, result[0][1], name)
name = 'Set (cyclical)'
result = devalue.parse([['Set', 0, 1], 42])
self.assertEqual(result[1], 42, name)
self.assertIs(result, result[0], name)
result = devalue.parse([[0]])
self.assertIs(result, result[0], 'Array (cyclical)')
name = 'Object (cyclical)'
result = devalue.parse([{'self': 0}])
self.assertIs(result, result['self'], name)
name = 'Object with null prototype (cyclical)'
result = devalue.parse([['null', 'self', 0]])
self.assertIs(result, result['self'], name)
name = 'Objects (cyclical)'
result = devalue.parse([[1, 2], {'second': 2}, {'first': 1}])
self.assertIs(result[0], result[1]['first'], name)
self.assertIs(result[1], result[0]['second'], name)
def test_devalue_parse_revivers(self):
self.assertEqual(
devalue.parse([['indirect', 1], {'a': 2}, 'b'], revivers={'indirect': lambda x: x}),
{'a': 'b'}, 'revivers (indirect)')
self.assertEqual(
devalue.parse([['parse', 1], '{"a":0}'], revivers={'parse': lambda x: json.loads(x)}),
{'a': 0}, 'revivers (parse)')
if __name__ == '__main__':
unittest.main()
+29 -34
View File
@@ -14,13 +14,13 @@ import json
from test.helper import ( from test.helper import (
assertGreaterEqual, assertGreaterEqual,
assertLessEqual,
expect_info_dict, expect_info_dict,
expect_warnings, expect_warnings,
get_params, get_params,
gettestcases, gettestcases,
getwebpagetestcases, getwebpagetestcases,
is_download_test, is_download_test,
report_warning,
try_rm, try_rm,
) )
@@ -66,10 +66,6 @@ tests_counter = collections.defaultdict(collections.Counter)
@is_download_test @is_download_test
class TestDownload(unittest.TestCase): class TestDownload(unittest.TestCase):
# Parallel testing in nosetests. See
# http://nose.readthedocs.org/en/latest/doc_tests/test_multiprocess/multiprocess.html
_multiprocess_shared_ = True
maxDiff = None maxDiff = None
COMPLETED_TESTS = {} COMPLETED_TESTS = {}
@@ -94,7 +90,7 @@ def generator(test_case, tname):
'playlist', [] if is_playlist else [test_case]) 'playlist', [] if is_playlist else [test_case])
def print_skipping(reason): def print_skipping(reason):
print('Skipping %s: %s' % (test_case['name'], reason)) print('Skipping {}: {}'.format(test_case['name'], reason))
self.skipTest(reason) self.skipTest(reason)
if not ie.working(): if not ie.working():
@@ -117,15 +113,18 @@ def generator(test_case, tname):
for other_ie in other_ies: for other_ie in other_ies:
if not other_ie.working(): if not other_ie.working():
print_skipping('test depends on %sIE, marked as not WORKING' % other_ie.ie_key()) print_skipping(f'test depends on {other_ie.ie_key()}IE, marked as not WORKING')
params = get_params(test_case.get('params', {})) params = get_params(test_case.get('params', {}))
params['outtmpl'] = tname + '_' + params['outtmpl'] params['outtmpl'] = tname + '_' + params['outtmpl']
if is_playlist and 'playlist' not in test_case: if is_playlist and 'playlist' not in test_case:
params.setdefault('extract_flat', 'in_playlist') params.setdefault('playlistend', max(
params.setdefault('playlistend', test_case.get( test_case.get('playlist_mincount', -1),
'playlist_mincount', test_case.get('playlist_count', -2) + 1)) test_case.get('playlist_count', -2) + 1,
test_case.get('playlist_maxcount', -2) + 1))
params.setdefault('skip_download', True) params.setdefault('skip_download', True)
if 'playlist_duration_sum' not in test_case:
params.setdefault('extract_flat', 'in_playlist')
ydl = YoutubeDL(params, auto_init=False) ydl = YoutubeDL(params, auto_init=False)
ydl.add_default_info_extractors() ydl.add_default_info_extractors()
@@ -148,10 +147,7 @@ def generator(test_case, tname):
return False return False
if err.__class__.__name__ == expected_exception: if err.__class__.__name__ == expected_exception:
return True return True
for exc in err.exc_info: return any(exc.__class__.__name__ == expected_exception for exc in err.exc_info)
if exc.__class__.__name__ == expected_exception:
return True
return False
def try_rm_tcs_files(tcs=None): def try_rm_tcs_files(tcs=None):
if tcs is None: if tcs is None:
@@ -163,6 +159,7 @@ def generator(test_case, tname):
try_rm(os.path.splitext(tc_filename)[0] + '.info.json') try_rm(os.path.splitext(tc_filename)[0] + '.info.json')
try_rm_tcs_files() try_rm_tcs_files()
try: try:
test_url = test_case['url']
try_num = 1 try_num = 1
while True: while True:
try: try:
@@ -170,7 +167,7 @@ def generator(test_case, tname):
# for outside error handling, and returns the exit code # for outside error handling, and returns the exit code
# instead of the result dict. # instead of the result dict.
res_dict = ydl.extract_info( res_dict = ydl.extract_info(
test_case['url'], test_url,
force_generic_extractor=params.get('force_generic_extractor', False)) force_generic_extractor=params.get('force_generic_extractor', False))
except (DownloadError, ExtractorError) as err: except (DownloadError, ExtractorError) as err:
# Check if the exception is not a network related one # Check if the exception is not a network related one
@@ -181,8 +178,7 @@ def generator(test_case, tname):
raise raise
if try_num == RETRIES: if try_num == RETRIES:
report_warning('%s failed due to network errors, skipping...' % tname) raise
return
print(f'Retrying: {try_num} failed tries\n\n##########\n\n') print(f'Retrying: {try_num} failed tries\n\n##########\n\n')
@@ -199,23 +195,23 @@ def generator(test_case, tname):
self.assertTrue('entries' in res_dict) self.assertTrue('entries' in res_dict)
expect_info_dict(self, res_dict, test_case.get('info_dict', {})) expect_info_dict(self, res_dict, test_case.get('info_dict', {}))
num_entries = len(res_dict.get('entries', []))
if 'playlist_mincount' in test_case: if 'playlist_mincount' in test_case:
mincount = test_case['playlist_mincount']
assertGreaterEqual( assertGreaterEqual(
self, self, num_entries, mincount,
len(res_dict['entries']), f'Expected at least {mincount} entries in playlist {test_url}, but got only {num_entries}')
test_case['playlist_mincount'],
'Expected at least %d in playlist %s, but got only %d' % (
test_case['playlist_mincount'], test_case['url'],
len(res_dict['entries'])))
if 'playlist_count' in test_case: if 'playlist_count' in test_case:
count = test_case['playlist_count']
got = num_entries if num_entries <= count else 'more'
self.assertEqual( self.assertEqual(
len(res_dict['entries']), num_entries, count,
test_case['playlist_count'], f'Expected exactly {count} entries in playlist {test_url}, but got {got}')
'Expected %d entries in playlist %s, but got %d.' % ( if 'playlist_maxcount' in test_case:
test_case['playlist_count'], maxcount = test_case['playlist_maxcount']
test_case['url'], assertLessEqual(
len(res_dict['entries']), self, num_entries, maxcount,
)) f'Expected at most {maxcount} entries in playlist {test_url}, but got more')
if 'playlist_duration_sum' in test_case: if 'playlist_duration_sum' in test_case:
got_duration = sum(e['duration'] for e in res_dict['entries']) got_duration = sum(e['duration'] for e in res_dict['entries'])
self.assertEqual( self.assertEqual(
@@ -244,9 +240,8 @@ def generator(test_case, tname):
got_fsize = os.path.getsize(tc_filename) got_fsize = os.path.getsize(tc_filename)
assertGreaterEqual( assertGreaterEqual(
self, got_fsize, expected_minsize, self, got_fsize, expected_minsize,
'Expected %s to be at least %s, but it\'s only %s ' % f'Expected {tc_filename} to be at least {format_bytes(expected_minsize)}, '
(tc_filename, format_bytes(expected_minsize), f'but it\'s only {format_bytes(got_fsize)} ')
format_bytes(got_fsize)))
if 'md5' in tc: if 'md5' in tc:
md5_for_file = _file_md5(tc_filename) md5_for_file = _file_md5(tc_filename)
self.assertEqual(tc['md5'], md5_for_file) self.assertEqual(tc['md5'], md5_for_file)
@@ -255,7 +250,7 @@ def generator(test_case, tname):
info_json_fn = os.path.splitext(tc_filename)[0] + '.info.json' info_json_fn = os.path.splitext(tc_filename)[0] + '.info.json'
self.assertTrue( self.assertTrue(
os.path.exists(info_json_fn), os.path.exists(info_json_fn),
'Missing info file %s' % info_json_fn) f'Missing info file {info_json_fn}')
with open(info_json_fn, encoding='utf-8') as infof: with open(info_json_fn, encoding='utf-8') as infof:
info_dict = json.load(infof) info_dict = json.load(infof)
expect_info_dict(self, info_dict, tc.get('info_dict', {})) expect_info_dict(self, info_dict, tc.get('info_dict', {}))
+6 -7
View File
@@ -15,7 +15,6 @@ import threading
from test.helper import http_server_port, try_rm from test.helper import http_server_port, try_rm
from yt_dlp import YoutubeDL from yt_dlp import YoutubeDL
from yt_dlp.downloader.http import HttpFD from yt_dlp.downloader.http import HttpFD
from yt_dlp.utils import encodeFilename
from yt_dlp.utils._utils import _YDLLogger as FakeLogger from yt_dlp.utils._utils import _YDLLogger as FakeLogger
TEST_DIR = os.path.dirname(os.path.abspath(__file__)) TEST_DIR = os.path.dirname(os.path.abspath(__file__))
@@ -38,9 +37,9 @@ class HTTPTestRequestHandler(http.server.BaseHTTPRequestHandler):
end = int(mobj.group(2)) end = int(mobj.group(2))
valid_range = start is not None and end is not None valid_range = start is not None and end is not None
if valid_range: if valid_range:
content_range = 'bytes %d-%d' % (start, end) content_range = f'bytes {start}-{end}'
if total: if total:
content_range += '/%d' % total content_range += f'/{total}'
self.send_header('Content-Range', content_range) self.send_header('Content-Range', content_range)
return (end - start + 1) if valid_range else total return (end - start + 1) if valid_range else total
@@ -82,12 +81,12 @@ class TestHttpFD(unittest.TestCase):
ydl = YoutubeDL(params) ydl = YoutubeDL(params)
downloader = HttpFD(ydl, params) downloader = HttpFD(ydl, params)
filename = 'testfile.mp4' filename = 'testfile.mp4'
try_rm(encodeFilename(filename)) try_rm(filename)
self.assertTrue(downloader.real_download(filename, { self.assertTrue(downloader.real_download(filename, {
'url': 'http://127.0.0.1:%d/%s' % (self.port, ep), 'url': f'http://127.0.0.1:{self.port}/{ep}',
}), ep) }), ep)
self.assertEqual(os.path.getsize(encodeFilename(filename)), TEST_SIZE, ep) self.assertEqual(os.path.getsize(filename), TEST_SIZE, ep)
try_rm(encodeFilename(filename)) try_rm(filename)
def download_all(self, params): def download_all(self, params):
for ep in ('regular', 'no-content-length', 'no-range', 'no-range-no-content-length'): for ep in ('regular', 'no-content-length', 'no-range', 'no-range-no-content-length'):
+376
View File
@@ -0,0 +1,376 @@
import abc
import base64
import contextlib
import functools
import json
import os
import random
import ssl
import threading
from http.server import BaseHTTPRequestHandler
from socketserver import ThreadingTCPServer
import pytest
from test.helper import http_server_port, verify_address_availability
from test.test_networking import TEST_DIR
from test.test_socks import IPv6ThreadingTCPServer
from yt_dlp.dependencies import urllib3
from yt_dlp.networking import Request
from yt_dlp.networking.exceptions import HTTPError, ProxyError, SSLError
class HTTPProxyAuthMixin:
def proxy_auth_error(self):
self.send_response(407)
self.send_header('Proxy-Authenticate', 'Basic realm="test http proxy"')
self.end_headers()
return False
def do_proxy_auth(self, username, password):
if username is None and password is None:
return True
proxy_auth_header = self.headers.get('Proxy-Authorization', None)
if proxy_auth_header is None:
return self.proxy_auth_error()
if not proxy_auth_header.startswith('Basic '):
return self.proxy_auth_error()
auth = proxy_auth_header[6:]
try:
auth_username, auth_password = base64.b64decode(auth).decode().split(':', 1)
except Exception:
return self.proxy_auth_error()
if auth_username != (username or '') or auth_password != (password or ''):
return self.proxy_auth_error()
return True
class HTTPProxyHandler(BaseHTTPRequestHandler, HTTPProxyAuthMixin):
def __init__(self, *args, proxy_info=None, username=None, password=None, request_handler=None, **kwargs):
self.username = username
self.password = password
self.proxy_info = proxy_info
super().__init__(*args, **kwargs)
def do_GET(self):
if not self.do_proxy_auth(self.username, self.password):
self.server.close_request(self.request)
return
if self.path.endswith('/proxy_info'):
payload = json.dumps(self.proxy_info or {
'client_address': self.client_address,
'connect': False,
'connect_host': None,
'connect_port': None,
'headers': dict(self.headers),
'path': self.path,
'proxy': ':'.join(str(y) for y in self.connection.getsockname()),
})
self.send_response(200)
self.send_header('Content-Type', 'application/json; charset=utf-8')
self.send_header('Content-Length', str(len(payload)))
self.end_headers()
self.wfile.write(payload.encode())
else:
self.send_response(404)
self.end_headers()
self.server.close_request(self.request)
if urllib3:
import urllib3.util.ssltransport
class SSLTransport(urllib3.util.ssltransport.SSLTransport):
"""
Modified version of urllib3 SSLTransport to support server side SSL
This allows us to chain multiple TLS connections.
"""
def __init__(self, socket, ssl_context, server_hostname=None, suppress_ragged_eofs=True, server_side=False):
self.incoming = ssl.MemoryBIO()
self.outgoing = ssl.MemoryBIO()
self.suppress_ragged_eofs = suppress_ragged_eofs
self.socket = socket
self.sslobj = ssl_context.wrap_bio(
self.incoming,
self.outgoing,
server_hostname=server_hostname,
server_side=server_side,
)
self._ssl_io_loop(self.sslobj.do_handshake)
@property
def _io_refs(self):
return self.socket._io_refs
@_io_refs.setter
def _io_refs(self, value):
self.socket._io_refs = value
def shutdown(self, *args, **kwargs):
self.socket.shutdown(*args, **kwargs)
else:
SSLTransport = None
class HTTPSProxyHandler(HTTPProxyHandler):
def __init__(self, request, *args, **kwargs):
certfn = os.path.join(TEST_DIR, 'testcert.pem')
sslctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
sslctx.load_cert_chain(certfn, None)
if isinstance(request, ssl.SSLSocket):
request = SSLTransport(request, ssl_context=sslctx, server_side=True)
else:
request = sslctx.wrap_socket(request, server_side=True)
super().__init__(request, *args, **kwargs)
class HTTPConnectProxyHandler(BaseHTTPRequestHandler, HTTPProxyAuthMixin):
protocol_version = 'HTTP/1.1'
default_request_version = 'HTTP/1.1'
def __init__(self, *args, username=None, password=None, request_handler=None, **kwargs):
self.username = username
self.password = password
self.request_handler = request_handler
super().__init__(*args, **kwargs)
def do_CONNECT(self):
if not self.do_proxy_auth(self.username, self.password):
self.server.close_request(self.request)
return
self.send_response(200)
self.end_headers()
proxy_info = {
'client_address': self.client_address,
'connect': True,
'connect_host': self.path.split(':')[0],
'connect_port': int(self.path.split(':')[1]),
'headers': dict(self.headers),
'path': self.path,
'proxy': ':'.join(str(y) for y in self.connection.getsockname()),
}
self.request_handler(self.request, self.client_address, self.server, proxy_info=proxy_info)
self.server.close_request(self.request)
class HTTPSConnectProxyHandler(HTTPConnectProxyHandler):
def __init__(self, request, *args, **kwargs):
certfn = os.path.join(TEST_DIR, 'testcert.pem')
sslctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
sslctx.load_cert_chain(certfn, None)
request = sslctx.wrap_socket(request, server_side=True)
self._original_request = request
super().__init__(request, *args, **kwargs)
def do_CONNECT(self):
super().do_CONNECT()
self.server.close_request(self._original_request)
@contextlib.contextmanager
def proxy_server(proxy_server_class, request_handler, bind_ip=None, **proxy_server_kwargs):
server = server_thread = None
try:
bind_address = bind_ip or '127.0.0.1'
server_type = ThreadingTCPServer if '.' in bind_address else IPv6ThreadingTCPServer
server = server_type(
(bind_address, 0), functools.partial(proxy_server_class, request_handler=request_handler, **proxy_server_kwargs))
server_port = http_server_port(server)
server_thread = threading.Thread(target=server.serve_forever)
server_thread.daemon = True
server_thread.start()
if '.' not in bind_address:
yield f'[{bind_address}]:{server_port}'
else:
yield f'{bind_address}:{server_port}'
finally:
server.shutdown()
server.server_close()
server_thread.join(2.0)
class HTTPProxyTestContext(abc.ABC):
REQUEST_HANDLER_CLASS = None
REQUEST_PROTO = None
def http_server(self, server_class, *args, **kwargs):
return proxy_server(server_class, self.REQUEST_HANDLER_CLASS, *args, **kwargs)
@abc.abstractmethod
def proxy_info_request(self, handler, target_domain=None, target_port=None, **req_kwargs) -> dict:
"""return a dict of proxy_info"""
class HTTPProxyHTTPTestContext(HTTPProxyTestContext):
# Standard HTTP Proxy for http requests
REQUEST_HANDLER_CLASS = HTTPProxyHandler
REQUEST_PROTO = 'http'
def proxy_info_request(self, handler, target_domain=None, target_port=None, **req_kwargs):
request = Request(f'http://{target_domain or "127.0.0.1"}:{target_port or "40000"}/proxy_info', **req_kwargs)
handler.validate(request)
return json.loads(handler.send(request).read().decode())
class HTTPProxyHTTPSTestContext(HTTPProxyTestContext):
# HTTP Connect proxy, for https requests
REQUEST_HANDLER_CLASS = HTTPSProxyHandler
REQUEST_PROTO = 'https'
def proxy_info_request(self, handler, target_domain=None, target_port=None, **req_kwargs):
request = Request(f'https://{target_domain or "127.0.0.1"}:{target_port or "40000"}/proxy_info', **req_kwargs)
handler.validate(request)
return json.loads(handler.send(request).read().decode())
CTX_MAP = {
'http': HTTPProxyHTTPTestContext,
'https': HTTPProxyHTTPSTestContext,
}
@pytest.fixture(scope='module')
def ctx(request):
return CTX_MAP[request.param]()
@pytest.mark.parametrize(
'handler', ['Urllib', 'Requests', 'CurlCFFI'], indirect=True)
@pytest.mark.parametrize('ctx', ['http'], indirect=True) # pure http proxy can only support http
class TestHTTPProxy:
def test_http_no_auth(self, handler, ctx):
with ctx.http_server(HTTPProxyHandler) as server_address:
with handler(proxies={ctx.REQUEST_PROTO: f'http://{server_address}'}) as rh:
proxy_info = ctx.proxy_info_request(rh)
assert proxy_info['proxy'] == server_address
assert proxy_info['connect'] is False
assert 'Proxy-Authorization' not in proxy_info['headers']
def test_http_auth(self, handler, ctx):
with ctx.http_server(HTTPProxyHandler, username='test', password='test') as server_address:
with handler(proxies={ctx.REQUEST_PROTO: f'http://test:test@{server_address}'}) as rh:
proxy_info = ctx.proxy_info_request(rh)
assert proxy_info['proxy'] == server_address
assert 'Proxy-Authorization' in proxy_info['headers']
def test_http_bad_auth(self, handler, ctx):
with ctx.http_server(HTTPProxyHandler, username='test', password='test') as server_address:
with handler(proxies={ctx.REQUEST_PROTO: f'http://test:bad@{server_address}'}) as rh:
with pytest.raises(HTTPError) as exc_info:
ctx.proxy_info_request(rh)
assert exc_info.value.response.status == 407
exc_info.value.response.close()
def test_http_source_address(self, handler, ctx):
with ctx.http_server(HTTPProxyHandler) as server_address:
source_address = f'127.0.0.{random.randint(5, 255)}'
verify_address_availability(source_address)
with handler(proxies={ctx.REQUEST_PROTO: f'http://{server_address}'},
source_address=source_address) as rh:
proxy_info = ctx.proxy_info_request(rh)
assert proxy_info['proxy'] == server_address
assert proxy_info['client_address'][0] == source_address
@pytest.mark.skip_handler('Urllib', 'urllib does not support https proxies')
def test_https(self, handler, ctx):
with ctx.http_server(HTTPSProxyHandler) as server_address:
with handler(verify=False, proxies={ctx.REQUEST_PROTO: f'https://{server_address}'}) as rh:
proxy_info = ctx.proxy_info_request(rh)
assert proxy_info['proxy'] == server_address
assert proxy_info['connect'] is False
assert 'Proxy-Authorization' not in proxy_info['headers']
@pytest.mark.skip_handler('Urllib', 'urllib does not support https proxies')
def test_https_verify_failed(self, handler, ctx):
with ctx.http_server(HTTPSProxyHandler) as server_address:
with handler(verify=True, proxies={ctx.REQUEST_PROTO: f'https://{server_address}'}) as rh:
# Accept SSLError as may not be feasible to tell if it is proxy or request error.
# note: if request proto also does ssl verification, this may also be the error of the request.
# Until we can support passing custom cacerts to handlers, we cannot properly test this for all cases.
with pytest.raises((ProxyError, SSLError)):
ctx.proxy_info_request(rh)
def test_http_with_idn(self, handler, ctx):
with ctx.http_server(HTTPProxyHandler) as server_address:
with handler(proxies={ctx.REQUEST_PROTO: f'http://{server_address}'}) as rh:
proxy_info = ctx.proxy_info_request(rh, target_domain='中文.tw')
assert proxy_info['proxy'] == server_address
assert proxy_info['path'].startswith('http://xn--fiq228c.tw')
assert proxy_info['headers']['Host'].split(':', 1)[0] == 'xn--fiq228c.tw'
@pytest.mark.parametrize(
'handler,ctx', [
('Requests', 'https'),
('CurlCFFI', 'https'),
], indirect=True)
class TestHTTPConnectProxy:
def test_http_connect_no_auth(self, handler, ctx):
with ctx.http_server(HTTPConnectProxyHandler) as server_address:
with handler(verify=False, proxies={ctx.REQUEST_PROTO: f'http://{server_address}'}) as rh:
proxy_info = ctx.proxy_info_request(rh)
assert proxy_info['proxy'] == server_address
assert proxy_info['connect'] is True
assert 'Proxy-Authorization' not in proxy_info['headers']
def test_http_connect_auth(self, handler, ctx):
with ctx.http_server(HTTPConnectProxyHandler, username='test', password='test') as server_address:
with handler(verify=False, proxies={ctx.REQUEST_PROTO: f'http://test:test@{server_address}'}) as rh:
proxy_info = ctx.proxy_info_request(rh)
assert proxy_info['proxy'] == server_address
assert 'Proxy-Authorization' in proxy_info['headers']
def test_http_connect_bad_auth(self, handler, ctx):
with ctx.http_server(HTTPConnectProxyHandler, username='test', password='test') as server_address:
with handler(verify=False, proxies={ctx.REQUEST_PROTO: f'http://test:bad@{server_address}'}) as rh:
with pytest.raises(ProxyError):
ctx.proxy_info_request(rh)
def test_http_connect_source_address(self, handler, ctx):
with ctx.http_server(HTTPConnectProxyHandler) as server_address:
source_address = f'127.0.0.{random.randint(5, 255)}'
verify_address_availability(source_address)
with handler(proxies={ctx.REQUEST_PROTO: f'http://{server_address}'},
source_address=source_address,
verify=False) as rh:
proxy_info = ctx.proxy_info_request(rh)
assert proxy_info['proxy'] == server_address
assert proxy_info['client_address'][0] == source_address
@pytest.mark.skipif(urllib3 is None, reason='requires urllib3 to test')
def test_https_connect_proxy(self, handler, ctx):
with ctx.http_server(HTTPSConnectProxyHandler) as server_address:
with handler(verify=False, proxies={ctx.REQUEST_PROTO: f'https://{server_address}'}) as rh:
proxy_info = ctx.proxy_info_request(rh)
assert proxy_info['proxy'] == server_address
assert proxy_info['connect'] is True
assert 'Proxy-Authorization' not in proxy_info['headers']
@pytest.mark.skipif(urllib3 is None, reason='requires urllib3 to test')
def test_https_connect_verify_failed(self, handler, ctx):
with ctx.http_server(HTTPSConnectProxyHandler) as server_address:
with handler(verify=True, proxies={ctx.REQUEST_PROTO: f'https://{server_address}'}) as rh:
# Accept SSLError as may not be feasible to tell if it is proxy or request error.
# note: if request proto also does ssl verification, this may also be the error of the request.
# Until we can support passing custom cacerts to handlers, we cannot properly test this for all cases.
with pytest.raises((ProxyError, SSLError)):
ctx.proxy_info_request(rh)
@pytest.mark.skipif(urllib3 is None, reason='requires urllib3 to test')
def test_https_connect_proxy_auth(self, handler, ctx):
with ctx.http_server(HTTPSConnectProxyHandler, username='test', password='test') as server_address:
with handler(verify=False, proxies={ctx.REQUEST_PROTO: f'https://test:test@{server_address}'}) as rh:
proxy_info = ctx.proxy_info_request(rh)
assert proxy_info['proxy'] == server_address
assert 'Proxy-Authorization' in proxy_info['headers']
+2 -2
View File
@@ -29,11 +29,11 @@ class WarningLogger:
@is_download_test @is_download_test
class TestIqiyiSDKInterpreter(unittest.TestCase): class TestIqiyiSDKInterpreter(unittest.TestCase):
def test_iqiyi_sdk_interpreter(self): def test_iqiyi_sdk_interpreter(self):
''' """
Test the functionality of IqiyiSDKInterpreter by trying to log in Test the functionality of IqiyiSDKInterpreter by trying to log in
If `sign` is incorrect, /validate call throws an HTTP 556 error If `sign` is incorrect, /validate call throws an HTTP 556 error
''' """
logger = WarningLogger() logger = WarningLogger()
ie = IqiyiIE(FakeYDL({'logger': logger})) ie = IqiyiIE(FakeYDL({'logger': logger}))
ie._perform_login('foo', 'bar') ie._perform_login('foo', 'bar')
+168 -2
View File
@@ -9,7 +9,7 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import math import math
from yt_dlp.jsinterp import JS_Undefined, JSInterpreter from yt_dlp.jsinterp import JS_Undefined, JSInterpreter, js_number_to_string
class NaN: class NaN:
@@ -92,6 +92,17 @@ class TestJSInterpreter(unittest.TestCase):
self._test('function f(){return 0 && 1 || 2;}', 2) self._test('function f(){return 0 && 1 || 2;}', 2)
self._test('function f(){return 0 ?? 42;}', 0) self._test('function f(){return 0 ?? 42;}', 0)
self._test('function f(){return "life, the universe and everything" < 42;}', False) self._test('function f(){return "life, the universe and everything" < 42;}', False)
self._test('function f(){return 0 - 7 * - 6;}', 42)
self._test('function f(){return true << "5";}', 32)
self._test('function f(){return true << true;}', 2)
self._test('function f(){return "19" & "21.9";}', 17)
self._test('function f(){return "19" & false;}', 0)
self._test('function f(){return "11.0" >> "2.1";}', 2)
self._test('function f(){return 5 ^ 9;}', 12)
self._test('function f(){return 0.0 << NaN}', 0)
self._test('function f(){return null << undefined}', 0)
# TODO: Does not work due to number too large
# self._test('function f(){return 21 << 4294967297}', 42)
def test_array_access(self): def test_array_access(self):
self._test('function f(){var x = [1,2,3]; x[0] = 4; x[0] = 5; x[2.0] = 7; return x;}', [5, 2, 7]) self._test('function f(){var x = [1,2,3]; x[0] = 4; x[0] = 5; x[2.0] = 7; return x;}', [5, 2, 7])
@@ -107,6 +118,7 @@ class TestJSInterpreter(unittest.TestCase):
self._test('function f(){var x = 20; x = 30 + 1; return x;}', 31) self._test('function f(){var x = 20; x = 30 + 1; return x;}', 31)
self._test('function f(){var x = 20; x += 30 + 1; return x;}', 51) self._test('function f(){var x = 20; x += 30 + 1; return x;}', 51)
self._test('function f(){var x = 20; x -= 30 + 1; return x;}', -11) self._test('function f(){var x = 20; x -= 30 + 1; return x;}', -11)
self._test('function f(){var x = 2; var y = ["a", "b"]; y[x%y["length"]]="z"; return y}', ['z', 'b'])
@unittest.skip('Not implemented') @unittest.skip('Not implemented')
def test_comments(self): def test_comments(self):
@@ -373,7 +385,161 @@ class TestJSInterpreter(unittest.TestCase):
@unittest.skip('Not implemented') @unittest.skip('Not implemented')
def test_packed(self): def test_packed(self):
jsi = JSInterpreter('''function f(p,a,c,k,e,d){while(c--)if(k[c])p=p.replace(new RegExp('\\b'+c.toString(a)+'\\b','g'),k[c]);return p}''') jsi = JSInterpreter('''function f(p,a,c,k,e,d){while(c--)if(k[c])p=p.replace(new RegExp('\\b'+c.toString(a)+'\\b','g'),k[c]);return p}''')
self.assertEqual(jsi.call_function('f', '''h 7=g("1j");7.7h({7g:[{33:"w://7f-7e-7d-7c.v.7b/7a/79/78/77/76.74?t=73&s=2s&e=72&f=2t&71=70.0.0.1&6z=6y&6x=6w"}],6v:"w://32.v.u/6u.31",16:"r%",15:"r%",6t:"6s",6r:"",6q:"l",6p:"l",6o:"6n",6m:\'6l\',6k:"6j",9:[{33:"/2u?b=6i&n=50&6h=w://32.v.u/6g.31",6f:"6e"}],1y:{6d:1,6c:\'#6b\',6a:\'#69\',68:"67",66:30,65:r,},"64":{63:"%62 2m%m%61%5z%5y%5x.u%5w%5v%5u.2y%22 2k%m%1o%22 5t%m%1o%22 5s%m%1o%22 2j%m%5r%22 16%m%5q%22 15%m%5p%22 5o%2z%5n%5m%2z",5l:"w://v.u/d/1k/5k.2y",5j:[]},\'5i\':{"5h":"5g"},5f:"5e",5d:"w://v.u",5c:{},5b:l,1x:[0.25,0.50,0.75,1,1.25,1.5,2]});h 1m,1n,5a;h 59=0,58=0;h 7=g("1j");h 2x=0,57=0,56=0;$.55({54:{\'53-52\':\'2i-51\'}});7.j(\'4z\',6(x){c(5>0&&x.1l>=5&&1n!=1){1n=1;$(\'q.4y\').4x(\'4w\')}});7.j(\'13\',6(x){2x=x.1l});7.j(\'2g\',6(x){2w(x)});7.j(\'4v\',6(){$(\'q.2v\').4u()});6 2w(x){$(\'q.2v\').4t();c(1m)19;1m=1;17=0;c(4s.4r===l){17=1}$.4q(\'/2u?b=4p&2l=1k&4o=2t-4n-4m-2s-4l&4k=&4j=&4i=&17=\'+17,6(2r){$(\'#4h\').4g(2r)});$(\'.3-8-4f-4e:4d("4c")\').2h(6(e){2q();g().4b(0);g().4a(l)});6 2q(){h $14=$("<q />").2p({1l:"49",16:"r%",15:"r%",48:0,2n:0,2o:47,46:"45(10%, 10%, 10%, 0.4)","44-43":"42"});$("<41 />").2p({16:"60%",15:"60%",2o:40,"3z-2n":"3y"}).3x({\'2m\':\'/?b=3w&2l=1k\',\'2k\':\'0\',\'2j\':\'2i\'}).2f($14);$14.2h(6(){$(3v).3u();g().2g()});$14.2f($(\'#1j\'))}g().13(0);}6 3t(){h 9=7.1b(2e);2d.2c(9);c(9.n>1){1r(i=0;i<9.n;i++){c(9[i].1a==2e){2d.2c(\'!!=\'+i);7.1p(i)}}}}7.j(\'3s\',6(){g().1h("/2a/3r.29","3q 10 28",6(){g().13(g().27()+10)},"2b");$("q[26=2b]").23().21(\'.3-20-1z\');g().1h("/2a/3p.29","3o 10 28",6(){h 12=g().27()-10;c(12<0)12=0;g().13(12)},"24");$("q[26=24]").23().21(\'.3-20-1z\');});6 1i(){}7.j(\'3n\',6(){1i()});7.j(\'3m\',6(){1i()});7.j("k",6(y){h 9=7.1b();c(9.n<2)19;$(\'.3-8-3l-3k\').3j(6(){$(\'#3-8-a-k\').1e(\'3-8-a-z\');$(\'.3-a-k\').p(\'o-1f\',\'11\')});7.1h("/3i/3h.3g","3f 3e",6(){$(\'.3-1w\').3d(\'3-8-1v\');$(\'.3-8-1y, .3-8-1x\').p(\'o-1g\',\'11\');c($(\'.3-1w\').3c(\'3-8-1v\')){$(\'.3-a-k\').p(\'o-1g\',\'l\');$(\'.3-a-k\').p(\'o-1f\',\'l\');$(\'.3-8-a\').1e(\'3-8-a-z\');$(\'.3-8-a:1u\').3b(\'3-8-a-z\')}3a{$(\'.3-a-k\').p(\'o-1g\',\'11\');$(\'.3-a-k\').p(\'o-1f\',\'11\');$(\'.3-8-a:1u\').1e(\'3-8-a-z\')}},"39");7.j("38",6(y){1d.37(\'1c\',y.9[y.36].1a)});c(1d.1t(\'1c\')){35("1s(1d.1t(\'1c\'));",34)}});h 18;6 1s(1q){h 9=7.1b();c(9.n>1){1r(i=0;i<9.n;i++){c(9[i].1a==1q){c(i==18){19}18=i;7.1p(i)}}}}',36,270,'|||jw|||function|player|settings|tracks|submenu||if||||jwplayer|var||on|audioTracks|true|3D|length|aria|attr|div|100|||sx|filemoon|https||event|active||false|tt|seek|dd|height|width|adb|current_audio|return|name|getAudioTracks|default_audio|localStorage|removeClass|expanded|checked|addButton|callMeMaybe|vplayer|0fxcyc2ajhp1|position|vvplay|vvad|220|setCurrentAudioTrack|audio_name|for|audio_set|getItem|last|open|controls|playbackRates|captions|rewind|icon|insertAfter||detach|ff00||button|getPosition|sec|png|player8|ff11|log|console|track_name|appendTo|play|click|no|scrolling|frameborder|file_code|src|top|zIndex|css|showCCform|data|1662367683|383371|dl|video_ad|doPlay|prevt|mp4|3E||jpg|thumbs|file|300|setTimeout|currentTrack|setItem|audioTrackChanged|dualSound|else|addClass|hasClass|toggleClass|Track|Audio|svg|dualy|images|mousedown|buttons|topbar|playAttemptFailed|beforePlay|Rewind|fr|Forward|ff|ready|set_audio_track|remove|this|upload_srt|prop|50px|margin|1000001|iframe|center|align|text|rgba|background|1000000|left|absolute|pause|setCurrentCaptions|Upload|contains|item|content|html|fviews|referer|prem|embed|3e57249ef633e0d03bf76ceb8d8a4b65|216|83|hash|view|get|TokenZir|window|hide|show|complete|slow|fadeIn|video_ad_fadein|time||cache|Cache|Content|headers|ajaxSetup|v2done|tott|vastdone2|vastdone1|vvbefore|playbackRateControls|cast|aboutlink|FileMoon|abouttext|UHD|1870|qualityLabels|sites|GNOME_POWER|link|2Fiframe|3C|allowfullscreen|22360|22640|22no|marginheight|marginwidth|2FGNOME_POWER|2F0fxcyc2ajhp1|2Fe|2Ffilemoon|2F|3A||22https|3Ciframe|code|sharing|fontOpacity|backgroundOpacity|Tahoma|fontFamily|303030|backgroundColor|FFFFFF|color|userFontScale|thumbnails|kind|0fxcyc2ajhp10000|url|get_slides|start|startparam|none|preload|html5|primary|hlshtml|androidhls|duration|uniform|stretching|0fxcyc2ajhp1_xt|image|2048|sp|6871|asn|127|srv|43200|_g3XlBcu2lmD9oDexD2NLWSmah2Nu3XcDrl93m9PwXY|m3u8||master|0fxcyc2ajhp1_x|00076|01|hls2|to|s01|delivery|storage|moon|sources|setup'''.split('|'))) self.assertEqual(jsi.call_function('f', '''h 7=g("1j");7.7h({7g:[{33:"w://7f-7e-7d-7c.v.7b/7a/79/78/77/76.74?t=73&s=2s&e=72&f=2t&71=70.0.0.1&6z=6y&6x=6w"}],6v:"w://32.v.u/6u.31",16:"r%",15:"r%",6t:"6s",6r:"",6q:"l",6p:"l",6o:"6n",6m:\'6l\',6k:"6j",9:[{33:"/2u?b=6i&n=50&6h=w://32.v.u/6g.31",6f:"6e"}],1y:{6d:1,6c:\'#6b\',6a:\'#69\',68:"67",66:30,65:r,},"64":{63:"%62 2m%m%61%5z%5y%5x.u%5w%5v%5u.2y%22 2k%m%1o%22 5t%m%1o%22 5s%m%1o%22 2j%m%5r%22 16%m%5q%22 15%m%5p%22 5o%2z%5n%5m%2z",5l:"w://v.u/d/1k/5k.2y",5j:[]},\'5i\':{"5h":"5g"},5f:"5e",5d:"w://v.u",5c:{},5b:l,1x:[0.25,0.50,0.75,1,1.25,1.5,2]});h 1m,1n,5a;h 59=0,58=0;h 7=g("1j");h 2x=0,57=0,56=0;$.55({54:{\'53-52\':\'2i-51\'}});7.j(\'4z\',6(x){c(5>0&&x.1l>=5&&1n!=1){1n=1;$(\'q.4y\').4x(\'4w\')}});7.j(\'13\',6(x){2x=x.1l});7.j(\'2g\',6(x){2w(x)});7.j(\'4v\',6(){$(\'q.2v\').4u()});6 2w(x){$(\'q.2v\').4t();c(1m)19;1m=1;17=0;c(4s.4r===l){17=1}$.4q(\'/2u?b=4p&2l=1k&4o=2t-4n-4m-2s-4l&4k=&4j=&4i=&17=\'+17,6(2r){$(\'#4h\').4g(2r)});$(\'.3-8-4f-4e:4d("4c")\').2h(6(e){2q();g().4b(0);g().4a(l)});6 2q(){h $14=$("<q />").2p({1l:"49",16:"r%",15:"r%",48:0,2n:0,2o:47,46:"45(10%, 10%, 10%, 0.4)","44-43":"42"});$("<41 />").2p({16:"60%",15:"60%",2o:40,"3z-2n":"3y"}).3x({\'2m\':\'/?b=3w&2l=1k\',\'2k\':\'0\',\'2j\':\'2i\'}).2f($14);$14.2h(6(){$(3v).3u();g().2g()});$14.2f($(\'#1j\'))}g().13(0);}6 3t(){h 9=7.1b(2e);2d.2c(9);c(9.n>1){1r(i=0;i<9.n;i++){c(9[i].1a==2e){2d.2c(\'!!=\'+i);7.1p(i)}}}}7.j(\'3s\',6(){g().1h("/2a/3r.29","3q 10 28",6(){g().13(g().27()+10)},"2b");$("q[26=2b]").23().21(\'.3-20-1z\');g().1h("/2a/3p.29","3o 10 28",6(){h 12=g().27()-10;c(12<0)12=0;g().13(12)},"24");$("q[26=24]").23().21(\'.3-20-1z\');});6 1i(){}7.j(\'3n\',6(){1i()});7.j(\'3m\',6(){1i()});7.j("k",6(y){h 9=7.1b();c(9.n<2)19;$(\'.3-8-3l-3k\').3j(6(){$(\'#3-8-a-k\').1e(\'3-8-a-z\');$(\'.3-a-k\').p(\'o-1f\',\'11\')});7.1h("/3i/3h.3g","3f 3e",6(){$(\'.3-1w\').3d(\'3-8-1v\');$(\'.3-8-1y, .3-8-1x\').p(\'o-1g\',\'11\');c($(\'.3-1w\').3c(\'3-8-1v\')){$(\'.3-a-k\').p(\'o-1g\',\'l\');$(\'.3-a-k\').p(\'o-1f\',\'l\');$(\'.3-8-a\').1e(\'3-8-a-z\');$(\'.3-8-a:1u\').3b(\'3-8-a-z\')}3a{$(\'.3-a-k\').p(\'o-1g\',\'11\');$(\'.3-a-k\').p(\'o-1f\',\'11\');$(\'.3-8-a:1u\').1e(\'3-8-a-z\')}},"39");7.j("38",6(y){1d.37(\'1c\',y.9[y.36].1a)});c(1d.1t(\'1c\')){35("1s(1d.1t(\'1c\'));",34)}});h 18;6 1s(1q){h 9=7.1b();c(9.n>1){1r(i=0;i<9.n;i++){c(9[i].1a==1q){c(i==18){19}18=i;7.1p(i)}}}}',36,270,'|||jw|||function|player|settings|tracks|submenu||if||||jwplayer|var||on|audioTracks|true|3D|length|aria|attr|div|100|||sx|filemoon|https||event|active||false|tt|seek|dd|height|width|adb|current_audio|return|name|getAudioTracks|default_audio|localStorage|removeClass|expanded|checked|addButton|callMeMaybe|vplayer|0fxcyc2ajhp1|position|vvplay|vvad|220|setCurrentAudioTrack|audio_name|for|audio_set|getItem|last|open|controls|playbackRates|captions|rewind|icon|insertAfter||detach|ff00||button|getPosition|sec|png|player8|ff11|log|console|track_name|appendTo|play|click|no|scrolling|frameborder|file_code|src|top|zIndex|css|showCCform|data|1662367683|383371|dl|video_ad|doPlay|prevt|mp4|3E||jpg|thumbs|file|300|setTimeout|currentTrack|setItem|audioTrackChanged|dualSound|else|addClass|hasClass|toggleClass|Track|Audio|svg|dualy|images|mousedown|buttons|topbar|playAttemptFailed|beforePlay|Rewind|fr|Forward|ff|ready|set_audio_track|remove|this|upload_srt|prop|50px|margin|1000001|iframe|center|align|text|rgba|background|1000000|left|absolute|pause|setCurrentCaptions|Upload|contains|item|content|html|fviews|referer|prem|embed|3e57249ef633e0d03bf76ceb8d8a4b65|216|83|hash|view|get|TokenZir|window|hide|show|complete|slow|fadeIn|video_ad_fadein|time||cache|Cache|Content|headers|ajaxSetup|v2done|tott|vastdone2|vastdone1|vvbefore|playbackRateControls|cast|aboutlink|FileMoon|abouttext|UHD|1870|qualityLabels|sites|GNOME_POWER|link|2Fiframe|3C|allowfullscreen|22360|22640|22no|marginheight|marginwidth|2FGNOME_POWER|2F0fxcyc2ajhp1|2Fe|2Ffilemoon|2F|3A||22https|3Ciframe|code|sharing|fontOpacity|backgroundOpacity|Tahoma|fontFamily|303030|backgroundColor|FFFFFF|color|userFontScale|thumbnails|kind|0fxcyc2ajhp10000|url|get_slides|start|startparam|none|preload|html5|primary|hlshtml|androidhls|duration|uniform|stretching|0fxcyc2ajhp1_xt|image|2048|sp|6871|asn|127|srv|43200|_g3XlBcu2lmD9oDexD2NLWSmah2Nu3XcDrl93m9PwXY|m3u8||master|0fxcyc2ajhp1_x|00076|01|hls2|to|s01|delivery|storage|moon|sources|setup'''.split('|'))) # noqa: SIM905
def test_join(self):
test_input = list('test')
tests = [
'function f(a, b){return a.join(b)}',
'function f(a, b){return Array.prototype.join.call(a, b)}',
'function f(a, b){return Array.prototype.join.apply(a, [b])}',
]
for test in tests:
jsi = JSInterpreter(test)
self._test(jsi, 'test', args=[test_input, ''])
self._test(jsi, 't-e-s-t', args=[test_input, '-'])
self._test(jsi, '', args=[[], '-'])
def test_split(self):
test_result = list('test')
tests = [
'function f(a, b){return a.split(b)}',
'function f(a, b){return a["split"](b)}',
'function f(a, b){let x = ["split"]; return a[x[0]](b)}',
'function f(a, b){return String.prototype.split.call(a, b)}',
'function f(a, b){return String.prototype.split.apply(a, [b])}',
]
for test in tests:
jsi = JSInterpreter(test)
self._test(jsi, test_result, args=['test', ''])
self._test(jsi, test_result, args=['t-e-s-t', '-'])
self._test(jsi, [''], args=['', '-'])
self._test(jsi, [], args=['', ''])
def test_slice(self):
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice()}', [0, 1, 2, 3, 4, 5, 6, 7, 8])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(0)}', [0, 1, 2, 3, 4, 5, 6, 7, 8])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(5)}', [5, 6, 7, 8])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(99)}', [])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(-2)}', [7, 8])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(-99)}', [0, 1, 2, 3, 4, 5, 6, 7, 8])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(0, 0)}', [])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(1, 0)}', [])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(0, 1)}', [0])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(3, 6)}', [3, 4, 5])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(1, -1)}', [1, 2, 3, 4, 5, 6, 7])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(-1, 1)}', [])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(-3, -1)}', [6, 7])
self._test('function f(){return "012345678".slice()}', '012345678')
self._test('function f(){return "012345678".slice(0)}', '012345678')
self._test('function f(){return "012345678".slice(5)}', '5678')
self._test('function f(){return "012345678".slice(99)}', '')
self._test('function f(){return "012345678".slice(-2)}', '78')
self._test('function f(){return "012345678".slice(-99)}', '012345678')
self._test('function f(){return "012345678".slice(0, 0)}', '')
self._test('function f(){return "012345678".slice(1, 0)}', '')
self._test('function f(){return "012345678".slice(0, 1)}', '0')
self._test('function f(){return "012345678".slice(3, 6)}', '345')
self._test('function f(){return "012345678".slice(1, -1)}', '1234567')
self._test('function f(){return "012345678".slice(-1, 1)}', '')
self._test('function f(){return "012345678".slice(-3, -1)}', '67')
def test_splice(self):
self._test('function f(){var T = ["0", "1", "2"]; T["splice"](2, 1, "0")[0]; return T }', ['0', '1', '0'])
def test_js_number_to_string(self):
for test, radix, expected in [
(0, None, '0'),
(-0, None, '0'),
(0.0, None, '0'),
(-0.0, None, '0'),
(math.nan, None, 'NaN'),
(-math.nan, None, 'NaN'),
(math.inf, None, 'Infinity'),
(-math.inf, None, '-Infinity'),
(10 ** 21.5, 8, '526665530627250154000000'),
(6, 2, '110'),
(254, 16, 'fe'),
(-10, 2, '-1010'),
(-0xff, 2, '-11111111'),
(0.1 + 0.2, 16, '0.4cccccccccccd'),
(1234.1234, 10, '1234.1234'),
# (1000000000000000128, 10, '1000000000000000100')
]:
assert js_number_to_string(test, radix) == expected
def test_extract_function(self):
jsi = JSInterpreter('function a(b) { return b + 1; }')
func = jsi.extract_function('a')
self.assertEqual(func([2]), 3)
def test_extract_function_with_global_stack(self):
jsi = JSInterpreter('function c(d) { return d + e + f + g; }')
func = jsi.extract_function('c', {'e': 10}, {'f': 100, 'g': 1000})
self.assertEqual(func([1]), 1111)
def test_extract_object(self):
jsi = JSInterpreter('var a={};a.xy={};var xy;var zxy={};xy={z:function(){return "abc"}};')
self.assertTrue('z' in jsi.extract_object('xy', None))
def test_increment_decrement(self):
self._test('function f() { var x = 1; return ++x; }', 2)
self._test('function f() { var x = 1; return x++; }', 1)
self._test('function f() { var x = 1; x--; return x }', 0)
self._test('function f() { var y; var x = 1; x++, --x, x--, x--, y="z", "abc", x++; return --x }', -1)
self._test('function f() { var a = "test--"; return a; }', 'test--')
self._test('function f() { var b = 1; var a = "b--"; return a; }', 'b--')
def test_nested_function_scoping(self):
self._test(R'''
function f() {
var g = function() {
var P = 2;
return P;
};
var P = 1;
g();
return P;
}
''', 1)
self._test(R'''
function f() {
var x = function() {
for (var w = 1, M = []; w < 2; w++) switch (w) {
case 1:
M.push("a");
case 2:
M.push("b");
}
return M
};
var w = "c";
var M = "d";
var y = x();
y.push(w);
y.push(M);
return y;
}
''', ['a', 'b', 'c', 'd'])
self._test(R'''
function f() {
var P, Q;
var z = 100;
var g = function() {
var P, Q; P = 2; Q = 15;
z = 0;
return P+Q;
};
P = 1; Q = 10;
var x = g(), y = 3;
return P+Q+x+y+z;
}
''', 31)
def test_undefined_varnames(self):
jsi = JSInterpreter('function f(){ var a; return [a, b]; }')
self._test(jsi, [JS_Undefined, JS_Undefined])
self.assertEqual(jsi._undefined_varnames, {'b'})
if __name__ == '__main__': if __name__ == '__main__':
+1 -1
View File
@@ -21,7 +21,7 @@ class TestNetRc(unittest.TestCase):
continue continue
self.assertTrue( self.assertTrue(
ie._NETRC_MACHINE, ie._NETRC_MACHINE,
'Extractor %s supports login, but is missing a _NETRC_MACHINE property' % ie.IE_NAME) f'Extractor {ie.IE_NAME} supports login, but is missing a _NETRC_MACHINE property')
if __name__ == '__main__': if __name__ == '__main__':
+735 -265
View File
File diff suppressed because it is too large Load Diff
+7 -8
View File
@@ -20,7 +20,6 @@ from yt_dlp.networking._helper import (
add_accept_encoding_header, add_accept_encoding_header,
get_redirect_method, get_redirect_method,
make_socks_proxy_opts, make_socks_proxy_opts,
select_proxy,
ssl_load_certs, ssl_load_certs,
) )
from yt_dlp.networking.exceptions import ( from yt_dlp.networking.exceptions import (
@@ -28,7 +27,7 @@ from yt_dlp.networking.exceptions import (
IncompleteRead, IncompleteRead,
) )
from yt_dlp.socks import ProxyType from yt_dlp.socks import ProxyType
from yt_dlp.utils.networking import HTTPHeaderDict from yt_dlp.utils.networking import HTTPHeaderDict, select_proxy
TEST_DIR = os.path.dirname(os.path.abspath(__file__)) TEST_DIR = os.path.dirname(os.path.abspath(__file__))
@@ -39,7 +38,7 @@ class TestNetworkingUtils:
proxies = { proxies = {
'all': 'socks5://example.com', 'all': 'socks5://example.com',
'http': 'http://example.com:1080', 'http': 'http://example.com:1080',
'no': 'bypass.example.com,yt-dl.org' 'no': 'bypass.example.com,yt-dl.org',
} }
assert select_proxy('https://example.com', proxies) == proxies['all'] assert select_proxy('https://example.com', proxies) == proxies['all']
@@ -54,7 +53,7 @@ class TestNetworkingUtils:
'port': 1080, 'port': 1080,
'rdns': True, 'rdns': True,
'username': None, 'username': None,
'password': None 'password': None,
}), }),
('socks5://user:@example.com:5555', { ('socks5://user:@example.com:5555', {
'proxytype': ProxyType.SOCKS5, 'proxytype': ProxyType.SOCKS5,
@@ -62,7 +61,7 @@ class TestNetworkingUtils:
'port': 5555, 'port': 5555,
'rdns': False, 'rdns': False,
'username': 'user', 'username': 'user',
'password': '' 'password': '',
}), }),
('socks4://u%40ser:pa%20ss@127.0.0.1:1080', { ('socks4://u%40ser:pa%20ss@127.0.0.1:1080', {
'proxytype': ProxyType.SOCKS4, 'proxytype': ProxyType.SOCKS4,
@@ -70,7 +69,7 @@ class TestNetworkingUtils:
'port': 1080, 'port': 1080,
'rdns': False, 'rdns': False,
'username': 'u@ser', 'username': 'u@ser',
'password': 'pa ss' 'password': 'pa ss',
}), }),
('socks4a://:pa%20ss@127.0.0.1', { ('socks4a://:pa%20ss@127.0.0.1', {
'proxytype': ProxyType.SOCKS4A, 'proxytype': ProxyType.SOCKS4A,
@@ -78,8 +77,8 @@ class TestNetworkingUtils:
'port': 1080, 'port': 1080,
'rdns': True, 'rdns': True,
'username': '', 'username': '',
'password': 'pa ss' 'password': 'pa ss',
}) }),
]) ])
def test_make_socks_proxy_opts(self, socks_proxy, expected): def test_make_socks_proxy_opts(self, socks_proxy, expected):
assert make_socks_proxy_opts(socks_proxy) == expected assert make_socks_proxy_opts(socks_proxy) == expected
+4 -4
View File
@@ -27,9 +27,9 @@ class TestOverwrites(unittest.TestCase):
[ [
sys.executable, 'yt_dlp/__main__.py', sys.executable, 'yt_dlp/__main__.py',
'-o', 'test.webm', '-o', 'test.webm',
'https://www.youtube.com/watch?v=jNQXAC9IVRw' 'https://www.youtube.com/watch?v=jNQXAC9IVRw',
], cwd=root_dir, stdout=subprocess.PIPE, stderr=subprocess.PIPE) ], cwd=root_dir, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
sout, serr = outp.communicate() sout, _ = outp.communicate()
self.assertTrue(b'has already been downloaded' in sout) self.assertTrue(b'has already been downloaded' in sout)
# if the file has no content, it has not been redownloaded # if the file has no content, it has not been redownloaded
self.assertTrue(os.path.getsize(download_file) < 1) self.assertTrue(os.path.getsize(download_file) < 1)
@@ -39,9 +39,9 @@ class TestOverwrites(unittest.TestCase):
[ [
sys.executable, 'yt_dlp/__main__.py', '--yes-overwrites', sys.executable, 'yt_dlp/__main__.py', '--yes-overwrites',
'-o', 'test.webm', '-o', 'test.webm',
'https://www.youtube.com/watch?v=jNQXAC9IVRw' 'https://www.youtube.com/watch?v=jNQXAC9IVRw',
], cwd=root_dir, stdout=subprocess.PIPE, stderr=subprocess.PIPE) ], cwd=root_dir, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
sout, serr = outp.communicate() sout, _ = outp.communicate()
self.assertTrue(b'has already been downloaded' not in sout) self.assertTrue(b'has already been downloaded' not in sout)
# if the file has no content, it has not been redownloaded # if the file has no content, it has not been redownloaded
self.assertTrue(os.path.getsize(download_file) > 1) self.assertTrue(os.path.getsize(download_file) > 1)
+182 -9
View File
@@ -10,41 +10,103 @@ TEST_DATA_DIR = Path(os.path.dirname(os.path.abspath(__file__)), 'testdata')
sys.path.append(str(TEST_DATA_DIR)) sys.path.append(str(TEST_DATA_DIR))
importlib.invalidate_caches() importlib.invalidate_caches()
from yt_dlp.plugins import PACKAGE_NAME, directories, load_plugins from yt_dlp.plugins import (
PACKAGE_NAME,
PluginSpec,
directories,
load_plugins,
load_all_plugins,
register_plugin_spec,
)
from yt_dlp.globals import (
extractors,
postprocessors,
plugin_dirs,
plugin_ies,
plugin_pps,
all_plugins_loaded,
plugin_specs,
)
EXTRACTOR_PLUGIN_SPEC = PluginSpec(
module_name='extractor',
suffix='IE',
destination=extractors,
plugin_destination=plugin_ies,
)
POSTPROCESSOR_PLUGIN_SPEC = PluginSpec(
module_name='postprocessor',
suffix='PP',
destination=postprocessors,
plugin_destination=plugin_pps,
)
def reset_plugins():
plugin_ies.value = {}
plugin_pps.value = {}
plugin_dirs.value = ['default']
plugin_specs.value = {}
all_plugins_loaded.value = False
# Clearing override plugins is probably difficult
for module_name in tuple(sys.modules):
for plugin_type in ('extractor', 'postprocessor'):
if module_name.startswith(f'{PACKAGE_NAME}.{plugin_type}.'):
del sys.modules[module_name]
importlib.invalidate_caches()
class TestPlugins(unittest.TestCase): class TestPlugins(unittest.TestCase):
TEST_PLUGIN_DIR = TEST_DATA_DIR / PACKAGE_NAME TEST_PLUGIN_DIR = TEST_DATA_DIR / PACKAGE_NAME
def setUp(self):
reset_plugins()
def tearDown(self):
reset_plugins()
def test_directories_containing_plugins(self): def test_directories_containing_plugins(self):
self.assertIn(self.TEST_PLUGIN_DIR, map(Path, directories())) self.assertIn(self.TEST_PLUGIN_DIR, map(Path, directories()))
def test_extractor_classes(self): def test_extractor_classes(self):
for module_name in tuple(sys.modules): plugins_ie = load_plugins(EXTRACTOR_PLUGIN_SPEC)
if module_name.startswith(f'{PACKAGE_NAME}.extractor'):
del sys.modules[module_name]
plugins_ie = load_plugins('extractor', 'IE')
self.assertIn(f'{PACKAGE_NAME}.extractor.normal', sys.modules.keys()) self.assertIn(f'{PACKAGE_NAME}.extractor.normal', sys.modules.keys())
self.assertIn('NormalPluginIE', plugins_ie.keys()) self.assertIn('NormalPluginIE', plugins_ie.keys())
# don't load modules with underscore prefix # don't load modules with underscore prefix
self.assertFalse( self.assertFalse(
f'{PACKAGE_NAME}.extractor._ignore' in sys.modules.keys(), f'{PACKAGE_NAME}.extractor._ignore' in sys.modules,
'loaded module beginning with underscore') 'loaded module beginning with underscore')
self.assertNotIn('IgnorePluginIE', plugins_ie.keys()) self.assertNotIn('IgnorePluginIE', plugins_ie.keys())
self.assertNotIn('IgnorePluginIE', plugin_ies.value)
# Don't load extractors with underscore prefix # Don't load extractors with underscore prefix
self.assertNotIn('_IgnoreUnderscorePluginIE', plugins_ie.keys()) self.assertNotIn('_IgnoreUnderscorePluginIE', plugins_ie.keys())
self.assertNotIn('_IgnoreUnderscorePluginIE', plugin_ies.value)
# Don't load extractors not specified in __all__ (if supplied) # Don't load extractors not specified in __all__ (if supplied)
self.assertNotIn('IgnoreNotInAllPluginIE', plugins_ie.keys()) self.assertNotIn('IgnoreNotInAllPluginIE', plugins_ie.keys())
self.assertNotIn('IgnoreNotInAllPluginIE', plugin_ies.value)
self.assertIn('InAllPluginIE', plugins_ie.keys()) self.assertIn('InAllPluginIE', plugins_ie.keys())
self.assertIn('InAllPluginIE', plugin_ies.value)
# Don't load override extractors
self.assertNotIn('OverrideGenericIE', plugins_ie.keys())
self.assertNotIn('OverrideGenericIE', plugin_ies.value)
self.assertNotIn('_UnderscoreOverrideGenericIE', plugins_ie.keys())
self.assertNotIn('_UnderscoreOverrideGenericIE', plugin_ies.value)
def test_postprocessor_classes(self): def test_postprocessor_classes(self):
plugins_pp = load_plugins('postprocessor', 'PP') plugins_pp = load_plugins(POSTPROCESSOR_PLUGIN_SPEC)
self.assertIn('NormalPluginPP', plugins_pp.keys()) self.assertIn('NormalPluginPP', plugins_pp.keys())
self.assertIn(f'{PACKAGE_NAME}.postprocessor.normal', sys.modules.keys())
self.assertIn('NormalPluginPP', plugin_pps.value)
def test_importing_zipped_module(self): def test_importing_zipped_module(self):
zip_path = TEST_DATA_DIR / 'zipped_plugins.zip' zip_path = TEST_DATA_DIR / 'zipped_plugins.zip'
@@ -57,10 +119,10 @@ class TestPlugins(unittest.TestCase):
package = importlib.import_module(f'{PACKAGE_NAME}.{plugin_type}') package = importlib.import_module(f'{PACKAGE_NAME}.{plugin_type}')
self.assertIn(zip_path / PACKAGE_NAME / plugin_type, map(Path, package.__path__)) self.assertIn(zip_path / PACKAGE_NAME / plugin_type, map(Path, package.__path__))
plugins_ie = load_plugins('extractor', 'IE') plugins_ie = load_plugins(EXTRACTOR_PLUGIN_SPEC)
self.assertIn('ZippedPluginIE', plugins_ie.keys()) self.assertIn('ZippedPluginIE', plugins_ie.keys())
plugins_pp = load_plugins('postprocessor', 'PP') plugins_pp = load_plugins(POSTPROCESSOR_PLUGIN_SPEC)
self.assertIn('ZippedPluginPP', plugins_pp.keys()) self.assertIn('ZippedPluginPP', plugins_pp.keys())
finally: finally:
@@ -68,6 +130,117 @@ class TestPlugins(unittest.TestCase):
os.remove(zip_path) os.remove(zip_path)
importlib.invalidate_caches() # reset the import caches importlib.invalidate_caches() # reset the import caches
def test_reloading_plugins(self):
reload_plugins_path = TEST_DATA_DIR / 'reload_plugins'
load_plugins(EXTRACTOR_PLUGIN_SPEC)
load_plugins(POSTPROCESSOR_PLUGIN_SPEC)
# Remove default folder and add reload_plugin path
sys.path.remove(str(TEST_DATA_DIR))
sys.path.append(str(reload_plugins_path))
importlib.invalidate_caches()
try:
for plugin_type in ('extractor', 'postprocessor'):
package = importlib.import_module(f'{PACKAGE_NAME}.{plugin_type}')
self.assertIn(reload_plugins_path / PACKAGE_NAME / plugin_type, map(Path, package.__path__))
plugins_ie = load_plugins(EXTRACTOR_PLUGIN_SPEC)
self.assertIn('NormalPluginIE', plugins_ie.keys())
self.assertTrue(
plugins_ie['NormalPluginIE'].REPLACED,
msg='Reloading has not replaced original extractor plugin')
self.assertTrue(
extractors.value['NormalPluginIE'].REPLACED,
msg='Reloading has not replaced original extractor plugin globally')
plugins_pp = load_plugins(POSTPROCESSOR_PLUGIN_SPEC)
self.assertIn('NormalPluginPP', plugins_pp.keys())
self.assertTrue(plugins_pp['NormalPluginPP'].REPLACED,
msg='Reloading has not replaced original postprocessor plugin')
self.assertTrue(
postprocessors.value['NormalPluginPP'].REPLACED,
msg='Reloading has not replaced original postprocessor plugin globally')
finally:
sys.path.remove(str(reload_plugins_path))
sys.path.append(str(TEST_DATA_DIR))
importlib.invalidate_caches()
def test_extractor_override_plugin(self):
load_plugins(EXTRACTOR_PLUGIN_SPEC)
from yt_dlp.extractor.generic import GenericIE
self.assertEqual(GenericIE.TEST_FIELD, 'override')
self.assertEqual(GenericIE.SECONDARY_TEST_FIELD, 'underscore-override')
self.assertEqual(GenericIE.IE_NAME, 'generic+override+underscore-override')
importlib.invalidate_caches()
# test that loading a second time doesn't wrap a second time
load_plugins(EXTRACTOR_PLUGIN_SPEC)
from yt_dlp.extractor.generic import GenericIE
self.assertEqual(GenericIE.IE_NAME, 'generic+override+underscore-override')
def test_load_all_plugin_types(self):
# no plugin specs registered
load_all_plugins()
self.assertNotIn(f'{PACKAGE_NAME}.extractor.normal', sys.modules.keys())
self.assertNotIn(f'{PACKAGE_NAME}.postprocessor.normal', sys.modules.keys())
register_plugin_spec(EXTRACTOR_PLUGIN_SPEC)
register_plugin_spec(POSTPROCESSOR_PLUGIN_SPEC)
load_all_plugins()
self.assertTrue(all_plugins_loaded.value)
self.assertIn(f'{PACKAGE_NAME}.extractor.normal', sys.modules.keys())
self.assertIn(f'{PACKAGE_NAME}.postprocessor.normal', sys.modules.keys())
def test_no_plugin_dirs(self):
register_plugin_spec(EXTRACTOR_PLUGIN_SPEC)
register_plugin_spec(POSTPROCESSOR_PLUGIN_SPEC)
plugin_dirs.value = []
load_all_plugins()
self.assertNotIn(f'{PACKAGE_NAME}.extractor.normal', sys.modules.keys())
self.assertNotIn(f'{PACKAGE_NAME}.postprocessor.normal', sys.modules.keys())
def test_set_plugin_dirs(self):
custom_plugin_dir = str(TEST_DATA_DIR / 'plugin_packages')
plugin_dirs.value = [custom_plugin_dir]
load_plugins(EXTRACTOR_PLUGIN_SPEC)
self.assertIn(f'{PACKAGE_NAME}.extractor.package', sys.modules.keys())
self.assertIn('PackagePluginIE', plugin_ies.value)
def test_invalid_plugin_dir(self):
plugin_dirs.value = ['invalid_dir']
with self.assertRaises(ValueError):
load_plugins(EXTRACTOR_PLUGIN_SPEC)
def test_append_plugin_dirs(self):
custom_plugin_dir = str(TEST_DATA_DIR / 'plugin_packages')
self.assertEqual(plugin_dirs.value, ['default'])
plugin_dirs.value.append(custom_plugin_dir)
self.assertEqual(plugin_dirs.value, ['default', custom_plugin_dir])
load_plugins(EXTRACTOR_PLUGIN_SPEC)
self.assertIn(f'{PACKAGE_NAME}.extractor.package', sys.modules.keys())
self.assertIn('PackagePluginIE', plugin_ies.value)
def test_get_plugin_spec(self):
register_plugin_spec(EXTRACTOR_PLUGIN_SPEC)
register_plugin_spec(POSTPROCESSOR_PLUGIN_SPEC)
self.assertEqual(plugin_specs.value.get('extractor'), EXTRACTOR_PLUGIN_SPEC)
self.assertEqual(plugin_specs.value.get('postprocessor'), POSTPROCESSOR_PLUGIN_SPEC)
self.assertIsNone(plugin_specs.value.get('invalid'))
if __name__ == '__main__': if __name__ == '__main__':
unittest.main() unittest.main()
+1 -1
View File
@@ -59,7 +59,7 @@ class TestPostHooks(unittest.TestCase):
def hook_three(self, filename): def hook_three(self, filename):
self.files.append(filename) self.files.append(filename)
raise Exception('Test exception for \'%s\'' % filename) raise Exception(f'Test exception for \'{filename}\'')
def tearDown(self): def tearDown(self):
for f in self.files: for f in self.files:
+106 -54
View File
@@ -8,8 +8,10 @@ import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import subprocess
from yt_dlp import YoutubeDL from yt_dlp import YoutubeDL
from yt_dlp.compat import compat_shlex_quote from yt_dlp.utils import shell_quote
from yt_dlp.postprocessor import ( from yt_dlp.postprocessor import (
ExecPP, ExecPP,
FFmpegThumbnailsConvertorPP, FFmpegThumbnailsConvertorPP,
@@ -47,7 +49,18 @@ class TestConvertThumbnail(unittest.TestCase):
print('Skipping: ffmpeg not found') print('Skipping: ffmpeg not found')
return return
file = 'test/testdata/thumbnails/foo %d bar/foo_%d.{}' test_data_dir = 'test/testdata/thumbnails'
generated_file = f'{test_data_dir}/empty.webp'
subprocess.check_call([
pp.executable, '-y', '-f', 'lavfi', '-i', 'color=c=black:s=320x320',
'-c:v', 'libwebp', '-pix_fmt', 'yuv420p', '-vframes', '1', generated_file,
], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
file = test_data_dir + '/foo %d bar/foo_%d.{}'
initial_file = file.format('webp')
os.replace(generated_file, initial_file)
tests = (('webp', 'png'), ('png', 'jpg')) tests = (('webp', 'png'), ('png', 'jpg'))
for inp, out in tests: for inp, out in tests:
@@ -55,17 +68,19 @@ class TestConvertThumbnail(unittest.TestCase):
if os.path.exists(out_file): if os.path.exists(out_file):
os.remove(out_file) os.remove(out_file)
pp.convert_thumbnail(file.format(inp), out) pp.convert_thumbnail(file.format(inp), out)
assert os.path.exists(out_file) self.assertTrue(os.path.exists(out_file))
for _, out in tests: for _, out in tests:
os.remove(file.format(out)) os.remove(file.format(out))
os.remove(initial_file)
class TestExec(unittest.TestCase): class TestExec(unittest.TestCase):
def test_parse_cmd(self): def test_parse_cmd(self):
pp = ExecPP(YoutubeDL(), '') pp = ExecPP(YoutubeDL(), '')
info = {'filepath': 'file name'} info = {'filepath': 'file name'}
cmd = 'echo %s' % compat_shlex_quote(info['filepath']) cmd = 'echo {}'.format(shell_quote(info['filepath']))
self.assertEqual(pp.parse_cmd('echo', info), cmd) self.assertEqual(pp.parse_cmd('echo', info), cmd)
self.assertEqual(pp.parse_cmd('echo {}', info), cmd) self.assertEqual(pp.parse_cmd('echo {}', info), cmd)
@@ -125,7 +140,8 @@ class TestModifyChaptersPP(unittest.TestCase):
self._remove_marked_arrange_sponsors_test_impl(chapters, chapters, []) self._remove_marked_arrange_sponsors_test_impl(chapters, chapters, [])
def test_remove_marked_arrange_sponsors_ChapterWithSponsors(self): def test_remove_marked_arrange_sponsors_ChapterWithSponsors(self):
chapters = self._chapters([70], ['c']) + [ chapters = [
*self._chapters([70], ['c']),
self._sponsor_chapter(10, 20, 'sponsor'), self._sponsor_chapter(10, 20, 'sponsor'),
self._sponsor_chapter(30, 40, 'preview'), self._sponsor_chapter(30, 40, 'preview'),
self._sponsor_chapter(50, 60, 'filler')] self._sponsor_chapter(50, 60, 'filler')]
@@ -136,7 +152,8 @@ class TestModifyChaptersPP(unittest.TestCase):
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, []) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
def test_remove_marked_arrange_sponsors_SponsorBlockChapters(self): def test_remove_marked_arrange_sponsors_SponsorBlockChapters(self):
chapters = self._chapters([70], ['c']) + [ chapters = [
*self._chapters([70], ['c']),
self._sponsor_chapter(10, 20, 'chapter', title='sb c1'), self._sponsor_chapter(10, 20, 'chapter', title='sb c1'),
self._sponsor_chapter(15, 16, 'chapter', title='sb c2'), self._sponsor_chapter(15, 16, 'chapter', title='sb c2'),
self._sponsor_chapter(30, 40, 'preview'), self._sponsor_chapter(30, 40, 'preview'),
@@ -149,10 +166,14 @@ class TestModifyChaptersPP(unittest.TestCase):
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, []) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
def test_remove_marked_arrange_sponsors_UniqueNamesForOverlappingSponsors(self): def test_remove_marked_arrange_sponsors_UniqueNamesForOverlappingSponsors(self):
chapters = self._chapters([120], ['c']) + [ chapters = [
self._sponsor_chapter(10, 45, 'sponsor'), self._sponsor_chapter(20, 40, 'selfpromo'), *self._chapters([120], ['c']),
self._sponsor_chapter(50, 70, 'sponsor'), self._sponsor_chapter(60, 85, 'selfpromo'), self._sponsor_chapter(10, 45, 'sponsor'),
self._sponsor_chapter(90, 120, 'selfpromo'), self._sponsor_chapter(100, 110, 'sponsor')] self._sponsor_chapter(20, 40, 'selfpromo'),
self._sponsor_chapter(50, 70, 'sponsor'),
self._sponsor_chapter(60, 85, 'selfpromo'),
self._sponsor_chapter(90, 120, 'selfpromo'),
self._sponsor_chapter(100, 110, 'sponsor')]
expected = self._chapters( expected = self._chapters(
[10, 20, 40, 45, 50, 60, 70, 85, 90, 100, 110, 120], [10, 20, 40, 45, 50, 60, 70, 85, 90, 100, 110, 120],
['c', '[SponsorBlock]: Sponsor', '[SponsorBlock]: Sponsor, Unpaid/Self Promotion', ['c', '[SponsorBlock]: Sponsor', '[SponsorBlock]: Sponsor, Unpaid/Self Promotion',
@@ -172,7 +193,8 @@ class TestModifyChaptersPP(unittest.TestCase):
chapters, self._chapters([40], ['c']), cuts) chapters, self._chapters([40], ['c']), cuts)
def test_remove_marked_arrange_sponsors_ChapterWithSponsorsAndCuts(self): def test_remove_marked_arrange_sponsors_ChapterWithSponsorsAndCuts(self):
chapters = self._chapters([70], ['c']) + [ chapters = [
*self._chapters([70], ['c']),
self._sponsor_chapter(10, 20, 'sponsor'), self._sponsor_chapter(10, 20, 'sponsor'),
self._sponsor_chapter(30, 40, 'selfpromo', remove=True), self._sponsor_chapter(30, 40, 'selfpromo', remove=True),
self._sponsor_chapter(50, 60, 'interaction')] self._sponsor_chapter(50, 60, 'interaction')]
@@ -185,24 +207,29 @@ class TestModifyChaptersPP(unittest.TestCase):
def test_remove_marked_arrange_sponsors_ChapterWithSponsorCutInTheMiddle(self): def test_remove_marked_arrange_sponsors_ChapterWithSponsorCutInTheMiddle(self):
cuts = [self._sponsor_chapter(20, 30, 'selfpromo', remove=True), cuts = [self._sponsor_chapter(20, 30, 'selfpromo', remove=True),
self._chapter(40, 50, remove=True)] self._chapter(40, 50, remove=True)]
chapters = self._chapters([70], ['c']) + [self._sponsor_chapter(10, 60, 'sponsor')] + cuts chapters = [
*self._chapters([70], ['c']),
self._sponsor_chapter(10, 60, 'sponsor'),
*cuts]
expected = self._chapters( expected = self._chapters(
[10, 40, 50], ['c', '[SponsorBlock]: Sponsor', 'c']) [10, 40, 50], ['c', '[SponsorBlock]: Sponsor', 'c'])
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
def test_remove_marked_arrange_sponsors_ChapterWithCutHidingSponsor(self): def test_remove_marked_arrange_sponsors_ChapterWithCutHidingSponsor(self):
cuts = [self._sponsor_chapter(20, 50, 'selfpromo', remove=True)] cuts = [self._sponsor_chapter(20, 50, 'selfpromo', remove=True)]
chapters = self._chapters([60], ['c']) + [ chapters = [
*self._chapters([60], ['c']),
self._sponsor_chapter(10, 20, 'intro'), self._sponsor_chapter(10, 20, 'intro'),
self._sponsor_chapter(30, 40, 'sponsor'), self._sponsor_chapter(30, 40, 'sponsor'),
self._sponsor_chapter(50, 60, 'outro'), self._sponsor_chapter(50, 60, 'outro'),
] + cuts *cuts]
expected = self._chapters( expected = self._chapters(
[10, 20, 30], ['c', '[SponsorBlock]: Intermission/Intro Animation', '[SponsorBlock]: Endcards/Credits']) [10, 20, 30], ['c', '[SponsorBlock]: Intermission/Intro Animation', '[SponsorBlock]: Endcards/Credits'])
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
def test_remove_marked_arrange_sponsors_ChapterWithAdjacentSponsors(self): def test_remove_marked_arrange_sponsors_ChapterWithAdjacentSponsors(self):
chapters = self._chapters([70], ['c']) + [ chapters = [
*self._chapters([70], ['c']),
self._sponsor_chapter(10, 20, 'sponsor'), self._sponsor_chapter(10, 20, 'sponsor'),
self._sponsor_chapter(20, 30, 'selfpromo'), self._sponsor_chapter(20, 30, 'selfpromo'),
self._sponsor_chapter(30, 40, 'interaction')] self._sponsor_chapter(30, 40, 'interaction')]
@@ -213,7 +240,8 @@ class TestModifyChaptersPP(unittest.TestCase):
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, []) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
def test_remove_marked_arrange_sponsors_ChapterWithAdjacentCuts(self): def test_remove_marked_arrange_sponsors_ChapterWithAdjacentCuts(self):
chapters = self._chapters([70], ['c']) + [ chapters = [
*self._chapters([70], ['c']),
self._sponsor_chapter(10, 20, 'sponsor'), self._sponsor_chapter(10, 20, 'sponsor'),
self._sponsor_chapter(20, 30, 'interaction', remove=True), self._sponsor_chapter(20, 30, 'interaction', remove=True),
self._chapter(30, 40, remove=True), self._chapter(30, 40, remove=True),
@@ -226,7 +254,8 @@ class TestModifyChaptersPP(unittest.TestCase):
chapters, expected, [self._chapter(20, 50, remove=True)]) chapters, expected, [self._chapter(20, 50, remove=True)])
def test_remove_marked_arrange_sponsors_ChapterWithOverlappingSponsors(self): def test_remove_marked_arrange_sponsors_ChapterWithOverlappingSponsors(self):
chapters = self._chapters([70], ['c']) + [ chapters = [
*self._chapters([70], ['c']),
self._sponsor_chapter(10, 30, 'sponsor'), self._sponsor_chapter(10, 30, 'sponsor'),
self._sponsor_chapter(20, 50, 'selfpromo'), self._sponsor_chapter(20, 50, 'selfpromo'),
self._sponsor_chapter(40, 60, 'interaction')] self._sponsor_chapter(40, 60, 'interaction')]
@@ -238,7 +267,8 @@ class TestModifyChaptersPP(unittest.TestCase):
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, []) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
def test_remove_marked_arrange_sponsors_ChapterWithOverlappingCuts(self): def test_remove_marked_arrange_sponsors_ChapterWithOverlappingCuts(self):
chapters = self._chapters([70], ['c']) + [ chapters = [
*self._chapters([70], ['c']),
self._sponsor_chapter(10, 30, 'sponsor', remove=True), self._sponsor_chapter(10, 30, 'sponsor', remove=True),
self._sponsor_chapter(20, 50, 'selfpromo', remove=True), self._sponsor_chapter(20, 50, 'selfpromo', remove=True),
self._sponsor_chapter(40, 60, 'interaction', remove=True)] self._sponsor_chapter(40, 60, 'interaction', remove=True)]
@@ -246,7 +276,8 @@ class TestModifyChaptersPP(unittest.TestCase):
chapters, self._chapters([20], ['c']), [self._chapter(10, 60, remove=True)]) chapters, self._chapters([20], ['c']), [self._chapter(10, 60, remove=True)])
def test_remove_marked_arrange_sponsors_ChapterWithRunsOfOverlappingSponsors(self): def test_remove_marked_arrange_sponsors_ChapterWithRunsOfOverlappingSponsors(self):
chapters = self._chapters([170], ['c']) + [ chapters = [
*self._chapters([170], ['c']),
self._sponsor_chapter(0, 30, 'intro'), self._sponsor_chapter(0, 30, 'intro'),
self._sponsor_chapter(20, 50, 'sponsor'), self._sponsor_chapter(20, 50, 'sponsor'),
self._sponsor_chapter(40, 60, 'selfpromo'), self._sponsor_chapter(40, 60, 'selfpromo'),
@@ -267,7 +298,8 @@ class TestModifyChaptersPP(unittest.TestCase):
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, []) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
def test_remove_marked_arrange_sponsors_ChapterWithRunsOfOverlappingCuts(self): def test_remove_marked_arrange_sponsors_ChapterWithRunsOfOverlappingCuts(self):
chapters = self._chapters([170], ['c']) + [ chapters = [
*self._chapters([170], ['c']),
self._chapter(0, 30, remove=True), self._chapter(0, 30, remove=True),
self._sponsor_chapter(20, 50, 'sponsor', remove=True), self._sponsor_chapter(20, 50, 'sponsor', remove=True),
self._chapter(40, 60, remove=True), self._chapter(40, 60, remove=True),
@@ -284,7 +316,8 @@ class TestModifyChaptersPP(unittest.TestCase):
chapters, self._chapters([20], ['c']), expected_cuts) chapters, self._chapters([20], ['c']), expected_cuts)
def test_remove_marked_arrange_sponsors_OverlappingSponsorsDifferentTitlesAfterCut(self): def test_remove_marked_arrange_sponsors_OverlappingSponsorsDifferentTitlesAfterCut(self):
chapters = self._chapters([60], ['c']) + [ chapters = [
*self._chapters([60], ['c']),
self._sponsor_chapter(10, 60, 'sponsor'), self._sponsor_chapter(10, 60, 'sponsor'),
self._sponsor_chapter(10, 40, 'intro'), self._sponsor_chapter(10, 40, 'intro'),
self._sponsor_chapter(30, 50, 'interaction'), self._sponsor_chapter(30, 50, 'interaction'),
@@ -297,7 +330,8 @@ class TestModifyChaptersPP(unittest.TestCase):
chapters, expected, [self._chapter(30, 50, remove=True)]) chapters, expected, [self._chapter(30, 50, remove=True)])
def test_remove_marked_arrange_sponsors_SponsorsNoLongerOverlapAfterCut(self): def test_remove_marked_arrange_sponsors_SponsorsNoLongerOverlapAfterCut(self):
chapters = self._chapters([70], ['c']) + [ chapters = [
*self._chapters([70], ['c']),
self._sponsor_chapter(10, 30, 'sponsor'), self._sponsor_chapter(10, 30, 'sponsor'),
self._sponsor_chapter(20, 50, 'interaction'), self._sponsor_chapter(20, 50, 'interaction'),
self._sponsor_chapter(30, 50, 'selfpromo', remove=True), self._sponsor_chapter(30, 50, 'selfpromo', remove=True),
@@ -310,7 +344,8 @@ class TestModifyChaptersPP(unittest.TestCase):
chapters, expected, [self._chapter(30, 50, remove=True)]) chapters, expected, [self._chapter(30, 50, remove=True)])
def test_remove_marked_arrange_sponsors_SponsorsStillOverlapAfterCut(self): def test_remove_marked_arrange_sponsors_SponsorsStillOverlapAfterCut(self):
chapters = self._chapters([70], ['c']) + [ chapters = [
*self._chapters([70], ['c']),
self._sponsor_chapter(10, 60, 'sponsor'), self._sponsor_chapter(10, 60, 'sponsor'),
self._sponsor_chapter(20, 60, 'interaction'), self._sponsor_chapter(20, 60, 'interaction'),
self._sponsor_chapter(30, 50, 'selfpromo', remove=True)] self._sponsor_chapter(30, 50, 'selfpromo', remove=True)]
@@ -321,7 +356,8 @@ class TestModifyChaptersPP(unittest.TestCase):
chapters, expected, [self._chapter(30, 50, remove=True)]) chapters, expected, [self._chapter(30, 50, remove=True)])
def test_remove_marked_arrange_sponsors_ChapterWithRunsOfOverlappingSponsorsAndCuts(self): def test_remove_marked_arrange_sponsors_ChapterWithRunsOfOverlappingSponsorsAndCuts(self):
chapters = self._chapters([200], ['c']) + [ chapters = [
*self._chapters([200], ['c']),
self._sponsor_chapter(10, 40, 'sponsor'), self._sponsor_chapter(10, 40, 'sponsor'),
self._sponsor_chapter(10, 30, 'intro'), self._sponsor_chapter(10, 30, 'intro'),
self._chapter(20, 30, remove=True), self._chapter(20, 30, remove=True),
@@ -347,8 +383,9 @@ class TestModifyChaptersPP(unittest.TestCase):
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, expected_cuts) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, expected_cuts)
def test_remove_marked_arrange_sponsors_SponsorOverlapsMultipleChapters(self): def test_remove_marked_arrange_sponsors_SponsorOverlapsMultipleChapters(self):
chapters = (self._chapters([20, 40, 60, 80, 100], ['c1', 'c2', 'c3', 'c4', 'c5']) chapters = [
+ [self._sponsor_chapter(10, 90, 'sponsor')]) *self._chapters([20, 40, 60, 80, 100], ['c1', 'c2', 'c3', 'c4', 'c5']),
self._sponsor_chapter(10, 90, 'sponsor')]
expected = self._chapters([10, 90, 100], ['c1', '[SponsorBlock]: Sponsor', 'c5']) expected = self._chapters([10, 90, 100], ['c1', '[SponsorBlock]: Sponsor', 'c5'])
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, []) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
@@ -359,9 +396,10 @@ class TestModifyChaptersPP(unittest.TestCase):
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
def test_remove_marked_arrange_sponsors_SponsorsWithinSomeChaptersAndOverlappingOthers(self): def test_remove_marked_arrange_sponsors_SponsorsWithinSomeChaptersAndOverlappingOthers(self):
chapters = (self._chapters([10, 40, 60, 80], ['c1', 'c2', 'c3', 'c4']) chapters = [
+ [self._sponsor_chapter(20, 30, 'sponsor'), *self._chapters([10, 40, 60, 80], ['c1', 'c2', 'c3', 'c4']),
self._sponsor_chapter(50, 70, 'selfpromo')]) self._sponsor_chapter(20, 30, 'sponsor'),
self._sponsor_chapter(50, 70, 'selfpromo')]
expected = self._chapters([10, 20, 30, 40, 50, 70, 80], expected = self._chapters([10, 20, 30, 40, 50, 70, 80],
['c1', 'c2', '[SponsorBlock]: Sponsor', 'c2', 'c3', ['c1', 'c2', '[SponsorBlock]: Sponsor', 'c2', 'c3',
'[SponsorBlock]: Unpaid/Self Promotion', 'c4']) '[SponsorBlock]: Unpaid/Self Promotion', 'c4'])
@@ -374,8 +412,9 @@ class TestModifyChaptersPP(unittest.TestCase):
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
def test_remove_marked_arrange_sponsors_ChaptersAfterLastSponsor(self): def test_remove_marked_arrange_sponsors_ChaptersAfterLastSponsor(self):
chapters = (self._chapters([20, 40, 50, 60], ['c1', 'c2', 'c3', 'c4']) chapters = [
+ [self._sponsor_chapter(10, 30, 'music_offtopic')]) *self._chapters([20, 40, 50, 60], ['c1', 'c2', 'c3', 'c4']),
self._sponsor_chapter(10, 30, 'music_offtopic')]
expected = self._chapters( expected = self._chapters(
[10, 30, 40, 50, 60], [10, 30, 40, 50, 60],
['c1', '[SponsorBlock]: Non-Music Section', 'c2', 'c3', 'c4']) ['c1', '[SponsorBlock]: Non-Music Section', 'c2', 'c3', 'c4'])
@@ -388,8 +427,9 @@ class TestModifyChaptersPP(unittest.TestCase):
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
def test_remove_marked_arrange_sponsors_SponsorStartsAtChapterStart(self): def test_remove_marked_arrange_sponsors_SponsorStartsAtChapterStart(self):
chapters = (self._chapters([10, 20, 40], ['c1', 'c2', 'c3']) chapters = [
+ [self._sponsor_chapter(20, 30, 'sponsor')]) *self._chapters([10, 20, 40], ['c1', 'c2', 'c3']),
self._sponsor_chapter(20, 30, 'sponsor')]
expected = self._chapters([10, 20, 30, 40], ['c1', 'c2', '[SponsorBlock]: Sponsor', 'c3']) expected = self._chapters([10, 20, 30, 40], ['c1', 'c2', '[SponsorBlock]: Sponsor', 'c3'])
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, []) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
@@ -400,8 +440,9 @@ class TestModifyChaptersPP(unittest.TestCase):
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
def test_remove_marked_arrange_sponsors_SponsorEndsAtChapterEnd(self): def test_remove_marked_arrange_sponsors_SponsorEndsAtChapterEnd(self):
chapters = (self._chapters([10, 30, 40], ['c1', 'c2', 'c3']) chapters = [
+ [self._sponsor_chapter(20, 30, 'sponsor')]) *self._chapters([10, 30, 40], ['c1', 'c2', 'c3']),
self._sponsor_chapter(20, 30, 'sponsor')]
expected = self._chapters([10, 20, 30, 40], ['c1', 'c2', '[SponsorBlock]: Sponsor', 'c3']) expected = self._chapters([10, 20, 30, 40], ['c1', 'c2', '[SponsorBlock]: Sponsor', 'c3'])
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, []) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
@@ -412,8 +453,9 @@ class TestModifyChaptersPP(unittest.TestCase):
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
def test_remove_marked_arrange_sponsors_SponsorCoincidesWithChapters(self): def test_remove_marked_arrange_sponsors_SponsorCoincidesWithChapters(self):
chapters = (self._chapters([10, 20, 30, 40], ['c1', 'c2', 'c3', 'c4']) chapters = [
+ [self._sponsor_chapter(10, 30, 'sponsor')]) *self._chapters([10, 20, 30, 40], ['c1', 'c2', 'c3', 'c4']),
self._sponsor_chapter(10, 30, 'sponsor')]
expected = self._chapters([10, 30, 40], ['c1', '[SponsorBlock]: Sponsor', 'c4']) expected = self._chapters([10, 30, 40], ['c1', '[SponsorBlock]: Sponsor', 'c4'])
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, []) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
@@ -424,8 +466,9 @@ class TestModifyChaptersPP(unittest.TestCase):
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
def test_remove_marked_arrange_sponsors_SponsorsAtVideoBoundaries(self): def test_remove_marked_arrange_sponsors_SponsorsAtVideoBoundaries(self):
chapters = (self._chapters([20, 40, 60], ['c1', 'c2', 'c3']) chapters = [
+ [self._sponsor_chapter(0, 10, 'intro'), self._sponsor_chapter(50, 60, 'outro')]) *self._chapters([20, 40, 60], ['c1', 'c2', 'c3']),
self._sponsor_chapter(0, 10, 'intro'), self._sponsor_chapter(50, 60, 'outro')]
expected = self._chapters( expected = self._chapters(
[10, 20, 40, 50, 60], ['[SponsorBlock]: Intermission/Intro Animation', 'c1', 'c2', 'c3', '[SponsorBlock]: Endcards/Credits']) [10, 20, 40, 50, 60], ['[SponsorBlock]: Intermission/Intro Animation', 'c1', 'c2', 'c3', '[SponsorBlock]: Endcards/Credits'])
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, []) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
@@ -437,8 +480,10 @@ class TestModifyChaptersPP(unittest.TestCase):
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
def test_remove_marked_arrange_sponsors_SponsorsOverlapChaptersAtVideoBoundaries(self): def test_remove_marked_arrange_sponsors_SponsorsOverlapChaptersAtVideoBoundaries(self):
chapters = (self._chapters([10, 40, 50], ['c1', 'c2', 'c3']) chapters = [
+ [self._sponsor_chapter(0, 20, 'intro'), self._sponsor_chapter(30, 50, 'outro')]) *self._chapters([10, 40, 50], ['c1', 'c2', 'c3']),
self._sponsor_chapter(0, 20, 'intro'),
self._sponsor_chapter(30, 50, 'outro')]
expected = self._chapters( expected = self._chapters(
[20, 30, 50], ['[SponsorBlock]: Intermission/Intro Animation', 'c2', '[SponsorBlock]: Endcards/Credits']) [20, 30, 50], ['[SponsorBlock]: Intermission/Intro Animation', 'c2', '[SponsorBlock]: Endcards/Credits'])
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, []) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
@@ -450,8 +495,10 @@ class TestModifyChaptersPP(unittest.TestCase):
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
def test_remove_marked_arrange_sponsors_EverythingSponsored(self): def test_remove_marked_arrange_sponsors_EverythingSponsored(self):
chapters = (self._chapters([10, 20, 30, 40], ['c1', 'c2', 'c3', 'c4']) chapters = [
+ [self._sponsor_chapter(0, 20, 'intro'), self._sponsor_chapter(20, 40, 'outro')]) *self._chapters([10, 20, 30, 40], ['c1', 'c2', 'c3', 'c4']),
self._sponsor_chapter(0, 20, 'intro'),
self._sponsor_chapter(20, 40, 'outro')]
expected = self._chapters([20, 40], ['[SponsorBlock]: Intermission/Intro Animation', '[SponsorBlock]: Endcards/Credits']) expected = self._chapters([20, 40], ['[SponsorBlock]: Intermission/Intro Animation', '[SponsorBlock]: Endcards/Credits'])
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, []) self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
@@ -491,38 +538,39 @@ class TestModifyChaptersPP(unittest.TestCase):
chapters, self._chapters([2.5], ['c2']), cuts) chapters, self._chapters([2.5], ['c2']), cuts)
def test_remove_marked_arrange_sponsors_TinyChaptersResultingFromSponsorOverlapAreIgnored(self): def test_remove_marked_arrange_sponsors_TinyChaptersResultingFromSponsorOverlapAreIgnored(self):
chapters = self._chapters([1, 3, 4], ['c1', 'c2', 'c3']) + [ chapters = [
*self._chapters([1, 3, 4], ['c1', 'c2', 'c3']),
self._sponsor_chapter(1.5, 2.5, 'sponsor')] self._sponsor_chapter(1.5, 2.5, 'sponsor')]
self._remove_marked_arrange_sponsors_test_impl( self._remove_marked_arrange_sponsors_test_impl(
chapters, self._chapters([1.5, 2.5, 4], ['c1', '[SponsorBlock]: Sponsor', 'c3']), []) chapters, self._chapters([1.5, 2.5, 4], ['c1', '[SponsorBlock]: Sponsor', 'c3']), [])
def test_remove_marked_arrange_sponsors_TinySponsorsOverlapsAreIgnored(self): def test_remove_marked_arrange_sponsors_TinySponsorsOverlapsAreIgnored(self):
chapters = self._chapters([2, 3, 5], ['c1', 'c2', 'c3']) + [ chapters = [
*self._chapters([2, 3, 5], ['c1', 'c2', 'c3']),
self._sponsor_chapter(1, 3, 'sponsor'), self._sponsor_chapter(1, 3, 'sponsor'),
self._sponsor_chapter(2.5, 4, 'selfpromo') self._sponsor_chapter(2.5, 4, 'selfpromo')]
]
self._remove_marked_arrange_sponsors_test_impl( self._remove_marked_arrange_sponsors_test_impl(
chapters, self._chapters([1, 3, 4, 5], [ chapters, self._chapters([1, 3, 4, 5], [
'c1', '[SponsorBlock]: Sponsor', '[SponsorBlock]: Unpaid/Self Promotion', 'c3']), []) 'c1', '[SponsorBlock]: Sponsor', '[SponsorBlock]: Unpaid/Self Promotion', 'c3']), [])
def test_remove_marked_arrange_sponsors_TinySponsorsPrependedToTheNextSponsor(self): def test_remove_marked_arrange_sponsors_TinySponsorsPrependedToTheNextSponsor(self):
chapters = self._chapters([4], ['c']) + [ chapters = [
*self._chapters([4], ['c']),
self._sponsor_chapter(1.5, 2, 'sponsor'), self._sponsor_chapter(1.5, 2, 'sponsor'),
self._sponsor_chapter(2, 4, 'selfpromo') self._sponsor_chapter(2, 4, 'selfpromo')]
]
self._remove_marked_arrange_sponsors_test_impl( self._remove_marked_arrange_sponsors_test_impl(
chapters, self._chapters([1.5, 4], ['c', '[SponsorBlock]: Unpaid/Self Promotion']), []) chapters, self._chapters([1.5, 4], ['c', '[SponsorBlock]: Unpaid/Self Promotion']), [])
def test_remove_marked_arrange_sponsors_SmallestSponsorInTheOverlapGetsNamed(self): def test_remove_marked_arrange_sponsors_SmallestSponsorInTheOverlapGetsNamed(self):
self._pp._sponsorblock_chapter_title = '[SponsorBlock]: %(name)s' self._pp._sponsorblock_chapter_title = '[SponsorBlock]: %(name)s'
chapters = self._chapters([10], ['c']) + [ chapters = [
*self._chapters([10], ['c']),
self._sponsor_chapter(2, 8, 'sponsor'), self._sponsor_chapter(2, 8, 'sponsor'),
self._sponsor_chapter(4, 6, 'selfpromo') self._sponsor_chapter(4, 6, 'selfpromo')]
]
self._remove_marked_arrange_sponsors_test_impl( self._remove_marked_arrange_sponsors_test_impl(
chapters, self._chapters([2, 4, 6, 8, 10], [ chapters, self._chapters([2, 4, 6, 8, 10], [
'c', '[SponsorBlock]: Sponsor', '[SponsorBlock]: Unpaid/Self Promotion', 'c', '[SponsorBlock]: Sponsor', '[SponsorBlock]: Unpaid/Self Promotion',
'[SponsorBlock]: Sponsor', 'c' '[SponsorBlock]: Sponsor', 'c',
]), []) ]), [])
def test_make_concat_opts_CommonCase(self): def test_make_concat_opts_CommonCase(self):
@@ -577,3 +625,7 @@ outpoint 10.000000
self.assertEqual( self.assertEqual(
r"'special '\'' characters '\'' galore'\'\'\'", r"'special '\'' characters '\'' galore'\'\'\'",
self._pp._quote_for_ffmpeg("special ' characters ' galore'''")) self._pp._quote_for_ffmpeg("special ' characters ' galore'''"))
if __name__ == '__main__':
unittest.main()
+71
View File
@@ -0,0 +1,71 @@
import collections
import pytest
from yt_dlp import YoutubeDL
from yt_dlp.cookies import YoutubeDLCookieJar
from yt_dlp.extractor.common import InfoExtractor
from yt_dlp.extractor.youtube.pot._provider import IEContentProviderLogger
from yt_dlp.extractor.youtube.pot.provider import PoTokenRequest, PoTokenContext
from yt_dlp.utils.networking import HTTPHeaderDict
class MockLogger(IEContentProviderLogger):
log_level = IEContentProviderLogger.LogLevel.TRACE
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.messages = collections.defaultdict(list)
def trace(self, message: str):
self.messages['trace'].append(message)
def debug(self, message: str):
self.messages['debug'].append(message)
def info(self, message: str):
self.messages['info'].append(message)
def warning(self, message: str, *, once=False):
self.messages['warning'].append(message)
def error(self, message: str):
self.messages['error'].append(message)
@pytest.fixture
def ie() -> InfoExtractor:
ydl = YoutubeDL()
return ydl.get_info_extractor('Youtube')
@pytest.fixture
def logger() -> MockLogger:
return MockLogger()
@pytest.fixture()
def pot_request() -> PoTokenRequest:
return PoTokenRequest(
context=PoTokenContext.GVS,
innertube_context={'client': {'clientName': 'WEB'}},
innertube_host='youtube.com',
session_index=None,
player_url=None,
is_authenticated=False,
video_webpage=None,
visitor_data='example-visitor-data',
data_sync_id='example-data-sync-id',
video_id='example-video-id',
request_cookiejar=YoutubeDLCookieJar(),
request_proxy=None,
request_headers=HTTPHeaderDict(),
request_timeout=None,
request_source_address=None,
request_verify_tls=True,
bypass_cache=False,
)
@@ -0,0 +1,117 @@
import threading
import time
from collections import OrderedDict
import pytest
from yt_dlp.extractor.youtube.pot._provider import IEContentProvider, BuiltinIEContentProvider
from yt_dlp.utils import bug_reports_message
from yt_dlp.extractor.youtube.pot._builtin.memory_cache import MemoryLRUPCP, memorylru_preference, initialize_global_cache
from yt_dlp.version import __version__
from yt_dlp.extractor.youtube.pot._registry import _pot_cache_providers, _pot_memory_cache
class TestMemoryLRUPCS:
def test_base_type(self):
assert issubclass(MemoryLRUPCP, IEContentProvider)
assert issubclass(MemoryLRUPCP, BuiltinIEContentProvider)
@pytest.fixture
def pcp(self, ie, logger) -> MemoryLRUPCP:
return MemoryLRUPCP(ie, logger, {}, initialize_cache=lambda max_size: (OrderedDict(), threading.Lock(), max_size))
def test_is_registered(self):
assert _pot_cache_providers.value.get('MemoryLRU') == MemoryLRUPCP
def test_initialization(self, pcp):
assert pcp.PROVIDER_NAME == 'memory'
assert pcp.PROVIDER_VERSION == __version__
assert pcp.BUG_REPORT_MESSAGE == bug_reports_message(before='')
assert pcp.is_available()
def test_store_and_get(self, pcp):
pcp.store('key1', 'value1', int(time.time()) + 60)
assert pcp.get('key1') == 'value1'
assert len(pcp.cache) == 1
def test_store_ignore_expired(self, pcp):
pcp.store('key1', 'value1', int(time.time()) - 1)
assert len(pcp.cache) == 0
assert pcp.get('key1') is None
assert len(pcp.cache) == 0
def test_store_override_existing_key(self, ie, logger):
MAX_SIZE = 2
pcp = MemoryLRUPCP(ie, logger, {}, initialize_cache=lambda max_size: (OrderedDict(), threading.Lock(), MAX_SIZE))
pcp.store('key1', 'value1', int(time.time()) + 60)
pcp.store('key2', 'value2', int(time.time()) + 60)
assert len(pcp.cache) == 2
pcp.store('key1', 'value2', int(time.time()) + 60)
# Ensure that the override key gets added to the end of the cache instead of in the same position
pcp.store('key3', 'value3', int(time.time()) + 60)
assert pcp.get('key1') == 'value2'
def test_store_ignore_expired_existing_key(self, pcp):
pcp.store('key1', 'value2', int(time.time()) + 60)
pcp.store('key1', 'value1', int(time.time()) - 1)
assert len(pcp.cache) == 1
assert pcp.get('key1') == 'value2'
assert len(pcp.cache) == 1
def test_get_key_expired(self, pcp):
pcp.store('key1', 'value1', int(time.time()) + 60)
assert pcp.get('key1') == 'value1'
assert len(pcp.cache) == 1
pcp.cache['key1'] = ('value1', int(time.time()) - 1)
assert pcp.get('key1') is None
assert len(pcp.cache) == 0
def test_lru_eviction(self, ie, logger):
MAX_SIZE = 2
provider = MemoryLRUPCP(ie, logger, {}, initialize_cache=lambda max_size: (OrderedDict(), threading.Lock(), MAX_SIZE))
provider.store('key1', 'value1', int(time.time()) + 5)
provider.store('key2', 'value2', int(time.time()) + 5)
assert len(provider.cache) == 2
assert provider.get('key1') == 'value1'
provider.store('key3', 'value3', int(time.time()) + 5)
assert len(provider.cache) == 2
assert provider.get('key2') is None
provider.store('key4', 'value4', int(time.time()) + 5)
assert len(provider.cache) == 2
assert provider.get('key1') is None
assert provider.get('key3') == 'value3'
assert provider.get('key4') == 'value4'
def test_delete(self, pcp):
pcp.store('key1', 'value1', int(time.time()) + 5)
assert len(pcp.cache) == 1
assert pcp.get('key1') == 'value1'
pcp.delete('key1')
assert len(pcp.cache) == 0
assert pcp.get('key1') is None
def test_use_global_cache_default(self, ie, logger):
pcp = MemoryLRUPCP(ie, logger, {})
assert pcp.max_size == _pot_memory_cache.value['max_size'] == 25
assert pcp.cache is _pot_memory_cache.value['cache']
assert pcp.lock is _pot_memory_cache.value['lock']
pcp2 = MemoryLRUPCP(ie, logger, {})
assert pcp.max_size == pcp2.max_size == _pot_memory_cache.value['max_size'] == 25
assert pcp.cache is pcp2.cache is _pot_memory_cache.value['cache']
assert pcp.lock is pcp2.lock is _pot_memory_cache.value['lock']
def test_fail_max_size_change_global(self, ie, logger):
pcp = MemoryLRUPCP(ie, logger, {})
assert pcp.max_size == _pot_memory_cache.value['max_size'] == 25
with pytest.raises(ValueError, match='Cannot change max_size of initialized global memory cache'):
initialize_global_cache(50)
assert pcp.max_size == _pot_memory_cache.value['max_size'] == 25
def test_memory_lru_preference(self, pcp, ie, pot_request):
assert memorylru_preference(pcp, pot_request) == 10000
+47
View File
@@ -0,0 +1,47 @@
import pytest
from yt_dlp.extractor.youtube.pot.provider import (
PoTokenContext,
)
from yt_dlp.extractor.youtube.pot.utils import get_webpo_content_binding, ContentBindingType
class TestGetWebPoContentBinding:
@pytest.mark.parametrize('client_name, context, is_authenticated, expected', [
*[(client, context, is_authenticated, expected) for client in [
'WEB', 'MWEB', 'TVHTML5', 'WEB_EMBEDDED_PLAYER', 'WEB_CREATOR', 'TVHTML5_SIMPLY_EMBEDDED_PLAYER', 'TVHTML5_SIMPLY']
for context, is_authenticated, expected in [
(PoTokenContext.GVS, False, ('example-visitor-data', ContentBindingType.VISITOR_DATA)),
(PoTokenContext.PLAYER, False, ('example-video-id', ContentBindingType.VIDEO_ID)),
(PoTokenContext.SUBS, False, ('example-video-id', ContentBindingType.VIDEO_ID)),
(PoTokenContext.GVS, True, ('example-data-sync-id', ContentBindingType.DATASYNC_ID)),
]],
('WEB_REMIX', PoTokenContext.GVS, False, ('example-visitor-data', ContentBindingType.VISITOR_DATA)),
('WEB_REMIX', PoTokenContext.PLAYER, False, ('example-visitor-data', ContentBindingType.VISITOR_DATA)),
('ANDROID', PoTokenContext.GVS, False, (None, None)),
('IOS', PoTokenContext.GVS, False, (None, None)),
])
def test_get_webpo_content_binding(self, pot_request, client_name, context, is_authenticated, expected):
pot_request.innertube_context['client']['clientName'] = client_name
pot_request.context = context
pot_request.is_authenticated = is_authenticated
assert get_webpo_content_binding(pot_request) == expected
def test_extract_visitor_id(self, pot_request):
pot_request.visitor_data = 'CgsxMjNhYmNYWVpfLSiA4s%2DqBg%3D%3D'
assert get_webpo_content_binding(pot_request, bind_to_visitor_id=True) == ('123abcXYZ_-', ContentBindingType.VISITOR_ID)
def test_invalid_visitor_id(self, pot_request):
# visitor id not alphanumeric (i.e. protobuf extraction failed)
pot_request.visitor_data = 'CggxMjM0NTY3OCiA4s-qBg%3D%3D'
assert get_webpo_content_binding(pot_request, bind_to_visitor_id=True) == (pot_request.visitor_data, ContentBindingType.VISITOR_DATA)
def test_no_visitor_id(self, pot_request):
pot_request.visitor_data = 'KIDiz6oG'
assert get_webpo_content_binding(pot_request, bind_to_visitor_id=True) == (pot_request.visitor_data, ContentBindingType.VISITOR_DATA)
def test_invalid_base64(self, pot_request):
pot_request.visitor_data = 'invalid-base64'
assert get_webpo_content_binding(pot_request, bind_to_visitor_id=True) == (pot_request.visitor_data, ContentBindingType.VISITOR_DATA)

Some files were not shown because too many files have changed in this diff Show More