468 Commits

Author SHA1 Message Date
KnugiHK
9d2e06f973 Merge branch 'main' of https://github.com/KnugiHK/Whatsapp-Chat-Exporter 2025-12-15 01:12:30 +08:00
KnugiHK
dffce977de Bump version to 0.12.1 2025-12-15 01:12:14 +08:00
KnugiHK
71ca293557 Add main entry point
Added a main entry point in __main__.py to allow running the exporter as a script. Required for standalone binary
2025-12-15 01:12:04 +08:00
Knugi
75720c6d0a Upgrade GitHub Actions to use version 6 2025-12-14 17:08:49 +00:00
Knugi
fa629503f7 Update Nuitka version and build commands in workflow 2025-12-14 09:43:50 +00:00
Knugi
f6442f9d73 Update Nuitka installation in CI workflow
Removed specific version for Nuitka installation.
2025-12-14 09:20:41 +00:00
Knugi
1d5bad92a7 Add new IV and DB entry to utility.py
Reported by @silasjelley
2025-11-07 13:13:14 +00:00
Knugi
09162bf522 Update README with usage notes and Android link
Added note about providing link for Android export instructions.
2025-10-20 05:55:09 +00:00
Knugi
7b66fe2ee2 Update LICENSE.django 2025-05-17 05:40:22 +00:00
Knugi
c70143fb4b Create codeql.yml 2025-05-11 10:26:48 +00:00
Knugi
9c9c4d9ad2 Update README.md 2025-05-11 10:21:37 +00:00
Knugi
4bd3c1d74a Update pull_request_template.md 2025-05-07 14:55:21 +00:00
Knugi
f7d1332a14 Update pull_request_template.md 2025-05-05 09:19:45 +00:00
Knugi
5291ed0d6f Update generate-website.yml 2025-05-04 08:10:17 +00:00
Knugi
cab54658ee Update generate-website.js 2025-05-04 08:05:22 +00:00
Knugi
96e5823faa Update LICENSE 2025-05-01 12:25:34 +00:00
Knugi
81f072f899 Update generate-website.js 2025-04-29 13:22:05 +00:00
Knugi
2d8960d5e3 Update README.md 2025-04-29 13:20:14 +00:00
Knugi
bacbcda474 Update README.md 2025-04-29 08:55:31 +00:00
Knugi
9cfbb560eb Update generate-website.yml 2025-04-29 08:52:32 +00:00
Knugi
c37e505408 Update generate-website.yml 2025-04-29 08:49:57 +00:00
KnugiHK
b3ce22ddbc Add docs.html to gh-page 2025-04-27 16:21:38 +08:00
Knugi
15d6674644 Delete CNAME 2025-04-27 08:16:50 +00:00
Knugi
07b525b0c6 Update README.md 2025-04-27 07:19:21 +00:00
KnugiHK
dc639d5dac Update pyproject.toml 2025-04-27 14:40:48 +08:00
KnugiHK
ae6a65f98d Update generate-website.js 2025-04-27 14:07:51 +08:00
KnugiHK
578c961932 Add workflow for generating website from readme 2025-04-27 13:00:51 +08:00
KnugiHK
82ac466527 Merge branch 'dev' 2025-04-27 11:34:45 +08:00
KnugiHK
4faf8e3e16 Bump version 2025-04-27 11:34:27 +08:00
KnugiHK
df6bc43aa9 Handle numbers prefixing the country code without + 2025-04-26 18:47:56 +08:00
KnugiHK
40dc3b657e Fixed the regex to match any prefix for TEL fields 2025-04-26 18:46:28 +08:00
KnugiHK
6dde72d330 Add testing for brazilian_number_processing.py 2025-04-26 18:46:10 +08:00
KnugiHK
eaba41b604 Add introduction to the script 2025-04-26 18:17:10 +08:00
KnugiHK
a22427e155 Rename the script 2025-04-26 18:16:57 +08:00
KnugiHK
e287ccb724 Bug fix on None metadata #148 2025-04-26 17:59:10 +08:00
Knugi
eb37c91eee Update README.md 2025-04-19 08:50:27 +00:00
Knugi
763b2e5c76 Update CONTRIBUTING.md 2025-04-19 08:49:34 +00:00
Knugi
9da1da402b Merge pull request #147 from NicksonYap/main
Add the fileID / SHA-1 for CallHistory.sqlite of WhatsApp for Business
2025-04-19 11:24:56 +08:00
Nickson Yap
7c7260893d Add the fileID / SHA-1 for CallHistory.sqlite of WhatsApp for Business 2025-04-18 03:02:10 +08:00
Knugi
60b8512dde Update README.md 2025-04-08 14:41:32 +00:00
KnugiHK
09503069b7 Fix name 'exit' is not defined (#107) 2025-04-08 22:35:16 +08:00
KnugiHK
c56682ff8d Add a potential solution for missing the database in iOS #125 2025-03-29 23:32:48 +08:00
Knugi
1c30dc0ed8 Update docs.html 2025-03-29 15:26:37 +00:00
Knugi
9adb1f9c08 Update README.md 2025-03-27 14:05:45 +00:00
Knugi
d0100ad904 Update docs.html 2025-03-15 07:14:05 +00:00
Knugi
4bafeb9b00 Update README.md 2025-03-11 14:30:06 +00:00
Knugi
538afef5b6 Update README.md 2025-03-11 13:59:03 +00:00
Knugi
6b98acdecf Update CONTRIBUTING.md 2025-03-10 13:32:41 +00:00
Knugi
17308d9727 Create CONTRIBUTING.md 2025-03-10 13:29:37 +00:00
Knugi
ed49633f9c Update README.md 2025-03-09 16:29:11 +00:00
Knugi
7ee61084c0 Create pull_request_template.md 2025-03-04 14:08:08 +00:00
Knugi
9b3e940a4f Merge pull request #119 from gamelaster/patch-1
Add extracting command for iTunes downloaded from Microsoft Store
2025-03-04 13:57:04 +00:00
KnugiHK
ec53ba61e3 The new path may not necessarily be used exclusively by iTunes from the MS Store 2025-03-04 21:53:43 +08:00
KnugiHK
d75c485a3d Update README.md 2025-03-02 15:45:28 +08:00
KnugiHK
0074acca7a Allow users to set the number of threads for bruteforcing offsets 2025-03-02 15:44:40 +08:00
KnugiHK
8f0a9c3cc5 Refactor android_crypt.py 2025-03-02 15:32:37 +08:00
KnugiHK
6a67f72ff3 Refactor ios_media_handler 2025-03-02 14:57:35 +08:00
KnugiHK
0ebd01444a Refactor android_handler 2025-03-02 14:17:22 +08:00
KnugiHK
8c9c43ef38 Merge branch 'dev' into refactoring 2025-03-02 13:35:13 +08:00
KnugiHK
1bb3f2ccea Skip generating chats that do not contain any message 2025-03-02 13:06:45 +08:00
KnugiHK
7c4705d149 Major refactoring
This commit does not refactor Android handler
2025-03-02 12:57:27 +08:00
KnugiHK
4a0be0233c Bug fix on model change for Message 2025-03-02 11:25:01 +08:00
KnugiHK
2290be751a Update ios_handler.py 2025-03-02 01:49:42 +08:00
KnugiHK
1ef223e238 Refactor the data model 2025-03-02 01:41:44 +08:00
KnugiHK
9f321384ec Make ChatStore.messages private 2025-03-02 00:52:28 +08:00
KnugiHK
4d04e51dda Refactor and add docstrings 2025-03-02 00:47:34 +08:00
KnugiHK
431dce7d24 Change package_url_json to a constant 2025-03-02 00:29:03 +08:00
KnugiHK
86cb44ced9 Add more docstrings 2025-03-02 00:28:47 +08:00
KnugiHK
272454c2ce Bug fix on missing _version_ variable, introduced in 0.11.0 2025-03-01 23:59:12 +08:00
Knugi
b08f958c2a Update compile-binary.yml 2025-03-01 04:23:02 +00:00
KnugiHK
6034937cf5 Terminate the process when unknown android backup format supplied 2025-03-01 12:18:43 +08:00
Knugi
2d7a377646 Update compile-binary.yml 2025-03-01 04:16:45 +00:00
Knugi
e23773e521 Update compile-binary.yml 2025-03-01 04:13:27 +00:00
Knugi
39a1e1dec0 Update compile-binary.yml 2025-03-01 04:09:26 +00:00
Knugi
2132bbbff8 Add vobject to dependency 2025-03-01 04:04:15 +00:00
Knugi
113e9c1c19 Update Nuitka 2025-03-01 04:02:33 +00:00
KnugiHK
457ab209c1 Bug fix on incorrectly positioned argument
This commit also made `dry_run` and `keyfile_stream` keyword arguments

Affects #130
2025-02-26 21:30:52 +08:00
KnugiHK
a7496f80a7 Update bplist.py 2025-02-26 21:20:17 +08:00
KnugiHK
b2bcf36622 Move Android backup decryption to a standalone module 2025-02-26 21:20:11 +08:00
KnugiHK
26abfdd570 Bug fix on argument positions 2025-02-26 21:14:50 +08:00
KnugiHK
b9f811c147 Add documentations, refactor and implement crypt15 key dynamical input 2025-02-22 18:14:15 +08:00
KnugiHK
d6b1d944bf Implement dry-run for decrypting Android backup #130 2025-02-21 22:47:52 +08:00
KnugiHK
8c85656831 Show different warning messages when enrich_from_vcards is set and contact db is empty 2025-02-20 23:46:26 +08:00
KnugiHK
db577c8de6 Merge branch 'main' into dev 2025-02-16 12:08:00 +08:00
KnugiHK
6896c9663e Bump version 2025-02-16 11:19:17 +08:00
KnugiHK
6dda2eb6d5 Fix handling of empty quoted_content 2025-02-16 11:17:38 +08:00
KnugiHK
1706f3e9e5 Fixed the inclusion of the template files 2025-02-16 10:44:53 +08:00
Knugi
63c27f63bd Update docs.html 2025-02-14 18:30:55 +00:00
KnugiHK
0cbae4d276 Create a script to process Brazilian numbers in vcards #127 2025-02-11 00:52:05 +08:00
KnugiHK
cfe04c8c0b Display the metadata from the messages sent by "me" (#69)
For now, only the time for "delivered" (android & ios) and "read" (android only)  is support.
2025-02-09 18:44:18 +08:00
KnugiHK
aaeff80547 Remove TODO flag as it is fixed already 2025-02-09 18:37:05 +08:00
KnugiHK
c8b71213ae Remove --iphone 2025-02-09 16:26:08 +08:00
KnugiHK
05505eb3ba Bump version 2025-02-09 16:23:04 +08:00
KnugiHK
88680042ba Merge branch 'dev' 2025-02-09 16:19:45 +08:00
KnugiHK
510b4a7e7d Implement quoted message preview for iOS reply bubble (#28) 2025-02-09 16:13:14 +08:00
KnugiHK
bb26d7b605 Fix the wrong behaviour for reply anchor introduced by the base tag 2025-02-09 15:41:15 +08:00
KnugiHK
dd75ec4b87 Implement backward navigation for splited files 2025-02-09 14:47:05 +08:00
KnugiHK
0b2dfa9aba Implement custom headline (#97) 2025-02-09 14:20:11 +08:00
KnugiHK
539a1d58b0 Add finish line 2025-02-09 14:09:39 +08:00
KnugiHK
f43e1f760d Lazy loading of video for new theme 2025-02-09 14:09:22 +08:00
KnugiHK
bfd172031c Distinguish between regular video and GIF (#103) 2025-02-09 13:54:48 +08:00
KnugiHK
17ec2ecf76 Update the layout of the footer 2025-02-09 13:23:54 +08:00
KnugiHK
f300e017ed Implement lazy loading for video (#103) 2025-02-09 13:23:23 +08:00
KnugiHK
bf993c5302 Change the column to determine if the chat should be filtered (#112) 2025-02-09 12:47:35 +08:00
Knugi
5b3d0e2b3a Update README.md 2025-01-08 15:57:04 +00:00
Knugi
ec7cafd6b6 Update README.md 2025-01-07 16:18:48 +00:00
KnugiHK
23af55d645 Implement empty chat filtering from SQL #112
This commit also removed the old empty chat filtering logic.
2025-01-04 18:18:34 +08:00
KnugiHK
92d710bce8 Differentiate group and personal calls 2025-01-02 20:57:28 +08:00
KnugiHK
7a1fa46368 Implement call log for iOS #122 2025-01-02 20:48:11 +08:00
KnugiHK
cf03bfba1b Bug fix on duplicated base name #126 2025-01-02 17:30:11 +08:00
KnugiHK
a0b8167121 Create a whatsapp-alike theme #97 2025-01-02 16:01:25 +08:00
KnugiHK
7117716e5b Add crypt14 offset 2024-12-19 19:13:21 +08:00
Knugi
a1f6320cd8 Update README.md 2024-12-09 14:12:26 +00:00
KnugiHK
37e329a051 Merge branch 'main' into dev 2024-12-09 22:11:29 +08:00
KnugiHK
a8bac8837e Automatically detect timezone offset when --time-offset is not provided #124 2024-12-08 20:57:38 +08:00
KnugiHK
82d2485778 Fixed the incorrect iOS timestamp #124 2024-12-08 20:42:33 +08:00
KnugiHK
209d5a7796 Migrate to pyproject.toml 2024-12-08 20:36:22 +08:00
KnugiHK
fef9684189 Remove __version__
Use importlib.metadata.version instead
2024-12-08 20:35:54 +08:00
Knugi
0d43d80e23 Update bug_report.md 2024-11-14 04:26:51 +00:00
Knugi
88c2abd5e7 Update bug_report.md 2024-11-14 04:25:59 +00:00
Knugi
379e4bbb7e Update README.md 2024-10-27 03:55:32 +00:00
KnugiHK
fa37dd4b2d Update setup.py 2024-10-24 19:51:00 +08:00
KnugiHK
afa6052a08 Add note 2024-10-24 19:39:19 +08:00
KnugiHK
bde3c18498 Bump version 2024-10-24 19:34:09 +08:00
KnugiHK
af3307825a Update docs 2024-10-24 19:33:54 +08:00
Marek Kraus
ecc7706959 Add extracting command for iTunes downloaded from Microsoft Store 2024-09-29 11:48:54 +02:00
KnugiHK
9b34f7ea6d Merge branch 'main' into dev 2024-09-24 23:37:51 +08:00
Knugi
280a1186d8 Update README.md 2024-09-24 15:37:27 +00:00
KnugiHK
30cff71e76 Add type hints 2024-09-24 23:31:59 +08:00
KnugiHK
05d21e3e5a Rename the functions for size conversion 2024-09-24 23:31:00 +08:00
KnugiHK
fb88c83ac4 Dynamically convert the size unit for outputting JSON 2024-09-24 23:29:11 +08:00
KnugiHK
ffb6aef96e Add handlers for the default and other call state #117 2024-09-24 23:16:40 +08:00
Knugi
77c5a3e20c Update README.md 2024-09-20 16:11:03 +00:00
Knugi
7b0965ac1d Update README.md 2024-09-20 13:25:13 +00:00
KnugiHK
d72b41da11 Wrap caption in a w3-container #111 2024-09-16 21:37:18 +08:00
KnugiHK
fed14ceb29 Bug fix on missing group name 2024-09-15 11:33:23 +08:00
KnugiHK
3e6fdaa126 Bug fix on empty file name for (mostly) ios 2024-09-15 11:09:06 +08:00
KnugiHK
04000c78e2 Implement chat filter with group msg sender for iOS #85 2024-09-15 10:55:43 +08:00
KnugiHK
75c429fe22 Remove blocked code (ios) 2024-09-15 10:43:44 +08:00
KnugiHK
9608fa387d Allow units in --split 2024-09-15 01:27:08 +08:00
KnugiHK
fc9c76c34c Implement chat filter with group msg sender for Android #85 2024-09-15 00:29:57 +08:00
KnugiHK
87b1fcc038 Show caption even if the the message is a metadata #111 2024-09-14 16:10:18 +08:00
KnugiHK
fe88f1b837 Make the iOS contact database optional #76 2024-09-14 15:22:53 +08:00
Knugi
af3d31f773 Update README.md 2024-09-13 12:37:18 +00:00
KnugiHK
df67a549c0 Bug fix on empty contact list when using --enrich-from-vcards 2024-09-11 00:23:13 +08:00
KnugiHK
884ccc4cc0 Bug fix on vobject being a necessary dependency 2024-09-11 00:07:39 +08:00
KnugiHK
484910cf5c Remove blocked code
No plan on implementing this feature
2024-09-10 00:15:25 +08:00
KnugiHK
a83c8eb17f Basic implementation on txt format output #108 2024-09-10 00:14:10 +08:00
KnugiHK
8ffa8cfcac Handle the wording of time unit in calls 2024-09-07 21:26:36 +08:00
KnugiHK
8fcd50d21b Implement unit conversion for picked up calls #115 2024-09-07 20:58:54 +08:00
KnugiHK
f91c527676 Adapting the function of converting size unit from stackoverflow 2024-09-07 20:45:26 +08:00
KnugiHK
f35bf24a5e Add space before <br> to prevent it from being recognized as part of a link #113 2024-09-07 20:30:22 +08:00
KnugiHK
e2684845b8 Bug fix on missing the last message when using --split #114 2024-09-07 18:57:16 +08:00
KnugiHK
df3333f948 Sanitize the file name for --per-chat options #86 2024-09-07 18:45:36 +08:00
KnugiHK
bd4ccbb8ac Add all forbidden options to an error message 2024-09-07 18:31:06 +08:00
KnugiHK
fb5a1c3e1f Bug fix on incomplete JSON output on --per-chat #86 2024-09-07 18:30:38 +08:00
Knugi
1760dea0f5 Merge pull request #106 from Bnaya/bnaya-iteration
[Feat] Import contact names from exported google vcards and more
2024-07-20 13:45:20 +08:00
Knugi
4fcb4df0a4 Merge branch 'dev' into bnaya-iteration 2024-07-20 13:43:32 +08:00
Bnaya Peretz
13904ea4d8 Fix cond 2024-07-15 08:48:15 +03:00
KnugiHK
8069882dc5 Update variable name for filter_empty 2024-07-13 21:54:13 +08:00
KnugiHK
d95b075ac0 Invert the --dont-filter-empty option to provide clarity 2024-07-13 18:57:16 +08:00
KnugiHK
ea01a727cf Fixed the wrong boolean value 2024-07-13 18:33:33 +08:00
Knugi
b2f679d975 Fix --avoid-encoding-json option
Co-authored-by: Bnaya Peretz <me@bnaya.net>
2024-07-13 10:25:44 +00:00
KnugiHK
0cf113561a Rename variable and files 2024-07-13 14:16:57 +08:00
KnugiHK
80bdc4414a Refactoring empty filtering 2024-07-13 14:12:21 +08:00
KnugiHK
09e5e1a756 Update readme for enriching contacts 2024-07-13 13:59:47 +08:00
KnugiHK
6e37061e71 Rename module 2024-07-13 13:52:07 +08:00
KnugiHK
b301dd22d0 Refactoring 2024-07-13 13:47:07 +08:00
KnugiHK
5b97d6013a Refactor core code of mapping vcards 2024-07-13 13:33:47 +08:00
KnugiHK
8f304f1c48 There should be no need for enumerate() 2024-07-13 12:43:03 +08:00
KnugiHK
7bb2fb2420 Refactoring --pretty-print-json and --avoid_encoding_json options 2024-07-13 12:28:01 +08:00
KnugiHK
83fefe585b Refactoring
The added code should be placed above any platform specific code
2024-07-13 12:18:18 +08:00
KnugiHK
4886587065 Make the message clear 2024-07-13 12:15:45 +08:00
KnugiHK
0423fdabda Update contacts_names_from_vcards_test.py 2024-07-13 12:12:55 +08:00
KnugiHK
823ed663e7 The check happened in __main__ already 2024-07-13 12:12:32 +08:00
Bnaya Peretz
be469aed93 Bnaya's assorted features 2024-07-13 12:00:35 +08:00
Knugi
b34045a59f Update bug_report.md 2024-07-02 17:08:04 +00:00
KnugiHK
3461ce3735 Handle the FileNotFoundError in decryption (#98) 2024-07-02 20:34:18 +08:00
KnugiHK
b0942d695b Add base tag to reduce the output size (#103) 2024-06-30 19:17:45 +08:00
KnugiHK
5449646a1b Add lazy loading to image (#103) 2024-06-30 19:17:40 +08:00
KnugiHK
6370b81299 Update import 2024-06-30 18:55:42 +08:00
KnugiHK
c69d053049 Import back the BPListReader 2024-06-30 18:18:26 +08:00
KnugiHK
b01d81ddec Lower the chunk size for decryption 2024-06-22 20:20:57 +08:00
KnugiHK
7e2800d89a Make necessary changes for adopting the latest iphone_backup_decrypt 2024-06-22 20:02:29 +08:00
Knugi
33763b5f41 Update compile-binary.yml 2024-06-09 07:36:58 +00:00
KnugiHK
f080e2d4ea Update compile-binary.yml 2024-06-08 18:54:53 +08:00
KnugiHK
00f666a3c0 Update python-publish.yml 2024-06-08 18:29:58 +08:00
KnugiHK
2ca064d111 Revert back to 3.10 as 3.11 and 3.12 failed 2024-06-08 18:24:05 +08:00
KnugiHK
3b54ca9d28 Change to use OIDC for publishing to PyPi 2024-06-08 18:20:44 +08:00
KnugiHK
03312da6ee Update Python to 12 for Nuitka 2024-06-08 17:57:48 +08:00
KnugiHK
c7e8a603c7 Undefined vCard File 2024-06-08 17:51:44 +08:00
KnugiHK
574b0393d8 Align vCard UX with iOS 2024-06-08 17:50:48 +08:00
KnugiHK
baa79a7b74 Merge branch 'pr/99' into dev 2024-06-08 17:38:51 +08:00
KnugiHK
d57ff29e71 Add link to vcard entry 2024-06-08 17:29:10 +08:00
jonx
2d4d934a91 Handle groups of VCards correctly 2024-05-03 02:36:17 +02:00
Knugi
9741cab078 Merge pull request #93 from mmmeeedddsss/main
Add support for separating media files per chat
2024-04-26 21:38:37 +08:00
KnugiHK
1e7687f8e8 Update the help text for --create-separated-media 2024-04-21 12:35:09 +08:00
KnugiHK
524b3a4034 Implement separate media for iOS also 2024-04-21 12:33:03 +08:00
KnugiHK
1ab4b24fa0 Fix typo 2024-04-21 12:03:41 +08:00
KnugiHK
8d003b217c Refactor a bit and use chat jid as the final fallback 2024-04-21 12:00:25 +08:00
KnugiHK
d754e6c279 Add Django license 2024-04-21 11:37:05 +08:00
Mert Tunc
0eebbcff21 Add support for separating media files per chat 2024-04-15 19:20:33 +03:00
KnugiHK
a569fb0875 Change exit to argparse error 2024-02-24 16:41:33 +08:00
KnugiHK
6e8e0d7f59 Implement per chat json #86 2024-02-24 16:26:15 +08:00
KnugiHK
c0a511adb3 Refactor imports 2024-02-13 16:01:21 +08:00
KnugiHK
e84640de1c Update description 2024-02-13 15:59:14 +08:00
KnugiHK
20199ed794 Rename files and names 2024-02-13 15:58:29 +08:00
KnugiHK
f4e610a953 Rename variable of the date filter in processes 2024-02-13 15:37:06 +08:00
KnugiHK
99a3a4bcd0 Refactor chat condition 2024-02-13 15:33:58 +08:00
KnugiHK
dedfce8feb Improve help menu 2024-02-13 15:11:53 +08:00
KnugiHK
54e0b43888 Improve redirection 2024-02-13 15:03:51 +08:00
KnugiHK
d5ea843286 Add chat filter 2024-02-13 14:52:15 +08:00
KnugiHK
b01fe0ab4a Bug fix on missing table join 2024-02-13 13:57:22 +08:00
KnugiHK
a7ccc3be66 Merge branch 'main' into dev 2024-02-12 18:08:59 +08:00
KnugiHK
07b1cf6a8a Wrong place 2024-02-12 18:08:45 +08:00
KnugiHK
2b49ac2e41 Add date filter for iOS #82 2024-02-12 18:00:26 +08:00
KnugiHK
2466e2542a Add date filter for Android #82 2024-02-12 17:31:29 +08:00
Knugi
af70f6f6f9 Update CNAME 2024-02-12 09:27:09 +00:00
KnugiHK
48c3fa965f Bug fix on wrong table name for old schema 2023-12-29 18:32:38 +08:00
KnugiHK
472c18448c Update the Python version for standalone binaries to 3.11 2023-12-17 14:53:18 +08:00
KnugiHK
810d8c7c8b Revert "Add macos-13-arm64"
This reverts commit f80be81ee6.
2023-12-17 14:53:09 +08:00
KnugiHK
f80be81ee6 Add macos-13-arm64 2023-12-17 14:43:20 +08:00
KnugiHK
0fcaa946e6 Add --no-deployment-flag=self-execution flag for -m option in binary 2023-12-17 14:40:47 +08:00
KnugiHK
1e7953e5fe Update .gitignore 2023-12-17 14:40:27 +08:00
KnugiHK
481656fdeb Bug fix for the different between key file and hex key 2023-12-17 14:40:08 +08:00
KnugiHK
3d155fb48f Bug fix on wrong timestamp variable 2023-12-17 14:37:59 +08:00
KnugiHK
f659a8c171 Make sure all messages are extracted chronologically #64 2023-12-07 23:17:50 +08:00
KnugiHK
3ffb63ed28 Update readme 2023-12-03 20:41:34 +08:00
KnugiHK
94956913e8 Add time zone offset to display time 2023-12-03 20:35:58 +08:00
KnugiHK
7b5a7419f1 Apply ZCONTACTJID as chat ID in vcard process 2023-12-03 20:21:19 +08:00
KnugiHK
d5cef051d3 Fix an incorrect variable 2023-12-03 20:17:28 +08:00
KnugiHK
f81f31d667 Fix attempt to #64 2023-12-03 14:19:49 +08:00
KnugiHK
8c617b721f Create bruteforce_crypt15.py 2023-12-03 13:49:57 +08:00
KnugiHK
0d626519ec Add support for decrypting contact db from crypt15 backup 2023-12-03 13:40:21 +08:00
Knugi
f39d448aa6 Update issue templates 2023-12-03 03:19:00 +00:00
Knugi
2dc433df7c Update issue templates 2023-12-03 03:17:41 +00:00
Knugi
75a8a2e8c5 Update issue templates 2023-12-03 03:16:52 +00:00
KnugiHK
3847836ed6 Fix: use a more reliable way to determine chat #61
This fix also changed how to determine if the message belongs to a group or PM.
2023-12-03 10:27:22 +08:00
KnugiHK
c27f5ee41c Merge branch 'dev' 2023-11-12 14:27:35 +08:00
KnugiHK
e6c43e7e35 Bump version 2023-11-12 14:27:03 +08:00
KnugiHK
c2fa18778f Merge branch 'dev' 2023-11-12 14:09:04 +08:00
KnugiHK
150180fdff Refactor 2023-10-29 18:12:17 +08:00
KnugiHK
86ea938323 Add fields for implementing read receipt 2023-10-29 18:12:16 +08:00
KnugiHK
7da71e84fe Bug fix on missing sender in group chats #67 2023-10-29 18:12:11 +08:00
KnugiHK
efd5ed80b2 Bug fix on ignoring system chat 2023-10-29 17:56:19 +08:00
KnugiHK
efea1d6165 Remove waste 2023-10-29 17:52:58 +08:00
KnugiHK
3082c83bc4 Convert all links to hyperlinks #69 2023-10-24 00:16:49 +08:00
KnugiHK
fc50415afd An attempt to fix #67 2023-10-24 00:05:15 +08:00
KnugiHK
be4adadbd8 Update __main__.py 2023-10-08 11:42:47 +08:00
KnugiHK
8eb05424fd Support preservation of file timestamp for encrypted backup 2023-10-08 11:42:39 +08:00
KnugiHK
380289d1c1 Bug fix on missing argument #65 2023-10-08 10:12:58 +08:00
KnugiHK
91ff882d15 Add option to preserve modify timestamp #65 2023-10-07 23:50:43 +08:00
Knugi
5aad65fff7 Merge pull request #60 from andrp92/dev
Feat: Support for WhatsApp Business (iPhone only)
2023-09-22 21:33:12 +08:00
KnugiHK
decea88028 Better UI
I am too lazy to switch branch.
2023-08-23 16:07:04 +08:00
KnugiHK
a08f44e6ed Fix the potential collision of Whatsapp and Whatsapp Business 2023-08-23 16:06:25 +08:00
Andres Rodriguez
dbd1802dd6 feat: use dynamic domains when encrypted 2023-08-18 01:51:13 -04:00
Andres Rodriguez
b9f123fbea fix: use corresponding identifier 2023-08-17 19:59:11 -04:00
KnugiHK
2944d00ca2 Refactoring so that no file needs to be introduced 2023-08-15 17:40:20 +08:00
KnugiHK
448ba892cc Change the option from --smb to --business 2023-08-15 17:33:56 +08:00
KnugiHK
a5cb46e095 Also make vcard path dynamic in Android 2023-08-15 16:22:43 +08:00
KnugiHK
ee4e95c75f Bug fix on the possible breakage mentioned in #60 2023-08-15 16:22:08 +08:00
Andres Rodriguez
f488894942 fix(whatsapp_business): missing domain ref 2023-08-15 00:46:54 -04:00
Andres Rodriguez
269a59c1e2 feat(whatsapp_business): made vcard path dynamic 2023-08-15 00:36:15 -04:00
Andres Rodriguez
d8b434e169 feat(whatsapp_business)!: args and default values for smb
Added corresponding logic for whatsapp business and default values.
2023-08-15 00:35:32 -04:00
Andres Rodriguez
326b99d860 feat(whatsapp_business)!: extraction script for smb
Duplicated original extract_iphone_media to include smb specfic fileIDs.
2023-08-15 00:32:07 -04:00
KnugiHK
bd2f063cc0 Bug fix on calls without chat #58 2023-08-12 15:51:05 +08:00
Knugi
736292538b Update the command for macOS #55 2023-07-25 08:53:04 +00:00
KnugiHK
d772efe779 Update the sample output screenshot 2023-06-25 20:01:06 +08:00
KnugiHK
282c99c7dd PyPi can't have git source as dependency 2023-06-25 13:57:11 +08:00
KnugiHK
8dec2a7e97 Bug fix on missing attribute in the Message class. 2023-06-25 13:30:44 +08:00
KnugiHK
f9dedc7930 Increase the row size 2023-06-25 13:14:39 +08:00
KnugiHK
e2f497dbb6 Bug fix on exported chat 2023-06-25 12:42:37 +08:00
KnugiHK
989bddca37 Limit the reply quote length 2023-06-21 17:24:50 +08:00
KnugiHK
40d060628f Coding style 2023-06-21 17:04:58 +08:00
KnugiHK
032af6cdcf Refactor 2023-06-21 17:01:14 +08:00
KnugiHK
e243abe2a4 Bug fix 2023-06-21 16:52:39 +08:00
KnugiHK
b8f0af5f31 Refactor 2023-06-21 16:24:36 +08:00
KnugiHK
030fef53e1 I found that old schema also has such tables 2023-06-20 19:53:05 +08:00
KnugiHK
3ed269e17f Support a lot of metadata in Android's new schema 2023-06-20 19:12:38 +08:00
KnugiHK
1e3ee5e322 Fix incorrect group message sender name 2023-06-20 16:03:24 +08:00
KnugiHK
6c740e69a5 Fix wrongly determined metadata 2023-06-20 15:32:05 +08:00
KnugiHK
828c8a1a72 Fix only one group chat is rendered when contact db is not present 2023-06-20 15:31:24 +08:00
KnugiHK
1fb8588752 Add more alias to wtsexporter command 2023-06-20 14:58:44 +08:00
KnugiHK
6636210e4c Fix the reply jump offset after introducing status 2023-06-20 14:52:32 +08:00
KnugiHK
cc0105647a Refactor a bit 2023-06-20 14:50:57 +08:00
KnugiHK
4fa360a389 No longer support direct execution on the script 2023-06-20 14:41:31 +08:00
KnugiHK
e5228855d2 Temporary workaround mentioned in #48 2023-06-20 14:39:54 +08:00
KnugiHK
db1cdf8189 Add contact's status if present (iOS) 2023-06-20 14:37:25 +08:00
KnugiHK
08ce61e68e Make them optional when importing! 2023-06-20 14:37:12 +08:00
KnugiHK
138dd5351f Add status to JSON import 2023-06-20 14:19:07 +08:00
KnugiHK
136152dc18 Add contact's status if present (Android) 2023-06-20 14:07:42 +08:00
KnugiHK
55bc62cdc1 Add fallback to Android contact name just like commit 5f6b764 2023-06-20 13:49:22 +08:00
KnugiHK
d430c7bfba Update wording on progress 2023-06-19 20:40:10 +08:00
KnugiHK
672b85474e Bug fix on the wrong type of media_wa_type in old Android schema 2023-06-19 20:26:13 +08:00
KnugiHK
525d88f2c6 Fix < Python 3.11 compatibility 2023-06-19 20:09:45 +08:00
KnugiHK
b57087794a Bump version for preparing next release 2023-06-19 17:56:24 +08:00
KnugiHK
be316ebb89 Add aliases to extras_require 2023-06-18 15:15:39 +08:00
KnugiHK
1078f7e5f7 Bug fix on sticker not resizing 2023-06-18 15:10:00 +08:00
KnugiHK
ba2a88067a Update setup.py 2023-06-18 14:36:15 +08:00
KnugiHK
af53ba978b Merge branch 'main' into dev 2023-06-16 19:27:56 +08:00
Knugi
d55a42a549 Merge pull request #36 from nahoj/patch-1
Where to find the key file
2023-06-16 19:27:34 +08:00
KnugiHK
506f8e89f4 Implement import json and output HTML 2023-06-16 19:19:33 +08:00
KnugiHK
fff11b26a5 Store the device type 2023-06-16 19:12:06 +08:00
KnugiHK
fa66ef3a52 Output all attribute to JSON file 2023-06-16 19:11:55 +08:00
Johan Grande
0b93ae567e rephrase as suggested 2023-06-16 12:38:38 +02:00
Johan Grande
c62e07cb0a Merge branch 'main' into patch-1 2023-06-16 12:33:17 +02:00
KnugiHK
317d785d50 Bug fix on raise of exception when "media_folder" does not exists on the filesystem 2023-06-16 18:27:49 +08:00
KnugiHK
38c1e47be9 Add iphone-backup-decrypt to setup.py as extra dependency 2023-06-16 17:57:33 +08:00
KnugiHK
ff95625edf Dropping Python 3.7 and adding Python 3.11 support 2023-06-16 17:50:08 +08:00
KnugiHK
14854193bd Merge branch 'main' into dev 2023-06-16 14:26:34 +08:00
KnugiHK
67c1b43669 Update README.md 2023-06-16 14:07:04 +08:00
KnugiHK
23046e01ba Add links to README 2023-06-16 13:48:36 +08:00
KnugiHK
c366e656af Further reduce the maximum length of vcard file name #51 2023-06-16 01:46:27 +08:00
KnugiHK
41f45fb07c PEP8 2023-06-16 01:43:43 +08:00
KnugiHK
be9e790b12 Better handling of binary message #44 2023-06-16 01:25:51 +08:00
KnugiHK
bfdc68cd6a Add autoescape to rendering 2023-06-16 01:10:24 +08:00
KnugiHK
594a04adbc Update the way to handle encrypted iOS backup file
Since this commit, iphone_backup_decrypt must be re-installed
2023-06-15 21:32:57 +08:00
KnugiHK
20b2eec047 Support Android call logs 2023-06-15 18:25:29 +08:00
KnugiHK
011c8ff1e7 Support missed call (PM) metadata for Android 2023-06-15 17:44:38 +08:00
KnugiHK
e4c47ea41f Update whatsapp.html 2023-06-15 17:01:13 +08:00
KnugiHK
c344f05b05 Bug fix on wrong alias 2023-06-15 17:00:55 +08:00
KnugiHK
88ef4989fc Fix wrong error message 2023-06-15 17:00:34 +08:00
KnugiHK
f7f6b01c86 Resize sticker 2023-06-15 16:59:54 +08:00
KnugiHK
a49a911e03 Replace image rendered in the HTML to thumbnail if possible 2023-06-15 16:19:35 +08:00
KnugiHK
3443143744 Restore code for downloading media from whatsapp server 2023-06-15 02:16:53 +08:00
KnugiHK
5f6b764bb9 Add fallback to iOS contact name
Fallback = the name set by the contact
2023-06-14 21:53:23 +08:00
KnugiHK
3940b2991f Try to reduce the size of the template 2023-06-13 21:23:44 +08:00
KnugiHK
dc1df8a03e Just coding style 2023-06-13 19:48:31 +08:00
KnugiHK
dd5ec2219c Sync changes 2023-06-13 19:46:09 +08:00
KnugiHK
e0c2cf5f66 Implement iOS avatar #48 2023-06-13 19:44:16 +08:00
KnugiHK
8cdb694a16 Modify the root directory name of iOS media 2023-06-13 17:00:40 +08:00
KnugiHK
8294f06587 Extract whole WhatsApp directory instead of wanted files only 2023-06-13 16:29:27 +08:00
KnugiHK
200dea218f Update help message 2023-06-13 14:01:10 +08:00
Knugi
df93033c6c Update README.md 2023-06-13 05:54:14 +00:00
KnugiHK
8f90733da2 Change "Gathering" to "Processing" 2023-06-11 01:35:40 +08:00
KnugiHK
3fdf6d0818 Update LICENSE 2023-06-11 01:33:09 +08:00
KnugiHK
2fa5c4268e Rewrite a bit 2023-06-11 01:22:21 +08:00
KnugiHK
ed658d78dc Fix incorrect media path on iOS #49 2023-06-11 01:21:07 +08:00
KnugiHK
0280325b4a Bug fix 2023-06-11 00:47:11 +08:00
KnugiHK
a42ec5d762 Beautify 2023-06-10 19:58:14 +08:00
KnugiHK
c419dd5d39 Raise error if time is not str and int 2023-06-10 19:54:09 +08:00
KnugiHK
8d036a6d87 Rewrite a bit 2023-06-10 19:45:29 +08:00
KnugiHK
42435c38cc Add <br> to newline 2023-06-10 19:32:48 +08:00
KnugiHK
32caab7c40 Distinguish between media missing and media omitted 2023-06-10 19:32:38 +08:00
KnugiHK
0897dc2897 Implement export TXT chat #22 2023-06-10 19:24:39 +08:00
KnugiHK
f63b180500 Implement splitted outputs #23 2023-06-08 18:16:47 +08:00
KnugiHK
dbdfdaedcf Refine code to use the data model 2023-06-08 17:51:57 +08:00
KnugiHK
0e802f4554 Remove old file 2023-06-08 16:50:33 +08:00
KnugiHK
41dd5e545f Make the not supported note looks less intimidating
#39
2023-06-08 15:46:12 +08:00
KnugiHK
8750315e8e Update __main__.py 2023-06-02 02:41:06 +08:00
KnugiHK
e9499c3bb7 Add highlighting when navigate to replied message 2023-06-02 02:41:01 +08:00
KnugiHK
80c3ed11f6 Partially implement reply feature in iOS
#28
2023-06-02 01:27:05 +08:00
KnugiHK
7c78cce221 Update link for reporting offsets 2023-06-01 22:26:32 +08:00
KnugiHK
32a312d332 Add offset mentioned in #46 2023-06-01 22:21:58 +08:00
KnugiHK
328f34e632 Merge branch 'main' into dev 2023-05-19 13:25:21 +08:00
KnugiHK
9ac8839ecc Workaround for non-UTF8 message
#44
2023-05-19 13:23:51 +08:00
Knugi
e7113d72d7 Update README.md 2023-05-18 09:34:59 +00:00
KnugiHK
0a0ae8cf15 Update python-publish.yml 2023-05-18 16:37:35 +08:00
KnugiHK
b1d8d173a2 Bug fix on too long vCard file name 2023-05-17 01:06:31 +08:00
KnugiHK
3bd6f288ea Bug fix file exist exception 2023-05-17 00:48:33 +08:00
KnugiHK
bf06795962 Add checksum on compiled binaries 2023-05-17 00:19:45 +08:00
KnugiHK
20d8e1384a Bump version 2023-05-17 00:08:52 +08:00
KnugiHK
6fd0e61b64 Update help test 2023-05-16 23:29:10 +08:00
KnugiHK
bbb47cd839 Bug fix 2023-05-16 23:25:12 +08:00
Knugi
c155064ae1 Update README.md 2023-05-16 13:38:16 +00:00
Knugi
d4efd919f9 Update python-publish.yml 2023-05-16 11:49:57 +00:00
Knugi
13d761286e Update compile-binary.yml 2023-05-16 11:48:25 +00:00
Knugi
a943808734 Update compile-binary.yml 2023-05-16 11:48:00 +00:00
KnugiHK
0495970c38 Bump version 2023-05-16 19:33:29 +08:00
KnugiHK
32b14dc392 Update README.md 2023-05-16 19:31:45 +08:00
KnugiHK
6bea8d07f4 Update setup.py 2023-05-05 14:03:40 +08:00
KnugiHK
69fdb61bae Merge branch 'dev' 2023-05-05 13:55:41 +08:00
KnugiHK
e1f160fc7c Update setup.py 2023-05-05 13:53:11 +08:00
KnugiHK
3b7e02ba31 Add update checking 2023-05-05 13:53:00 +08:00
KnugiHK
bdb7d80831 Merge branch 'dev' 2023-05-05 10:42:48 +08:00
Knugi
ea7e019adc Update README.md
Make the command compatible with both Linux and Windows
2023-04-25 05:41:27 +00:00
KnugiHK
c7a01bb9c0 Handle deleted message in new schema
Related to #39 and #9
2023-04-25 13:02:20 +08:00
KnugiHK
7c0b90d458 Another attempt to fix the previous bug 2023-04-09 02:38:26 +08:00
KnugiHK
84383e1d9d Revert "Bug fix on empty vcf contact name"
This reverts commit 06a1d34567.
2023-04-09 02:37:04 +08:00
KnugiHK
06a1d34567 Bug fix on empty vcf contact name 2023-04-09 02:23:50 +08:00
KnugiHK
b371587d65 Bug fix on ChatStore initialization 2023-04-09 02:05:17 +08:00
KnugiHK
3e7d7916a7 PEP8 2023-03-28 12:23:46 +08:00
Knugi
17997e840f Update README.md 2023-03-28 04:21:09 +00:00
KnugiHK
640acb3f86 Move some utility functions to a separated python file 2023-03-25 18:33:22 +08:00
KnugiHK
cdfaf69f7a Add an option to disable html output 2023-03-25 18:26:03 +08:00
KnugiHK
ee5f8b82be Add suggestion to CryptX 2023-03-25 12:11:00 +08:00
KnugiHK
b9fa36acb4 Raise error if the error is not expected 2023-03-25 11:55:18 +08:00
KnugiHK
fb88124d21 Merge old and new schema processing logic 2023-03-25 11:51:27 +08:00
KnugiHK
b5effbd512 Change autobuffer to preload for video and audio tag 2023-03-25 11:46:52 +08:00
KnugiHK
430a5eccb8 Merge branch 'pr/26' into dev 2023-03-25 10:37:47 +08:00
GoComputing
8f0511a6e2 Re-enabled the HTML generation 2023-03-24 19:43:27 +01:00
Johan Grande
8380487e44 Where to find the key file 2023-03-17 12:04:40 +01:00
Knugi
45666d8878 Add Python 3.10 to classifiers 2023-02-26 07:43:36 +00:00
KnugiHK
2ba55719f1 Add #32 to common offset 2023-02-26 15:31:03 +08:00
KnugiHK
2d23052758 Improve argument parser 2023-02-13 16:31:35 +08:00
KnugiHK
10875060c9 Deprecate --iphone and replace with --ios 2023-02-13 16:15:54 +08:00
KnugiHK
0bb99d59e0 Prepare for size control of output file 2023-02-13 16:08:21 +08:00
KnugiHK
9178e5326b Transit from optparse to argparse 2023-02-13 12:53:20 +08:00
KnugiHK
26320413e8 Add offline availability of w3css 2023-02-13 12:23:43 +08:00
KnugiHK
a275a0f40c Why is this line not in last commit... 2023-02-13 00:27:55 +08:00
KnugiHK
4cb4ac3e7b Bug fix for sender name in group chat
#9
2023-02-13 00:25:31 +08:00
Knugi
4139cab00f Fix binary 2023-02-12 11:51:43 +00:00
Knugi
c1964bc2cd Prepare for standalone binary
https://github.com/KnugiHK/Whatsapp-Chat-Exporter/issues/29
2023-02-12 18:11:38 +08:00
KnugiHK
dab0493354 Revert "Prepare for standalone binary"
This reverts commit 4d6c80b561.
2023-02-12 18:11:00 +08:00
Knugi
e0c464c8d8 Update compile-binary.yml 2023-02-12 18:01:29 +08:00
KnugiHK
92d339d1c0 Implement standalone binary compilation 2023-02-12 18:01:29 +08:00
Knugi
4d6c80b561 Prepare for standalone binary
https://github.com/KnugiHK/Whatsapp-Chat-Exporter/issues/29
2023-02-12 18:01:29 +08:00
Knugi
d46a42a097 Update compile-binary.yml 2023-02-12 09:56:55 +00:00
KnugiHK
7cd259143a Implement standalone binary compilation 2023-02-12 17:54:32 +08:00
Knugi
726812a5f7 Prepare for standalone binary
https://github.com/KnugiHK/Whatsapp-Chat-Exporter/issues/29
2023-02-12 08:15:40 +00:00
KnugiHK
6fddc1c23a Merge branch 'main' into dev 2023-01-31 17:56:53 +08:00
KnugiHK
77ceaa25dd Bump version 2023-01-31 17:52:34 +08:00
KnugiHK
e09f18e2f2 Minor fix 2023-01-31 17:52:12 +08:00
KnugiHK
23114572bd Forgot to change the variable lol 2023-01-31 17:52:12 +08:00
KnugiHK
2f04b69f38 A more concrete way to determine database offset 2023-01-31 17:52:12 +08:00
KnugiHK
e7c246822b Link to the file intead of showing the path directly
Not tested
Ref: https://github.com/KnugiHK/Whatsapp-Chat-Exporter/issues/15
2023-01-31 17:52:12 +08:00
KnugiHK
2a215d024f Bug fix
Duplicated folder creation
https://github.com/KnugiHK/Whatsapp-Chat-Exporter/issues/14
2023-01-31 17:52:12 +08:00
KnugiHK
f267f53007 Remove unused dependencies 2023-01-31 17:52:12 +08:00
KnugiHK
3a30dfc800 Bump version 2023-01-31 17:52:12 +08:00
KnugiHK
9600da59ae Correct the default in #25 2023-01-31 17:25:59 +08:00
KnugiHK
26b58843fb Add message 2023-01-31 16:46:22 +08:00
KnugiHK
60575c7989 Implement #25
Copying media folder to the output directory will be the default starting from this commit.
2023-01-31 16:34:34 +08:00
KnugiHK
14b1cb7fde Minor fix 2023-01-30 18:53:01 +08:00
GoComputing
92b8903521 Fixed JSON export
Added serialization to the classes 'ChatStore' and 'Messages' so that they can be
JSON serialized.
2023-01-28 20:51:40 +01:00
KnugiHK
d3892a4e4f Fix caption part 2022-12-23 17:28:23 +08:00
KnugiHK
b37c13434e Merge branch 'dev' into message_table 2022-12-23 16:49:37 +08:00
KnugiHK
4b357d5ea9 Update the import of Crypt to latest one for message 2022-12-23 16:49:28 +08:00
KnugiHK
6407ba2136 Adopt the latest version 2022-12-21 21:45:20 +08:00
KnugiHK
f87108dadc Some left-over 2022-12-21 21:42:54 +08:00
KnugiHK
6ca7e81484 Support new WhatsApp database schema
https://github.com/KnugiHK/Whatsapp-Chat-Exporter/issues/9
2022-12-21 21:28:54 +08:00
KnugiHK
41d3659269 Prepare for porting 2022-12-21 20:16:37 +08:00
Knugi
580eaddb24 Delete old_README.md 2022-10-24 03:59:42 +00:00
Knugi
77b4b784d3 Update README.md 2022-10-24 03:59:24 +00:00
KnugiHK
d9a77e0eec Forgot to change the variable lol 2022-09-05 12:49:12 +08:00
KnugiHK
876729eb81 A more concrete way to determine database offset 2022-09-05 12:48:36 +08:00
KnugiHK
48f667d02b Implement exporting 64-digit crypt15 encryption key
https://github.com/KnugiHK/Whatsapp-Chat-Exporter/issues/20
2022-09-05 12:16:07 +08:00
KnugiHK
422ab2f784 Link to the file intead of showing the path directly
Not tested
Ref: https://github.com/KnugiHK/Whatsapp-Chat-Exporter/issues/15
2022-07-03 12:30:01 +08:00
KnugiHK
996ee65525 Bug fix
Duplicated folder creation
https://github.com/KnugiHK/Whatsapp-Chat-Exporter/issues/14
2022-05-25 18:28:07 +08:00
KnugiHK
042f6f9024 Remove unused dependencies 2022-05-23 20:12:50 +08:00
KnugiHK
507e88d9c3 Merge branch 'dev' 2022-05-09 18:31:03 +08:00
KnugiHK
60e1e7d3eb Bump version 2022-05-09 18:28:13 +08:00
Knugi
774fb6d781 Merge pull request #11 from asla9709/dev
Fixed bug where blank VCard media_name would crash the program.
2022-05-09 10:03:04 +00:00
Aakif Aslam
3ef3b02230 Fixed bug where blank VCard media_name would crash the program. 2022-04-24 18:01:32 -04:00
Knugi
07cc0f3571 Update README.md 2022-04-04 07:57:12 +00:00
KnugiHK
a1319eb835 Exclude default conversation from results 2022-04-01 17:17:43 +08:00
KnugiHK
8cbb0af43a Oh. I missed this change 2022-03-04 14:06:19 +08:00
KnugiHK
28c4a7b99f Prepare for new function 2022-03-04 14:03:15 +08:00
KnugiHK
e4c9d42927 Update __init__.py 2022-03-04 13:48:12 +08:00
KnugiHK
c274b6b1c0 Merge branch 'dev' 2022-03-04 13:47:54 +08:00
KnugiHK
eec739d7cf Add crypt15 dependency to android_backup 2022-03-04 13:47:17 +08:00
Knugi
3d7dca0682 Delete _config.yml 2022-02-25 08:57:31 +00:00
Knugi
24f7837171 Create python-publish.yml 2022-02-22 14:01:36 +00:00
KnugiHK
15201acbe6 Bump version 2022-02-22 21:53:13 +08:00
Knugi
6fd290efd8 typo 2022-02-22 13:52:06 +00:00
Knugi
691bfe31c8 Update README.md 2022-02-22 13:51:00 +00:00
KnugiHK
64eb2bcb9d Add some aliases 2022-02-22 21:42:58 +08:00
KnugiHK
1bc4a8c5b9 Create android_structure_backup_crypt15.png 2022-02-22 21:42:50 +08:00
Knugi
8a621827ff Update README.md 2022-02-22 13:42:16 +00:00
KnugiHK
227f438404 Add one more offset 2022-02-22 21:22:07 +08:00
KnugiHK
3e71817778 Make the brute-force more sensitive and bug fix 2022-02-22 21:19:19 +08:00
KnugiHK
08c5979eed Support Hex key for Crypt15 2022-02-22 18:59:04 +08:00
KnugiHK
0e6319eb4e Support crypt15 2022-02-22 18:33:54 +08:00
KnugiHK
734bb78cd8 Implement offset brute forcing
Not tested yet
2022-02-22 02:08:44 +08:00
KnugiHK
a522eb2034 PEP8 2022-01-17 13:01:10 +08:00
KnugiHK
9fe6a0d2a8 Refactoring 2022-01-17 12:06:15 +08:00
KnugiHK
c73eabe2a4 Clearer error message in decompressing decrypted backup 2022-01-17 10:46:02 +08:00
KnugiHK
1faf111e64 Handle case that the database file does not exist and clearer exit code 2021-12-30 11:59:30 +08:00
KnugiHK
9140c07feb Prevent PermissionError from raising 2021-12-28 20:04:40 +08:00
42 changed files with 6749 additions and 1229 deletions

37
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@@ -0,0 +1,37 @@
---
name: Bug report
about: Create a report to help us improve
title: "[BUG]"
labels: ''
assignees: ''
---
# Must have
- WhatsApp version: [WhatsApp version]
- OS: [Android/iOS] - [version]
- Platform: [Linux/Windows/MacOS]
- Exporter's branch and version: [main/dev] - [exporter version]
**Describe the bug**
A clear and concise description of what the bug is.
If it is an error yield by Python, please also provide the trackback
```
[trackback here]
```
# Nice to have
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Additional context**
Add any other context about the problem here.

View File

@@ -0,0 +1,17 @@
---
name: Feature request
about: Suggest an idea for this project
title: "[FEATURE]"
labels: ''
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Additional context**
Add any other context or screenshots about the feature request here.

20
.github/docs.html vendored Normal file
View File

@@ -0,0 +1,20 @@
<!DOCTYPE html>
<html>
<head>
<script type="text/javascript">
destination = {
"filter": "Filter",
"date": "Filters#date-filters",
"chat": "Filters#chat-filter",
"osl": "Open-Source-Licenses",
"iose2e": "iOS-Usage#encrypted-iosipados-backup",
null: ""
};
const dest = new URLSearchParams(window.location.search).get('dest');
window.location.href = `https://github.com/KnugiHK/WhatsApp-Chat-Exporter/wiki/${destination[dest]}`;
</script>
</head>
<body>
<p>If the redirection doesn't work, you can find the documentation at <a href="https://github.com/KnugiHK/WhatsApp-Chat-Exporter/wiki">https://github.com/KnugiHK/WhatsApp-Chat-Exporter/wiki</a>.</p>
</body>
</html>

489
.github/generate-website.js vendored Normal file
View File

@@ -0,0 +1,489 @@
const fs = require('fs-extra');
const marked = require('marked');
const path = require('path');
const markedAlert = require('marked-alert');
fs.ensureDirSync('docs');
fs.ensureDirSync('docs/imgs');
if (fs.existsSync('imgs')) {
fs.copySync('imgs', 'docs/imgs');
}
if (fs.existsSync('.github/docs.html')) {
fs.copySync('.github/docs.html', 'docs/docs.html');
}
const readmeContent = fs.readFileSync('README.md', 'utf8');
const toc = `<div class="table-of-contents">
<h3>Table of Contents</h3>
<ul>
<li><a href="#intro">Introduction</a></li>
<li><a href="#usage">Usage</a></li>
<li><a href="#todo">To Do</a></li>
<li><a href="#legal">Legal Stuff & Disclaimer</a></li>
</ul>
</div>
`
const generateHTML = (content) =>
`<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="title" content="WhatsApp Chat Exporter">
<meta name="description" content="Export your WhatsApp conversations from Android and iOS/iPadOS devices to HTML, JSON, or text formats. Supports encrypted backups (Crypt12, Crypt14, Crypt15) and customizable templates.">
<meta name="keywords" content="WhatsApp, WhatsApp Chat Exporter, WhatsApp export tool, WhatsApp backup decryption, Crypt12, Crypt14, Crypt15, WhatsApp database parser, WhatsApp chat history, HTML export, JSON export, text export, customizable templates, media handling, vCard import, Python tool, open source, MIT license">
<meta name="robots" content="index, follow">
<meta name="author" content="KnugiHK">
<meta name="license" content="MIT">
<meta name="generator" content="Python">
<title>WhatsApp Chat Exporter</title>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.4.0/css/all.min.css">
<style>
:root {
--primary-color: #128C7E;
--secondary-color: #25D366;
--dark-color: #075E54;
--light-color: #DCF8C6;
--text-color: #333;
--light-text: #777;
--code-bg: #f6f8fa;
--border-color: #e1e4e8;
}
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
line-height: 1.6;
color: var(--text-color);
background-color: #f9f9f9;
}
.container {
max-width: 1200px;
margin: 0 auto;
padding: 0 20px;
}
header {
background-color: var(--primary-color);
color: white;
padding: 60px 0 40px;
text-align: center;
box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1);
}
header h1 {
font-size: 2.8rem;
margin-bottom: 16px;
}
.badges {
margin: 20px 0;
display: flex;
justify-content: center;
flex-wrap: wrap;
gap: 10px;
}
.badge {
display: inline-block;
margin: 5px;
}
.tagline {
font-size: 1.2rem;
max-width: 800px;
margin: 0 auto;
padding: 0 20px;
}
.main-content {
background: white;
padding: 40px 0;
margin: 0;
}
.inner-content {
padding: 0 30px;
max-width: 900px;
margin: 0 auto;
}
h2 {
color: var(--dark-color);
margin: 30px 0 15px;
padding-bottom: 8px;
border-bottom: 2px solid var(--light-color);
font-size: 1.8rem;
}
h3 {
color: var(--dark-color);
margin: 25px 0 15px;
font-size: 1.4rem;
}
h4 {
color: var(--dark-color);
margin: 20px 0 10px;
font-size: 1.2rem;
}
p, ul, ol {
margin-bottom: 16px;
}
ul, ol {
padding-left: 25px;
}
a {
color: var(--primary-color);
text-decoration: none;
}
a:hover {
text-decoration: underline;
}
.alert {
background-color: #f8f9fa;
border-left: 4px solid #f0ad4e;
padding: 15px;
margin-bottom: 20px;
border-radius: 3px;
}
.alert--tip {
border-color: var(--secondary-color);
background-color: rgba(37, 211, 102, 0.1);
}
.alert--note {
border-color: #0088cc;
background-color: rgba(0, 136, 204, 0.1);
}
.markdown-alert {
background-color: #f8f9fa;
border-left: 4px solid #f0ad4e;
padding: 15px;
margin-bottom: 20px;
border-radius: 3px;
}
.markdown-alert-note {
border-color: #0088cc;
background-color: rgba(0, 136, 204, 0.1);
}
.markdown-alert-tip {
border-color: var(--secondary-color);
background-color: rgba(37, 211, 102, 0.1);
}
.markdown-alert-important {
border-color: #d9534f;
background-color: rgba(217, 83, 79, 0.1);
}
.markdown-alert-warning {
border-color: #f0ad4e;
background-color: rgba(240, 173, 78, 0.1);
}
.markdown-alert-caution {
border-color: #ff9800;
background-color: rgba(255, 152, 0, 0.1);
}
.markdown-alert p {
margin: 0;
}
.markdown-alert-title {
font-weight: 600;
margin-bottom: 8px;
display: flex;
align-items: center;
gap: 8px;
}
pre {
background-color: var(--code-bg);
border-radius: 6px;
padding: 16px;
overflow-x: auto;
margin: 16px 0;
border: 1px solid var(--border-color);
}
code {
font-family: SFMono-Regular, Consolas, Liberation Mono, Menlo, monospace;
font-size: 85%;
background-color: var(--code-bg);
padding: 0.2em 0.4em;
border-radius: 3px;
}
pre code {
padding: 0;
background-color: transparent;
}
.screenshot {
max-width: 100%;
border-radius: 8px;
box-shadow: 0 4px 8px rgba(0, 0, 0, 0.1);
margin: 20px 0;
border: 1px solid var(--border-color);
}
.feature-grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(280px, 1fr));
gap: 20px;
margin: 30px 0;
}
.feature-card {
background: white;
border-radius: 8px;
box-shadow: 0 2px 5px rgba(0, 0, 0, 0.1);
padding: 20px;
border: 1px solid var(--border-color);
transition: transform 0.3s ease;
}
.feature-card:hover {
transform: translateY(-5px);
box-shadow: 0 5px 15px rgba(0, 0, 0, 0.1);
}
.feature-icon {
font-size: 2rem;
color: var(--primary-color);
margin-bottom: 15px;
}
.feature-title {
font-weight: 600;
margin-bottom: 10px;
}
footer {
background-color: var(--dark-color);
color: white;
text-align: center;
padding: 30px 0;
margin-top: 50px;
}
.btn {
display: inline-block;
background-color: var(--primary-color);
color: white;
padding: 10px 20px;
border-radius: 4px;
text-decoration: none;
font-weight: 500;
transition: background-color 0.3s ease;
margin: 5px;
}
.btn:hover {
background-color: var(--dark-color);
text-decoration: none;
}
.btn-secondary {
background-color: white;
color: var(--primary-color);
border: 1px solid var(--primary-color);
}
.btn-secondary:hover {
background-color: var(--light-color);
color: var(--dark-color);
}
.action-buttons {
margin: 30px 0;
text-align: center;
}
.table-of-contents {
background-color: #f8f9fa;
border: 1px solid var(--border-color);
border-radius: 6px;
padding: 15px 25px;
margin: 30px 0;
}
.table-of-contents h3 {
margin-top: 0;
margin-bottom: 10px;
}
.table-of-contents ul {
margin-bottom: 0;
}
.help-text {
color: var(--light-text);
font-size: 0.9rem;
}
.device-section {
padding: 15px;
border: 1px solid var(--border-color);
border-radius: 6px;
margin-bottom: 20px;
background-color: #fff;
}
@media (max-width: 768px) {
header {
padding: 40px 0 30px;
}
header h1 {
font-size: 2.2rem;
}
.tagline {
font-size: 1.1rem;
}
.feature-grid {
grid-template-columns: 1fr;
}
}
</style>
</head>
<body>
<header>
<div class="container">
<h1>WhatsApp Chat Exporter</h1>
<div class="badges">
<a href="https://pypi.org/project/whatsapp-chat-exporter/" class="badge"><img src="https://img.shields.io/pypi/v/whatsapp-chat-exporter?label=Latest%20in%20PyPI" alt="Latest in PyPI"></a>
<a href="https://github.com/KnugiHK/WhatsApp-Chat-Exporter/blob/main/LICENSE" class="badge"><img src="https://img.shields.io/pypi/l/whatsapp-chat-exporter?color=427B93" alt="License MIT"></a>
<a href="https://pypi.org/project/Whatsapp-Chat-Exporter/" class="badge"><img src="https://img.shields.io/pypi/pyversions/Whatsapp-Chat-Exporter" alt="Python"></a>
<a href="https://matrix.to/#/#wtsexporter:matrix.org" class="badge"><img src="https://img.shields.io/matrix/wtsexporter:matrix.org.svg?label=Matrix%20Chat%20Room" alt="Matrix Chat Room"></a>
</div>
<p class="tagline">A customizable Android and iPhone Whatsapp database parser that will give you the history of your Whatsapp conversations in HTML and JSON</p>
<div class="action-buttons">
<a href="https://github.com/KnugiHK/WhatsApp-Chat-Exporter" class="btn"><i class="fab fa-github"></i> GitHub</a>
<a href="https://pypi.org/project/whatsapp-chat-exporter/" class="btn btn-secondary"><i class="fab fa-python"></i> PyPI</a>
</div>
</div>
</header>
<div class="main-content">
<div class="inner-content">
<section id="features">
<h2>Key Features</h2>
<div class="feature-grid">
<div class="feature-card">
<div class="feature-icon"><i class="fas fa-mobile-alt"></i></div>
<h3 class="feature-title">Cross-Platform</h3>
<p>Support for both Android and iOS/iPadOS WhatsApp databases</p>
</div>
<div class="feature-card">
<div class="feature-icon"><i class="fas fa-lock"></i></div>
<h3 class="feature-title">Backup Decryption</h3>
<p>Support for Crypt12, Crypt14, and Crypt15 (End-to-End) encrypted backups</p>
</div>
<div class="feature-card">
<div class="feature-icon"><i class="fas fa-file-export"></i></div>
<h3 class="feature-title">Multiple Formats</h3>
<p>Export your chats in HTML, JSON, and text formats</p>
</div>
<div class="feature-card">
<div class="feature-icon"><i class="fas fa-paint-brush"></i></div>
<h3 class="feature-title">Customizable</h3>
<p>Use custom HTML templates and styling for your chat exports</p>
</div>
<div class="feature-card">
<div class="feature-icon"><i class="fas fa-images"></i></div>
<h3 class="feature-title">Media Support</h3>
<p>Properly handles and organizes your media files in the exports</p>
</div>
<div class="feature-card">
<div class="feature-icon"><i class="fas fa-filter"></i></div>
<h3 class="feature-title">Filtering Options</h3>
<p>Filter chats by date, phone number, and more</p>
</div>
</div>
</section>
<div class="readme-content">
${content}
</div>
<div class="action-buttons">
<a href="https://github.com/KnugiHK/WhatsApp-Chat-Exporter" class="btn"><i class="fab fa-github"></i> View on GitHub</a>
<a href="https://pypi.org/project/whatsapp-chat-exporter/" class="btn btn-secondary"><i class="fab fa-python"></i> PyPI Package</a>
</div>
</div>
</div>
<footer>
<div class="container">
<p>© 2021-${new Date().getFullYear()} WhatsApp Chat Exporter</p>
<p>Licensed under MIT License</p>
<p>
<a href="https://github.com/KnugiHK/WhatsApp-Chat-Exporter" style="color: white; margin: 0 10px;"><i class="fab fa-github fa-lg"></i></a>
<a href="https://matrix.to/#/#wtsexporter:matrix.org" style="color: white; margin: 0 10px;"><i class="fas fa-comments fa-lg"></i></a>
</p>
<p><small>Last updated: ${new Date().toLocaleDateString()}</small></p>
</div>
</footer>
<script>
// Simple script to handle smooth scrolling for anchor links
document.querySelectorAll('a[href^="#"]').forEach(anchor => {
anchor.addEventListener('click', function(e) {
e.preventDefault();
const targetId = this.getAttribute('href');
const targetElement = document.querySelector(targetId);
if (targetElement) {
window.scrollTo({
top: targetElement.offsetTop - 20,
behavior: 'smooth'
});
}
});
});
</script>
</body>
</html>
`;
const processedContent = readmeContent.replace(/\[!\[.*?\]\(.*?\)\]\(.*?\)/g, '')
const htmlContent = marked.use(markedAlert()).parse(processedContent, {
gfm: true,
breaks: true,
renderer: new marked.Renderer()
});
const finalHTML = generateHTML(htmlContent);
fs.writeFileSync('docs/index.html', finalHTML);
console.log('Website generated successfully!');

11
.github/pull_request_template.md vendored Normal file
View File

@@ -0,0 +1,11 @@
# Important Note
**All PRs (except for changes unrelated to source files) should target and start from the `dev` branch.**
## Related Issue
- Please put a reference to the related issue here (e.g., `Fixes #123` or `Closes #456`), if there are any.
## Description of Changes
- Briefly describe the changes made in this PR. Explain the purpose, the implementation details, and any important information that reviewers should be aware of.

100
.github/workflows/codeql.yml vendored Normal file
View File

@@ -0,0 +1,100 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL Advanced"
on:
push:
branches: [ "main", "dev" ]
pull_request:
branches: [ "main", "dev" ]
schedule:
- cron: '25 21 * * 5'
jobs:
analyze:
name: Analyze (${{ matrix.language }})
# Runner size impacts CodeQL analysis time. To learn more, please see:
# - https://gh.io/recommended-hardware-resources-for-running-codeql
# - https://gh.io/supported-runners-and-hardware-resources
# - https://gh.io/using-larger-runners (GitHub.com only)
# Consider using larger runners or machines with greater resources for possible analysis time improvements.
runs-on: ${{ (matrix.language == 'swift' && 'macos-latest') || 'ubuntu-latest' }}
permissions:
# required for all workflows
security-events: write
# required to fetch internal or private CodeQL packs
packages: read
# only required for workflows in private repositories
actions: read
contents: read
strategy:
fail-fast: false
matrix:
include:
- language: actions
build-mode: none
- language: python
build-mode: none
# CodeQL supports the following values keywords for 'language': 'actions', 'c-cpp', 'csharp', 'go', 'java-kotlin', 'javascript-typescript', 'python', 'ruby', 'swift'
# Use `c-cpp` to analyze code written in C, C++ or both
# Use 'java-kotlin' to analyze code written in Java, Kotlin or both
# Use 'javascript-typescript' to analyze code written in JavaScript, TypeScript or both
# To learn more about changing the languages that are analyzed or customizing the build mode for your analysis,
# see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning.
# If you are analyzing a compiled language, you can modify the 'build-mode' for that language to customize how
# your codebase is analyzed, see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages
steps:
- name: Checkout repository
uses: actions/checkout@v4
# Add any setup steps before running the `github/codeql-action/init` action.
# This includes steps like installing compilers or runtimes (`actions/setup-node`
# or others). This is typically only required for manual builds.
# - name: Setup runtime (example)
# uses: actions/setup-example@v1
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# For more details on CodeQL's query packs, refer to: https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
# queries: security-extended,security-and-quality
# If the analyze step fails for one of the languages you are analyzing with
# "We were unable to automatically build your code", modify the matrix above
# to set the build mode to "manual" for that language. Then modify this step
# to build your code.
# Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
- if: matrix.build-mode == 'manual'
shell: bash
run: |
echo 'If you are using a "manual" build mode for one or more of the' \
'languages you are analyzing, replace this with the commands to build' \
'your code, for example:'
echo ' make bootstrap'
echo ' make release'
exit 1
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
with:
category: "/language:${{matrix.language}}"

84
.github/workflows/compile-binary.yml vendored Normal file
View File

@@ -0,0 +1,84 @@
name: Compile standalone binary
on:
release:
types: [published]
workflow_dispatch:
permissions:
contents: read
jobs:
linux:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.13'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pycryptodome vobject javaobj-py3 ordered-set zstandard nuitka==2.8.9
pip install .
- name: Build binary with Nuitka
run: |
python -m nuitka --onefile \
--include-data-file=./Whatsapp_Chat_Exporter/whatsapp.html=./Whatsapp_Chat_Exporter/whatsapp.html \
--assume-yes-for-downloads Whatsapp_Chat_Exporter --output-filename=wtsexporter_linux_x64
sha256sum wtsexporter_linux_x64
- uses: actions/upload-artifact@v6
with:
name: binary-linux
path: |
./wtsexporter_linux_x64
windows:
runs-on: windows-latest
steps:
- uses: actions/checkout@v6
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.13'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pycryptodome vobject javaobj-py3 ordered-set zstandard nuitka==2.8.9
pip install .
- name: Build binary with Nuitka
run: |
python -m nuitka --onefile --include-data-file=./Whatsapp_Chat_Exporter/whatsapp.html=./Whatsapp_Chat_Exporter/whatsapp.html --assume-yes-for-downloads Whatsapp_Chat_Exporter --output-filename=wtsexporter
copy wtsexporter.exe wtsexporter_x64.exe
Get-FileHash wtsexporter_x64.exe
- uses: actions/upload-artifact@v6
with:
name: binary-windows
path: |
.\wtsexporter_x64.exe
macos:
runs-on: macos-latest
steps:
- uses: actions/checkout@v6
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.13'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pycryptodome vobject javaobj-py3 ordered-set zstandard nuitka==2.8.9
pip install .
- name: Build binary with Nuitka
run: |
python -m nuitka --onefile \
--include-data-file=./Whatsapp_Chat_Exporter/whatsapp.html=./Whatsapp_Chat_Exporter/whatsapp.html \
--assume-yes-for-downloads Whatsapp_Chat_Exporter --output-filename=wtsexporter_macos_x64
shasum -a 256 wtsexporter_macos_x64
- uses: actions/upload-artifact@v6
with:
name: binary-macos
path: |
./wtsexporter_macos_x64

43
.github/workflows/generate-website.yml vendored Normal file
View File

@@ -0,0 +1,43 @@
name: Generate Website from README
on:
push:
branches:
- main
paths:
- 'README.md'
- '.github/workflows/generate-website.yml'
- '.github/generate-website.js'
- '.github/docs.html'
workflow_dispatch:
permissions:
contents: write
pages: write
jobs:
build-and-deploy:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '22'
- name: Install dependencies
run: npm install marked fs-extra marked-alert
- name: Generate website from README
run: |
node .github/generate-website.js
echo 'wts.knugi.dev' > ./docs/CNAME
- name: Deploy to gh-pages
if: github.ref == 'refs/heads/main' # Ensure deployment only happens from main
uses: peaceiris/actions-gh-pages@v4
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./docs
publish_branch: gh-pages

37
.github/workflows/python-publish.yml vendored Normal file
View File

@@ -0,0 +1,37 @@
# This workflow will upload a Python Package using Twine when a release is created
# For more information see: https://help.github.com/en/actions/language-and-framework-guides/using-python-with-github-actions#publishing-to-package-registries
# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.
name: Upload Python Package
on:
release:
types: [published]
workflow_dispatch:
permissions:
id-token: write
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.x'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build
- name: Build package
run: python -m build
- name: Publish package
uses: pypa/gh-action-pypi-publish@release/v1

18
.gitignore vendored
View File

@@ -127,3 +127,21 @@ dmypy.json
# Pyre type checker
.pyre/
# Nuitka
*.build/
*.dist/
*.onefile-build/
*.exe
__main__
# Dev time intermidiates & temp files
result/
WhatsApp/
/*.db
/*.db-*
/myout
/msgstore.db
/myout-json
.vscode/

1
CNAME
View File

@@ -1 +0,0 @@
wts.knugi.com

63
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,63 @@
# Contributing Guidelines
*Pull requests, bug reports, and all other forms of contribution are welcomed and highly encouraged!*
> **This guide serves to set clear expectations for everyone involved with the project so that we can improve it together while also creating a welcoming space for everyone to participate. Following these guidelines will help ensure a positive experience for contributors and maintainers.**
<sub> Maintainer's note: I aim to keep things simple and flexible, without imposing too many restrictions, while still ensuring its useful for the project. </sub>
## :book: Code of Conduct
There isn't an official code of conduct at the moment, and we hope it won't be necessary. The rule is simple: be reasonable and treat others with respect!
## :bulb: Asking Questions
While there is no formal support from the maintainer, they are happy to help if you provide enough information. However, please note:
If you feel the questions or difficulties you're encountering aren't related to the software itself, please [open a discussion thread](https://github.com/KnugiHK/WhatsApp-Chat-Exporter/discussions/new/choose). Do not open an issue just to ask a question. While asking questions in the project issues is not strictly prohibited, any issues that don't qualify as genuine problems will be converted into discussion threads.
Hopefully, the community will be able to offer assistance as well. You can check out the article [How do I ask a good question?](https://stackoverflow.com/help/how-to-ask) on StackOverflow to learn how to craft questions that encourage more people to respond.
## :inbox_tray: Opening an Issue
Before [creating an issue](https://help.github.com/en/github/managing-your-work-on-github/creating-an-issue), check if you are using the latest version of the project. If you are not up-to-date, see if updating fixes your issue first.
### :lock: Reporting Security Issues
Please report any vulnerability to [GitHub Security Advisory](https://github.com/KnugiHK/WhatsApp-Chat-Exporter/security/advisories/new). **Do not** file a public issue for security vulnerabilities.
### :beetle: Bug Reports and Feature Requests
- **Do not open a duplicate issue!** Search through existing issues to see if your issue or request has previously been reported. If your issue exists, comment with any additional information you have. You may simply note "I have this problem too/I want this feature too", which helps prioritize the most common problems and requests.
- **Fully complete the provided issue template.** The issue templates request all the information we need to quickly and efficiently address your issue. Be clear, concise, and descriptive. Provide as much information as you can, including steps to reproduce, stack traces, compiler errors, library versions, OS versions, and screenshots (if applicable). This will assist the maintainer in efficiently triaging your issues and isolating the problems.
- For feature requests, be specific about the proposed outcome and how it fits with the existing features. If possible, include implementation details.
Note that feature requests may be out of scope for the project, and if accepted, we cannot commit to a specific timeline for implementation.
## :repeat: Submitting Pull Requests
- **Smaller is better.** Submit **one** pull request per bug fix or feature. A pull request should contain isolated changes pertaining to a single bug fix or feature implementation. **Do not** refactor or reformat code that is unrelated to your change. It is better to **submit many small pull requests** rather than a single large one. Enormous pull requests will take enormous amounts of time to review, or may be rejected altogether.
- **Coordinate bigger changes.** For large and non-trivial changes, open an issue to discuss a strategy with the maintainers. Otherwise, you risk doing a lot of work for nothing!
- **Follow PEP8.** Python code should follow PEP8 formatting and styling guidelines. Consider using automated tools like [autopep8](https://github.com/hhatto/autopep8) or [flake8](https://github.com/PyCQA/flake8) to ensure your code adheres to these standards.
- **[Resolve any merge conflicts](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/resolving-a-merge-conflict-on-github)** that occur.
- Use spaces, not tabs.
- Make sure all commits work with the new template — the old one is being deprecated.
## :memo: Copyright
This repository is licensed under the MIT License. **Any contributions you submit will be licensed under the same terms.**
By contributing, you confirm that your contributions do not infringe on the rights of others.
If your contribution includes code from other open-source projects, ensure that their licenses are compatible with this one. For example, code licensed under the GPL cannot be included in this project.
## :pray: Credit
This contribution guidelines is remixed from [jessesquires/.github:CONTRIBUTING.md](https://github.com/jessesquires/.github/blob/main/CONTRIBUTING.md) which also incorporated other works. *We commend them for their efforts to facilitate collaboration in their projects.*

View File

@@ -1,6 +1,6 @@
MIT License
Copyright (c) 2021 Knugi
Copyright (c) 2021-2025 Knugi
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

32
LICENSE.django Normal file
View File

@@ -0,0 +1,32 @@
The Whatsapp Chat Exporter is licensed under the MIT license. For more information,
refer to https://github.com/KnugiHK/WhatsApp-Chat-Exporter/wiki/Open-Source-Licenses.
------
Copyright (c) Django Software Foundation and individual contributors.
All rights reserved.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Neither the name of Django nor the names of its contributors may be used
to endorse or promote products derived from this software without
specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

224
README.md
View File

@@ -1,29 +1,44 @@
# Whatsapp-Chat-Exporter
[![Latest in Pypi](https://img.shields.io/pypi/v/whatsapp-chat-exporter?label=Latest%20in%20Pypi)](https://pypi.org/project/whatsapp-chat-exporter/)
![License MIT](https://img.shields.io/pypi/l/whatsapp-chat-exporter)
[![Latest in PyPI](https://img.shields.io/pypi/v/whatsapp-chat-exporter?label=Latest%20in%20PyPI)](https://pypi.org/project/whatsapp-chat-exporter/)
[![License MIT](https://img.shields.io/pypi/l/whatsapp-chat-exporter?color=427B93)](https://github.com/KnugiHK/WhatsApp-Chat-Exporter/blob/main/LICENSE)
[![Python](https://img.shields.io/pypi/pyversions/Whatsapp-Chat-Exporter)](https://pypi.org/project/Whatsapp-Chat-Exporter/)
[![Matrix Chat Room](https://img.shields.io/matrix/wtsexporter:matrix.org.svg?label=Matrix%20Chat%20Room)](https://matrix.to/#/#wtsexporter:matrix.org)
[![Since 2021](https://img.shields.io/github/created-at/knugihk/WhatsApp-Chat-Exporter?label=Since&color=purple)](https://wts.knugi.dev)
A customizable Android and iPhone Whatsapp database parser that will give you the history of your Whatsapp conversations in HTML and JSON.
**If you plan to uninstall WhatsApp or delete your WhatsApp account, please make a backup of your WhatsApp database. You may want to use this exporter again on the same database in the future as the exporter develops**
A customizable Android and iPhone Whatsapp database parser that will give you the history of your Whatsapp conversations in HTML and JSON. Inspired by [Telegram Chat Export Tool](https://telegram.org/blog/export-and-more).
> [!TIP]
> If you plan to uninstall WhatsApp or delete your WhatsApp account, please make a backup of your WhatsApp database. You may want to use this exporter again on the same database in the future as the exporter develops.
If you would like to support this project, all you need to do is to contribute or share this project! If you think otherwise and want to make a donation, please refer to the [Donation Guide](https://blog.knugi.com/DONATE.html).
To contribute, see the [Contributing Guidelines](https://github.com/KnugiHK/WhatsApp-Chat-Exporter/blob/main/CONTRIBUTING.md).
# Usage
**If you want to use the old release (< 0.5) of the exporter, please follow the [old usage guide](https://github.com/KnugiHK/Whatsapp-Chat-Exporter/blob/main/old_README.md#usage)**
> [!NOTE]
> Usage in README may be removed in the future. Check the usage in [Wiki](https://github.com/KnugiHK/Whatsapp-Chat-Exporter/wiki)
>
> Click [here](https://github.com/KnugiHK/WhatsApp-Chat-Exporter/wiki/Android-Usage#crypt15-end-to-end-encrypted-backup) for the most trivia way for exporting from Android
First, install the exporter by:
```shell
pip install whatsapp-chat-exporter
pip install whatsapp-chat-exporter[android_backup] & :: Optional, if you want it to support decrypting Android WhatsApp backup.
pip install whatsapp-chat-exporter[android_backup] :; # Optional, if you want it to support decrypting Android WhatsApp backup.
```
Then, create a working directory in somewhere you want
```shell
mkdir working_wts
cd working_wts
```
> [!TIP]
> macOS users should grant *Full Disk Access* to Terminal in the *Security & Privacy* settings before using the exporter.
## Working with Android
### Unencrypted WhatsApp database
Extract the WhatsApp database with whatever means, one possible means is to use the [WhatsApp-Key-DB-Extractor](https://github.com/KnugiHK/WhatsApp-Key-DB-Extractor)
Extract the WhatsApp database with whatever means, one possible means is to use the [WhatsApp-Key-DB-Extractor](https://github.com/KnugiHK/WhatsApp-Key-DB-Extractor). Note that the extractor only works on Android 4.0 to 13.
After you obtain your WhatsApp databse, copy the WhatsApp database and media folder to the working directory. The database is called msgstore.db. If you also want the name of your contacts, get the contact database, which is called wa.db. And copy the WhatsApp (Media) directory from your phone directly.
After you obtain your WhatsApp database, copy the WhatsApp database and media folder to the working directory. The database is called msgstore.db. If you also want the name of your contacts, get the contact database, which is called wa.db. And copy the WhatsApp (Media) directory from your phone directly.
And now, you should have something like this in the working directory.
@@ -33,12 +48,26 @@ Simply invoke the following command from shell.
```sh
wtsexporter -a
```
#### Enriching Contact from vCard
The default WhatsApp contact database typically contained contact names extracted from your phone, which the exporter used to map your chats. However, in some reported cases, the database may have never been populated. In such case, you can export your contacts to a vCard file from your phone or a cloud provider like Google Contacts. Then, install the necessary dependency and run the following command from the shell:
```sh
pip install whatsapp-chat-exporter["vcards"]
wtsexporter -a --enrich-from-vcard contacts.vcf --default-country-code 852
```
### Encrypted Android WhatsApp Backup
In order to support the decryption, install pycryptodome if it is not installed
```sh
pip install pycryptodome
pip install pycryptodome # Or
pip install whatsapp-chat-exporter["android_backup"] # install along with this software
```
> [!TIP]
> Crypt15 is now the easiest way to decrypt a backup. If you have the 32 bytes hex key generated when you enable End-to-End encrypted backup, you can use it to decrypt the backup. If you do not have the 32 bytes hex key, you can still use the key file extracted just like extacting key file for Crypt12 and Crypt14 to decrypt the backup.
#### Crypt12 or Crypt14
You will need the decryption key file from your phone. If you have root access, you can find it as `/data/data/com.whatsapp/files/key`. Otherwise, if you used WhatsApp-Key-DB-Extractor before, it will appear in the WhatsApp backup directory as `WhatsApp/Databases/.nomedia`.
Place the decryption key file (key) and the encrypted WhatsApp Backup (msgstore.db.crypt14) in the working directory. If you also want the name of your contacts, get the contact database, which is called wa.db. And copy the WhatsApp (Media) directory from your phone directly.
And now, you should have something like this in the working directory.
@@ -50,19 +79,60 @@ Simply invoke the following command from shell.
wtsexporter -a -k key -b msgstore.db.crypt14
```
## Working with iPhone
Do an iPhone Backup with iTunes first.
### Encrypted iPhone Backup
**If you are working on unencrypted iPhone backup, skip this**
#### Crypt15 (End-to-End Encrypted Backup)
To support Crypt15 backup, install javaobj-py3 if it is not installed
```sh
pip install javaobj-py3 # Or
pip install whatsapp-chat-exporter["crypt15"] # install along with this software
```
Before proceeding with this method, you must first create an end-to-end encrypted backup. For detailed instructions, refer to [WhatsApp's help center](https://faq.whatsapp.com/490592613091019).
If you want to work on an encrypted iPhone Backup, you should install iphone_backup_decrypt from [KnugiHK/iphone_backup_decrypt](https://github.com/KnugiHK/iphone_backup_decrypt) before you run the extract_iphone_media.py.
Once you have copied the backup files to your computer, place the encrypted WhatsApp backup file (msgstore.db.crypt15) into the working directory. If you also wish to include your contacts' names, obtain the contact database file, named wa.db. Additionally, copy the WhatsApp Media folder directly from your phone.
If you do not have the 32 bytes hex key (64 hexdigits), place the decryption key file (encrypted_backup.key) extracted from Android. If you gave the 32 bytes hex key, simply put the key in the shell.
Now, you should have something like this in the working directory (if you do not have 32 bytes hex key).
![Android folder structure with WhatsApp Crypt15 Backup](imgs/android_structure_backup_crypt15.png)
##### Extracting
If you do not have 32 bytes hex key but have the key file available, simply invoke the following command from shell.
```sh
wtsexporter -a -k encrypted_backup.key -b msgstore.db.crypt15
```
If you have the 32 bytes hex key, simply put the hex key in the -k option and invoke the command from shell like this:
```sh
wtsexporter -a -k 133735053b5204b08e5c3823423399aa30ff061435ab89bc4e6713969cda1337 -b msgstore.db.crypt15
```
## Working with iOS/iPadOS (iPhone or iPad)
Do an iPhone/iPad Backup with iTunes/Finder first.
* iPhone backup on Mac: https://support.apple.com/HT211229
* iPhone backup on Windows: https://support.apple.com/HT212156
* iPad backup: https://support.apple.com/guide/ipad/ipad9a74df05xx/ipados
### Encrypted iOS/iPadOS Backup
> [!NOTE]
> If you are working on unencrypted iOS/iPadOS backup, skip this.
If you want to work on an encrypted iOS/iPadOS Backup, you should install iphone_backup_decrypt from [KnugiHK/iphone_backup_decrypt](https://github.com/KnugiHK/iphone_backup_decrypt) before you run the extract_iphone_media.py.
```sh
pip install git+https://github.com/KnugiHK/iphone_backup_decrypt
```
> [!NOTE]
> You will need to disable the built-in end-to-end encryption for WhatsApp backups. See [WhatsApp's FAQ](https://faq.whatsapp.com/490592613091019#turn-off-end-to-end-encrypted-backup) for how to do it.
### Extracting
Simply invoke the following command from shell, remember to replace the username and device id correspondingly in the command.
```sh
To extract messages from iOS/iPadOS backups, run the following command in the shell, making sure to replace the username and device ID with the correct values. Keep in mind that there are at least two possible paths for the backups on Windows.
#### Windows
```powershell
# Possible path one
wtsexporter -i -b "C:\Users\[Username]\AppData\Roaming\Apple Computer\MobileSync\Backup\[device id]"
# Possible path two
wtsexporter -i -b "C:\Users\[Username]\Apple\MobileSync\Backup\[device id]"
```
#### Mac
```sh
wtsexporter -i -b ~/Library/Application\ Support/MobileSync/Backup/[device id]
```
## Results
@@ -77,32 +147,114 @@ After extracting, you will get these:
Invoke the wtsexporter with --help option will show you all options available.
```sh
> wtsexporter --help
Usage: wtsexporter [options]
usage: wtsexporter [-h] [-a] [-i] [-e EXPORTED] [-w WA] [-m MEDIA] [-b BACKUP] [-d DB] [-k [KEY]]
[--call-db [CALL_DB_IOS]] [--wab WAB] [-o OUTPUT] [-j [JSON]] [--txt [TEXT_FORMAT]] [--no-html]
[--size [SIZE]] [--avoid-encoding-json] [--pretty-print-json [PRETTY_PRINT_JSON]] [--per-chat]
[--import] [-t TEMPLATE] [--offline OFFLINE] [--no-avatar] [--experimental-new-theme]
[--headline HEADLINE] [-c] [--create-separated-media] [--time-offset {-12 to 14}] [--date DATE]
[--date-format FORMAT] [--include [phone number ...]] [--exclude [phone number ...]]
[--dont-filter-empty] [--enrich-from-vcards ENRICH_FROM_VCARDS]
[--default-country-code DEFAULT_COUNTRY_CODE] [-s] [--check-update] [--assume-first-as-me]
[--business] [--decrypt-chunk-size DECRYPT_CHUNK_SIZE]
[--max-bruteforce-worker MAX_BRUTEFORCE_WORKER]
Options:
--version show program's version number and exit
A customizable Android and iOS/iPadOS WhatsApp database parser that will give you the history of your WhatsApp
conversations in HTML and JSON. Android Backup Crypt12, Crypt14 and Crypt15 supported.
options:
-h, --help show this help message and exit
Device Type:
-a, --android Define the target as Android
-i, --iphone Define the target as iPhone
-w WA, --wa=WA Path to contact database
-m MEDIA, --media=MEDIA
Path to WhatsApp media folder
-b BACKUP, --backup=BACKUP
Path to Android (must be used together with -k)/iPhone
WhatsApp backup
-o OUTPUT, --output=OUTPUT
Output to specific directory
-j, --json Save the result to a single JSON file
-d DB, --db=DB Path to database file
-k KEY, --key=KEY Path to key file
-t TEMPLATE, --template=TEMPLATE
-i, --ios Define the target as iPhone/iPad
-e, --exported EXPORTED
Define the target as exported chat file and specify the path to the file
Input Files:
-w, --wa WA Path to contact database (default: wa.db/ContactsV2.sqlite)
-m, --media MEDIA Path to WhatsApp media folder (default: WhatsApp)
-b, --backup BACKUP Path to Android (must be used together with -k)/iOS WhatsApp backup
-d, --db DB Path to database file (default: msgstore.db/7c7fba66680ef796b916b067077cc246adacf01d)
-k, --key [KEY] Path to key file. If this option is set for crypt15 backup but nothing is specified, you will
be prompted to enter the key.
--call-db [CALL_DB_IOS]
Path to call database (default: 1b432994e958845fffe8e2f190f26d1511534088) iOS only
--wab, --wa-backup WAB
Path to contact database in crypt15 format
Output Options:
-o, --output OUTPUT Output to specific directory (default: result)
-j, --json [JSON] Save the result to a single JSON file (default if present: result.json)
--txt [TEXT_FORMAT] Export chats in text format similar to what WhatsApp officially provided (default if present:
result/)
--no-html Do not output html files
--size, --output-size, --split [SIZE]
Maximum (rough) size of a single output file in bytes, 0 for auto
JSON Options:
--avoid-encoding-json
Don't encode non-ascii characters in the output JSON files
--pretty-print-json [PRETTY_PRINT_JSON]
Pretty print the output JSON.
--per-chat Output the JSON file per chat
--import Import JSON file and convert to HTML output
HTML Options:
-t, --template TEMPLATE
Path to custom HTML template
--offline OFFLINE Relative path to offline static files
--no-avatar Do not render avatar in HTML output
--experimental-new-theme
Use the newly designed WhatsApp-alike theme
--headline HEADLINE The custom headline for the HTML output. Use '??' as a placeholder for the chat name
Media Handling:
-c, --move-media Move the media directory to output directory if the flag is set, otherwise copy it
--create-separated-media
Create a copy of the media seperated per chat in <MEDIA>/separated/ directory
Filtering Options:
--time-offset {-12 to 14}
Offset in hours (-12 to 14) for time displayed in the output
--date DATE The date filter in specific format (inclusive)
--date-format FORMAT The date format for the date filter
--include [phone number ...]
Include chats that match the supplied phone number
--exclude [phone number ...]
Exclude chats that match the supplied phone number
--dont-filter-empty By default, the exporter will not render chats with no valid message. Setting this flag will
cause the exporter to render those. This is useful if chat(s) are missing from the output
Contact Enrichment:
--enrich-from-vcards ENRICH_FROM_VCARDS
Path to an exported vcf file from Google contacts export. Add names missing from WhatsApp's
default database
--default-country-code DEFAULT_COUNTRY_CODE
Use with --enrich-from-vcards. When numbers in the vcf file does not have a country code, this
will be used. 1 is for US, 66 for Thailand etc. Most likely use the number of your own country
Miscellaneous:
-s, --showkey Show the HEX key used to decrypt the database
--check-update Check for updates (require Internet access)
--assume-first-as-me Assume the first message in a chat as sent by me (must be used together with -e)
--business Use Whatsapp Business default files (iOS only)
--decrypt-chunk-size DECRYPT_CHUNK_SIZE
Specify the chunk size for decrypting iOS backup, which may affect the decryption speed.
--max-bruteforce-worker MAX_BRUTEFORCE_WORKER
Specify the maximum number of worker for bruteforce decryption.
WhatsApp Chat Exporter: 0.12.1 Licensed with MIT. See https://wts.knugi.dev/docs?dest=osl for all open source
licenses.
```
# To do
1. Reply in iPhone
# Legal Stuff & Disclaimer
# Copyright
This is a MIT licensed project.
The Telegram Desktop's export is the reference for whatsapp.html in this repo
The Telegram Desktop's export is the reference for whatsapp.html in this repo.
`bplist.py` was released by Vladimir "Farcaller" Pouzanov under MIT license.
Please also refer to any files prefixed with `LICENSE` to obtain copies of the various licenses.
WhatsApp Chat Exporter is not affiliated, associated, authorized, endorsed by, or in any way officially connected with the WhatsApp LLC, or any of its subsidiaries or its affiliates. The official WhatsApp LLC website can be found at https://www.whatsapp.com/.

View File

@@ -1 +0,0 @@
__version__ = "0.7.0"

View File

@@ -1,163 +1,736 @@
from .__init__ import __version__
from Whatsapp_Chat_Exporter import extract, extract_iphone
from Whatsapp_Chat_Exporter import extract_iphone_media
from optparse import OptionParser
#!/usr/bin/python3
import io
import os
import sqlite3
import shutil
import json
import string
import glob
import importlib.metadata
from Whatsapp_Chat_Exporter import android_crypt, exported_handler, android_handler
from Whatsapp_Chat_Exporter import ios_handler, ios_media_handler
from Whatsapp_Chat_Exporter.data_model import ChatCollection, ChatStore
from Whatsapp_Chat_Exporter.utility import APPLE_TIME, Crypt, check_update, DbType
from Whatsapp_Chat_Exporter.utility import readable_to_bytes, sanitize_filename
from Whatsapp_Chat_Exporter.utility import import_from_json, bytes_to_readable
from argparse import ArgumentParser, SUPPRESS
from datetime import datetime
from getpass import getpass
from sys import exit
from typing import Tuple, Optional, List, Dict, Any, Union
# Try to import vobject for contacts processing
try:
import vobject
except ModuleNotFoundError:
vcards_deps_installed = False
else:
from Whatsapp_Chat_Exporter.vcards_contacts import ContactsFromVCards
vcards_deps_installed = True
def setup_argument_parser() -> ArgumentParser:
"""Set up and return the argument parser with all options."""
parser = ArgumentParser(
description='A customizable Android and iOS/iPadOS WhatsApp database parser that '
'will give you the history of your WhatsApp conversations in HTML '
'and JSON. Android Backup Crypt12, Crypt14 and Crypt15 supported.',
epilog=f'WhatsApp Chat Exporter: {importlib.metadata.version("whatsapp_chat_exporter")} Licensed with MIT. See '
'https://wts.knugi.dev/docs?dest=osl for all open source licenses.'
)
# Device type arguments
device_group = parser.add_argument_group('Device Type')
device_group.add_argument(
'-a', '--android', dest='android', default=False, action='store_true',
help="Define the target as Android"
)
device_group.add_argument(
'-i', '--ios', dest='ios', default=False, action='store_true',
help="Define the target as iPhone/iPad"
)
device_group.add_argument(
"-e", "--exported", dest="exported", default=None,
help="Define the target as exported chat file and specify the path to the file"
)
# Input file paths
input_group = parser.add_argument_group('Input Files')
input_group.add_argument(
"-w", "--wa", dest="wa", default=None,
help="Path to contact database (default: wa.db/ContactsV2.sqlite)"
)
input_group.add_argument(
"-m", "--media", dest="media", default=None,
help="Path to WhatsApp media folder (default: WhatsApp)"
)
input_group.add_argument(
"-b", "--backup", dest="backup", default=None,
help="Path to Android (must be used together with -k)/iOS WhatsApp backup"
)
input_group.add_argument(
"-d", "--db", dest="db", default=None,
help="Path to database file (default: msgstore.db/7c7fba66680ef796b916b067077cc246adacf01d)"
)
input_group.add_argument(
"-k", "--key", dest="key", default=None, nargs='?',
help="Path to key file. If this option is set for crypt15 backup but nothing is specified, you will be prompted to enter the key."
)
input_group.add_argument(
"--call-db", dest="call_db_ios", nargs='?', default=None, type=str,
const="1b432994e958845fffe8e2f190f26d1511534088",
help="Path to call database (default: 1b432994e958845fffe8e2f190f26d1511534088) iOS only"
)
input_group.add_argument(
"--wab", "--wa-backup", dest="wab", default=None,
help="Path to contact database in crypt15 format"
)
# Output options
output_group = parser.add_argument_group('Output Options')
output_group.add_argument(
"-o", "--output", dest="output", default="result",
help="Output to specific directory (default: result)"
)
output_group.add_argument(
'-j', '--json', dest='json', nargs='?', default=None, type=str, const="result.json",
help="Save the result to a single JSON file (default if present: result.json)"
)
output_group.add_argument(
"--txt", dest="text_format", nargs='?', default=None, type=str, const="result",
help="Export chats in text format similar to what WhatsApp officially provided (default if present: result/)"
)
output_group.add_argument(
"--no-html", dest="no_html", default=False, action='store_true',
help="Do not output html files"
)
output_group.add_argument(
"--size", "--output-size", "--split", dest="size", nargs='?', const=0, default=None,
help="Maximum (rough) size of a single output file in bytes, 0 for auto"
)
# JSON formatting options
json_group = parser.add_argument_group('JSON Options')
json_group.add_argument(
'--avoid-encoding-json', dest='avoid_encoding_json', default=False, action='store_true',
help="Don't encode non-ascii characters in the output JSON files"
)
json_group.add_argument(
'--pretty-print-json', dest='pretty_print_json', default=None, nargs='?', const=2, type=int,
help="Pretty print the output JSON."
)
json_group.add_argument(
"--per-chat", dest="json_per_chat", default=False, action='store_true',
help="Output the JSON file per chat"
)
json_group.add_argument(
"--import", dest="import_json", default=False, action='store_true',
help="Import JSON file and convert to HTML output"
)
# HTML options
html_group = parser.add_argument_group('HTML Options')
html_group.add_argument(
"-t", "--template", dest="template", default=None,
help="Path to custom HTML template"
)
html_group.add_argument(
"--embedded", dest="embedded", default=False, action='store_true',
help=SUPPRESS or "Embed media into HTML file (not yet implemented)"
)
html_group.add_argument(
"--offline", dest="offline", default=None,
help="Relative path to offline static files"
)
html_group.add_argument(
"--no-avatar", dest="no_avatar", default=False, action='store_true',
help="Do not render avatar in HTML output"
)
html_group.add_argument(
"--experimental-new-theme", dest="whatsapp_theme", default=False, action='store_true',
help="Use the newly designed WhatsApp-alike theme"
)
html_group.add_argument(
"--headline", dest="headline", default="Chat history with ??",
help="The custom headline for the HTML output. Use '??' as a placeholder for the chat name"
)
# Media handling
media_group = parser.add_argument_group('Media Handling')
media_group.add_argument(
"-c", "--move-media", dest="move_media", default=False, action='store_true',
help="Move the media directory to output directory if the flag is set, otherwise copy it"
)
media_group.add_argument(
"--create-separated-media", dest="separate_media", default=False, action='store_true',
help="Create a copy of the media seperated per chat in <MEDIA>/separated/ directory"
)
# Filtering options
filter_group = parser.add_argument_group('Filtering Options')
filter_group.add_argument(
"--time-offset", dest="timezone_offset", default=0, type=int, choices=range(-12, 15),
metavar="{-12 to 14}", help="Offset in hours (-12 to 14) for time displayed in the output"
)
filter_group.add_argument(
"--date", dest="filter_date", default=None, metavar="DATE",
help="The date filter in specific format (inclusive)"
)
filter_group.add_argument(
"--date-format", dest="filter_date_format", default="%Y-%m-%d %H:%M", metavar="FORMAT",
help="The date format for the date filter"
)
filter_group.add_argument(
"--include", dest="filter_chat_include", nargs='*', metavar="phone number",
help="Include chats that match the supplied phone number"
)
filter_group.add_argument(
"--exclude", dest="filter_chat_exclude", nargs='*', metavar="phone number",
help="Exclude chats that match the supplied phone number"
)
filter_group.add_argument(
"--dont-filter-empty", dest="filter_empty", default=True, action='store_false',
help=("By default, the exporter will not render chats with no valid message. "
"Setting this flag will cause the exporter to render those. "
"This is useful if chat(s) are missing from the output")
)
# Contact enrichment
contact_group = parser.add_argument_group('Contact Enrichment')
contact_group.add_argument(
"--enrich-from-vcards", dest="enrich_from_vcards", default=None,
help="Path to an exported vcf file from Google contacts export. Add names missing from WhatsApp's default database"
)
contact_group.add_argument(
"--default-country-code", dest="default_country_code", default=None,
help="Use with --enrich-from-vcards. When numbers in the vcf file does not have a country code, this will be used. 1 is for US, 66 for Thailand etc. Most likely use the number of your own country"
)
# Miscellaneous
misc_group = parser.add_argument_group('Miscellaneous')
misc_group.add_argument(
"-s", "--showkey", dest="showkey", default=False, action='store_true',
help="Show the HEX key used to decrypt the database"
)
misc_group.add_argument(
"--check-update", dest="check_update", default=False, action='store_true',
help="Check for updates (require Internet access)"
)
misc_group.add_argument(
"--assume-first-as-me", dest="assume_first_as_me", default=False, action='store_true',
help="Assume the first message in a chat as sent by me (must be used together with -e)"
)
misc_group.add_argument(
"--business", dest="business", default=False, action='store_true',
help="Use Whatsapp Business default files (iOS only)"
)
misc_group.add_argument(
"--decrypt-chunk-size", dest="decrypt_chunk_size", default=1 * 1024 * 1024, type=int,
help="Specify the chunk size for decrypting iOS backup, which may affect the decryption speed."
)
misc_group.add_argument(
"--max-bruteforce-worker", dest="max_bruteforce_worker", default=10, type=int,
help="Specify the maximum number of worker for bruteforce decryption."
)
return parser
def validate_args(parser: ArgumentParser, args) -> None:
"""Validate command line arguments and modify them if needed."""
# Basic validation checks
if args.android and args.ios and args.exported and args.import_json:
parser.error("You must define only one device type.")
if not args.android and not args.ios and not args.exported and not args.import_json:
parser.error("You must define the device type.")
if args.no_html and not args.json and not args.text_format:
parser.error("You must either specify a JSON output file, text file output directory or enable HTML output.")
if args.import_json and (args.android or args.ios or args.exported or args.no_html):
parser.error("You can only use --import with -j and without --no-html, -a, -i, -e.")
elif args.import_json and not os.path.isfile(args.json):
parser.error("JSON file not found.")
if args.android and args.business:
parser.error("WhatsApp Business is only available on iOS for now.")
if "??" not in args.headline:
parser.error("--headline must contain '??' for replacement.")
# JSON validation
if args.json_per_chat and args.json and (
(args.json.endswith(".json") and os.path.isfile(args.json)) or
(not args.json.endswith(".json") and os.path.isfile(args.json))
):
parser.error("When --per-chat is enabled, the destination of --json must be a directory.")
# vCards validation
if args.enrich_from_vcards is not None and args.default_country_code is None:
parser.error("When --enrich-from-vcards is provided, you must also set --default-country-code")
# Size validation
if args.size is not None and not isinstance(args.size, int) and not args.size.isnumeric():
try:
args.size = readable_to_bytes(args.size)
except ValueError:
parser.error("The value for --split must be ended in pure bytes or with a proper unit (e.g., 1048576 or 1MB)")
# Date filter validation and processing
if args.filter_date is not None:
process_date_filter(parser, args)
# Crypt15 key validation
if args.key is None and args.backup is not None and args.backup.endswith("crypt15"):
args.key = getpass("Enter your encryption key: ")
# Theme validation
if args.whatsapp_theme:
args.template = "whatsapp_new.html"
# Chat filter validation
if args.filter_chat_include is not None and args.filter_chat_exclude is not None:
parser.error("Chat inclusion and exclusion filters cannot be used together.")
validate_chat_filters(parser, args.filter_chat_include)
validate_chat_filters(parser, args.filter_chat_exclude)
def validate_chat_filters(parser: ArgumentParser, chat_filter: Optional[List[str]]) -> None:
"""Validate chat filters to ensure they contain only phone numbers."""
if chat_filter is not None:
for chat in chat_filter:
if not chat.isnumeric():
parser.error("Enter a phone number in the chat filter. See https://wts.knugi.dev/docs?dest=chat")
def process_date_filter(parser: ArgumentParser, args) -> None:
"""Process and validate date filter arguments."""
if " - " in args.filter_date:
start, end = args.filter_date.split(" - ")
start = int(datetime.strptime(start, args.filter_date_format).timestamp())
end = int(datetime.strptime(end, args.filter_date_format).timestamp())
if start < 1009843200 or end < 1009843200:
parser.error("WhatsApp was first released in 2009...")
if start > end:
parser.error("The start date cannot be a moment after the end date.")
if args.android:
args.filter_date = f"BETWEEN {start}000 AND {end}000"
elif args.ios:
args.filter_date = f"BETWEEN {start - APPLE_TIME} AND {end - APPLE_TIME}"
else:
process_single_date_filter(parser, args)
def process_single_date_filter(parser: ArgumentParser, args) -> None:
"""Process single date comparison filters."""
if len(args.filter_date) < 3:
parser.error("Unsupported date format. See https://wts.knugi.dev/docs?dest=date")
_timestamp = int(datetime.strptime(args.filter_date[2:], args.filter_date_format).timestamp())
if _timestamp < 1009843200:
parser.error("WhatsApp was first released in 2009...")
if args.filter_date[:2] == "> ":
if args.android:
args.filter_date = f">= {_timestamp}000"
elif args.ios:
args.filter_date = f">= {_timestamp - APPLE_TIME}"
elif args.filter_date[:2] == "< ":
if args.android:
args.filter_date = f"<= {_timestamp}000"
elif args.ios:
args.filter_date = f"<= {_timestamp - APPLE_TIME}"
else:
parser.error("Unsupported date format. See https://wts.knugi.dev/docs?dest=date")
def setup_contact_store(args) -> Optional['ContactsFromVCards']:
"""Set up and return a contact store if needed."""
if args.enrich_from_vcards is not None:
if not vcards_deps_installed:
print(
"You don't have the dependency to enrich contacts with vCard.\n"
"Read more on how to deal with enriching contacts:\n"
"https://github.com/KnugiHK/Whatsapp-Chat-Exporter/blob/main/README.md#usage"
)
exit(1)
contact_store = ContactsFromVCards()
contact_store.load_vcf_file(args.enrich_from_vcards, args.default_country_code)
return contact_store
return None
def decrypt_android_backup(args) -> int:
"""Decrypt Android backup files and return error code."""
if args.key is None or args.backup is None:
print("You must specify the backup file with -b and a key with -k")
return 1
print("Decryption key specified, decrypting WhatsApp backup...")
# Determine crypt type
if "crypt12" in args.backup:
crypt = Crypt.CRYPT12
elif "crypt14" in args.backup:
crypt = Crypt.CRYPT14
elif "crypt15" in args.backup:
crypt = Crypt.CRYPT15
else:
print("Unknown backup format. The backup file must be crypt12, crypt14 or crypt15.")
return 1
# Get key
keyfile_stream = False
if not os.path.isfile(args.key) and all(char in string.hexdigits for char in args.key.replace(" ", "")):
key = bytes.fromhex(args.key.replace(" ", ""))
else:
key = open(args.key, "rb")
keyfile_stream = True
# Read backup
db = open(args.backup, "rb").read()
# Process WAB if provided
error_wa = 0
if args.wab:
wab = open(args.wab, "rb").read()
error_wa = android_crypt.decrypt_backup(
wab,
key,
args.wa,
crypt,
args.showkey,
DbType.CONTACT,
keyfile_stream=keyfile_stream,
max_worker=args.max_bruteforce_worker
)
if isinstance(key, io.IOBase):
key.seek(0)
# Decrypt message database
error_message = android_crypt.decrypt_backup(
db,
key,
args.db,
crypt,
args.showkey,
DbType.MESSAGE,
keyfile_stream=keyfile_stream,
max_worker=args.max_bruteforce_worker
)
# Handle errors
if error_wa != 0:
return error_wa
return error_message
def handle_decrypt_error(error: int) -> None:
"""Handle decryption errors with appropriate messages."""
if error == 1:
print("Dependencies of decrypt_backup and/or extract_encrypted_key"
" are not present. For details, see README.md.")
exit(3)
elif error == 2:
print("Failed when decompressing the decrypted backup. "
"Possibly incorrect offsets used in decryption.")
exit(4)
else:
print("Unknown error occurred.", error)
exit(5)
def process_contacts(args, data: ChatCollection, contact_store=None) -> None:
"""Process contacts from the database."""
contact_db = args.wa if args.wa else "wa.db" if args.android else "ContactsV2.sqlite"
if os.path.isfile(contact_db):
with sqlite3.connect(contact_db) as db:
db.row_factory = sqlite3.Row
if args.android:
android_handler.contacts(db, data, args.enrich_from_vcards)
else:
ios_handler.contacts(db, data)
def process_messages(args, data: ChatCollection) -> None:
"""Process messages, media and vcards from the database."""
msg_db = args.db if args.db else "msgstore.db" if args.android else args.identifiers.MESSAGE
if not os.path.isfile(msg_db):
print(
"The message database does not exist. You may specify the path "
"to database file with option -d or check your provided path."
)
exit(6)
filter_chat = (args.filter_chat_include, args.filter_chat_exclude)
with sqlite3.connect(msg_db) as db:
db.row_factory = sqlite3.Row
# Process messages
if args.android:
message_handler = android_handler
else:
message_handler = ios_handler
message_handler.messages(
db, data, args.media, args.timezone_offset,
args.filter_date, filter_chat, args.filter_empty
)
# Process media
message_handler.media(
db, data, args.media, args.filter_date,
filter_chat, args.filter_empty, args.separate_media
)
# Process vcards
message_handler.vcard(
db, data, args.media, args.filter_date,
filter_chat, args.filter_empty
)
# Process calls
process_calls(args, db, data, filter_chat)
def process_calls(args, db, data: ChatCollection, filter_chat) -> None:
"""Process call history if available."""
if args.android:
android_handler.calls(db, data, args.timezone_offset, filter_chat)
elif args.ios and args.call_db_ios is not None:
with sqlite3.connect(args.call_db_ios) as cdb:
cdb.row_factory = sqlite3.Row
ios_handler.calls(cdb, data, args.timezone_offset, filter_chat)
def handle_media_directory(args) -> None:
"""Handle media directory copying or moving."""
if os.path.isdir(args.media):
media_path = os.path.join(args.output, args.media)
if os.path.isdir(media_path):
print("\nWhatsApp directory already exists in output directory. Skipping...", end="\n")
else:
if args.move_media:
try:
print("\nMoving media directory...", end="\n")
shutil.move(args.media, f"{args.output}/")
except PermissionError:
print("\nCannot remove original WhatsApp directory. "
"Perhaps the directory is opened?", end="\n")
else:
print("\nCopying media directory...", end="\n")
shutil.copytree(args.media, media_path)
def create_output_files(args, data: ChatCollection, contact_store=None) -> None:
"""Create output files in the specified formats."""
# Create HTML files if requested
if not args.no_html:
# Enrich from vcards if available
if contact_store and not contact_store.is_empty():
contact_store.enrich_from_vcards(data)
android_handler.create_html(
data,
args.output,
args.template,
args.embedded,
args.offline,
args.size,
args.no_avatar,
args.whatsapp_theme,
args.headline
)
# Create text files if requested
if args.text_format:
print("Writing text file...")
android_handler.create_txt(data, args.text_format)
# Create JSON files if requested
if args.json and not args.import_json:
export_json(args, data, contact_store)
def export_json(args, data: ChatCollection, contact_store=None) -> None:
"""Export data to JSON format."""
# Enrich from vcards if available
if contact_store and not contact_store.is_empty():
contact_store.enrich_from_vcards(data)
# Convert ChatStore objects to JSON
if isinstance(data.get(next(iter(data), None)), ChatStore):
data = {jik: chat.to_json() for jik, chat in data.items()}
# Export as a single file or per chat
if not args.json_per_chat:
export_single_json(args, data)
else:
export_multiple_json(args, data)
def export_single_json(args, data: Dict) -> None:
"""Export data to a single JSON file."""
with open(args.json, "w") as f:
json_data = json.dumps(
data,
ensure_ascii=not args.avoid_encoding_json,
indent=args.pretty_print_json
)
print(f"\nWriting JSON file...({bytes_to_readable(len(json_data))})")
f.write(json_data)
def export_multiple_json(args, data: Dict) -> None:
"""Export data to multiple JSON files, one per chat."""
# Adjust output path if needed
json_path = args.json[:-5] if args.json.endswith(".json") else args.json
# Create directory if it doesn't exist
if not os.path.isdir(json_path):
os.makedirs(json_path, exist_ok=True)
# Export each chat
total = len(data.keys())
for index, jik in enumerate(data.keys()):
if data[jik]["name"] is not None:
contact = data[jik]["name"].replace('/', '')
else:
contact = jik.replace('+', '')
with open(f"{json_path}/{sanitize_filename(contact)}.json", "w") as f:
file_content = json.dumps(
{jik: data[jik]},
ensure_ascii=not args.avoid_encoding_json,
indent=args.pretty_print_json
)
f.write(file_content)
print(f"Writing JSON file...({index + 1}/{total})", end="\r")
print()
def process_exported_chat(args, data: ChatCollection) -> None:
"""Process an exported chat file."""
exported_handler.messages(args.exported, data, args.assume_first_as_me)
if not args.no_html:
android_handler.create_html(
data,
args.output,
args.template,
args.embedded,
args.offline,
args.size,
args.no_avatar,
args.whatsapp_theme,
args.headline
)
# Copy files to output directory
for file in glob.glob(r'*.*'):
shutil.copy(file, args.output)
def main():
parser = OptionParser(version=f"Whatsapp Chat Exporter: {__version__}")
parser.add_option(
'-a',
'--android',
dest='android',
default=False,
action='store_true',
help="Define the target as Android")
parser.add_option(
'-i',
'--iphone',
dest='iphone',
default=False,
action='store_true',
help="Define the target as iPhone")
parser.add_option(
"-w",
"--wa",
dest="wa",
default=None,
help="Path to contact database")
parser.add_option(
"-m",
"--media",
dest="media",
default=None,
help="Path to WhatsApp media folder")
parser.add_option(
"-b",
"--backup",
dest="backup",
default=None,
help="Path to Android (must be used together "
"with -k)/iPhone WhatsApp backup")
parser.add_option(
"-o",
"--output",
dest="output",
default="result",
help="Output to specific directory")
parser.add_option(
'-j',
'--json',
dest='json',
default=False,
action='store_true',
help="Save the result to a single JSON file")
parser.add_option(
'-d',
'--db',
dest='db',
default=None,
help="Path to database file")
parser.add_option(
'-k',
'--key',
dest='key',
default=None,
help="Path to key file"
)
parser.add_option(
"-t",
"--template",
dest="template",
default=None,
help="Path to custom HTML template")
(options, args) = parser.parse_args()
if options.android and options.iphone:
print("You must define only one device type.")
exit()
if not options.android and not options.iphone:
print("You must define the device type.")
exit()
data = {}
if options.android:
contacts = extract.contacts
messages = extract.messages
media = extract.media
vcard = extract.vcard
create_html = extract.create_html
if options.db is None:
msg_db = "msgstore.db"
else:
msg_db = options.db
if options.key is not None:
if options.backup is None:
print("You must specify the backup file with -b")
return False
print("Decryption key specified, decrypting WhatsApp backup...")
key = open(options.key, "rb").read()
db = open(options.backup, "rb").read()
is_crypt14 = False if "crypt12" in options.backup else True
if not extract.decrypt_backup(db, key, msg_db, is_crypt14):
print("Dependencies of decrypt_backup are not "
"present. For details, see README.md")
return False
if options.wa is None:
contact_db = "wa.db"
else:
contact_db = options.wa
if options.media is None:
options.media = "WhatsApp"
if len(args) == 1:
msg_db = args[0]
if os.path.isfile(contact_db):
with sqlite3.connect(contact_db) as db:
contacts(db, data)
elif options.iphone:
messages = extract_iphone.messages
media = extract_iphone.media
vcard = extract_iphone.vcard
create_html = extract_iphone.create_html
if options.backup is not None:
extract_iphone_media.extract_media(options.backup)
if options.db is None:
msg_db = "7c7fba66680ef796b916b067077cc246adacf01d"
else:
msg_db = options.db
if options.wa is None:
contact_db = "ContactsV2.sqlite"
else:
contact_db = options.wa
if options.media is None:
options.media = "Message"
if len(args) == 1:
msg_db = args[0]
if os.path.isfile(msg_db):
with sqlite3.connect(msg_db) as db:
messages(db, data)
media(db, data, options.media)
vcard(db, data)
create_html(data, options.output, options.template)
if not os.path.isdir(f"{options.output}/{options.media}"):
shutil.move(options.media, f"{options.output}/")
if options.json:
with open("result.json", "w") as f:
data = json.dumps(data)
print(f"\nWriting JSON file...({int(len(data)/1024/1024)}MB)")
f.write(data)
"""Main function to run the WhatsApp Chat Exporter."""
# Set up and parse arguments
parser = setup_argument_parser()
args = parser.parse_args()
# Check for updates
if args.check_update:
exit(check_update())
# Validate arguments
validate_args(parser, args)
# Create output directory if it doesn't exist
os.makedirs(args.output, exist_ok=True)
# Initialize data collection
data = ChatCollection()
# Set up contact store for vCard enrichment if needed
contact_store = setup_contact_store(args)
if args.import_json:
# Import from JSON
import_from_json(args.json, data)
android_handler.create_html(
data,
args.output,
args.template,
args.embedded,
args.offline,
args.size,
args.no_avatar,
args.whatsapp_theme,
args.headline
)
elif args.exported:
# Process exported chat
process_exported_chat(args, data)
else:
print()
# Process Android or iOS data
if args.android:
# Set default media path if not provided
if args.media is None:
args.media = "WhatsApp"
# Set default DB paths if not provided
if args.db is None:
args.db = "msgstore.db"
if args.wa is None:
args.wa = "wa.db"
# Decrypt backup if needed
if args.key is not None:
error = decrypt_android_backup(args)
if error != 0:
handle_decrypt_error(error)
elif args.ios:
# Set up identifiers based on business flag
if args.business:
from Whatsapp_Chat_Exporter.utility import WhatsAppBusinessIdentifier as identifiers
else:
from Whatsapp_Chat_Exporter.utility import WhatsAppIdentifier as identifiers
args.identifiers = identifiers
# Set default media path if not provided
if args.media is None:
args.media = identifiers.DOMAIN
# Extract media from backup if needed
if args.backup is not None:
if not os.path.isdir(args.media):
ios_media_handler.extract_media(args.backup, identifiers, args.decrypt_chunk_size)
else:
print("WhatsApp directory already exists, skipping WhatsApp file extraction.")
# Set default DB paths if not provided
if args.db is None:
args.db = identifiers.MESSAGE
if args.wa is None:
args.wa = "ContactsV2.sqlite"
# Process contacts
process_contacts(args, data, contact_store)
# Process messages, media, and calls
process_messages(args, data)
# Create output files
create_output_files(args, data, contact_store)
# Handle media directory
handle_media_directory(args)
print("Everything is done!")
print("Everything is done!")
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,328 @@
import hmac
import io
import zlib
import concurrent.futures
from typing import Tuple, Union
from hashlib import sha256
from sys import exit
from Whatsapp_Chat_Exporter.utility import CRYPT14_OFFSETS, Crypt, DbType
try:
import zlib
from Crypto.Cipher import AES
except ModuleNotFoundError:
support_backup = False
else:
support_backup = True
try:
import javaobj
except ModuleNotFoundError:
support_crypt15 = False
else:
support_crypt15 = True
class DecryptionError(Exception):
"""Base class for decryption-related exceptions."""
pass
class InvalidKeyError(DecryptionError):
"""Raised when the provided key is invalid."""
pass
class InvalidFileFormatError(DecryptionError):
"""Raised when the input file format is invalid."""
pass
class OffsetNotFoundError(DecryptionError):
"""Raised when the correct offsets for decryption cannot be found."""
pass
def _derive_main_enc_key(key_stream: bytes) -> Tuple[bytes, bytes]:
"""
Derive the main encryption key for the given key stream.
Args:
key_stream (bytes): The key stream to generate HMAC of HMAC.
Returns:
Tuple[bytes, bytes]: A tuple containing the main encryption key and the original key stream.
"""
intermediate_hmac = hmac.new(b'\x00' * 32, key_stream, sha256).digest()
key = hmac.new(intermediate_hmac, b"backup encryption\x01", sha256).digest()
return key, key_stream
def _extract_enc_key(keyfile: bytes) -> Tuple[bytes, bytes]:
"""
Extract the encryption key from the keyfile.
Args:
keyfile (bytes): The keyfile containing the encrypted key.
Returns:
Tuple[bytes, bytes]: values from _derive_main_enc_key()
"""
key_stream = b''.join([byte.to_bytes(1, "big", signed=True) for byte in javaobj.loads(keyfile)])
return _derive_main_enc_key(key_stream)
def brute_force_offset(max_iv: int = 200, max_db: int = 200):
"""
Brute force the offsets for IV and database start position in WhatsApp backup files.
Args:
max_iv (int, optional): Maximum value to try for IV offset. Defaults to 200.
max_db (int, optional): Maximum value to try for database start offset. Defaults to 200.
Yields:
tuple: A tuple containing:
- int: Start position of IV
- int: End position of IV (start + 16)
- int: Start position of database
"""
for iv in range(0, max_iv):
for db in range(0, max_db):
yield iv, iv + 16, db
def _decrypt_database(db_ciphertext: bytes, main_key: bytes, iv: bytes) -> bytes:
"""Decrypt and decompress a database chunk.
Args:
db_ciphertext (bytes): The encrypted chunk of the database.
main_key (bytes): The main decryption key.
iv (bytes): The initialization vector.
Returns:
bytes: The decrypted and decompressed database.
Raises:
zlib.error: If decompression fails.
ValueError: if the plaintext is not a SQLite database.
"""
cipher = AES.new(main_key, AES.MODE_GCM, iv)
db_compressed = cipher.decrypt(db_ciphertext)
db = zlib.decompress(db_compressed)
if db[0:6].upper() != b"SQLITE":
raise ValueError(
"The plaintext is not a SQLite database. Ensure you are using the correct key."
)
return db
def _decrypt_crypt14(database: bytes, main_key: bytes, max_worker: int = 10) -> bytes:
"""Decrypt a crypt14 database using multithreading for brute-force offset detection.
Args:
database (bytes): The encrypted database.
main_key (bytes): The decryption key.
max_worker (int, optional): The maximum number of threads to use for brute force. Defaults to 10.
Returns:
bytes: The decrypted database.
Raises:
InvalidFileFormatError: If the file is too small.
OffsetNotFoundError: If no valid offsets are found.
"""
if len(database) < 191:
raise InvalidFileFormatError("The crypt14 file must be at least 191 bytes")
# Attempt known offsets first
for offsets in CRYPT14_OFFSETS:
iv = database[offsets["iv"]:offsets["iv"] + 16]
db_ciphertext = database[offsets["db"]:]
try:
return _decrypt_database(db_ciphertext, main_key, iv)
except (zlib.error, ValueError):
pass # Try next offset
print("Common offsets failed. Initiating brute-force with multithreading...")
# Convert brute force generator into a list for parallel processing
offset_combinations = list(brute_force_offset())
def attempt_decrypt(offset_tuple):
"""Attempt decryption with the given offsets."""
start_iv, end_iv, start_db = offset_tuple
iv = database[start_iv:end_iv]
db_ciphertext = database[start_db:]
try:
db = _decrypt_database(db_ciphertext, main_key, iv)
print(
f"The offsets of your IV and database are {start_iv} and "
f"{start_db}, respectively. To include your offsets in the "
"program, please report it by creating an issue on GitHub: "
"https://github.com/KnugiHK/Whatsapp-Chat-Exporter/discussions/47"
"\nShutting down other threads..."
)
return db
except (zlib.error, ValueError):
return None # Decryption failed, move to next
with concurrent.futures.ThreadPoolExecutor(max_worker) as executor:
future_to_offset = {executor.submit(attempt_decrypt, offset): offset for offset in offset_combinations}
try:
for future in concurrent.futures.as_completed(future_to_offset):
result = future.result()
if result is not None:
# Shutdown remaining threads
executor.shutdown(wait=False, cancel_futures=True)
return result
except KeyboardInterrupt:
print("\nBrute force interrupted by user (Ctrl+C). Exiting gracefully...")
executor.shutdown(wait=False, cancel_futures=True)
exit(1)
raise OffsetNotFoundError("Could not find the correct offsets for decryption.")
def _decrypt_crypt12(database: bytes, main_key: bytes) -> bytes:
"""Decrypt a crypt12 database.
Args:
database (bytes): The encrypted database.
main_key (bytes): The decryption key.
Returns:
bytes: The decrypted database.
Raises:
ValueError: If the file format is invalid or the signature mismatches.
"""
if len(database) < 67:
raise InvalidFileFormatError("The crypt12 file must be at least 67 bytes")
t2 = database[3:35]
iv = database[51:67]
db_ciphertext = database[67:-20]
return _decrypt_database(db_ciphertext, main_key, iv)
def _decrypt_crypt15(database: bytes, main_key: bytes, db_type: DbType) -> bytes:
"""Decrypt a crypt15 database.
Args:
database (bytes): The encrypted database.
main_key (bytes): The decryption key.
db_type (DbType): The type of database.
Returns:
bytes: The decrypted database.
Raises:
ValueError: If the file format is invalid or the signature mismatches.
"""
if not support_crypt15:
raise RuntimeError("Crypt15 is not supported")
if len(database) < 131:
raise InvalidFileFormatError("The crypt15 file must be at least 131 bytes")
if db_type == DbType.MESSAGE:
iv = database[8:24]
db_offset = database[0] + 2
elif db_type == DbType.CONTACT:
iv = database[7:23]
db_offset = database[0] + 1
else:
raise ValueError(f"Invalid db_type: {db_type}")
db_ciphertext = database[db_offset:]
return _decrypt_database(db_ciphertext, main_key, iv)
def decrypt_backup(
database: bytes,
key: Union[str, io.IOBase],
output: str = None,
crypt: Crypt = Crypt.CRYPT14,
show_crypt15: bool = False,
db_type: DbType = DbType.MESSAGE,
*,
dry_run: bool = False,
keyfile_stream: bool = False,
max_worker: int = 10
) -> int:
"""
Decrypt the WhatsApp backup database.
Args:
database (bytes): The encrypted database file.
key (str or io.IOBase): The key to decrypt the database.
output (str, optional): The path to save the decrypted database. Defaults to None.
crypt (Crypt, optional): The encryption version of the database. Defaults to Crypt.CRYPT14.
show_crypt15 (bool, optional): Whether to show the HEX key of the crypt15 backup. Defaults to False.
db_type (DbType, optional): The type of database (MESSAGE or CONTACT). Defaults to DbType.MESSAGE.
dry_run (bool, optional): Whether to perform a dry run. Defaults to False.
keyfile_stream (bool, optional): Whether the key is a key stream. Defaults to False.
Returns:
int: The status code of the decryption process (0 for success).
Raises:
ValueError: If the key is invalid or output file not provided when dry_run is False.
DecryptionError: for errors during decryption
RuntimeError: for dependency errors
"""
if not support_backup:
raise RuntimeError("Dependencies for backup decryption are not available.")
if not dry_run and output is None:
raise ValueError(
"The path to the decrypted database must be specified unless dry_run is true."
)
if isinstance(key, io.IOBase):
key = key.read()
if crypt is not Crypt.CRYPT15 and len(key) != 158:
raise InvalidKeyError("The key file must be 158 bytes")
#signature check, this is check is used in crypt 12 and 14
if crypt != Crypt.CRYPT15:
t1 = key[30:62]
if t1 != database[15:47] and crypt == Crypt.CRYPT14:
raise ValueError("The signature of key file and backup file mismatch")
if t1 != database[3:35] and crypt == Crypt.CRYPT12:
raise ValueError("The signature of key file and backup file mismatch")
if crypt == Crypt.CRYPT15:
if keyfile_stream:
main_key, hex_key = _extract_enc_key(key)
else:
main_key, hex_key = _derive_main_enc_key(key)
if show_crypt15:
hex_key_str = ' '.join([hex_key.hex()[c:c+4] for c in range(0, len(hex_key.hex()), 4)])
print(f"The HEX key of the crypt15 backup is: {hex_key_str}")
else:
main_key = key[126:]
try:
if crypt == Crypt.CRYPT14:
db = _decrypt_crypt14(database, main_key, max_worker)
elif crypt == Crypt.CRYPT12:
db = _decrypt_crypt12(database, main_key)
elif crypt == Crypt.CRYPT15:
db = _decrypt_crypt15(database, main_key, db_type)
else:
raise ValueError(f"Unsupported crypt type: {crypt}")
except (InvalidFileFormatError, OffsetNotFoundError, ValueError) as e:
raise DecryptionError(f"Decryption failed: {e}") from e
if not dry_run:
with open(output, "wb") as f:
f.write(db)
return 0

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,292 @@
#################################################################################
# Copyright (C) 2009-2011 Vladimir "Farcaller" Pouzanov <farcaller@gmail.com> #
# #
# Permission is hereby granted, free of charge, to any person obtaining a copy #
# of this software and associated documentation files (the "Software"), to deal #
# in the Software without restriction, including without limitation the rights #
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell #
# copies of the Software, and to permit persons to whom the Software is #
# furnished to do so, subject to the following conditions: #
# #
# The above copyright notice and this permission notice shall be included in #
# all copies or substantial portions of the Software. #
# #
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR #
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, #
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE #
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER #
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, #
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN #
# THE SOFTWARE. #
#################################################################################
import struct
import codecs
from datetime import datetime, timedelta
class BPListWriter(object):
def __init__(self, objects):
self.bplist = ""
self.objects = objects
def binary(self):
'''binary -> string
Generates bplist
'''
self.data = 'bplist00'
# TODO: flatten objects and count max length size
# TODO: write objects and save offsets
# TODO: write offsets
# TODO: write metadata
return self.data
def write(self, filename):
'''
Writes bplist to file
'''
if self.bplist != "":
pass
# TODO: save self.bplist to file
else:
raise Exception('BPlist not yet generated')
class BPListReader(object):
def __init__(self, s):
self.data = s
self.objects = []
self.resolved = {}
def __unpackIntStruct(self, sz, s):
'''__unpackIntStruct(size, string) -> int
Unpacks the integer of given size (1, 2 or 4 bytes) from string
'''
if sz == 1:
ot = '!B'
elif sz == 2:
ot = '!H'
elif sz == 4:
ot = '!I'
elif sz == 8:
ot = '!Q'
else:
raise Exception('int unpack size '+str(sz)+' unsupported')
return struct.unpack(ot, s)[0]
def __unpackInt(self, offset):
'''__unpackInt(offset) -> int
Unpacks int field from plist at given offset
'''
return self.__unpackIntMeta(offset)[1]
def __unpackIntMeta(self, offset):
'''__unpackIntMeta(offset) -> (size, int)
Unpacks int field from plist at given offset and returns its size and value
'''
obj_header = self.data[offset]
obj_type, obj_info = (obj_header & 0xF0), (obj_header & 0x0F)
int_sz = 2**obj_info
return int_sz, self.__unpackIntStruct(int_sz, self.data[offset+1:offset+1+int_sz])
def __resolveIntSize(self, obj_info, offset):
'''__resolveIntSize(obj_info, offset) -> (count, offset)
Calculates count of objref* array entries and returns count and offset to first element
'''
if obj_info == 0x0F:
ofs, obj_count = self.__unpackIntMeta(offset+1)
objref = offset+2+ofs
else:
obj_count = obj_info
objref = offset+1
return obj_count, objref
def __unpackFloatStruct(self, sz, s):
'''__unpackFloatStruct(size, string) -> float
Unpacks the float of given size (4 or 8 bytes) from string
'''
if sz == 4:
ot = '!f'
elif sz == 8:
ot = '!d'
else:
raise Exception('float unpack size '+str(sz)+' unsupported')
return struct.unpack(ot, s)[0]
def __unpackFloat(self, offset):
'''__unpackFloat(offset) -> float
Unpacks float field from plist at given offset
'''
obj_header = self.data[offset]
obj_type, obj_info = (obj_header & 0xF0), (obj_header & 0x0F)
int_sz = 2**obj_info
return int_sz, self.__unpackFloatStruct(int_sz, self.data[offset+1:offset+1+int_sz])
def __unpackDate(self, offset):
td = int(struct.unpack(">d", self.data[offset+1:offset+9])[0])
return datetime(year=2001,month=1,day=1) + timedelta(seconds=td)
def __unpackItem(self, offset):
'''__unpackItem(offset)
Unpacks and returns an item from plist
'''
obj_header = self.data[offset]
obj_type, obj_info = (obj_header & 0xF0), (obj_header & 0x0F)
if obj_type == 0x00:
if obj_info == 0x00: # null 0000 0000
return None
elif obj_info == 0x08: # bool 0000 1000 // false
return False
elif obj_info == 0x09: # bool 0000 1001 // true
return True
elif obj_info == 0x0F: # fill 0000 1111 // fill byte
raise Exception("0x0F Not Implemented") # this is really pad byte, FIXME
else:
raise Exception('unpack item type '+str(obj_header)+' at '+str(offset)+ 'failed')
elif obj_type == 0x10: # int 0001 nnnn ... // # of bytes is 2^nnnn, big-endian bytes
return self.__unpackInt(offset)
elif obj_type == 0x20: # real 0010 nnnn ... // # of bytes is 2^nnnn, big-endian bytes
return self.__unpackFloat(offset)
elif obj_type == 0x30: # date 0011 0011 ... // 8 byte float follows, big-endian bytes
return self.__unpackDate(offset)
elif obj_type == 0x40: # data 0100 nnnn [int] ... // nnnn is number of bytes unless 1111 then int count follows, followed by bytes
obj_count, objref = self.__resolveIntSize(obj_info, offset)
return self.data[objref:objref+obj_count] # XXX: we return data as str
elif obj_type == 0x50: # string 0101 nnnn [int] ... // ASCII string, nnnn is # of chars, else 1111 then int count, then bytes
obj_count, objref = self.__resolveIntSize(obj_info, offset)
return self.data[objref:objref+obj_count]
elif obj_type == 0x60: # string 0110 nnnn [int] ... // Unicode string, nnnn is # of chars, else 1111 then int count, then big-endian 2-byte uint16_t
obj_count, objref = self.__resolveIntSize(obj_info, offset)
return self.data[objref:objref+obj_count*2].decode('utf-16be')
elif obj_type == 0x80: # uid 1000 nnnn ... // nnnn+1 is # of bytes
# FIXME: Accept as a string for now
obj_count, objref = self.__resolveIntSize(obj_info, offset)
return self.data[objref:objref+obj_count]
elif obj_type == 0xA0: # array 1010 nnnn [int] objref* // nnnn is count, unless '1111', then int count follows
obj_count, objref = self.__resolveIntSize(obj_info, offset)
arr = []
for i in range(obj_count):
arr.append(self.__unpackIntStruct(self.object_ref_size, self.data[objref+i*self.object_ref_size:objref+i*self.object_ref_size+self.object_ref_size]))
return arr
elif obj_type == 0xC0: # set 1100 nnnn [int] objref* // nnnn is count, unless '1111', then int count follows
# XXX: not serializable via apple implementation
raise Exception("0xC0 Not Implemented") # FIXME: implement
elif obj_type == 0xD0: # dict 1101 nnnn [int] keyref* objref* // nnnn is count, unless '1111', then int count follows
obj_count, objref = self.__resolveIntSize(obj_info, offset)
keys = []
for i in range(obj_count):
keys.append(self.__unpackIntStruct(self.object_ref_size, self.data[objref+i*self.object_ref_size:objref+i*self.object_ref_size+self.object_ref_size]))
values = []
objref += obj_count*self.object_ref_size
for i in range(obj_count):
values.append(self.__unpackIntStruct(self.object_ref_size, self.data[objref+i*self.object_ref_size:objref+i*self.object_ref_size+self.object_ref_size]))
dic = {}
for i in range(obj_count):
dic[keys[i]] = values[i]
return dic
else:
raise Exception('don\'t know how to unpack obj type '+hex(obj_type)+' at '+str(offset))
def __resolveObject(self, idx):
try:
return self.resolved[idx]
except KeyError:
obj = self.objects[idx]
if type(obj) == list:
newArr = []
for i in obj:
newArr.append(self.__resolveObject(i))
self.resolved[idx] = newArr
return newArr
if type(obj) == dict:
newDic = {}
for k,v in obj.items():
key_resolved = self.__resolveObject(k)
if isinstance(key_resolved, str):
rk = key_resolved
else:
rk = codecs.decode(key_resolved, "utf-8")
rv = self.__resolveObject(v)
newDic[rk] = rv
self.resolved[idx] = newDic
return newDic
else:
self.resolved[idx] = obj
return obj
def parse(self):
# read header
if self.data[:8] != b'bplist00':
raise Exception('Bad magic')
# read trailer
self.offset_size, self.object_ref_size, self.number_of_objects, self.top_object, self.table_offset = struct.unpack('!6xBB4xI4xI4xI', self.data[-32:])
#print "** plist offset_size:",self.offset_size,"objref_size:",self.object_ref_size,"num_objs:",self.number_of_objects,"top:",self.top_object,"table_ofs:",self.table_offset
# read offset table
self.offset_table = self.data[self.table_offset:-32]
self.offsets = []
ot = self.offset_table
for i in range(self.number_of_objects):
offset_entry = ot[:self.offset_size]
ot = ot[self.offset_size:]
self.offsets.append(self.__unpackIntStruct(self.offset_size, offset_entry))
#print "** plist offsets:",self.offsets
# read object table
self.objects = []
k = 0
for i in self.offsets:
obj = self.__unpackItem(i)
#print "** plist unpacked",k,type(obj),obj,"at",i
k += 1
self.objects.append(obj)
# rebuild object tree
#for i in range(len(self.objects)):
# self.__resolveObject(i)
# return root object
return self.__resolveObject(self.top_object)
@classmethod
def plistWithString(cls, s):
parser = cls(s)
return parser.parse()
# helpers for testing
def plist(obj):
from Foundation import NSPropertyListSerialization, NSPropertyListBinaryFormat_v1_0
b = NSPropertyListSerialization.dataWithPropertyList_format_options_error_(obj, NSPropertyListBinaryFormat_v1_0, 0, None)
return str(b.bytes())
def unplist(s):
from Foundation import NSData, NSPropertyListSerialization
d = NSData.dataWithBytes_length_(s, len(s))
return NSPropertyListSerialization.propertyListWithData_options_format_error_(d, 0, None, None)
if __name__ == "__main__":
import os
import sys
import json
file_path = sys.argv[1]
with open(file_path, "rb") as fp:
data = fp.read()
out = BPListReader(data).parse()
with open(file_path + ".json", "w") as fp:
json.dump(out, indent=4)

View File

@@ -0,0 +1,313 @@
import os
from datetime import datetime, tzinfo, timedelta
from typing import MutableMapping, Union, Optional, Dict, Any
class Timing:
"""
Handles timestamp formatting with timezone support.
"""
def __init__(self, timezone_offset: Optional[int]) -> None:
"""
Initialize Timing object.
Args:
timezone_offset (Optional[int]): Hours offset from UTC
"""
self.timezone_offset = timezone_offset
def format_timestamp(self, timestamp: Optional[Union[int, float]], format: str) -> Optional[str]:
"""
Format a timestamp with the specified format string.
Args:
timestamp (Optional[Union[int, float]]): Unix timestamp to format
format (str): strftime format string
Returns:
Optional[str]: Formatted timestamp string, or None if timestamp is None
"""
if timestamp:
timestamp = timestamp / 1000 if timestamp > 9999999999 else timestamp
return datetime.fromtimestamp(timestamp, TimeZone(self.timezone_offset)).strftime(format)
return None
class TimeZone(tzinfo):
"""
Custom timezone class with fixed offset.
"""
def __init__(self, offset: int) -> None:
"""
Initialize TimeZone object.
Args:
offset (int): Hours offset from UTC
"""
self.offset = offset
def utcoffset(self, dt: Optional[datetime]) -> timedelta:
"""Get UTC offset."""
return timedelta(hours=self.offset)
def dst(self, dt: Optional[datetime]) -> timedelta:
"""Get DST offset (always 0)."""
return timedelta(0)
class ChatCollection(MutableMapping):
"""
A collection of chats that provides dictionary-like access with additional chat management methods.
Inherits from MutableMapping to implement a custom dictionary-like behavior.
"""
def __init__(self) -> None:
"""Initialize an empty chat collection."""
self._chats: Dict[str, ChatStore] = {}
def __getitem__(self, key: str) -> 'ChatStore':
"""Get a chat by its ID. Required for dict-like access."""
return self._chats[key]
def __setitem__(self, key: str, value: 'ChatStore') -> None:
"""Set a chat by its ID. Required for dict-like access."""
if not isinstance(value, ChatStore):
raise TypeError("Value must be a ChatStore object")
self._chats[key] = value
def __delitem__(self, key: str) -> None:
"""Delete a chat by its ID. Required for dict-like access."""
del self._chats[key]
def __iter__(self):
"""Iterate over chat IDs. Required for dict-like access."""
return iter(self._chats)
def __len__(self) -> int:
"""Get number of chats. Required for dict-like access."""
return len(self._chats)
def get_chat(self, chat_id: str) -> Optional['ChatStore']:
"""
Get a chat by its ID.
Args:
chat_id (str): The ID of the chat to retrieve
Returns:
Optional['ChatStore']: The chat if found, None otherwise
"""
return self._chats.get(chat_id)
def add_chat(self, chat_id: str, chat: 'ChatStore') -> None:
"""
Add a new chat to the collection.
Args:
chat_id (str): The ID for the chat
chat (ChatStore): The chat to add
Raises:
TypeError: If chat is not a ChatStore object
"""
if not isinstance(chat, ChatStore):
raise TypeError("Chat must be a ChatStore object")
self._chats[chat_id] = chat
return self._chats[chat_id]
def remove_chat(self, chat_id: str) -> None:
"""
Remove a chat from the collection.
Args:
chat_id (str): The ID of the chat to remove
"""
if chat_id in self._chats:
del self._chats[chat_id]
def items(self):
"""Get chat items (id, chat) pairs."""
return self._chats.items()
def values(self):
"""Get all chats."""
return self._chats.values()
def keys(self):
"""Get all chat IDs."""
return self._chats.keys()
def to_dict(self) -> Dict[str, Any]:
"""
Convert the collection to a dictionary.
Returns:
Dict[str, Any]: Dictionary representation of all chats
"""
return {chat_id: chat.to_json() for chat_id, chat in self._chats.items()}
class ChatStore:
"""
Stores chat information and messages.
"""
def __init__(self, type: str, name: Optional[str] = None, media: Optional[str] = None) -> None:
"""
Initialize ChatStore object.
Args:
type (str): Device type (IOS or ANDROID)
name (Optional[str]): Chat name
media (Optional[str]): Path to media folder
Raises:
TypeError: If name is not a string or None
"""
if name is not None and not isinstance(name, str):
raise TypeError("Name must be a string or None")
self.name = name
self._messages: Dict[str, 'Message'] = {}
self.type = type
if media is not None:
from Whatsapp_Chat_Exporter.utility import Device
if self.type == Device.IOS:
self.my_avatar = os.path.join(media, "Media/Profile/Photo.jpg")
elif self.type == Device.ANDROID:
self.my_avatar = None # TODO: Add Android support
else:
self.my_avatar = None
else:
self.my_avatar = None
self.their_avatar = None
self.their_avatar_thumb = None
self.status = None
self.media_base = ""
def __len__(self) -> int:
"""Get number of chats. Required for dict-like access."""
return len(self._messages)
def add_message(self, id: str, message: 'Message') -> None:
"""Add a message to the chat store."""
if not isinstance(message, Message):
raise TypeError("message must be a Message object")
self._messages[id] = message
def get_message(self, id: str) -> 'Message':
"""Get a message from the chat store."""
return self._messages.get(id)
def delete_message(self, id: str) -> None:
"""Delete a message from the chat store."""
if id in self._messages:
del self._messages[id]
def to_json(self) -> Dict[str, Any]:
"""Convert chat store to JSON-serializable dict."""
return {
'name': self.name,
'type': self.type,
'my_avatar': self.my_avatar,
'their_avatar': self.their_avatar,
'their_avatar_thumb': self.their_avatar_thumb,
'status': self.status,
'messages': {id: msg.to_json() for id, msg in self._messages.items()}
}
def get_last_message(self) -> 'Message':
"""Get the most recent message in the chat."""
return tuple(self._messages.values())[-1]
def items(self):
"""Get message items pairs."""
return self._messages.items()
def values(self):
"""Get all messages in the chat."""
return self._messages.values()
def keys(self):
"""Get all message keys in the chat."""
return self._messages.keys()
class Message:
"""
Represents a single message in a chat.
"""
def __init__(
self,
*,
from_me: Union[bool, int],
timestamp: int,
time: Union[int, float, str],
key_id: int,
received_timestamp: int,
read_timestamp: int,
timezone_offset: int = 0,
message_type: Optional[int] = None
) -> None:
"""
Initialize Message object.
Args:
from_me (Union[bool, int]): Whether message was sent by the user
timestamp (int): Message timestamp
time (Union[int, float, str]): Message time
key_id (int): Message unique identifier
received_timestamp (int): When message was received
read_timestamp (int): When message was read
timezone_offset (int, optional): Hours offset from UTC. Defaults to 0
message_type (Optional[int], optional): Type of message. Defaults to None
Raises:
TypeError: If time is not a string or number
"""
self.from_me = bool(from_me)
self.timestamp = timestamp / 1000 if timestamp > 9999999999 else timestamp
timing = Timing(timezone_offset)
if isinstance(time, (int, float)):
self.time = timing.format_timestamp(self.timestamp, "%H:%M")
elif isinstance(time, str):
self.time = time
else:
raise TypeError("Time must be a string or number")
self.media = False
self.key_id = key_id
self.meta = False
self.data = None
self.sender = None
self.safe = False
self.mime = None
self.message_type = message_type,
self.received_timestamp = timing.format_timestamp(received_timestamp, "%Y/%m/%d %H:%M")
self.read_timestamp = timing.format_timestamp(read_timestamp, "%Y/%m/%d %H:%M")
# Extra attributes
self.reply = None
self.quoted_data = None
self.caption = None
self.thumb = None # Android specific
self.sticker = False
def to_json(self) -> Dict[str, Any]:
"""Convert message to JSON-serializable dict."""
return {
'from_me': self.from_me,
'timestamp': self.timestamp,
'time': self.time,
'media': self.media,
'key_id': self.key_id,
'meta': self.meta,
'data': self.data,
'sender': self.sender,
'safe': self.safe,
'mime': self.mime,
'reply': self.reply,
'quoted_data': self.quoted_data,
'caption': self.caption,
'thumb': self.thumb,
'sticker': self.sticker
}

View File

@@ -0,0 +1,181 @@
#!/usr/bin/python3
import os
from datetime import datetime
from mimetypes import MimeTypes
from Whatsapp_Chat_Exporter.data_model import ChatStore, Message
from Whatsapp_Chat_Exporter.utility import Device
def messages(path, data, assume_first_as_me=False):
"""
Extracts messages from an exported WhatsApp chat file.
Args:
path: Path to the exported chat file
data: Data container object to store the parsed chat
assume_first_as_me: If True, assumes the first message is sent from the user without asking
Returns:
Updated data container with extracted messages
"""
# Create a new chat in the data container
chat = data.add_chat("ExportedChat", ChatStore(Device.EXPORTED))
you = "" # Will store the username of the current user
user_identification_done = False # Flag to track if user identification has been done
# First pass: count total lines for progress reporting
with open(path, "r", encoding="utf8") as file:
total_row_number = sum(1 for _ in file)
# Second pass: process the messages
with open(path, "r", encoding="utf8") as file:
for index, line in enumerate(file):
you, user_identification_done = process_line(
line, index, chat, path, you,
assume_first_as_me, user_identification_done
)
# Show progress
if index % 1000 == 0:
print(f"Processing messages & media...({index}/{total_row_number})", end="\r")
print(f"Processing messages & media...({total_row_number}/{total_row_number})")
return data
def process_line(line, index, chat, file_path, you, assume_first_as_me, user_identification_done):
"""
Process a single line from the chat file
Returns:
Tuple of (updated_you_value, updated_user_identification_done_flag)
"""
parts = line.split(" - ", 1)
# Check if this is a new message (has timestamp format)
if len(parts) > 1:
time = parts[0]
you, user_identification_done = process_new_message(
time, parts[1], index, chat, you, file_path,
assume_first_as_me, user_identification_done
)
else:
# This is a continuation of the previous message
process_message_continuation(line, index, chat)
return you, user_identification_done
def process_new_message(time, content, index, chat, you, file_path,
assume_first_as_me, user_identification_done):
"""
Process a line that contains a new message
Returns:
Tuple of (updated_you_value, updated_user_identification_done_flag)
"""
# Create a new message
msg = Message(
from_me=False, # Will be updated later if needed
timestamp=datetime.strptime(time, "%d/%m/%Y, %H:%M").timestamp(),
time=time.split(", ")[1].strip(),
key_id=index,
received_timestamp=None,
read_timestamp=None
)
# Check if this is a system message (no name:message format)
if ":" not in content:
msg.data = content
msg.meta = True
else:
# Process user message
name, message = content.strip().split(":", 1)
# Handle user identification
if you == "":
if chat.name is None:
# First sender identification
if not user_identification_done:
if not assume_first_as_me:
# Ask only once if this is the user
you = prompt_for_user_identification(name)
user_identification_done = True
else:
you = name
user_identification_done = True
else:
# If we know the chat name, anyone else must be "you"
if name != chat.name:
you = name
# Set the chat name if needed
if chat.name is None and name != you:
chat.name = name
# Determine if this message is from the current user
msg.from_me = (name == you)
# Process message content
process_message_content(msg, message, file_path)
chat.add_message(index, msg)
return you, user_identification_done
def process_message_content(msg, message, file_path):
"""Process and set the content of a message based on its type"""
if "<Media omitted>" in message:
msg.data = "The media is omitted in the chat"
msg.mime = "media"
msg.meta = True
elif "(file attached)" in message:
process_attached_file(msg, message, file_path)
else:
msg.data = message.replace("\r\n", "<br>").replace("\n", "<br>")
def process_attached_file(msg, message, file_path):
"""Process an attached file in a message"""
mime = MimeTypes()
msg.media = True
# Extract file path and check if it exists
file_name = message.split("(file attached)")[0].strip()
attached_file_path = os.path.join(os.path.dirname(file_path), file_name)
if os.path.isfile(attached_file_path):
msg.data = attached_file_path
guess = mime.guess_type(attached_file_path)[0]
msg.mime = guess if guess is not None else "application/octet-stream"
else:
msg.data = "The media is missing"
msg.mime = "media"
msg.meta = True
def process_message_continuation(line, index, chat):
"""Process a line that continues a previous message"""
# Find the previous message
lookback = index - 1
while lookback not in chat.keys():
lookback -= 1
msg = chat.get_message(lookback)
# Add the continuation line to the message
if msg.media:
msg.caption = line.strip()
else:
msg.data += "<br>" + line.strip()
def prompt_for_user_identification(name):
"""Ask the user if the given name is their username"""
while True:
ans = input(f"Is '{name}' you? (Y/N)").lower()
if ans == "y":
return name
elif ans == "n":
return ""

View File

@@ -1,429 +0,0 @@
#!/usr/bin/python3
import sqlite3
import json
import jinja2
import os
import requests
import shutil
import re
import pkgutil
from pathlib import Path
from bleach import clean as sanitize
from markupsafe import Markup
from datetime import datetime
from mimetypes import MimeTypes
try:
import zlib
from Crypto.Cipher import AES
except ModuleNotFoundError:
support_backup = False
else:
support_backup = True
def sanitize_except(html):
return Markup(sanitize(html, tags=["br"]))
def determine_day(last, current):
last = datetime.fromtimestamp(last).date()
current = datetime.fromtimestamp(current).date()
if last == current:
return None
else:
return current
def decrypt_backup(database, key, output, crypt14=True):
if not support_backup:
return False
if len(key) != 158:
raise ValueError("The key file must be 158 bytes")
t1 = key[30:62]
if crypt14:
if len(database) < 191:
raise ValueError("The crypt14 file must be at least 191 bytes")
t2 = database[15:47]
iv = database[67:83]
db_ciphertext = database[191:]
else:
if len(database) < 67:
raise ValueError("The crypt12 file must be at least 67 bytes")
t2 = database[3:35]
iv = database[51:67]
db_ciphertext = database[67:-20]
if t1 != t2:
raise ValueError("The signature of key file and backup file mismatch")
main_key = key[126:]
cipher = AES.new(main_key, AES.MODE_GCM, iv)
db_compressed = cipher.decrypt(db_ciphertext)
db = zlib.decompress(db_compressed)
if db[0:6].upper() == b"SQLITE":
with open(output, "wb") as f:
f.write(db)
return True
else:
raise ValueError("The plaintext is not a SQLite database. Did you use the key to encrypt something...")
def contacts(db, data):
# Get contacts
c = db.cursor()
c.execute("""SELECT count() FROM wa_contacts""")
total_row_number = c.fetchone()[0]
print(f"Gathering contacts...({total_row_number})")
c.execute("""SELECT jid, display_name FROM wa_contacts; """)
row = c.fetchone()
while row is not None:
data[row[0]] = {"name": row[1], "messages": {}}
row = c.fetchone()
def messages(db, data):
# Get message history
c = db.cursor()
c.execute("""SELECT count() FROM messages""")
total_row_number = c.fetchone()[0]
print(f"Gathering messages...(0/{total_row_number})", end="\r")
phone_number_re = re.compile(r"[0-9]+@s.whatsapp.net")
c.execute("""SELECT messages.key_remote_jid,
messages._id,
messages.key_from_me,
messages.timestamp,
messages.data,
messages.status,
messages.edit_version,
messages.thumb_image,
messages.remote_resource,
messages.media_wa_type,
messages.latitude,
messages.longitude,
messages_quotes.key_id as quoted,
messages.key_id,
messages_quotes.data,
messages.media_caption
FROM messages
LEFT JOIN messages_quotes
ON messages.quoted_row_id = messages_quotes._id;""")
i = 0
content = c.fetchone()
while content is not None:
if content[0] not in data:
data[content[0]] = {"name": None, "messages": {}}
data[content[0]]["messages"][content[1]] = {
"from_me": bool(content[2]),
"timestamp": content[3]/1000,
"time": datetime.fromtimestamp(content[3]/1000).strftime("%H:%M"),
"media": False,
"key_id": content[13],
"meta": False,
"data": None
}
if "-" in content[0] and content[2] == 0:
name = None
if content[8] in data:
name = data[content[8]]["name"]
if "@" in content[8]:
fallback = content[8].split('@')[0]
else:
fallback = None
else:
fallback = None
data[content[0]]["messages"][content[1]]["sender"] = name or fallback
else:
data[content[0]]["messages"][content[1]]["sender"] = None
if content[12] is not None:
data[content[0]]["messages"][content[1]]["reply"] = content[12]
data[content[0]]["messages"][content[1]]["quoted_data"] = content[14]
else:
data[content[0]]["messages"][content[1]]["reply"] = None
if content[15] is not None:
data[content[0]]["messages"][content[1]]["caption"] = content[15]
else:
data[content[0]]["messages"][content[1]]["caption"] = None
if content[5] == 6:
if "-" in content[0]:
# Is Group
if content[4] is not None:
try:
int(content[4])
except ValueError:
msg = f"The group name changed to {content[4]}"
data[content[0]]["messages"][content[1]]["data"] = msg
data[content[0]]["messages"][content[1]]["meta"] = True
else:
del data[content[0]]["messages"][content[1]]
else:
thumb_image = content[7]
if thumb_image is not None:
if b"\x00\x00\x01\x74\x00\x1A" in thumb_image:
# Add user
added = phone_number_re.search(
thumb_image.decode("unicode_escape"))[0]
if added in data:
name_right = data[added]["name"]
else:
name_right = added.split('@')[0]
if content[8] is not None:
if content[8] in data:
name_left = data[content[8]]["name"]
else:
name_left = content[8].split('@')[0]
msg = f"{name_left} added {name_right or 'You'}"
else:
msg = f"Added {name_right or 'You'}"
elif b"\xac\xed\x00\x05\x74\x00" in thumb_image:
# Changed number
original = content[8].split('@')[0]
changed = thumb_image[7:].decode().split('@')[0]
msg = f"{original} changed to {changed}"
data[content[0]]["messages"][content[1]]["data"] = msg
data[content[0]]["messages"][content[1]]["meta"] = True
else:
if content[4] is None:
del data[content[0]]["messages"][content[1]]
else:
# Private chat
if content[4] is None and content[7] is None:
del data[content[0]]["messages"][content[1]]
else:
if content[2] == 1:
if content[5] == 5 and content[6] == 7:
msg = "Message deleted"
data[content[0]]["messages"][content[1]]["meta"] = True
else:
if content[9] == "5":
msg = f"Location shared: {content[10], content[11]}"
data[content[0]]["messages"][content[1]]["meta"] = True
else:
msg = content[4]
if msg is not None:
if "\r\n" in msg:
msg = msg.replace("\r\n", "<br>")
if "\n" in msg:
msg = msg.replace("\n", "<br>")
else:
if content[5] == 0 and content[6] == 7:
msg = "Message deleted"
data[content[0]]["messages"][content[1]]["meta"] = True
else:
if content[9] == "5":
msg = f"Location shared: {content[10], content[11]}"
data[content[0]]["messages"][content[1]]["meta"] = True
else:
msg = content[4]
if msg is not None:
if "\r\n" in msg:
msg = msg.replace("\r\n", "<br>")
if "\n" in msg:
msg = msg.replace("\n", "<br>")
data[content[0]]["messages"][content[1]]["data"] = msg
i += 1
if i % 1000 == 0:
print(f"Gathering messages...({i}/{total_row_number})", end="\r")
content = c.fetchone()
print(f"Gathering messages...({total_row_number}/{total_row_number})", end="\r")
def media(db, data, media_folder):
# Get media
c = db.cursor()
c.execute("""SELECT count() FROM message_media""")
total_row_number = c.fetchone()[0]
print(f"\nGathering media...(0/{total_row_number})", end="\r")
i = 0
c.execute("""SELECT messages.key_remote_jid,
message_row_id,
file_path,
message_url,
mime_type,
media_key
FROM message_media
INNER JOIN messages
ON message_media.message_row_id = messages._id
ORDER BY messages.key_remote_jid ASC""")
content = c.fetchone()
mime = MimeTypes()
while content is not None:
file_path = f"{media_folder}/{content[2]}"
data[content[0]]["messages"][content[1]]["media"] = True
if os.path.isfile(file_path):
data[content[0]]["messages"][content[1]]["data"] = file_path
if content[4] is None:
guess = mime.guess_type(file_path)[0]
if guess is not None:
data[content[0]]["messages"][content[1]]["mime"] = guess
else:
data[content[0]]["messages"][content[1]]["mime"] = "data/data"
else:
data[content[0]]["messages"][content[1]]["mime"] = content[4]
else:
# if "https://mmg" in content[4]:
# try:
# r = requests.get(content[3])
# if r.status_code != 200:
# raise RuntimeError()
# except:
# data[content[0]]["messages"][content[1]]["data"] = "{The media is missing}"
# data[content[0]]["messages"][content[1]]["media"] = True
# data[content[0]]["messages"][content[1]]["mime"] = "media"
# else:
data[content[0]]["messages"][content[1]]["data"] = "The media is missing"
data[content[0]]["messages"][content[1]]["mime"] = "media"
data[content[0]]["messages"][content[1]]["meta"] = True
i += 1
if i % 100 == 0:
print(f"Gathering media...({i}/{total_row_number})", end="\r")
content = c.fetchone()
print(
f"Gathering media...({total_row_number}/{total_row_number})", end="\r")
def vcard(db, data):
c = db.cursor()
c.execute("""SELECT message_row_id,
messages.key_remote_jid,
vcard,
messages.media_name
FROM messages_vcards
INNER JOIN messages
ON messages_vcards.message_row_id = messages._id
ORDER BY messages.key_remote_jid ASC;""")
rows = c.fetchall()
total_row_number = len(rows)
print(f"\nGathering vCards...(0/{total_row_number})", end="\r")
base = "WhatsApp/vCards"
if not os.path.isdir(base):
Path(base).mkdir(parents=True, exist_ok=True)
for index, row in enumerate(rows):
file_name = "".join(x for x in row[3] if x.isalnum())
file_path = f"{base}/{file_name}.vcf"
if not os.path.isfile(file_path):
with open(file_path, "w", encoding="utf-8") as f:
f.write(row[2])
data[row[1]]["messages"][row[0]]["data"] = row[3] + \
"The vCard file cannot be displayed here, " \
f"however it should be located at {file_path}"
data[row[1]]["messages"][row[0]]["mime"] = "text/x-vcard"
data[row[1]]["messages"][row[0]]["meta"] = True
print(f"Gathering vCards...({index + 1}/{total_row_number})", end="\r")
def create_html(data, output_folder, template=None):
if template is None:
template_dir = os.path.dirname(__file__)
template_file = "whatsapp.html"
else:
template_dir = os.path.dirname(template)
template_file = os.path.basename(template)
templateLoader = jinja2.FileSystemLoader(searchpath=template_dir)
templateEnv = jinja2.Environment(loader=templateLoader)
templateEnv.globals.update(determine_day=determine_day)
templateEnv.filters['sanitize_except'] = sanitize_except
template = templateEnv.get_template(template_file)
total_row_number = len(data)
print(f"\nCreating HTML...(0/{total_row_number})", end="\r")
if not os.path.isdir(output_folder):
os.mkdir(output_folder)
for current, contact in enumerate(data):
if len(data[contact]["messages"]) == 0:
continue
phone_number = contact.split('@')[0]
if "-" in contact:
file_name = ""
else:
file_name = phone_number
if data[contact]["name"] is not None:
if file_name != "":
file_name += "-"
file_name += data[contact]["name"].replace("/", "-")
name = data[contact]["name"]
else:
name = phone_number
safe_file_name = ''
safe_file_name = "".join(x for x in file_name if x.isalnum() or x in "- ")
with open(f"{output_folder}/{safe_file_name}.html", "w", encoding="utf-8") as f:
f.write(
template.render(
name=name,
msgs=data[contact]["messages"].values(),
my_avatar=None,
their_avatar=f"WhatsApp/Avatars/{contact}.j"
)
)
if current % 10 == 0:
print(f"Creating HTML...({current}/{total_row_number})", end="\r")
print(f"Creating HTML...({total_row_number}/{total_row_number})", end="\r")
if __name__ == "__main__":
from optparse import OptionParser
parser = OptionParser()
parser.add_option(
"-w",
"--wa",
dest="wa",
default="wa.db",
help="Path to contact database")
parser.add_option(
"-m",
"--media",
dest="media",
default="WhatsApp",
help="Path to WhatsApp media folder"
)
# parser.add_option(
# "-t",
# "--template",
# dest="html",
# default="wa.db",
# help="Path to HTML template")
(options, args) = parser.parse_args()
msg_db = "msgstore.db"
output_folder = "temp"
contact_db = options.wa
media_folder = options.media
if len(args) == 1:
msg_db = args[0]
elif len(args) == 2:
msg_db = args[0]
output_folder = args[1]
data = {}
if os.path.isfile(contact_db):
with sqlite3.connect(contact_db) as db:
contacts(db, data)
if os.path.isfile(msg_db):
with sqlite3.connect(msg_db) as db:
messages(db, data)
media(db, data, media_folder)
vcard(db, data)
create_html(data, output_folder)
if not os.path.isdir(f"{output_folder}/WhatsApp"):
shutil.move(media_folder, f"{output_folder}/")
with open("result.json", "w") as f:
data = json.dumps(data)
print(f"\nWriting JSON file...({int(len(data)/1024/1024)}MB)")
f.write(data)
print("Everything is done!")

View File

@@ -1,337 +0,0 @@
#!/usr/bin/python3
import sqlite3
import json
import jinja2
import os
import requests
import shutil
import pkgutil
from pathlib import Path
from bleach import clean as sanitize
from markupsafe import Markup
from datetime import datetime
from mimetypes import MimeTypes
APPLE_TIME = datetime.timestamp(datetime(2001, 1, 1))
def sanitize_except(html):
return Markup(sanitize(html, tags=["br"]))
def determine_day(last, current):
last = datetime.fromtimestamp(last).date()
current = datetime.fromtimestamp(current).date()
if last == current:
return None
else:
return current
def messages(db, data):
c = db.cursor()
# Get contacts
c.execute("""SELECT count() FROM ZWACHATSESSION""")
total_row_number = c.fetchone()[0]
print(f"Gathering contacts...({total_row_number})")
c.execute("""SELECT ZCONTACTJID, ZPARTNERNAME FROM ZWACHATSESSION; """)
row = c.fetchone()
while row is not None:
data[row[0]] = {"name": row[1], "messages": {}}
row = c.fetchone()
# Get message history
c.execute("""SELECT count() FROM ZWAMESSAGE""")
total_row_number = c.fetchone()[0]
print(f"Gathering messages...(0/{total_row_number})", end="\r")
c.execute("""SELECT COALESCE(ZFROMJID, ZTOJID),
ZWAMESSAGE.Z_PK,
ZISFROMME,
ZMESSAGEDATE,
ZTEXT,
ZMESSAGETYPE,
ZWAGROUPMEMBER.ZMEMBERJID
FROM main.ZWAMESSAGE
LEFT JOIN main.ZWAGROUPMEMBER
ON main.ZWAMESSAGE.ZGROUPMEMBER = main.ZWAGROUPMEMBER.Z_PK;""")
i = 0
content = c.fetchone()
while content is not None:
if content[0] not in data:
data[content[0]] = {"name": None, "messages": {}}
ts = APPLE_TIME + content[3]
data[content[0]]["messages"][content[1]] = {
"from_me": bool(content[2]),
"timestamp": ts,
"time": datetime.fromtimestamp(ts).strftime("%H:%M"),
"media": False,
"reply": None,
"caption": None,
"meta": False,
"data": None
}
if "-" in content[0] and content[2] == 0:
name = None
if content[6] is not None:
if content[6] in data:
name = data[content[6]]["name"]
if "@" in content[6]:
fallback = content[6].split('@')[0]
else:
fallback = None
else:
fallback = None
data[content[0]]["messages"][content[1]]["sender"] = name or fallback
else:
data[content[0]]["messages"][content[1]]["sender"] = None
if content[5] == 6:
# Metadata
if "-" in content[0]:
# Group
if content[4] is not None:
# Chnaged name
try:
int(content[4])
except ValueError:
msg = f"The group name changed to {content[4]}"
data[content[0]]["messages"][content[1]]["data"] = msg
data[content[0]]["messages"][content[1]]["meta"] = True
else:
del data[content[0]]["messages"][content[1]]
else:
data[content[0]]["messages"][content[1]]["data"] = None
else:
data[content[0]]["messages"][content[1]]["data"] = None
else:
# real message
if content[2] == 1:
if content[5] == 14:
msg = "Message deleted"
data[content[0]]["messages"][content[1]]["meta"] = True
else:
msg = content[4]
if msg is not None:
if "\r\n" in msg:
msg = msg.replace("\r\n", "<br>")
if "\n" in msg:
msg = msg.replace("\n", "<br>")
else:
if content[5] == 14:
msg = "Message deleted"
data[content[0]]["messages"][content[1]]["meta"] = True
else:
msg = content[4]
if msg is not None:
if "\r\n" in msg:
msg = msg.replace("\r\n", "<br>")
if "\n" in msg:
msg = msg.replace("\n", "<br>")
data[content[0]]["messages"][content[1]]["data"] = msg
i += 1
if i % 1000 == 0:
print(f"Gathering messages...({i}/{total_row_number})", end="\r")
content = c.fetchone()
print(
f"Gathering messages...({total_row_number}/{total_row_number})", end="\r")
def media(db, data, media_folder):
c = db.cursor()
# Get media
c.execute("""SELECT count() FROM ZWAMEDIAITEM""")
total_row_number = c.fetchone()[0]
print(f"\nGathering media...(0/{total_row_number})", end="\r")
i = 0
c.execute("""SELECT COALESCE(ZWAMESSAGE.ZFROMJID, ZWAMESSAGE.ZTOJID) as _id,
ZMESSAGE,
ZMEDIALOCALPATH,
ZMEDIAURL,
ZVCARDSTRING,
ZMEDIAKEY,
ZTITLE
FROM ZWAMEDIAITEM
INNER JOIN ZWAMESSAGE
ON ZWAMEDIAITEM.ZMESSAGE = ZWAMESSAGE.Z_PK
WHERE ZMEDIALOCALPATH IS NOT NULL
ORDER BY _id ASC""")
content = c.fetchone()
mime = MimeTypes()
while content is not None:
file_path = f"{media_folder}/{content[2]}"
data[content[0]]["messages"][content[1]]["media"] = True
if os.path.isfile(file_path):
data[content[0]]["messages"][content[1]]["data"] = file_path
if content[4] is None:
guess = mime.guess_type(file_path)[0]
if guess is not None:
data[content[0]]["messages"][content[1]]["mime"] = guess
else:
data[content[0]]["messages"][content[1]]["mime"] = "data/data"
else:
data[content[0]]["messages"][content[1]]["mime"] = content[4]
else:
# if "https://mmg" in content[4]:
# try:
# r = requests.get(content[3])
# if r.status_code != 200:
# raise RuntimeError()
# except:
# data[content[0]]["messages"][content[1]]["data"] = "{The media is missing}"
# data[content[0]]["messages"][content[1]]["mime"] = "media"
# else:
data[content[0]]["messages"][content[1]]["data"] = "The media is missing"
data[content[0]]["messages"][content[1]]["mime"] = "media"
data[content[0]]["messages"][content[1]]["meta"] = True
if content[6] is not None:
data[content[0]]["messages"][content[1]]["caption"] = content[6]
i += 1
if i % 100 == 0:
print(f"Gathering media...({i}/{total_row_number})", end="\r")
content = c.fetchone()
print(
f"Gathering media...({total_row_number}/{total_row_number})", end="\r")
def vcard(db, data):
c = db.cursor()
c.execute("""SELECT DISTINCT ZWAVCARDMENTION.ZMEDIAITEM,
ZWAMEDIAITEM.ZMESSAGE,
COALESCE(ZWAMESSAGE.ZFROMJID,
ZWAMESSAGE.ZTOJID) as _id,
ZVCARDNAME,
ZVCARDSTRING
FROM ZWAVCARDMENTION
INNER JOIN ZWAMEDIAITEM
ON ZWAVCARDMENTION.ZMEDIAITEM = ZWAMEDIAITEM.Z_PK
INNER JOIN ZWAMESSAGE
ON ZWAMEDIAITEM.ZMESSAGE = ZWAMESSAGE.Z_PK""")
rows = c.fetchall()
total_row_number = len(rows)
print(f"\nGathering vCards...(0/{total_row_number})", end="\r")
base = "Message/vCards"
if not os.path.isdir(base):
Path(base).mkdir(parents=True, exist_ok=True)
for index, row in enumerate(rows):
file_name = "".join(x for x in row[3] if x.isalnum())
file_path = f"{base}/{file_name[:200]}.vcf"
if not os.path.isfile(file_path):
with open(file_path, "w", encoding="utf-8") as f:
f.write(row[4])
data[row[2]]["messages"][row[1]]["data"] = row[3] + \
"The vCard file cannot be displayed here, " \
f"however it should be located at {file_path}"
data[row[2]]["messages"][row[1]]["mime"] = "text/x-vcard"
data[row[2]]["messages"][row[1]]["media"] = True
data[row[2]]["messages"][row[1]]["meta"] = True
print(f"Gathering vCards...({index + 1}/{total_row_number})", end="\r")
def create_html(data, output_folder, template=None):
if template is None:
template_dir = os.path.dirname(__file__)
template_file = "whatsapp.html"
else:
template_dir = os.path.dirname(template)
template_file = os.path.basename(template)
templateLoader = jinja2.FileSystemLoader(searchpath=template_dir)
templateEnv = jinja2.Environment(loader=templateLoader)
templateEnv.globals.update(determine_day=determine_day)
templateEnv.filters['sanitize_except'] = sanitize_except
template = templateEnv.get_template(template_file)
total_row_number = len(data)
print(f"\nCreating HTML...(0/{total_row_number})", end="\r")
if not os.path.isdir(output_folder):
os.mkdir(output_folder)
for current, contact in enumerate(data):
if len(data[contact]["messages"]) == 0:
continue
phone_number = contact.split('@')[0]
if "-" in contact:
file_name = ""
else:
file_name = phone_number
if data[contact]["name"] is not None:
if file_name != "":
file_name += "-"
file_name += data[contact]["name"].replace("/", "-")
name = data[contact]["name"]
else:
name = phone_number
safe_file_name = ''
safe_file_name = "".join(x for x in file_name if x.isalnum() or x in "- ")
with open(f"{output_folder}/{safe_file_name}.html", "w", encoding="utf-8") as f:
f.write(
template.render(
name=name,
msgs=data[contact]["messages"].values(),
my_avatar=None,
their_avatar=f"WhatsApp/Avatars/{contact}.j"
)
)
if current % 10 == 0:
print(f"Creating HTML...({current}/{total_row_number})", end="\r")
print(f"Creating HTML...({total_row_number}/{total_row_number})", end="\r")
if __name__ == "__main__":
from optparse import OptionParser
parser = OptionParser()
parser.add_option(
"-w",
"--wa",
dest="wa",
default="wa.db",
help="Path to contact database")
parser.add_option(
"-m",
"--media",
dest="media",
default="Message",
help="Path to WhatsApp media folder"
)
# parser.add_option(
# "-t",
# "--template",
# dest="html",
# default="wa.db",
# help="Path to HTML template")
(options, args) = parser.parse_args()
msg_db = "7c7fba66680ef796b916b067077cc246adacf01d"
output_folder = "temp"
contact_db = options.wa
media_folder = options.media
if len(args) == 1:
msg_db = args[0]
elif len(args) == 2:
msg_db = args[0]
output_folder = args[1]
data = {}
if os.path.isfile(msg_db):
with sqlite3.connect(msg_db) as db:
messages(db, data)
media(db, data, media_folder)
vcard(db, data)
create_html(data, output_folder)
if not os.path.isdir(f"{output_folder}/WhatsApp"):
shutil.move(media_folder, f"{output_folder}/")
with open("result.json", "w") as f:
data = json.dumps(data)
print(f"\nWriting JSON file...({int(len(data)/1024/1024)}MB)")
f.write(data)
print("Everything is done!")

View File

@@ -1,133 +0,0 @@
#!/usr/bin/python3
import shutil
import sqlite3
import os
import getpass
try:
from iphone_backup_decrypt import EncryptedBackup, RelativePath
except ModuleNotFoundError:
support_encrypted = False
else:
support_encrypted = True
def extract_encrypted(base_dir, password):
backup = EncryptedBackup(backup_directory=base_dir, passphrase=password)
print("Decrypting WhatsApp database...")
backup.extract_file(relative_path=RelativePath.WHATSAPP_MESSAGES,
output_filename="7c7fba66680ef796b916b067077cc246adacf01d")
backup.extract_file(relative_path=RelativePath.WHATSAPP_CONTACTS,
output_filename="ContactsV2.sqlite")
data = backup.execute_sql("""SELECT count()
FROM Files
WHERE relativePath
LIKE 'Message/Media/%'"""
)
total_row_number = data[0][0]
print(f"Gathering media...(0/{total_row_number})", end="\r")
data = backup.execute_sql("""SELECT fileID,
relativePath,
flags,
file
FROM Files
WHERE relativePath
LIKE 'Message/Media/%'"""
)
if not os.path.isdir("Message"):
os.mkdir("Message")
if not os.path.isdir("Message/Media"):
os.mkdir("Message/Media")
i = 0
for row in data:
destination = row[1]
hashes = row[0]
folder = hashes[:2]
flags = row[2]
file = row[3]
if flags == 2:
try:
os.mkdir(destination)
except FileExistsError:
pass
elif flags == 1:
decrypted = backup.decrypt_inner_file(file_id=hashes, file_bplist=file)
with open(destination, "wb") as f:
f.write(decrypted)
i += 1
if i % 100 == 0:
print(f"Gathering media...({i}/{total_row_number})", end="\r")
print(f"Gathering media...({total_row_number}/{total_row_number})", end="\r")
def is_encrypted(base_dir):
with sqlite3.connect(f"{base_dir}/Manifest.db") as f:
c = f.cursor()
try:
c.execute("""SELECT count()
FROM Files
""")
except sqlite3.DatabaseError:
return True
else:
return False
def extract_media(base_dir):
if is_encrypted(base_dir):
if not support_encrypted:
print("You don't have the dependencies to handle encrypted backup.")
print("Read more on how to deal with encrypted backup:")
print("https://github.com/KnugiHK/Whatsapp-Chat-Exporter/blob/main/README.md#usage")
return False
password = getpass.getpass("Enter the password:")
extract_encrypted(base_dir, password)
else:
wts_db = os.path.join(base_dir, "7c/7c7fba66680ef796b916b067077cc246adacf01d")
if not os.path.isfile(wts_db):
print("WhatsApp database not found.")
exit()
else:
shutil.copyfile(wts_db, "7c7fba66680ef796b916b067077cc246adacf01d")
with sqlite3.connect(f"{base_dir}/Manifest.db") as manifest:
c = manifest.cursor()
c.execute("""SELECT count()
FROM Files
WHERE relativePath
LIKE 'Message/Media/%'""")
total_row_number = c.fetchone()[0]
print(f"Gathering media...(0/{total_row_number})", end="\r")
c.execute("""SELECT fileID,
relativePath,
flags
FROM Files
WHERE relativePath
LIKE 'Message/Media/%'""")
row = c.fetchone()
if not os.path.isdir("Message"):
os.mkdir("Message")
if not os.path.isdir("Message/Media"):
os.mkdir("Message/Media")
i = 0
while row is not None:
destination = row[1]
hashes = row[0]
folder = hashes[:2]
flags = row[2]
if flags == 2:
os.mkdir(destination)
elif flags == 1:
shutil.copyfile(f"{base_dir}/{folder}/{hashes}", destination)
i += 1
if i % 100 == 0:
print(f"Gathering media...({i}/{total_row_number})", end="\r")
row = c.fetchone()
print(f"Gathering media...({total_row_number}/{total_row_number})", end="\r")
if __name__ == "__main__":
from optparse import OptionParser
parser = OptionParser()
(_, args) = parser.parse_args()
base_dir = args[0]
extract_media(base_dir)

View File

@@ -0,0 +1,602 @@
#!/usr/bin/python3
import os
import shutil
from glob import glob
from pathlib import Path
from mimetypes import MimeTypes
from markupsafe import escape as htmle
from Whatsapp_Chat_Exporter.data_model import ChatStore, Message
from Whatsapp_Chat_Exporter.utility import APPLE_TIME, CURRENT_TZ_OFFSET, get_chat_condition
from Whatsapp_Chat_Exporter.utility import bytes_to_readable, convert_time_unit, slugify, Device
def contacts(db, data):
"""Process WhatsApp contacts with status information."""
c = db.cursor()
c.execute("""SELECT count() FROM ZWAADDRESSBOOKCONTACT WHERE ZABOUTTEXT IS NOT NULL""")
total_row_number = c.fetchone()[0]
print(f"Pre-processing contacts...({total_row_number})")
c.execute("""SELECT ZWHATSAPPID, ZABOUTTEXT FROM ZWAADDRESSBOOKCONTACT WHERE ZABOUTTEXT IS NOT NULL""")
content = c.fetchone()
while content is not None:
zwhatsapp_id = content["ZWHATSAPPID"]
if not zwhatsapp_id.endswith("@s.whatsapp.net"):
zwhatsapp_id += "@s.whatsapp.net"
current_chat = ChatStore(Device.IOS)
current_chat.status = content["ZABOUTTEXT"]
data.add_chat(zwhatsapp_id, current_chat)
content = c.fetchone()
def process_contact_avatars(current_chat, media_folder, contact_id):
"""Process and assign avatar images for a contact."""
path = f'{media_folder}/Media/Profile/{contact_id.split("@")[0]}'
avatars = glob(f"{path}*")
if 0 < len(avatars) <= 1:
current_chat.their_avatar = avatars[0]
else:
for avatar in avatars:
if avatar.endswith(".thumb") and current_chat.their_avatar_thumb is None:
current_chat.their_avatar_thumb = avatar
elif avatar.endswith(".jpg") and current_chat.their_avatar is None:
current_chat.their_avatar = avatar
def get_contact_name(content):
"""Determine the appropriate contact name based on push name and partner name."""
is_phone = content["ZPARTNERNAME"].replace("+", "").replace(" ", "").isdigit()
if content["ZPUSHNAME"] is None or (content["ZPUSHNAME"] and not is_phone):
return content["ZPARTNERNAME"]
else:
return content["ZPUSHNAME"]
def messages(db, data, media_folder, timezone_offset, filter_date, filter_chat, filter_empty):
"""Process WhatsApp messages and contacts from the database."""
c = db.cursor()
cursor2 = db.cursor()
# Build the chat filter conditions
chat_filter_include = get_chat_condition(filter_chat[0], True, ["ZWACHATSESSION.ZCONTACTJID", "ZMEMBERJID"], "ZGROUPINFO", "ios")
chat_filter_exclude = get_chat_condition(filter_chat[1], False, ["ZWACHATSESSION.ZCONTACTJID", "ZMEMBERJID"], "ZGROUPINFO", "ios")
date_filter = f'AND ZMESSAGEDATE {filter_date}' if filter_date is not None else ''
# Process contacts first
contact_query = f"""
SELECT count()
FROM (SELECT DISTINCT ZCONTACTJID,
ZPARTNERNAME,
ZWAPROFILEPUSHNAME.ZPUSHNAME
FROM ZWACHATSESSION
INNER JOIN ZWAMESSAGE
ON ZWAMESSAGE.ZCHATSESSION = ZWACHATSESSION.Z_PK
LEFT JOIN ZWAPROFILEPUSHNAME
ON ZWACHATSESSION.ZCONTACTJID = ZWAPROFILEPUSHNAME.ZJID
LEFT JOIN ZWAGROUPMEMBER
ON ZWAMESSAGE.ZGROUPMEMBER = ZWAGROUPMEMBER.Z_PK
WHERE 1=1
{chat_filter_include}
{chat_filter_exclude}
GROUP BY ZCONTACTJID);
"""
c.execute(contact_query)
total_row_number = c.fetchone()[0]
print(f"Processing contacts...({total_row_number})")
# Get distinct contacts
contacts_query = f"""
SELECT DISTINCT ZCONTACTJID,
ZPARTNERNAME,
ZWAPROFILEPUSHNAME.ZPUSHNAME
FROM ZWACHATSESSION
INNER JOIN ZWAMESSAGE
ON ZWAMESSAGE.ZCHATSESSION = ZWACHATSESSION.Z_PK
LEFT JOIN ZWAPROFILEPUSHNAME
ON ZWACHATSESSION.ZCONTACTJID = ZWAPROFILEPUSHNAME.ZJID
LEFT JOIN ZWAGROUPMEMBER
ON ZWAMESSAGE.ZGROUPMEMBER = ZWAGROUPMEMBER.Z_PK
WHERE 1=1
{chat_filter_include}
{chat_filter_exclude}
GROUP BY ZCONTACTJID;
"""
c.execute(contacts_query)
# Process each contact
content = c.fetchone()
while content is not None:
contact_name = get_contact_name(content)
contact_id = content["ZCONTACTJID"]
# Add or update chat
if contact_id not in data:
current_chat = data.add_chat(contact_id, ChatStore(Device.IOS, contact_name, media_folder))
else:
current_chat = data.get_chat(contact_id)
current_chat.name = contact_name
current_chat.my_avatar = os.path.join(media_folder, "Media/Profile/Photo.jpg")
# Process avatar images
process_contact_avatars(current_chat, media_folder, contact_id)
content = c.fetchone()
# Get message count
message_count_query = f"""
SELECT count()
FROM ZWAMESSAGE
INNER JOIN ZWACHATSESSION
ON ZWAMESSAGE.ZCHATSESSION = ZWACHATSESSION.Z_PK
LEFT JOIN ZWAGROUPMEMBER
ON ZWAMESSAGE.ZGROUPMEMBER = ZWAGROUPMEMBER.Z_PK
WHERE 1=1
{date_filter}
{chat_filter_include}
{chat_filter_exclude}
"""
c.execute(message_count_query)
total_row_number = c.fetchone()[0]
print(f"Processing messages...(0/{total_row_number})", end="\r")
# Fetch messages
messages_query = f"""
SELECT ZCONTACTJID,
ZWAMESSAGE.Z_PK,
ZISFROMME,
ZMESSAGEDATE,
ZTEXT,
ZMESSAGETYPE,
ZWAGROUPMEMBER.ZMEMBERJID,
ZMETADATA,
ZSTANZAID,
ZGROUPINFO,
ZSENTDATE
FROM ZWAMESSAGE
LEFT JOIN ZWAGROUPMEMBER
ON ZWAMESSAGE.ZGROUPMEMBER = ZWAGROUPMEMBER.Z_PK
LEFT JOIN ZWAMEDIAITEM
ON ZWAMESSAGE.Z_PK = ZWAMEDIAITEM.ZMESSAGE
INNER JOIN ZWACHATSESSION
ON ZWAMESSAGE.ZCHATSESSION = ZWACHATSESSION.Z_PK
WHERE 1=1
{date_filter}
{chat_filter_include}
{chat_filter_exclude}
ORDER BY ZMESSAGEDATE ASC;
"""
c.execute(messages_query)
# Process each message
i = 0
content = c.fetchone()
while content is not None:
contact_id = content["ZCONTACTJID"]
message_pk = content["Z_PK"]
is_group_message = content["ZGROUPINFO"] is not None
# Ensure chat exists
if contact_id not in data:
current_chat = data.add_chat(contact_id, ChatStore(Device.IOS))
process_contact_avatars(current_chat, media_folder, contact_id)
else:
current_chat = data.get_chat(contact_id)
# Create message object
ts = APPLE_TIME + content["ZMESSAGEDATE"]
message = Message(
from_me=content["ZISFROMME"],
timestamp=ts,
time=ts,
key_id=content["ZSTANZAID"][:17],
timezone_offset=timezone_offset if timezone_offset else CURRENT_TZ_OFFSET,
message_type=content["ZMESSAGETYPE"],
received_timestamp=APPLE_TIME + content["ZSENTDATE"] if content["ZSENTDATE"] else None,
read_timestamp=None # TODO: Add timestamp
)
# Process message data
invalid = process_message_data(message, content, is_group_message, data, cursor2)
# Add valid messages to chat
if not invalid:
current_chat.add_message(message_pk, message)
# Update progress
i += 1
if i % 1000 == 0:
print(f"Processing messages...({i}/{total_row_number})", end="\r")
content = c.fetchone()
print(f"Processing messages...({total_row_number}/{total_row_number})", end="\r")
def process_message_data(message, content, is_group_message, data, cursor2):
"""Process and set message data from content row."""
# Handle group sender info
if is_group_message and content["ZISFROMME"] == 0:
name = None
if content["ZMEMBERJID"] is not None:
if content["ZMEMBERJID"] in data:
name = data.get_chat(content["ZMEMBERJID"]).name
if "@" in content["ZMEMBERJID"]:
fallback = content["ZMEMBERJID"].split('@')[0]
else:
fallback = None
else:
fallback = None
message.sender = name or fallback
else:
message.sender = None
# Handle metadata messages
if content["ZMESSAGETYPE"] == 6:
return process_metadata_message(message, content, is_group_message)
# Handle quoted replies
if content["ZMETADATA"] is not None and content["ZMETADATA"].startswith(b"\x2a\x14") and False:
quoted = content["ZMETADATA"][2:19]
message.reply = quoted.decode()
cursor2.execute(f"""SELECT ZTEXT
FROM ZWAMESSAGE
WHERE ZSTANZAID LIKE '{message.reply}%'""")
quoted_content = cursor2.fetchone()
if quoted_content and "ZTEXT" in quoted_content:
message.quoted_data = quoted_content["ZTEXT"]
else:
message.quoted_data = None
# Handle stickers
if content["ZMESSAGETYPE"] == 15:
message.sticker = True
# Process message text
process_message_text(message, content)
return False # Message is valid
def process_metadata_message(message, content, is_group_message):
"""Process metadata messages (action_type 6)."""
if is_group_message:
# Group
if content["ZTEXT"] is not None:
# Changed name
try:
int(content["ZTEXT"])
except ValueError:
msg = f"The group name changed to {content['ZTEXT']}"
message.data = msg
message.meta = True
return False # Valid message
else:
return True # Invalid message
else:
message.data = None
return False
else:
message.data = None
return False
def process_message_text(message, content):
"""Process and format message text content."""
if content["ZISFROMME"] == 1:
if content["ZMESSAGETYPE"] == 14:
msg = "Message deleted"
message.meta = True
else:
msg = content["ZTEXT"]
if msg is not None:
msg = msg.replace("\r\n", "<br>").replace("\n", "<br>")
else:
if content["ZMESSAGETYPE"] == 14:
msg = "Message deleted"
message.meta = True
else:
msg = content["ZTEXT"]
if msg is not None:
msg = msg.replace("\r\n", "<br>").replace("\n", "<br>")
message.data = msg
def media(db, data, media_folder, filter_date, filter_chat, filter_empty, separate_media=False):
"""Process media files from WhatsApp messages."""
c = db.cursor()
# Build filter conditions
chat_filter_include = get_chat_condition(filter_chat[0], True, ["ZWACHATSESSION.ZCONTACTJID","ZMEMBERJID"], "ZGROUPINFO", "ios")
chat_filter_exclude = get_chat_condition(filter_chat[1], False, ["ZWACHATSESSION.ZCONTACTJID", "ZMEMBERJID"], "ZGROUPINFO", "ios")
date_filter = f'AND ZMESSAGEDATE {filter_date}' if filter_date is not None else ''
# Get media count
media_count_query = f"""
SELECT count()
FROM ZWAMEDIAITEM
INNER JOIN ZWAMESSAGE
ON ZWAMEDIAITEM.ZMESSAGE = ZWAMESSAGE.Z_PK
INNER JOIN ZWACHATSESSION
ON ZWAMESSAGE.ZCHATSESSION = ZWACHATSESSION.Z_PK
LEFT JOIN ZWAGROUPMEMBER
ON ZWAMESSAGE.ZGROUPMEMBER = ZWAGROUPMEMBER.Z_PK
WHERE 1=1
{date_filter}
{chat_filter_include}
{chat_filter_exclude}
"""
c.execute(media_count_query)
total_row_number = c.fetchone()[0]
print(f"\nProcessing media...(0/{total_row_number})", end="\r")
# Fetch media items
media_query = f"""
SELECT ZCONTACTJID,
ZMESSAGE,
ZMEDIALOCALPATH,
ZMEDIAURL,
ZVCARDSTRING,
ZMEDIAKEY,
ZTITLE
FROM ZWAMEDIAITEM
INNER JOIN ZWAMESSAGE
ON ZWAMEDIAITEM.ZMESSAGE = ZWAMESSAGE.Z_PK
INNER JOIN ZWACHATSESSION
ON ZWAMESSAGE.ZCHATSESSION = ZWACHATSESSION.Z_PK
LEFT JOIN ZWAGROUPMEMBER
ON ZWAMESSAGE.ZGROUPMEMBER = ZWAGROUPMEMBER.Z_PK
WHERE ZMEDIALOCALPATH IS NOT NULL
{date_filter}
{chat_filter_include}
{chat_filter_exclude}
ORDER BY ZCONTACTJID ASC
"""
c.execute(media_query)
# Process each media item
mime = MimeTypes()
i = 0
content = c.fetchone()
while content is not None:
process_media_item(content, data, media_folder, mime, separate_media)
# Update progress
i += 1
if i % 100 == 0:
print(f"Processing media...({i}/{total_row_number})", end="\r")
content = c.fetchone()
print(f"Processing media...({total_row_number}/{total_row_number})", end="\r")
def process_media_item(content, data, media_folder, mime, separate_media):
"""Process a single media item."""
file_path = f"{media_folder}/Message/{content['ZMEDIALOCALPATH']}"
current_chat = data.get_chat(content["ZCONTACTJID"])
message = current_chat.get_message(content["ZMESSAGE"])
message.media = True
if current_chat.media_base == "":
current_chat.media_base = media_folder + "/"
if os.path.isfile(file_path):
message.data = '/'.join(file_path.split("/")[1:])
# Set MIME type
if content["ZVCARDSTRING"] is None:
guess = mime.guess_type(file_path)[0]
message.mime = guess if guess is not None else "application/octet-stream"
else:
message.mime = content["ZVCARDSTRING"]
# Handle separate media option
if separate_media:
chat_display_name = slugify(current_chat.name or message.sender or content["ZCONTACTJID"].split('@')[0], True)
current_filename = file_path.split("/")[-1]
new_folder = os.path.join(media_folder, "separated", chat_display_name)
Path(new_folder).mkdir(parents=True, exist_ok=True)
new_path = os.path.join(new_folder, current_filename)
shutil.copy2(file_path, new_path)
message.data = '/'.join(new_path.split("\\")[1:])
else:
# Handle missing media
message.data = "The media is missing"
message.mime = "media"
message.meta = True
# Add caption if available
if content["ZTITLE"] is not None:
message.caption = content["ZTITLE"]
def vcard(db, data, media_folder, filter_date, filter_chat, filter_empty):
"""Process vCard contacts from WhatsApp messages."""
c = db.cursor()
# Build filter conditions
chat_filter_include = get_chat_condition(filter_chat[0], True, ["ZCONTACTJID", "ZMEMBERJID"], "ZGROUPINFO", "ios")
chat_filter_exclude = get_chat_condition(filter_chat[1], False, ["ZCONTACTJID", "ZMEMBERJID"], "ZGROUPINFO", "ios")
date_filter = f'AND ZWAMESSAGE.ZMESSAGEDATE {filter_date}' if filter_date is not None else ''
# Fetch vCard mentions
vcard_query = f"""
SELECT DISTINCT ZWAVCARDMENTION.ZMEDIAITEM,
ZWAMEDIAITEM.ZMESSAGE,
ZCONTACTJID,
ZVCARDNAME,
ZVCARDSTRING
FROM ZWAVCARDMENTION
INNER JOIN ZWAMEDIAITEM
ON ZWAVCARDMENTION.ZMEDIAITEM = ZWAMEDIAITEM.Z_PK
INNER JOIN ZWAMESSAGE
ON ZWAMEDIAITEM.ZMESSAGE = ZWAMESSAGE.Z_PK
INNER JOIN ZWACHATSESSION
ON ZWAMESSAGE.ZCHATSESSION = ZWACHATSESSION.Z_PK
LEFT JOIN ZWAGROUPMEMBER
ON ZWAMESSAGE.ZGROUPMEMBER = ZWAGROUPMEMBER.Z_PK
WHERE 1=1
{date_filter}
{chat_filter_include}
{chat_filter_exclude}
"""
c.execute(vcard_query)
contents = c.fetchall()
total_row_number = len(contents)
print(f"\nProcessing vCards...(0/{total_row_number})", end="\r")
# Create vCards directory
path = f'{media_folder}/Message/vCards'
Path(path).mkdir(parents=True, exist_ok=True)
# Process each vCard
for index, content in enumerate(contents):
process_vcard_item(content, path, data)
print(f"Processing vCards...({index + 1}/{total_row_number})", end="\r")
def process_vcard_item(content, path, data):
"""Process a single vCard item."""
file_paths = []
vcard_names = content["ZVCARDNAME"].split("_$!<Name-Separator>!$_")
vcard_strings = content["ZVCARDSTRING"].split("_$!<VCard-Separator>!$_")
# If this is a list of contacts
if len(vcard_names) > len(vcard_strings):
vcard_names.pop(0) # Dismiss the first element, which is the group name
# Save each vCard file
for name, vcard_string in zip(vcard_names, vcard_strings):
file_name = "".join(x for x in name if x.isalnum())
file_name = file_name.encode('utf-8')[:230].decode('utf-8', 'ignore')
file_path = os.path.join(path, f"{file_name}.vcf")
file_paths.append(file_path)
if not os.path.isfile(file_path):
with open(file_path, "w", encoding="utf-8") as f:
f.write(vcard_string)
# Create vCard summary and update message
vcard_summary = "This media include the following vCard file(s):<br>"
vcard_summary += " | ".join([f'<a href="{htmle(fp)}">{htmle(name)}</a>' for name, fp in zip(vcard_names, file_paths)])
message = data.get_chat(content["ZCONTACTJID"]).get_message(content["ZMESSAGE"])
message.data = vcard_summary
message.mime = "text/x-vcard"
message.media = True
message.meta = True
message.safe = True
def calls(db, data, timezone_offset, filter_chat):
"""Process WhatsApp call records."""
c = db.cursor()
# Build filter conditions
chat_filter_include = get_chat_condition(filter_chat[0], True, ["ZGROUPCALLCREATORUSERJIDSTRING"], None, "ios")
chat_filter_exclude = get_chat_condition(filter_chat[1], False, ["ZGROUPCALLCREATORUSERJIDSTRING"], None, "ios")
# Get call count
call_count_query = f"""
SELECT count()
FROM ZWACDCALLEVENT
WHERE 1=1
{chat_filter_include}
{chat_filter_exclude}
"""
c.execute(call_count_query)
total_row_number = c.fetchone()[0]
if total_row_number == 0:
return
print(f"\nProcessing calls...({total_row_number})", end="\r")
# Fetch call records
calls_query = f"""
SELECT ZCALLIDSTRING,
ZGROUPCALLCREATORUSERJIDSTRING,
ZGROUPJIDSTRING,
ZDATE,
ZOUTCOME,
ZBYTESRECEIVED + ZBYTESSENT AS bytes_transferred,
ZDURATION,
ZVIDEO,
ZMISSED,
ZINCOMING
FROM ZWACDCALLEVENT
INNER JOIN ZWAAGGREGATECALLEVENT
ON ZWACDCALLEVENT.Z1CALLEVENTS = ZWAAGGREGATECALLEVENT.Z_PK
WHERE 1=1
{chat_filter_include}
{chat_filter_exclude}
"""
c.execute(calls_query)
# Create calls chat
chat = ChatStore(Device.ANDROID, "WhatsApp Calls")
# Process each call
content = c.fetchone()
while content is not None:
process_call_record(content, chat, data, timezone_offset)
content = c.fetchone()
# Add calls chat to data
data.add_chat("000000000000000", chat)
def process_call_record(content, chat, data, timezone_offset):
"""Process a single call record."""
ts = APPLE_TIME + int(content["ZDATE"])
call = Message(
from_me=content["ZINCOMING"] == 0,
timestamp=ts,
time=ts,
key_id=content["ZCALLIDSTRING"],
timezone_offset=timezone_offset if timezone_offset else CURRENT_TZ_OFFSET
)
# Set sender info
_jid = content["ZGROUPCALLCREATORUSERJIDSTRING"]
name = data.get_chat(_jid).name if _jid in data else None
if _jid is not None and "@" in _jid:
fallback = _jid.split('@')[0]
else:
fallback = None
call.sender = name or fallback
# Set call metadata
call.meta = True
call.data = format_call_data(call, content)
# Add call to chat
chat.add_message(call.key_id, call)
def format_call_data(call, content):
"""Format call data message based on call attributes."""
# Basic call info
call_data = (
f"A {'group ' if content['ZGROUPJIDSTRING'] is not None else ''}"
f"{'video' if content['ZVIDEO'] == 1 else 'voice'} "
f"call {'to' if call.from_me else 'from'} "
f"{call.sender} was "
)
# Call outcome
if content['ZOUTCOME'] in (1, 4):
call_data += "not answered." if call.from_me else "missed."
elif content['ZOUTCOME'] == 2:
call_data += "failed."
elif content['ZOUTCOME'] == 0:
call_time = convert_time_unit(int(content['ZDURATION']))
call_bytes = bytes_to_readable(content['bytes_transferred'])
call_data += (
f"initiated and lasted for {call_time} "
f"with {call_bytes} data transferred."
)
else:
call_data += "in an unknown state."
return call_data

View File

@@ -0,0 +1,232 @@
#!/usr/bin/python3
import shutil
import sqlite3
import os
import getpass
from sys import exit
from Whatsapp_Chat_Exporter.utility import WhatsAppIdentifier
from Whatsapp_Chat_Exporter.bplist import BPListReader
try:
from iphone_backup_decrypt import EncryptedBackup, RelativePath
except ModuleNotFoundError:
support_encrypted = False
else:
support_encrypted = True
class BackupExtractor:
"""
A class to handle the extraction of WhatsApp data from iOS backups,
including encrypted and unencrypted backups.
"""
def __init__(self, base_dir, identifiers, decrypt_chunk_size):
self.base_dir = base_dir
self.identifiers = identifiers
self.decrypt_chunk_size = decrypt_chunk_size
def extract(self):
"""
Extracts WhatsApp data from the backup based on whether it's encrypted or not.
"""
if self._is_encrypted():
self._extract_encrypted_backup()
else:
self._extract_unencrypted_backup()
def _is_encrypted(self):
"""
Checks if the iOS backup is encrypted.
Returns:
bool: True if encrypted, False otherwise.
"""
with sqlite3.connect(os.path.join(self.base_dir, "Manifest.db")) as db:
c = db.cursor()
try:
c.execute("SELECT count() FROM Files")
c.fetchone() # Execute and fetch to trigger potential errors
except (sqlite3.OperationalError, sqlite3.DatabaseError):
return True
else:
return False
def _extract_encrypted_backup(self):
"""
Handles the extraction of data from an encrypted iOS backup.
"""
if not support_encrypted:
print("You don't have the dependencies to handle encrypted backup.")
print("Read more on how to deal with encrypted backup:")
print("https://github.com/KnugiHK/Whatsapp-Chat-Exporter/blob/main/README.md#usage")
return
print("Encryption detected on the backup!")
password = getpass.getpass("Enter the password for the backup:")
self._decrypt_backup(password)
self._extract_decrypted_files()
def _decrypt_backup(self, password):
"""
Decrypts the iOS backup using the provided password.
Args:
password (str): The password for the encrypted backup.
"""
print("Trying to decrypt the iOS backup...", end="")
self.backup = EncryptedBackup(
backup_directory=self.base_dir,
passphrase=password,
cleanup=False,
check_same_thread=False,
decrypt_chunk_size=self.decrypt_chunk_size,
)
print("Done\nDecrypting WhatsApp database...", end="")
try:
self.backup.extract_file(
relative_path=RelativePath.WHATSAPP_MESSAGES,
domain_like=self.identifiers.DOMAIN,
output_filename=self.identifiers.MESSAGE,
)
self.backup.extract_file(
relative_path=RelativePath.WHATSAPP_CONTACTS,
domain_like=self.identifiers.DOMAIN,
output_filename=self.identifiers.CONTACT,
)
self.backup.extract_file(
relative_path=RelativePath.WHATSAPP_CALLS,
domain_like=self.identifiers.DOMAIN,
output_filename=self.identifiers.CALL,
)
except ValueError:
print("Failed to decrypt backup: incorrect password?")
exit(7)
except FileNotFoundError:
print(
"Essential WhatsApp files are missing from the iOS backup. "
"Perhapse you enabled end-to-end encryption for the backup? "
"See https://wts.knugi.dev/docs.html?dest=iose2e"
)
exit(6)
else:
print("Done")
def _extract_decrypted_files(self):
"""Extract all WhatsApp files after decryption"""
def extract_progress_handler(file_id, domain, relative_path, n, total_files):
if n % 100 == 0:
print(f"Decrypting and extracting files...({n}/{total_files})", end="\r")
return True
self.backup.extract_files(
domain_like=self.identifiers.DOMAIN,
output_folder=self.identifiers.DOMAIN,
preserve_folders=True,
filter_callback=extract_progress_handler
)
print(f"All required files are decrypted and extracted. ", end="\n")
def _extract_unencrypted_backup(self):
"""
Handles the extraction of data from an unencrypted iOS backup.
"""
self._copy_whatsapp_databases()
self._extract_media_files()
def _copy_whatsapp_databases(self):
"""
Copies the WhatsApp message, contact, and call databases to the working directory.
"""
wts_db_path = os.path.join(self.base_dir, self.identifiers.MESSAGE[:2], self.identifiers.MESSAGE)
contact_db_path = os.path.join(self.base_dir, self.identifiers.CONTACT[:2], self.identifiers.CONTACT)
call_db_path = os.path.join(self.base_dir, self.identifiers.CALL[:2], self.identifiers.CALL)
if not os.path.isfile(wts_db_path):
if self.identifiers is WhatsAppIdentifier:
print("WhatsApp database not found.")
else:
print("WhatsApp Business database not found.")
print(
"Essential WhatsApp files are missing from the iOS backup. "
"Perhapse you enabled end-to-end encryption for the backup? "
"See https://wts.knugi.dev/docs.html?dest=iose2e"
)
exit(1)
else:
shutil.copyfile(wts_db_path, self.identifiers.MESSAGE)
if not os.path.isfile(contact_db_path):
print("Contact database not found. Skipping...")
else:
shutil.copyfile(contact_db_path, self.identifiers.CONTACT)
if not os.path.isfile(call_db_path):
print("Call database not found. Skipping...")
else:
shutil.copyfile(call_db_path, self.identifiers.CALL)
def _extract_media_files(self):
"""
Extracts media files from the unencrypted backup.
"""
_wts_id = self.identifiers.DOMAIN
with sqlite3.connect(os.path.join(self.base_dir, "Manifest.db")) as manifest:
manifest.row_factory = sqlite3.Row
c = manifest.cursor()
c.execute(f"SELECT count() FROM Files WHERE domain = '{_wts_id}'")
total_row_number = c.fetchone()[0]
print(f"Extracting WhatsApp files...(0/{total_row_number})", end="\r")
c.execute(
f"""
SELECT fileID, relativePath, flags, file AS metadata,
ROW_NUMBER() OVER(ORDER BY relativePath) AS _index
FROM Files
WHERE domain = '{_wts_id}'
ORDER BY relativePath
"""
)
if not os.path.isdir(_wts_id):
os.mkdir(_wts_id)
row = c.fetchone()
while row is not None:
if not row["relativePath"]: # Skip empty relative paths
row = c.fetchone()
continue
destination = os.path.join(_wts_id, row["relativePath"])
hashes = row["fileID"]
folder = hashes[:2]
flags = row["flags"]
if flags == 2: # Directory
try:
os.mkdir(destination)
except FileExistsError:
pass
elif flags == 1: # File
shutil.copyfile(os.path.join(self.base_dir, folder, hashes), destination)
metadata = BPListReader(row["metadata"]).parse()
creation = metadata["$objects"][1]["Birth"]
modification = metadata["$objects"][1]["LastModified"]
os.utime(destination, (modification, modification))
if row["_index"] % 100 == 0:
print(f"Extracting WhatsApp files...({row['_index']}/{total_row_number})", end="\r")
row = c.fetchone()
print(f"Extracting WhatsApp files...({total_row_number}/{total_row_number})", end="\n")
def extract_media(base_dir, identifiers, decrypt_chunk_size):
"""
Extracts WhatsApp data (media, messages, contacts, calls) from an iOS backup.
Args:
base_dir (str): The path to the iOS backup directory.
identifiers (WhatsAppIdentifier): An object containing WhatsApp file identifiers.
decrypt_chunk_size (int): The chunk size for decryption.
"""
extractor = BackupExtractor(base_dir, identifiers, decrypt_chunk_size)
extractor.extract()

View File

@@ -0,0 +1,567 @@
import sqlite3
import jinja2
import json
import os
import unicodedata
import re
import math
from bleach import clean as sanitize
from markupsafe import Markup
from datetime import datetime, timedelta
from enum import IntEnum
from Whatsapp_Chat_Exporter.data_model import ChatStore
from typing import Dict, List, Optional, Tuple
try:
from enum import StrEnum, IntEnum
except ImportError:
# < Python 3.11
# This should be removed when the support for Python 3.10 ends.
from enum import Enum
class StrEnum(str, Enum):
pass
class IntEnum(int, Enum):
pass
MAX_SIZE = 4 * 1024 * 1024 # Default 4MB
ROW_SIZE = 0x3D0
CURRENT_TZ_OFFSET = datetime.now().astimezone().utcoffset().seconds / 3600
def convert_time_unit(time_second: int) -> str:
"""Converts a time duration in seconds to a human-readable string.
Args:
time_second: The time duration in seconds.
Returns:
str: A human-readable string representing the time duration.
"""
time = str(timedelta(seconds=time_second))
if "day" not in time:
if time_second < 1:
time = "less than a second"
elif time_second == 1:
time = "a second"
elif time_second < 60:
time = time[5:][1 if time_second < 10 else 0:] + " seconds"
elif time_second == 60:
time = "a minute"
elif time_second < 3600:
time = time[2:] + " minutes"
elif time_second == 3600:
time = "an hour"
else:
time += " hour"
return time
def bytes_to_readable(size_bytes: int) -> str:
"""Converts a file size in bytes to a human-readable string with units.
From https://stackoverflow.com/a/14822210/9478891
Authors: james-sapam & other contributors
Licensed under CC BY-SA 3.0
See git commit logs for changes, if any.
Args:
size_bytes: The file size in bytes.
Returns:
A human-readable string representing the file size.
"""
if size_bytes == 0:
return "0B"
size_name = ("B", "KB", "MB", "GB", "TB", "PB", "EB", "ZB", "YB")
i = int(math.floor(math.log(size_bytes, 1024)))
p = math.pow(1024, i)
s = round(size_bytes / p, 2)
return "%s %s" % (s, size_name[i])
def readable_to_bytes(size_str: str) -> int:
"""Converts a human-readable file size string to bytes.
Args:
size_str: The human-readable file size string (e.g., "1024KB", "1MB", "2GB").
Returns:
The file size in bytes.
Raises:
ValueError: If the input string is invalid.
"""
SIZE_UNITS = {
'B': 1,
'KB': 1024,
'MB': 1024**2,
'GB': 1024**3,
'TB': 1024**4,
'PB': 1024**5,
'EB': 1024**6,
'ZB': 1024**7,
'YB': 1024**8
}
size_str = size_str.upper().strip()
number, unit = size_str[:-2].strip(), size_str[-2:].strip()
if unit not in SIZE_UNITS or not number.isnumeric():
raise ValueError("Invalid input for size_str. Example: 1024GB")
return int(number) * SIZE_UNITS[unit]
def sanitize_except(html: str) -> Markup:
"""Sanitizes HTML, only allowing <br> tag.
Args:
html: The HTML string to sanitize.
Returns:
A Markup object containing the sanitized HTML.
"""
return Markup(sanitize(html, tags=["br"]))
def determine_day(last: int, current: int) -> Optional[datetime.date]:
"""Determines if the day has changed between two timestamps. Exposed to Jinja's environment.
Args:
last: The timestamp of the previous message.
current: The timestamp of the current message.
Returns:
The date of the current message if it's a different day than the last message, otherwise None.
"""
last = datetime.fromtimestamp(last).date()
current = datetime.fromtimestamp(current).date()
if last == current:
return None
else:
return current
def check_update():
import urllib.request
import json
import importlib
from sys import platform
PACKAGE_JSON = "https://pypi.org/pypi/whatsapp-chat-exporter/json"
try:
raw = urllib.request.urlopen(PACKAGE_JSON)
except Exception:
print("Failed to check for updates.")
return 1
else:
with raw:
package_info = json.load(raw)
latest_version = tuple(map(int, package_info["info"]["version"].split(".")))
__version__ = importlib.metadata.version("whatsapp_chat_exporter")
current_version = tuple(map(int, __version__.split(".")))
if current_version < latest_version:
print("===============Update===============")
print("A newer version of WhatsApp Chat Exporter is available.")
print("Current version: " + __version__)
print("Latest version: " + package_info["info"]["version"])
if platform == "win32":
print("Update with: pip install --upgrade whatsapp-chat-exporter")
else:
print("Update with: pip3 install --upgrade whatsapp-chat-exporter")
print("====================================")
else:
print("You are using the latest version of WhatsApp Chat Exporter.")
return 0
def rendering(
output_file_name,
template,
name,
msgs,
contact,
w3css,
chat,
headline,
next=False,
previous=False
):
if chat.their_avatar_thumb is None and chat.their_avatar is not None:
their_avatar_thumb = chat.their_avatar
else:
their_avatar_thumb = chat.their_avatar_thumb
if "??" not in headline:
raise ValueError("Headline must contain '??' to replace with name")
headline = headline.replace("??", name)
with open(output_file_name, "w", encoding="utf-8") as f:
f.write(
template.render(
name=name,
msgs=msgs,
my_avatar=chat.my_avatar,
their_avatar=chat.their_avatar,
their_avatar_thumb=their_avatar_thumb,
w3css=w3css,
next=next,
previous=previous,
status=chat.status,
media_base=chat.media_base,
headline=headline
)
)
class Device(StrEnum):
IOS = "ios"
ANDROID = "android"
EXPORTED = "exported"
def import_from_json(json_file: str, data: Dict[str, ChatStore]):
"""Imports chat data from a JSON file into the data dictionary.
Args:
json_file: The path to the JSON file.
data: The dictionary to store the imported chat data.
"""
from Whatsapp_Chat_Exporter.data_model import ChatStore, Message
with open(json_file, "r") as f:
temp_data = json.loads(f.read())
total_row_number = len(tuple(temp_data.keys()))
print(f"Importing chats from JSON...(0/{total_row_number})", end="\r")
for index, (jid, chat_data) in enumerate(temp_data.items()):
chat = ChatStore(chat_data.get("type"), chat_data.get("name"))
chat.my_avatar = chat_data.get("my_avatar")
chat.their_avatar = chat_data.get("their_avatar")
chat.their_avatar_thumb = chat_data.get("their_avatar_thumb")
chat.status = chat_data.get("status")
for id, msg in chat_data.get("messages").items():
message = Message(
from_me=msg["from_me"],
timestamp=msg["timestamp"],
time=msg["time"],
key_id=msg["key_id"],
received_timestamp=msg.get("received_timestamp"),
read_timestamp=msg.get("read_timestamp")
)
message.media = msg.get("media")
message.meta = msg.get("meta")
message.data = msg.get("data")
message.sender = msg.get("sender")
message.safe = msg.get("safe")
message.mime = msg.get("mime")
message.reply = msg.get("reply")
message.quoted_data = msg.get("quoted_data")
message.caption = msg.get("caption")
message.thumb = msg.get("thumb")
message.sticker = msg.get("sticker")
chat.add_message(id, message)
data[jid] = chat
print(f"Importing chats from JSON...({index + 1}/{total_row_number})", end="\r")
def sanitize_filename(file_name: str) -> str:
"""Sanitizes a filename by removing invalid and unsafe characters.
Args:
file_name: The filename to sanitize.
Returns:
The sanitized filename.
"""
return "".join(x for x in file_name if x.isalnum() or x in "- ")
def get_file_name(contact: str, chat: ChatStore) -> Tuple[str, str]:
"""Generates a sanitized filename and contact name for a chat.
Args:
contact: The contact identifier (e.g., a phone number or group ID).
chat: The ChatStore object for the chat.
Returns:
A tuple containing the sanitized filename and the contact name.
Raises:
ValueError: If the contact format is unexpected.
"""
if "@" not in contact and contact not in ("000000000000000", "000000000000001", "ExportedChat"):
raise ValueError("Unexpected contact format: " + contact)
phone_number = contact.split('@')[0]
if "-" in contact and chat.name is not None:
file_name = ""
else:
file_name = phone_number
if chat.name is not None:
if file_name != "":
file_name += "-"
file_name += chat.name.replace("/", "-").replace("\\", "-")
name = chat.name
else:
name = phone_number
return sanitize_filename(file_name), name
def get_cond_for_empty(enable: bool, jid_field: str, broadcast_field: str) -> str:
"""Generates a SQL condition for filtering empty chats.
Args:
enable: True to include non-empty chats, False to include empty chats.
jid_field: The name of the JID field in the SQL query.
broadcast_field: The column name of the broadcast field in the SQL query.
Returns:
A SQL condition string.
"""
return f"AND (chat.hidden=0 OR {jid_field}='status@broadcast' OR {broadcast_field}>0)" if enable else ""
def get_chat_condition(filter: Optional[List[str]], include: bool, columns: List[str], jid: Optional[str] = None, platform: Optional[str] = None) -> str:
"""Generates a SQL condition for filtering chats based on inclusion or exclusion criteria.
Args:
filter: A list of phone numbers to include or exclude.
include: True to include chats that match the filter, False to exclude them.
columns: A list of column names to check against the filter.
jid: The JID column name (used for group identification).
platform: The platform ("android" or "ios") for platform-specific JID queries.
Returns:
A SQL condition string.
Raises:
ValueError: If the column count is invalid or an unsupported platform is provided.
"""
if filter is not None:
conditions = []
if len(columns) < 2 and jid is not None:
raise ValueError("There must be at least two elements in argument columns if jid is not None")
if jid is not None:
if platform == "android":
is_group = f"{jid}.type == 1"
elif platform == "ios":
is_group = f"{jid} IS NOT NULL"
else:
raise ValueError("Only android and ios are supported for argument platform if jid is not None")
for index, chat in enumerate(filter):
if include:
conditions.append(f"{' OR' if index > 0 else ''} {columns[0]} LIKE '%{chat}%'")
if len(columns) > 1:
conditions.append(f" OR ({columns[1]} LIKE '%{chat}%' AND {is_group})")
else:
conditions.append(f"{' AND' if index > 0 else ''} {columns[0]} NOT LIKE '%{chat}%'")
if len(columns) > 1:
conditions.append(f" AND ({columns[1]} NOT LIKE '%{chat}%' AND {is_group})")
return f"AND ({' '.join(conditions)})"
else:
return ""
# Android Specific
CRYPT14_OFFSETS = (
{"iv": 67, "db": 191},
{"iv": 67, "db": 190},
{"iv": 66, "db": 99},
{"iv": 67, "db": 193},
{"iv": 67, "db": 194},
{"iv": 67, "db": 158},
{"iv": 67, "db": 196}
)
class Crypt(IntEnum):
CRYPT15 = 15
CRYPT14 = 14
CRYPT12 = 12
class DbType(StrEnum):
MESSAGE = "message"
CONTACT = "contact"
def determine_metadata(content: sqlite3.Row, init_msg: Optional[str]) -> Optional[str]:
"""Determines the metadata of a message.
Args:
content (sqlite3.Row): A row from the messages table.
init_msg (Optional[str]): The initial message, if any.
Returns:
The metadata as a string or None if the type is unsupported.
"""
msg = init_msg if init_msg else ""
if content["is_me_joined"] == 1: # Override
return f"You were added into the group by {msg}"
if content["action_type"] == 1:
msg += f''' changed the group name to "{content['data']}"'''
elif content["action_type"] == 4:
msg += " was added to the group"
elif content["action_type"] == 5:
msg += " left the group"
elif content["action_type"] == 6:
msg += f" changed the group icon"
elif content["action_type"] == 7:
msg = "You were removed"
elif content["action_type"] == 8:
msg += ("WhatsApp Internal Error Occurred: "
"you cannot send message to this group")
elif content["action_type"] == 9:
msg += " created a broadcast channel"
elif content["action_type"] == 10:
try:
old = content['old_jid'].split('@')[0]
new = content['new_jid'].split('@')[0]
except (AttributeError, IndexError):
return None
else:
msg = f"{old} changed their number to {new}"
elif content["action_type"] == 11:
msg += f''' created a group with name: "{content['data']}"'''
elif content["action_type"] == 12:
msg += f" added someone" # TODO: Find out who
elif content["action_type"] == 13:
return # Someone left the group
elif content["action_type"] == 14:
msg += f" removed someone" # TODO: Find out who
elif content["action_type"] == 15:
return # Someone promoted someone as an admin
elif content["action_type"] == 18:
if msg != "You":
msg = f"The security code between you and {msg} changed"
else:
msg = "The security code in this chat changed"
elif content["action_type"] == 19:
msg = "This chat is now end-to-end encrypted"
elif content["action_type"] == 20:
msg = "Someone joined this group by using a invite link" # TODO: Find out who
elif content["action_type"] == 27:
msg += " changed the group description to:<br>"
msg += (content['data'] or "Unknown").replace("\n", '<br>')
elif content["action_type"] == 28:
try:
old = content['old_jid'].split('@')[0]
new = content['new_jid'].split('@')[0]
except (AttributeError, IndexError):
return None
else:
msg = f"{old} changed their number to {new}"
elif content["action_type"] == 46:
return # Voice message in PM??? Seems no need to handle.
elif content["action_type"] == 47:
msg = "The contact is an official business account"
elif content["action_type"] == 50:
msg = "The contact's account type changed from business to standard"
elif content["action_type"] == 56:
msg = "Messgae timer was enabled/updated/disabled"
elif content["action_type"] == 57:
if msg != "You":
msg = f"The security code between you and {msg} changed"
else:
msg = "The security code in this chat changed"
elif content["action_type"] == 58:
msg = "You blocked this contact"
elif content["action_type"] == 67:
return # (PM) this contact use secure service from Facebook???
elif content["action_type"] == 69:
return # (PM) this contact use secure service from Facebook??? What's the difference with 67????
else:
return # Unsupported
return msg
def get_status_location(output_folder: str, offline_static: str) -> str:
"""
Gets the location of the W3.CSS file, either from web or local storage.
Args:
output_folder (str): The folder where offline static files will be stored.
offline_static (str): The subfolder name for static files. If falsy, returns web URL.
Returns:
str: The path or URL to the W3.CSS file.
"""
w3css = "https://www.w3schools.com/w3css/4/w3.css"
if not offline_static:
return w3css
import urllib.request
static_folder = os.path.join(output_folder, offline_static)
if not os.path.isdir(static_folder):
os.mkdir(static_folder)
w3css_path = os.path.join(static_folder, "w3.css")
if not os.path.isfile(w3css_path):
with urllib.request.urlopen(w3css) as resp:
with open(w3css_path, "wb") as f: f.write(resp.read())
w3css = os.path.join(offline_static, "w3.css")
def setup_template(template: Optional[str], no_avatar: bool, experimental: bool = False) -> jinja2.Template:
"""
Sets up the Jinja2 template environment and loads the template.
Args:
template (Optional[str]): Path to custom template file. If None, uses default template.
no_avatar (bool): Whether to disable avatar display in the template.
experimental (bool, optional): Whether to use experimental template features. Defaults to False.
Returns:
jinja2.Template: The configured Jinja2 template object.
"""
if template is None or experimental:
template_dir = os.path.dirname(__file__)
template_file = "whatsapp.html" if not experimental else template
else:
template_dir = os.path.dirname(template)
template_file = os.path.basename(template)
template_loader = jinja2.FileSystemLoader(searchpath=template_dir)
template_env = jinja2.Environment(loader=template_loader, autoescape=True)
template_env.globals.update(
determine_day=determine_day,
no_avatar=no_avatar
)
template_env.filters['sanitize_except'] = sanitize_except
return template_env.get_template(template_file)
# iOS Specific
APPLE_TIME = 978307200
def slugify(value: str, allow_unicode: bool = False) -> str:
"""
Convert text to ASCII-only slugs for URL-safe strings.
Taken from https://github.com/django/django/blob/master/django/utils/text.py
Args:
value (str): The string to convert to a slug.
allow_unicode (bool, optional): Whether to allow Unicode characters. Defaults to False.
Returns:
str: The slugified string with only alphanumerics, underscores, or hyphens.
"""
value = str(value)
if allow_unicode:
value = unicodedata.normalize('NFKC', value)
else:
value = unicodedata.normalize('NFKD', value).encode('ascii', 'ignore').decode('ascii')
value = re.sub(r'[^\w\s-]', '', value.lower())
return re.sub(r'[-\s]+', '-', value).strip('-_')
class WhatsAppIdentifier(StrEnum):
MESSAGE = "7c7fba66680ef796b916b067077cc246adacf01d" # AppDomainGroup-group.net.whatsapp.WhatsApp.shared-ChatStorage.sqlite
CONTACT = "b8548dc30aa1030df0ce18ef08b882cf7ab5212f" # AppDomainGroup-group.net.whatsapp.WhatsApp.shared-ContactsV2.sqlite
CALL = "1b432994e958845fffe8e2f190f26d1511534088" # AppDomainGroup-group.net.whatsapp.WhatsApp.shared-CallHistory.sqlite
DOMAIN = "AppDomainGroup-group.net.whatsapp.WhatsApp.shared"
class WhatsAppBusinessIdentifier(StrEnum):
MESSAGE = "724bd3b98b18518b455a87c1f3ac3a0d189c4466" # AppDomainGroup-group.net.whatsapp.WhatsAppSMB.shared-ChatStorage.sqlite
CONTACT = "d7246a707f51ddf8b17ee2dddabd9e0a4da5c552" # AppDomainGroup-group.net.whatsapp.WhatsAppSMB.shared-ContactsV2.sqlite
CALL = "b463f7c4365eefc5a8723930d97928d4e907c603" # AppDomainGroup-group.net.whatsapp.WhatsAppSMB.shared-CallHistory.sqlite
DOMAIN = "AppDomainGroup-group.net.whatsapp.WhatsAppSMB.shared"
class JidType(IntEnum):
PM = 0
GROUP = 1
SYSTEM_BROADCAST = 5
STATUS = 11

View File

@@ -0,0 +1,82 @@
import vobject
from typing import List, TypedDict
class ExportedContactNumbers(TypedDict):
full_name: str
numbers: List[str]
class ContactsFromVCards:
def __init__(self) -> None:
self.contact_mapping = []
def is_empty(self):
return self.contact_mapping == []
def load_vcf_file(self, vcf_file_path: str, default_country_code: str):
self.contact_mapping = read_vcards_file(vcf_file_path, default_country_code)
def enrich_from_vcards(self, chats):
for number, name in self.contact_mapping:
# short number must be a bad contact, lets skip it
if len(number) <= 5:
continue
for chat in filter_chats_by_prefix(chats, number).values():
if not hasattr(chat, 'name') or (hasattr(chat, 'name') and chat.name is None):
setattr(chat, 'name', name)
def read_vcards_file(vcf_file_path, default_country_code: str):
contacts = []
with open(vcf_file_path, mode="r", encoding="utf-8") as f:
reader = vobject.readComponents(f)
for row in reader:
if hasattr(row, 'fn'):
name = str(row.fn.value)
elif hasattr(row, 'n'):
name = str(row.n.value)
else:
name = None
if not hasattr(row, 'tel') or name is None:
continue
contact: ExportedContactNumbers = {
"full_name": name,
"numbers": list(map(lambda tel: tel.value, row.tel_list)),
}
contacts.append(contact)
return map_number_to_name(contacts, default_country_code)
def filter_chats_by_prefix(chats, prefix: str):
return {k: v for k, v in chats.items() if k.startswith(prefix)}
def map_number_to_name(contacts, default_country_code: str):
mapping = []
for contact in contacts:
for index, num in enumerate(contact['numbers']):
normalized = normalize_number(num, default_country_code)
if len(contact['numbers']) > 1:
name = f"{contact['full_name']} ({index+1})"
else:
name = contact['full_name']
mapping.append((normalized, name))
return mapping
def normalize_number(number: str, country_code: str):
# Clean the number
number = ''.join(c for c in number if c.isdigit() or c == "+")
# A number that starts with a + or 00 means it already have a country code
for starting_char in ('+', "00"):
if number.startswith(starting_char):
return number[len(starting_char):]
# leading zero should be removed
if starting_char == '0':
number = number[1:]
return country_code + number # fall back

View File

@@ -0,0 +1,20 @@
# from contacts_names_from_vcards import readVCardsFile
from Whatsapp_Chat_Exporter.vcards_contacts import normalize_number, read_vcards_file
def test_readVCardsFile():
assert len(read_vcards_file("contacts.vcf", "973")) > 0
def test_create_number_to_name_dicts():
pass
def test_fuzzy_match_numbers():
pass
def test_normalize_number():
assert normalize_number('0531234567', '1') == '1531234567'
assert normalize_number('001531234567', '2') == '1531234567'
assert normalize_number('+1531234567', '34') == '1531234567'
assert normalize_number('053(123)4567', '34') == '34531234567'
assert normalize_number('0531-234-567', '58') == '58531234567'

View File

@@ -2,11 +2,10 @@
<html>
<head>
<title>Whatsapp - {{ name }}</title>
<link rel="stylesheet" href="https://www.w3schools.com/w3css/4/w3.css">
<meta charset="UTF-8">
<link rel="stylesheet" href="{{w3css}}">
<style>
@import url('https://fonts.googleapis.com/css2?family=Noto+Sans+HK:wght@300;400&display=swap');
html {
font-family: 'Noto Sans HK', sans-serif;
html, body {
font-size: 12px;
scroll-behavior: smooth;
}
@@ -21,7 +20,6 @@
}
footer {
border-top: 2px solid #e3e6e7;
font-size: 2em;
padding: 20px 0 20px 0;
}
article {
@@ -34,128 +32,233 @@
img, video {
max-width:100%;
}
a.anchor {
display: block;
position: relative;
top: -100px;
visibility: hidden;
}
div.reply{
font-size: 13px;
text-decoration: none;
}
div:target::before {
content: '';
display: block;
height: 115px;
margin-top: -115px;
visibility: hidden;
}
div:target {
border-style: solid;
border-width: 2px;
animation: border-blink 0.5s steps(1) 5;
border-color: rgba(0,0,0,0)
}
table {
width: 100%;
}
@keyframes border-blink {
0% {
border-color: #2196F3;
}
50% {
border-color: rgba(0,0,0,0);
}
}
.avatar {
border-radius:50%;
overflow:hidden;
max-width: 64px;
max-height: 64px;
}
.name {
color: #3892da;
}
.pad-left-10 {
padding-left: 10px;
}
.pad-right-10 {
padding-right: 10px;
}
.reply_link {
color: #168acc;
}
.blue {
color: #70777a;
}
.sticker {
max-width: 100px !important;
max-height: 100px !important;
}
</style>
<base href="{{ media_base }}" target="_blank">
</head>
<body>
<header class="w3-center w3-top">Chat history with {{ name }}</header>
<header class="w3-center w3-top">
{{ headline }}
{% if status is not none %}
<br>
<span class="w3-small">{{ status }}</span>
{% endif %}
</header>
<article class="w3-container">
<div class="table" style="width:100%">
<div class="table">
{% set last = {'last': 946688461.001} %}
{% for msg in msgs -%}
<div class="w3-row" style="padding-bottom: 10px">
<a class="anchor" id="{{ msg.key_id }}"></a>
<div class="w3-row w3-padding-small w3-margin-bottom" id="{{ msg.key_id }}">
{% if determine_day(last.last, msg.timestamp) is not none %}
<div class="w3-center" style="color:#70777c;padding: 10px 0 10px 0;">{{ determine_day(last.last, msg.timestamp) }}</div>
<div class="w3-center w3-padding-16 blue">{{ determine_day(last.last, msg.timestamp) }}</div>
{% if last.update({'last': msg.timestamp}) %}{% endif %}
{% endif %}
{% if msg.from_me == true %}
<div class="w3-row">
<div style="float: left; color:#70777c;">{{ msg.time }}</div>
<div style="padding-left: 10px; text-align: right; color: #3892da;">You</div>
<div class="w3-left blue">{{ msg.time }}</div>
<div class="name w3-right-align pad-left-10">You</div>
</div>
<div class="w3-row">
{% if not no_avatar and my_avatar is not none %}
<div class="w3-col m10 l10">
<div style="text-align: right;">
{% else %}
<div class="w3-col m12 l12">
{% endif %}
<div class="w3-right-align">
{% if msg.reply is not none %}
<div class="reply">
<span style="color: #70777a;">Replying to </span>
<a href="#{{msg.reply}}" style="color: #168acc;">"{{ msg.quoted_data or 'media' }}"</a>
<span class="blue">Replying to </span>
<a href="#{{msg.reply}}" target="_self" class="reply_link no-base">
{% if msg.quoted_data is not none %}
"{{msg.quoted_data}}"
{% else %}
this message
{% endif %}
</a>
</div>
{% endif %}
{% if msg.meta == true or msg.media == false and msg.data is none %}
<div style="text-align: center;" class="w3-panel w3-border-blue w3-pale-blue w3-rightbar w3-leftbar">
<p>{{ msg.data or 'This message is not supported' }}</p>
</div>
<div class="w3-panel w3-border-blue w3-pale-blue w3-rightbar w3-leftbar w3-threequarter w3-center">
{% if msg.safe %}
<p>{{ msg.data | safe or 'Not supported WhatsApp internal message' }}</p>
{% else %}
<p>{{ msg.data or 'Not supported WhatsApp internal message' }}</p>
{% endif %}
</div>
{% if msg.caption is not none %}
<div class="w3-container">
{{ msg.caption | urlize(none, true, '_blank') }}
</div>
{% endif %}
{% else %}
{% if msg.media == false %}
{{ msg.data | sanitize_except() }}
{{ msg.data | sanitize_except() | urlize(none, true, '_blank') }}
{% else %}
{% if "image/" in msg.mime %}
<a href="{{ msg.data }}"><img src="{{ msg.data }}" /></a>
<a href="{{ msg.data }}">
<img src="{{ msg.thumb if msg.thumb is not none else msg.data }}" {{ 'class="sticker"' | safe if msg.sticker }} loading="lazy"/>
</a>
{% elif "audio/" in msg.mime %}
<audio controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
</audio>
{% elif "video/" in msg.mime %}
<video controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
<video class="lazy" autobuffer {% if msg.message_type|int == 13 or msg.message_type|int == 11 %}autoplay muted loop playsinline{%else%}controls{% endif %}>
<source type="{{ msg.mime }}" data-src="{{ msg.data }}" />
</video>
{% elif "/" in msg.mime %}
<div style="text-align: center;" class="w3-panel w3-border-blue w3-pale-blue w3-rightbar w3-leftbar">
<p>The file cannot be displayed here, however it should be located at {{ msg.data }}</p>
<div class="w3-panel w3-border-blue w3-pale-blue w3-rightbar w3-leftbar w3-threequarter w3-center">
<p>The file cannot be displayed here, however it should be located at <a href="./{{ msg.data }}">here</a></p>
</div>
{% else %}
{% filter escape %}{{ msg.data }}{% endfilter %}
{% endif %}
{% if msg.caption is not none %}
<br>
{{ msg.caption }}
<div class="w3-container">
{{ msg.caption | urlize(none, true, '_blank') }}
</div>
{% endif %}
{% endif %}
{% endif %}
{% endif %}
</div>
</div>
<div class="w3-col m2 l2" style="padding-left: 10px"><img src="{{ my_avatar }}" onerror="this.style.display='none'"></div>
{% if not no_avatar and my_avatar is not none %}
<div class="w3-col m2 l2 pad-left-10">
<a href="{{ my_avatar }}">
<img src="{{ my_avatar }}" onerror="this.style.display='none'" class="avatar" loading="lazy">
</a>
</div>
{% endif %}
</div>
{% else %}
<div class="w3-row">
<div style="padding-right: 10px; float: left; color: #3892da;">
<div class="w3-left pad-right-10 name">
{% if msg.sender is not none %}
{{ msg.sender }}
{% else %}
{{ name }}
{% endif %}
</div>
<div style="text-align: right; color:#70777c;">{{ msg.time }}</div>
<div class="w3-right-align blue">{{ msg.time }}</div>
</div>
<div class="w3-row">
<div class="w3-col m2 l2"><img src="{{ their_avatar }}" onerror="this.style.display='none'"></div>
{% if not no_avatar %}
<div class="w3-col m2 l2">
{% if their_avatar is not none %}
<a href="{{ their_avatar }}"><img src="{{ their_avatar_thumb or '' }}" onerror="this.style.display='none'" class="avatar" loading="lazy"></a>
{% else %}
<img src="{{ their_avatar_thumb or '' }}" onerror="this.style.display='none'" class="avatar" loading="lazy">
{% endif %}
</div>
<div class="w3-col m10 l10">
<div style="text-align: left;">
{% else %}
<div class="w3-col m12 l12">
{% endif %}
<div class="w3-left-align">
{% if msg.reply is not none %}
<div class="reply">
<span style="color: #70777a;">Replying to </span>
<a href="#{{msg.reply}}" style="color: #168acc;">"{{ msg.quoted_data or 'media' }}"</a>
<span class="blue">Replying to </span>
<a href="#{{msg.reply}}" target="_self" class="reply_link no-base">
{% if msg.quoted_data is not none %}
"{{msg.quoted_data}}"
{% else %}
this message
{% endif %}
</a>
</div>
{% endif %}
{% if msg.meta == true or msg.media == false and msg.data is none %}
<div style="text-align: center;" class="w3-panel w3-border-blue w3-pale-blue w3-rightbar w3-leftbar">
<p>{{ msg.data or 'This message is not supported' }}</p>
<div class="w3-panel w3-border-blue w3-pale-blue w3-rightbar w3-leftbar w3-threequarter w3-center">
{% if msg.safe %}
<p>{{ msg.data | safe or 'Not supported WhatsApp internal message' }}</p>
{% else %}
<p>{{ msg.data or 'Not supported WhatsApp internal message' }}</p>
{% endif %}
</div>
{% if msg.caption is not none %}
<div class="w3-container">
{{ msg.caption | urlize(none, true, '_blank') }}
</div>
{% endif %}
{% else %}
{% if msg.media == false %}
{{ msg.data | sanitize_except() }}
{{ msg.data | sanitize_except() | urlize(none, true, '_blank') }}
{% else %}
{% if "image/" in msg.mime %}
<a href="{{ msg.data }}"><img src="{{ msg.data }}" /></a>
<a href="{{ msg.data }}">
<img src="{{ msg.thumb if msg.thumb is not none else msg.data }}" {{ 'class="sticker"' | safe if msg.sticker }} loading="lazy"/>
</a>
{% elif "audio/" in msg.mime %}
<audio controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
</audio>
{% elif "video/" in msg.mime %}
<video controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
<video class="lazy" autobuffer {% if msg.message_type|int == 13 or msg.message_type|int == 11 %}autoplay muted loop playsinline{%else%}controls{% endif %}>
<source type="{{ msg.mime }}" data-src="{{ msg.data }}" />
</video>
{% elif "/" in msg.mime %}
<div style="text-align: center;" class="w3-panel w3-border-blue w3-pale-blue w3-rightbar w3-leftbar">
<p>The file cannot be displayed here, however it should be located at {{ msg.data }}</p>
<div class="w3-panel w3-border-blue w3-pale-blue w3-rightbar w3-leftbar w3-threequarter w3-center">
<p>The file cannot be displayed here, however it should be located at <a href="./{{ msg.data }}">here</a></p>
</div>
{% else %}
{% filter escape %}{{ msg.data }}{% endfilter %}
{% endif %}
{% if msg.caption is not none %}
<br>
{{ msg.caption }}
<div class="w3-container">
{{ msg.caption | urlize(none, true, '_blank') }}
</div>
{% endif %}
{% endif %}
{% endif %}
@@ -168,7 +271,59 @@
</div>
</article>
<footer class="w3-center">
End of history
<h2>
{% if previous %}
<a href="./{{ previous }}" target="_self">Previous</a>
{% endif %}
<h2>
{% if next %}
<a href="./{{ next }}" target="_self">Next</a>
{% else %}
End of History
{% endif %}
</h2>
<br>
Portions of this page are reproduced from <a href="https://web.dev/articles/lazy-loading-video">work</a> created and <a href="https://developers.google.com/readme/policies">shared by Google</a> and used according to terms described in the <a href="https://www.apache.org/licenses/LICENSE-2.0">Apache 2.0 License</a>.
</footer>
<script>
document.addEventListener("DOMContentLoaded", function() {
var lazyVideos = [].slice.call(document.querySelectorAll("video.lazy"));
if ("IntersectionObserver" in window) {
var lazyVideoObserver = new IntersectionObserver(function(entries, observer) {
entries.forEach(function(video) {
if (video.isIntersecting) {
for (var source in video.target.children) {
var videoSource = video.target.children[source];
if (typeof videoSource.tagName === "string" && videoSource.tagName === "SOURCE") {
videoSource.src = videoSource.dataset.src;
}
}
video.target.load();
video.target.classList.remove("lazy");
lazyVideoObserver.unobserve(video.target);
}
});
});
lazyVideos.forEach(function(lazyVideo) {
lazyVideoObserver.observe(lazyVideo);
});
}
});
</script>
<script>
// Prevent the <base> tag from affecting links with the class "no-base"
document.querySelectorAll('.no-base').forEach(link => {
link.addEventListener('click', function(event) {
const href = this.getAttribute('href');
if (href.startsWith('#')) {
window.location.hash = href;
event.preventDefault();
}
});
});
</script>
</body>
</html>

View File

@@ -0,0 +1,467 @@
<!DOCTYPE html>
<html>
<head>
<title>Whatsapp - {{ name }}</title>
<meta charset="UTF-8">
<script src="https://cdn.tailwindcss.com"></script>
<script>
tailwind.config = {
theme: {
extend: {
colors: {
whatsapp: {
light: '#e7ffdb',
DEFAULT: '#25D366',
dark: '#075E54',
chat: '#efeae2',
'chat-light': '#f0f2f5',
}
}
}
}
}
</script>
<style>
body, html {
height: 100%;
margin: 0;
padding: 0;
scroll-behavior: smooth !important;
}
.chat-list {
height: calc(100vh - 120px);
overflow-y: auto;
}
.message-list {
height: calc(100vh - 90px);
overflow-y: auto;
}
@media (max-width: 640px) {
.chat-list, .message-list {
height: calc(100vh - 108px);
}
}
header {
position: fixed;
z-index: 20;
border-bottom: 2px solid #e3e6e7;
font-size: 2em;
font-weight: bolder;
background-color: white;
padding: 20px 0 20px 0;
}
footer {
margin-top: 10px;
border-top: 2px solid #e3e6e7;
padding: 20px 0 20px 0;
}
article {
width:430px;
margin: auto;
z-index:10;
font-size: 15px;
word-wrap: break-word;
}
img, video, audio{
max-width:100%;
box-sizing: border-box;
}
div.reply{
font-size: 13px;
text-decoration: none;
}
div:target::before {
content: '';
display: block;
height: 115px;
margin-top: -115px;
visibility: hidden;
}
div:target {
animation: 3s highlight;
}
.avatar {
border-radius:50%;
overflow:hidden;
max-width: 64px;
max-height: 64px;
}
.name {
color: #3892da;
}
.pad-left-10 {
padding-left: 10px;
}
.pad-right-10 {
padding-right: 10px;
}
.reply_link {
color: #168acc;
}
.blue {
color: #70777a;
}
.sticker {
max-width: 100px !important;
max-height: 100px !important;
}
@keyframes highlight {
from {
background-color: rgba(37, 211, 102, 0.1);
}
to {
background-color: transparent;
}
}
.search-input {
transform: translateY(-100%);
transition: transform 0.3s ease-in-out;
}
.search-input.active {
transform: translateY(0);
}
.reply-box:active {
background-color:rgb(200 202 205 / var(--tw-bg-opacity, 1));
}
.info-box-tooltip {
--tw-translate-x: -50%;
transform: translate(var(--tw-translate-x), var(--tw-translate-y)) rotate(var(--tw-rotate)) skewX(var(--tw-skew-x)) skewY(var(--tw-skew-y)) scaleX(var(--tw-scale-x)) scaleY(var(--tw-scale-y));
}
</style>
<script>
function search(event) {
keywords = document.getElementById("mainHeaderSearchInput").value;
hits = [];
document.querySelectorAll(".message-text").forEach(elem => {
if (elem.innerText.trim().includes(keywords)){
hits.push(elem.parentElement.parentElement.id);
}
})
console.log(hits);
}
</script>
<base href="{{ media_base }}" target="_blank">
</head>
<body>
<article class="h-screen bg-whatsapp-chat-light">
<div class="w-full flex flex-col">
<div class="p-3 bg-whatsapp-dark flex items-center justify-between border-l border-[#d1d7db]">
<div class="flex items-center">
{% if not no_avatar %}
<div class="w3-col m2 l2">
{% if their_avatar is not none %}
<a href="{{ their_avatar }}"><img src="{{ their_avatar_thumb or '' }}" onerror="this.style.display='none'" class="w-10 h-10 rounded-full mr-3" loading="lazy"></a>
{% else %}
<img src="{{ their_avatar_thumb or '' }}" onerror="this.style.display='none'" class="w-10 h-10 rounded-full mr-3" loading="lazy">
{% endif %}
</div>
{% endif %}
<div>
<h2 class="text-white font-medium">{{ headline }}</h2>
{% if status is not none %}<p class="text-[#8696a0] text-xs">{{ status }}</p>{% endif %}
</div>
</div>
<div class="flex space-x-4">
<!-- <button id="searchButton">
<svg xmlns="http://www.w3.org/2000/svg" class="h-5 w-5 text-[#aebac1]" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M21 21l-6-6m2-5a7 7 0 11-14 0 7 7 0 0114 0z" />
</svg>
</button> -->
<!-- <svg xmlns="http://www.w3.org/2000/svg" class="h-5 w-5 text-[#aebac1]" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M15 19l-7-7 7-7" />
</svg> -->
{% if previous %}
<a href="./{{ previous }}" target="_self">
<svg xmlns="http://www.w3.org/2000/svg" class="h-5 w-5 text-[#aebac1]" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M15 5l-7 7 7 7" />
</svg>
</a>
{% endif %}
{% if next %}
<a href="./{{ next }}" target="_self">
<svg xmlns="http://www.w3.org/2000/svg" class="h-5 w-5 text-[#aebac1]" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 5l7 7-7 7" />
</svg>
</a>
{% endif %}
</div>
<!-- Search Input Overlay -->
<div id="mainSearchInput" class="search-input absolute article top-0 bg-whatsapp-dark p-3 flex items-center space-x-3">
<button id="closeMainSearch" class="text-[#aebac1]">
<svg xmlns="http://www.w3.org/2000/svg" class="h-6 w-6" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M15 19l-7-7 7-7" />
</svg>
</button>
<input type="text" placeholder="Search..." class="flex-1 bg-[#1f2c34] text-white rounded-lg px-3 py-1 focus:outline-none" id="mainHeaderSearchInput" onkeyup="search(event)">
</div>
</div>
</div>
<div class="flex-1 p-5 message-list">
<div class="flex flex-col space-y-2">
<!--Date-->
{% set last = {'last': 946688461.001} %}
{% for msg in msgs -%}
{% if determine_day(last.last, msg.timestamp) is not none %}
<div class="flex justify-center">
<div class="bg-[#e1f2fb] rounded-lg px-2 py-1 text-xs text-[#54656f]">
{{ determine_day(last.last, msg.timestamp) }}
</div>
</div>
{% if last.update({'last': msg.timestamp}) %}{% endif %}
{% endif %}
<!--Actual messages-->
{% if msg.from_me == true %}
<div class="flex justify-end items-center group" id="{{ msg.key_id }}">
<div class="opacity-0 group-hover:opacity-100 transition-opacity duration-200 relative mr-2">
<div class="relative">
<div class="relative group/tooltip">
<svg xmlns="http://www.w3.org/2000/svg" class="h-5 w-5 text-[#8696a0] hover:text-[#54656f] cursor-pointer" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<use href="#info-icon"></use>
</svg>
<div class="absolute bottom-full info-box-tooltip mb-2 hidden group-hover/tooltip:block z-50">
<div class="bg-black text-white text-xs rounded py-1 px-2 whitespace-nowrap">
Delivered at {{msg.received_timestamp or 'unknown'}}
{% if msg.read_timestamp is not none %}
<br>Read at {{ msg.read_timestamp }}
{% endif %}
</div>
<div class="absolute top-full right-3 -mt-1 border-4 border-transparent border-t-black"></div>
</div>
</div>
</div>
</div>
<div class="bg-whatsapp-light rounded-lg p-2 max-w-[80%] shadow-sm">
{% if msg.reply is not none %}
<a href="#{{msg.reply}}" target="_self" class="no-base">
<div class="mb-2 p-1 bg-whatsapp-chat-light rounded border-l-4 border-whatsapp text-sm reply-box">
<p class="text-whatsapp font-medium text-xs">Replying to</p>
<p class="text-[#111b21] text-xs truncate">
{% if msg.quoted_data is not none %}
"{{msg.quoted_data}}"
{% else %}
this message
{% endif %}
</p>
</div>
</a>
{% endif %}
<p class="text-[#111b21] text-sm message-text">
{% if msg.meta == true or msg.media == false and msg.data is none %}
<div class="flex justify-center mb-2">
<div class="bg-[#FFF3C5] rounded-lg px-3 py-2 text-sm text-[#856404] flex items-center">
{% if msg.safe %}
{{ msg.data | safe or 'Not supported WhatsApp internal message' }}
{% else %}
{{ msg.data or 'Not supported WhatsApp internal message' }}
{% endif %}
</div>
</div>
{% if msg.caption is not none %}
<p>{{ msg.caption | urlize(none, true, '_blank') }}</p>
{% endif %}
{% else %}
{% if msg.media == false %}
{{ msg.data | sanitize_except() | urlize(none, true, '_blank') }}
{% else %}
{% if "image/" in msg.mime %}
<a href="{{ msg.data }}">
<img src="{{ msg.thumb if msg.thumb is not none else msg.data }}" {{ 'class="sticker"' | safe if msg.sticker }} loading="lazy"/>
</a>
{% elif "audio/" in msg.mime %}
<audio controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
</audio>
{% elif "video/" in msg.mime %}
<video class="lazy" autobuffer {% if msg.message_type|int == 13 or msg.message_type|int == 11 %}autoplay muted loop playsinline{%else%}controls{% endif %}>
<source type="{{ msg.mime }}" data-src="{{ msg.data }}" />
</video>
{% elif "/" in msg.mime %}
The file cannot be displayed here, however it should be located at <a href="./{{ msg.data }}">here</a>
{% else %}
{% filter escape %}{{ msg.data }}{% endfilter %}
{% endif %}
{% if msg.caption is not none %}
{{ msg.caption | urlize(none, true, '_blank') }}
{% endif %}
{% endif %}
{% endif %}
</p>
<p class="text-[10px] text-[#667781] text-right mt-1">{{ msg.time }}</p>
</div>
</div>
{% else %}
<div class="flex justify-start items-center group" id="{{ msg.key_id }}">
<div class="bg-white rounded-lg p-2 max-w-[80%] shadow-sm">
{% if msg.reply is not none %}
<a href="#{{msg.reply}}" target="_self" class="no-base">
<div class="mb-2 p-1 bg-whatsapp-chat-light rounded border-l-4 border-whatsapp text-sm reply-box">
<p class="text-whatsapp font-medium text-xs">Replying to</p>
<p class="text-[#808080] text-xs truncate">
{% if msg.quoted_data is not none %}
{{msg.quoted_data}}
{% else %}
this message
{% endif %}
</p>
</div>
</a>
{% endif %}
<p class="text-[#111b21] text-sm">
{% if msg.meta == true or msg.media == false and msg.data is none %}
<div class="flex justify-center mb-2">
<div class="bg-[#FFF3C5] rounded-lg px-3 py-2 text-sm text-[#856404] flex items-center">
{% if msg.safe %}
{{ msg.data | safe or 'Not supported WhatsApp internal message' }}
{% else %}
{{ msg.data or 'Not supported WhatsApp internal message' }}
{% endif %}
</div>
</div>
{% if msg.caption is not none %}
<p>{{ msg.caption | urlize(none, true, '_blank') }}</p>
{% endif %}
{% else %}
{% if msg.media == false %}
{{ msg.data | sanitize_except() | urlize(none, true, '_blank') }}
{% else %}
{% if "image/" in msg.mime %}
<a href="{{ msg.data }}">
<img src="{{ msg.thumb if msg.thumb is not none else msg.data }}" {{ 'class="sticker"' | safe if msg.sticker }} loading="lazy"/>
</a>
{% elif "audio/" in msg.mime %}
<audio controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
</audio>
{% elif "video/" in msg.mime %}
<video class="lazy" autobuffer {% if msg.message_type|int == 13 or msg.message_type|int == 11 %}autoplay muted loop playsinline{%else%}controls{% endif %}>
<source type="{{ msg.mime }}" data-src="{{ msg.data }}" />
</video>
{% elif "/" in msg.mime %}
The file cannot be displayed here, however it should be located at <a href="./{{ msg.data }}">here</a>
{% else %}
{% filter escape %}{{ msg.data }}{% endfilter %}
{% endif %}
{% if msg.caption is not none %}
{{ msg.caption | urlize(none, true, '_blank') }}
{% endif %}
{% endif %}
{% endif %}
</p>
<div class="flex items-baseline text-[10px] text-[#667781] mt-1 gap-2">
<span class="flex-shrink-0">
{% if msg.sender is not none %}
{{ msg.sender }}
{% endif %}
</span>
<span class="flex-grow min-w-[4px]"></span>
<span class="flex-shrink-0">{{ msg.time }}</span>
</div>
</div>
<!-- <div class="opacity-0 group-hover:opacity-100 transition-opacity duration-200 relative ml-2">
<div class="relative">
<div class="relative group/tooltip">
<svg xmlns="http://www.w3.org/2000/svg" class="h-5 w-5 text-[#8696a0] hover:text-[#54656f] cursor-pointer" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<use href="#info-icon"></use>
</svg>
<div class="absolute bottom-full info-box-tooltip mb-2 hidden group-hover/tooltip:block z-50">
<div class="bg-black text-white text-xs rounded py-1 px-2 whitespace-nowrap">
Received at {{msg.received_timestamp or 'unknown'}}
</div>
<div class="absolute top-full right-3 ml-1 border-4 border-transparent border-t-black"></div>
</div>
</div>
</div>
</div> -->
</div>
{% endif %}
{% endfor %}
</div>
<footer>
<h2 class="text-center">
{% if not next %}
End of History
{% endif %}
</h2>
<br>
Portions of this page are reproduced from <a href="https://web.dev/articles/lazy-loading-video">work</a> created and <a href="https://developers.google.com/readme/policies">shared by Google</a> and used according to terms described in the <a href="https://www.apache.org/licenses/LICENSE-2.0">Apache 2.0 License</a>.
</footer>
<svg style="display: none;">
<!-- Tooltip info icon -->
<symbol id="info-icon" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13 16h-1v-4h-1m1-4h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z" />
</symbol>
</svg>
</div>
</article>
</body>
<script>
// Search functionality
const searchButton = document.getElementById('searchButton');
const mainSearchInput = document.getElementById('mainSearchInput');
const closeMainSearch = document.getElementById('closeMainSearch');
const mainHeaderSearchInput = document.getElementById('mainHeaderSearchInput');
// Function to show search input
const showSearch = () => {
mainSearchInput.classList.add('active');
mainHeaderSearchInput.focus();
};
// Function to hide search input
const hideSearch = () => {
mainSearchInput.classList.remove('active');
mainHeaderSearchInput.value = '';
};
// Event listeners
searchButton.addEventListener('click', showSearch);
closeMainSearch.addEventListener('click', hideSearch);
// Handle ESC key
document.addEventListener('keydown', (event) => {
if (event.key === 'Escape' && mainSearchInput.classList.contains('active')) {
hideSearch();
}
});
</script>
<script>
document.addEventListener("DOMContentLoaded", function() {
var lazyVideos = [].slice.call(document.querySelectorAll("video.lazy"));
if ("IntersectionObserver" in window) {
var lazyVideoObserver = new IntersectionObserver(function(entries, observer) {
entries.forEach(function(video) {
if (video.isIntersecting) {
for (var source in video.target.children) {
var videoSource = video.target.children[source];
if (typeof videoSource.tagName === "string" && videoSource.tagName === "SOURCE") {
videoSource.src = videoSource.dataset.src;
}
}
video.target.load();
video.target.classList.remove("lazy");
lazyVideoObserver.unobserve(video.target);
}
});
});
lazyVideos.forEach(function(lazyVideo) {
lazyVideoObserver.observe(lazyVideo);
});
}
});
</script>
<script>
// Prevent the <base> tag from affecting links with the class "no-base"
document.querySelectorAll('.no-base').forEach(link => {
link.addEventListener('click', function(event) {
const href = this.getAttribute('href');
if (href.startsWith('#')) {
window.location.hash = href;
event.preventDefault();
}
});
});
</script>
</html>

View File

@@ -1 +0,0 @@
theme: jekyll-theme-cayman

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 36 KiB

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 38 KiB

After

Width:  |  Height:  |  Size: 126 KiB

View File

@@ -1,34 +0,0 @@
# Whatsapp-Chat-Exporter
A Whatsapp database parser that will give you the history of your Whatsapp conversations in HTML and JSON
**If you plan to uninstall WhatsApp or delete your WhatsApp account, please make a backup of your WhatsApp database. You may want to use this exporter again on the same database in the future as the exporter develops**
# Usage
First, clone this repo, and copy all py and html files to a working directory if you want to do so.
```shell
git clone https://github.com/KnugiHK/Whatsapp-Chat-Exporter.git
```
Then, ready your WhatsApp database, place them in the root of working directory.
* For Android, it is called msgstore.db. If you want name of your contacts, get the contact database, which is called wa.db.
* For iPhone, it is called 7c7fba66680ef796b916b067077cc246adacf01d (YES, a hash).
Next, ready your media folder, place it in the root of working directory.
* For Android, copy the WhatsApp directory from your phone directly.
* For iPhone, run the extract_iphone_media.py, and you will get a folder called Message.
```
python extract_iphone_media.py "C:\Users\[Username]\AppData\Roaming\Apple Computer\MobileSync\Backup\[device id]"
```
And now, you should have something like this:
![Folder structure](imgs/structure.png)
Last, run the script regarding the type of phone.
```
python extract.py & :: Android
python extract_iphone.py & :: iPhone
```
And you will get these:
#### Private Message
![Private Message](imgs/pm.png)
#### Group Message
![Group Message](imgs/group.png)

62
pyproject.toml Normal file
View File

@@ -0,0 +1,62 @@
[build-system]
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "whatsapp-chat-exporter"
version = "0.12.1"
description = "A Whatsapp database parser that provides history of your Whatsapp conversations in HTML and JSON. Android, iOS, iPadOS, Crypt12, Crypt14, Crypt15 supported."
readme = "README.md"
authors = [
{ name = "KnugiHK", email = "hello@knugi.com" }
]
license = { text = "MIT" }
keywords = [
"android", "ios", "parsing", "history", "iphone", "message", "crypt15",
"customizable", "whatsapp", "android-backup", "messages", "crypt14",
"crypt12", "whatsapp-chat-exporter", "whatsapp-export", "iphone-backup",
"whatsapp-database", "whatsapp-database-parser", "whatsapp-conversations"
]
classifiers = [
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
"Development Status :: 4 - Beta",
"Environment :: Console",
"Intended Audience :: End Users/Desktop",
"Topic :: Communications :: Chat",
"Topic :: Utilities",
"Topic :: Database"
]
requires-python = ">=3.9"
dependencies = [
"jinja2",
"bleach"
]
[project.optional-dependencies]
android_backup = ["pycryptodome", "javaobj-py3"]
crypt12 = ["pycryptodome"]
crypt14 = ["pycryptodome"]
crypt15 = ["pycryptodome", "javaobj-py3"]
all = ["pycryptodome", "javaobj-py3", "vobject"]
everything = ["pycryptodome", "javaobj-py3", "vobject"]
backup = ["pycryptodome", "javaobj-py3"]
vcards = ["vobject", "pycryptodome", "javaobj-py3"]
[project.scripts]
wtsexporter = "Whatsapp_Chat_Exporter.__main__:main"
waexporter = "Whatsapp_Chat_Exporter.__main__:main"
whatsapp-chat-exporter = "Whatsapp_Chat_Exporter.__main__:main"
[tool.setuptools.packages.find]
where = ["."]
include = ["Whatsapp_Chat_Exporter"]
[tool.setuptools.package-data]
Whatsapp_Chat_Exporter = ["*.html"]

View File

@@ -0,0 +1,115 @@
"""
This script processes a VCARD file to standardize telephone entries and add a second TEL line with the modified number (removing the extra ninth digit) for contacts with 9-digit subscribers.
It handles numbers that may already include a "+55" prefix and ensures that the output format is consistent.
Contributed by @magpires https://github.com/KnugiHK/WhatsApp-Chat-Exporter/issues/127#issuecomment-2646660625
"""
import re
import argparse
def process_phone_number(raw_phone):
"""
Process the raw phone string from the VCARD and return two formatted numbers:
- The original formatted number, and
- A modified formatted number with the extra (ninth) digit removed, if applicable.
Desired output:
For a number with a 9-digit subscriber:
Original: "+55 {area} {first 5 of subscriber}-{last 4 of subscriber}"
Modified: "+55 {area} {subscriber[1:5]}-{subscriber[5:]}"
For example, for an input that should represent "027912345678", the outputs are:
"+55 27 91234-5678" and "+55 27 1234-5678"
This function handles numbers that may already include a "+55" prefix.
It expects that after cleaning, a valid number (without the country code) should have either 10 digits
(2 for area + 8 for subscriber) or 11 digits (2 for area + 9 for subscriber).
If extra digits are present, it takes the last 11 (or 10) digits.
"""
# Store the original input for processing
number_to_process = raw_phone.strip()
# Remove all non-digit characters
digits = re.sub(r'\D', '', number_to_process)
# If the number starts with '55', remove it for processing
if digits.startswith("55") and len(digits) > 11:
digits = digits[2:]
# Remove trunk zero if present
if digits.startswith("0"):
digits = digits[1:]
# After cleaning, we expect a valid number to have either 10 or 11 digits
# If there are extra digits, use the last 11 (for a 9-digit subscriber) or last 10 (for an 8-digit subscriber)
if len(digits) > 11:
# Here, we assume the valid number is the last 11 digits
digits = digits[-11:]
elif len(digits) > 10 and len(digits) < 11:
# In some cases with an 8-digit subscriber, take the last 10 digits
digits = digits[-10:]
# Check if we have a valid number after processing
if len(digits) not in (10, 11):
return None, None
area = digits[:2]
subscriber = digits[2:]
if len(subscriber) == 9:
# Format the original number (5-4 split, e.g., "91234-5678")
orig_subscriber = f"{subscriber[:5]}-{subscriber[5:]}"
# Create a modified version: drop the first digit of the subscriber to form an 8-digit subscriber (4-4 split)
mod_subscriber = f"{subscriber[1:5]}-{subscriber[5:]}"
original_formatted = f"+55 {area} {orig_subscriber}"
modified_formatted = f"+55 {area} {mod_subscriber}"
elif len(subscriber) == 8:
original_formatted = f"+55 {area} {subscriber[:4]}-{subscriber[4:]}"
modified_formatted = None
else:
# This shouldn't happen given the earlier check, but just to be safe
return None, None
return original_formatted, modified_formatted
def process_vcard(input_vcard, output_vcard):
"""
Process a VCARD file to standardize telephone entries and add a second TEL line
with the modified number (removing the extra ninth digit) for contacts with 9-digit subscribers.
"""
with open(input_vcard, 'r', encoding='utf-8') as file:
lines = file.readlines()
output_lines = []
# Regex to capture any telephone line.
# It matches lines starting with "TEL:" or "TEL;TYPE=..." or with prefixes like "item1.TEL:".
phone_pattern = re.compile(r'^(?P<prefix>.*TEL(?:;TYPE=[^:]+)?):(?P<number>.*)$')
for line in lines:
stripped_line = line.rstrip("\n")
match = phone_pattern.match(stripped_line)
if match:
raw_phone = match.group("number").strip()
orig_formatted, mod_formatted = process_phone_number(raw_phone)
if orig_formatted:
# Always output using the standardized prefix.
output_lines.append(f"TEL;TYPE=CELL:{orig_formatted}\n")
else:
output_lines.append(line)
if mod_formatted:
output_lines.append(f"TEL;TYPE=CELL:{mod_formatted}\n")
else:
output_lines.append(line)
with open(output_vcard, 'w', encoding='utf-8') as file:
file.writelines(output_lines)
if __name__ == '__main__':
parser = argparse.ArgumentParser(
description="Process a VCARD file to standardize telephone entries and add a second TEL line with the modified number (removing the extra ninth digit) for contacts with 9-digit subscribers."
)
parser.add_argument('input_vcard', type=str, help='Input VCARD file')
parser.add_argument('output_vcard', type=str, help='Output VCARD file')
args = parser.parse_args()
process_vcard(args.input_vcard, args.output_vcard)
print(f"VCARD processed and saved to {args.output_vcard}")

View File

@@ -0,0 +1,269 @@
import subprocess
import unittest
import tempfile
import os
from unittest.mock import patch
from brazilian_number_processing import process_phone_number, process_vcard
class TestVCardProcessor(unittest.TestCase):
def test_process_phone_number(self):
"""Test the process_phone_number function with various inputs."""
# Test cases for 9-digit subscriber numbers
test_cases_9_digit = [
# Standard 11-digit number (2 area + 9 subscriber)
("27912345678", "+55 27 91234-5678", "+55 27 1234-5678"),
# With country code prefix
("5527912345678", "+55 27 91234-5678", "+55 27 1234-5678"),
# With plus in country code
("+5527912345678", "+55 27 91234-5678", "+55 27 1234-5678"),
# With spaces and formatting
("+55 27 9 1234-5678", "+55 27 91234-5678", "+55 27 1234-5678"),
# With trunk zero
("027912345678", "+55 27 91234-5678", "+55 27 1234-5678"),
# With country code and trunk zero
("+55027912345678", "+55 27 91234-5678", "+55 27 1234-5678"),
# With extra digits at the beginning (should use last 11)
("99927912345678", "+55 27 91234-5678", "+55 27 1234-5678"),
# With extra non-digit characters
("+55-27-9.1234_5678", "+55 27 91234-5678", "+55 27 1234-5678"),
]
# Test cases for 8-digit subscriber numbers
test_cases_8_digit = [
# Standard 10-digit number (2 area + 8 subscriber)
("2712345678", "+55 27 1234-5678", None),
# With country code prefix
("552712345678", "+55 27 1234-5678", None),
# With plus in country code
("+552712345678", "+55 27 1234-5678", None),
# With spaces and formatting
("+55 27 1234-5678", "+55 27 1234-5678", None),
# With trunk zero
("02712345678", "+55 27 1234-5678", None),
# With country code and trunk zero
("+55 0 27 1234-5678", "+55 27 1234-5678", None),
]
# Edge cases
edge_cases = [
# Too few digits
("271234567", None, None),
# Empty string
("", None, None),
# Non-numeric characters only
("abc-def+ghi", None, None),
# Single digit
("1", None, None),
# Unusual formatting but valid number
("(+55) [27] 9.1234_5678", "+55 27 91234-5678", "+55 27 1234-5678"),
]
# Run tests for all cases
all_cases = test_cases_9_digit + test_cases_8_digit + edge_cases
for raw_phone, expected_orig, expected_mod in all_cases:
with self.subTest(raw_phone=raw_phone):
orig, mod = process_phone_number(raw_phone)
self.assertEqual(orig, expected_orig)
self.assertEqual(mod, expected_mod)
def test_process_vcard(self):
"""Test the process_vcard function with various VCARD formats."""
# Test case 1: Standard TEL entries
vcard1 = """BEGIN:VCARD
VERSION:3.0
N:Doe;John;;;
FN:John Doe
TEL:+5527912345678
TEL:+552712345678
END:VCARD
"""
expected1 = """BEGIN:VCARD
VERSION:3.0
N:Doe;John;;;
FN:John Doe
TEL;TYPE=CELL:+55 27 91234-5678
TEL;TYPE=CELL:+55 27 1234-5678
TEL;TYPE=CELL:+55 27 1234-5678
END:VCARD
"""
# Test case 2: TEL entries with TYPE attributes
vcard2 = """BEGIN:VCARD
VERSION:3.0
N:Smith;Jane;;;
FN:Jane Smith
TEL;TYPE=CELL:+5527912345678
TEL;TYPE=HOME:+552712345678
END:VCARD
"""
expected2 = """BEGIN:VCARD
VERSION:3.0
N:Smith;Jane;;;
FN:Jane Smith
TEL;TYPE=CELL:+55 27 91234-5678
TEL;TYPE=CELL:+55 27 1234-5678
TEL;TYPE=CELL:+55 27 1234-5678
END:VCARD
"""
# Test case 3: Complex TEL entries with prefixes
vcard3 = """BEGIN:VCARD
VERSION:3.0
N:Brown;Robert;;;
FN:Robert Brown
item1.TEL:+5527912345678
item2.TEL;TYPE=CELL:+552712345678
END:VCARD
"""
expected3 = """BEGIN:VCARD
VERSION:3.0
N:Brown;Robert;;;
FN:Robert Brown
TEL;TYPE=CELL:+55 27 91234-5678
TEL;TYPE=CELL:+55 27 1234-5678
TEL;TYPE=CELL:+55 27 1234-5678
END:VCARD
"""
# Test case 4: Mixed valid and invalid phone numbers
vcard4 = """BEGIN:VCARD
VERSION:3.0
N:White;Alice;;;
FN:Alice White
TEL:123
TEL:+5527912345678
END:VCARD
"""
expected4 = """BEGIN:VCARD
VERSION:3.0
N:White;Alice;;;
FN:Alice White
TEL:123
TEL;TYPE=CELL:+55 27 91234-5678
TEL;TYPE=CELL:+55 27 1234-5678
END:VCARD
"""
# Test case 5: Multiple contacts with different formats
vcard5 = """BEGIN:VCARD
VERSION:3.0
N:Johnson;Mike;;;
FN:Mike Johnson
TEL:27912345678
END:VCARD
BEGIN:VCARD
VERSION:3.0
N:Williams;Sarah;;;
FN:Sarah Williams
TEL;TYPE=CELL:2712345678
END:VCARD
"""
expected5 = """BEGIN:VCARD
VERSION:3.0
N:Johnson;Mike;;;
FN:Mike Johnson
TEL;TYPE=CELL:+55 27 91234-5678
TEL;TYPE=CELL:+55 27 1234-5678
END:VCARD
BEGIN:VCARD
VERSION:3.0
N:Williams;Sarah;;;
FN:Sarah Williams
TEL;TYPE=CELL:+55 27 1234-5678
END:VCARD
"""
# Test case 6: VCARD with no phone numbers
vcard6 = """BEGIN:VCARD
VERSION:3.0
N:Davis;Tom;;;
FN:Tom Davis
EMAIL:tom@example.com
END:VCARD
"""
expected6 = """BEGIN:VCARD
VERSION:3.0
N:Davis;Tom;;;
FN:Tom Davis
EMAIL:tom@example.com
END:VCARD
"""
test_cases = [
(vcard1, expected1),
(vcard2, expected2),
(vcard3, expected3),
(vcard4, expected4),
(vcard5, expected5),
(vcard6, expected6)
]
for i, (input_vcard, expected_output) in enumerate(test_cases):
with self.subTest(case=i+1):
# Create temporary files for input and output
with tempfile.NamedTemporaryFile(mode='w+', delete=False, encoding='utf-8') as input_file:
input_file.write(input_vcard)
input_path = input_file.name
output_path = input_path + '.out'
try:
# Process the VCARD
process_vcard(input_path, output_path)
# Read and verify the output
with open(output_path, 'r', encoding='utf-8') as output_file:
actual_output = output_file.read()
self.assertEqual(actual_output, expected_output)
finally:
# Clean up temporary files
if os.path.exists(input_path):
os.unlink(input_path)
if os.path.exists(output_path):
os.unlink(output_path)
def test_script_argument_handling(self):
"""Test the script's command-line argument handling."""
test_input = """BEGIN:VCARD
VERSION:3.0
N:Test;User;;;
FN:User Test
TEL:+5527912345678
END:VCARD
"""
# Create a temporary input file
with tempfile.NamedTemporaryFile(mode='w+', delete=False, encoding='utf-8') as input_file:
input_file.write(test_input)
input_path = input_file.name
output_path = input_path + '.out'
try:
test_args = ['python' if os.name == 'nt' else 'python3', 'brazilian_number_processing.py', input_path, output_path]
# We're just testing that the argument parsing works
subprocess.call(
test_args,
stdout=subprocess.DEVNULL,
stderr=subprocess.STDOUT
)
# Check if the output file was created
self.assertTrue(os.path.exists(output_path))
finally:
# Clean up temporary files
if os.path.exists(input_path):
os.unlink(input_path)
if os.path.exists(output_path):
os.unlink(output_path)
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,49 @@
import hmac
import javaobj
import zlib
from Crypto.Cipher import AES
from hashlib import sha256
from sys import exit
def _generate_hmac_of_hmac(key_stream):
key = hmac.new(
hmac.new(
b'\x00' * 32,
key_stream,
sha256
).digest(),
b"backup encryption\x01",
sha256
)
return key.digest(), key_stream
def _extract_encrypted_key(keyfile):
key_stream = b""
for byte in javaobj.loads(keyfile):
key_stream += byte.to_bytes(1, "big", signed=True)
return _generate_hmac_of_hmac(key_stream)
key = open("encrypted_backup.key", "rb").read()
database = open("wa.db.crypt15", "rb").read()
main_key, hex_key = _extract_encrypted_key(key)
for i in range(100):
iv = database[i:i+16]
for j in range(100):
cipher = AES.new(main_key, AES.MODE_GCM, iv)
db_ciphertext = database[j:]
db_compressed = cipher.decrypt(db_ciphertext)
try:
db = zlib.decompress(db_compressed)
except zlib.error:
...
else:
if db[0:6] == b"SQLite":
print(f"Found!\nIV: {i}\nOffset: {j}")
print(db_compressed[:10])
exit()
print("Not found! Try to increase maximum search.")

View File

@@ -1,51 +0,0 @@
import setuptools
from re import search
with open("README.md", "r") as fh:
long_description = fh.read()
with open("Whatsapp_Chat_Exporter/__init__.py", encoding="utf8") as f:
version = search(r'__version__ = "(.*?)"', f.read()).group(1)
setuptools.setup(
name="whatsapp-chat-exporter",
version=version,
author="KnugiHK",
author_email="info@knugi.com",
description="A Whatsapp database parser that will give you the "
"history of your Whatsapp conversations in HTML and JSON.",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://github.com/KnugiHK/Whatsapp-Chat-Exporter",
packages=setuptools.find_packages(),
package_data={
'': ['whatsapp.html']
},
classifiers=[
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
"Development Status :: 4 - Beta",
"Environment :: Console",
"Intended Audience :: End Users/Desktop",
"Topic :: Communications :: Chat",
"Topic :: Utilities",
"Topic :: Database"
],
python_requires='>=3.7',
install_requires=[
'jinja2',
'bleach'
],
extras_require={
'android_backup': ["pycryptodome"]
},
entry_points={
"console_scripts": [
"wtsexporter = Whatsapp_Chat_Exporter.__main__:main"
]
}
)