283 Commits
0.3 ... 0.9.7

Author SHA1 Message Date
KnugiHK
c27f5ee41c Merge branch 'dev' 2023-11-12 14:27:35 +08:00
KnugiHK
e6c43e7e35 Bump version 2023-11-12 14:27:03 +08:00
KnugiHK
c2fa18778f Merge branch 'dev' 2023-11-12 14:09:04 +08:00
KnugiHK
150180fdff Refactor 2023-10-29 18:12:17 +08:00
KnugiHK
86ea938323 Add fields for implementing read receipt 2023-10-29 18:12:16 +08:00
KnugiHK
7da71e84fe Bug fix on missing sender in group chats #67 2023-10-29 18:12:11 +08:00
KnugiHK
efd5ed80b2 Bug fix on ignoring system chat 2023-10-29 17:56:19 +08:00
KnugiHK
efea1d6165 Remove waste 2023-10-29 17:52:58 +08:00
KnugiHK
3082c83bc4 Convert all links to hyperlinks #69 2023-10-24 00:16:49 +08:00
KnugiHK
fc50415afd An attempt to fix #67 2023-10-24 00:05:15 +08:00
KnugiHK
be4adadbd8 Update __main__.py 2023-10-08 11:42:47 +08:00
KnugiHK
8eb05424fd Support preservation of file timestamp for encrypted backup 2023-10-08 11:42:39 +08:00
KnugiHK
380289d1c1 Bug fix on missing argument #65 2023-10-08 10:12:58 +08:00
KnugiHK
91ff882d15 Add option to preserve modify timestamp #65 2023-10-07 23:50:43 +08:00
Knugi
5aad65fff7 Merge pull request #60 from andrp92/dev
Feat: Support for WhatsApp Business (iPhone only)
2023-09-22 21:33:12 +08:00
KnugiHK
decea88028 Better UI
I am too lazy to switch branch.
2023-08-23 16:07:04 +08:00
KnugiHK
a08f44e6ed Fix the potential collision of Whatsapp and Whatsapp Business 2023-08-23 16:06:25 +08:00
Andres Rodriguez
dbd1802dd6 feat: use dynamic domains when encrypted 2023-08-18 01:51:13 -04:00
Andres Rodriguez
b9f123fbea fix: use corresponding identifier 2023-08-17 19:59:11 -04:00
KnugiHK
2944d00ca2 Refactoring so that no file needs to be introduced 2023-08-15 17:40:20 +08:00
KnugiHK
448ba892cc Change the option from --smb to --business 2023-08-15 17:33:56 +08:00
KnugiHK
a5cb46e095 Also make vcard path dynamic in Android 2023-08-15 16:22:43 +08:00
KnugiHK
ee4e95c75f Bug fix on the possible breakage mentioned in #60 2023-08-15 16:22:08 +08:00
Andres Rodriguez
f488894942 fix(whatsapp_business): missing domain ref 2023-08-15 00:46:54 -04:00
Andres Rodriguez
269a59c1e2 feat(whatsapp_business): made vcard path dynamic 2023-08-15 00:36:15 -04:00
Andres Rodriguez
d8b434e169 feat(whatsapp_business)!: args and default values for smb
Added corresponding logic for whatsapp business and default values.
2023-08-15 00:35:32 -04:00
Andres Rodriguez
326b99d860 feat(whatsapp_business)!: extraction script for smb
Duplicated original extract_iphone_media to include smb specfic fileIDs.
2023-08-15 00:32:07 -04:00
KnugiHK
bd2f063cc0 Bug fix on calls without chat #58 2023-08-12 15:51:05 +08:00
Knugi
736292538b Update the command for macOS #55 2023-07-25 08:53:04 +00:00
KnugiHK
d772efe779 Update the sample output screenshot 2023-06-25 20:01:06 +08:00
KnugiHK
282c99c7dd PyPi can't have git source as dependency 2023-06-25 13:57:11 +08:00
KnugiHK
8dec2a7e97 Bug fix on missing attribute in the Message class. 2023-06-25 13:30:44 +08:00
KnugiHK
f9dedc7930 Increase the row size 2023-06-25 13:14:39 +08:00
KnugiHK
e2f497dbb6 Bug fix on exported chat 2023-06-25 12:42:37 +08:00
KnugiHK
989bddca37 Limit the reply quote length 2023-06-21 17:24:50 +08:00
KnugiHK
40d060628f Coding style 2023-06-21 17:04:58 +08:00
KnugiHK
032af6cdcf Refactor 2023-06-21 17:01:14 +08:00
KnugiHK
e243abe2a4 Bug fix 2023-06-21 16:52:39 +08:00
KnugiHK
b8f0af5f31 Refactor 2023-06-21 16:24:36 +08:00
KnugiHK
030fef53e1 I found that old schema also has such tables 2023-06-20 19:53:05 +08:00
KnugiHK
3ed269e17f Support a lot of metadata in Android's new schema 2023-06-20 19:12:38 +08:00
KnugiHK
1e3ee5e322 Fix incorrect group message sender name 2023-06-20 16:03:24 +08:00
KnugiHK
6c740e69a5 Fix wrongly determined metadata 2023-06-20 15:32:05 +08:00
KnugiHK
828c8a1a72 Fix only one group chat is rendered when contact db is not present 2023-06-20 15:31:24 +08:00
KnugiHK
1fb8588752 Add more alias to wtsexporter command 2023-06-20 14:58:44 +08:00
KnugiHK
6636210e4c Fix the reply jump offset after introducing status 2023-06-20 14:52:32 +08:00
KnugiHK
cc0105647a Refactor a bit 2023-06-20 14:50:57 +08:00
KnugiHK
4fa360a389 No longer support direct execution on the script 2023-06-20 14:41:31 +08:00
KnugiHK
e5228855d2 Temporary workaround mentioned in #48 2023-06-20 14:39:54 +08:00
KnugiHK
db1cdf8189 Add contact's status if present (iOS) 2023-06-20 14:37:25 +08:00
KnugiHK
08ce61e68e Make them optional when importing! 2023-06-20 14:37:12 +08:00
KnugiHK
138dd5351f Add status to JSON import 2023-06-20 14:19:07 +08:00
KnugiHK
136152dc18 Add contact's status if present (Android) 2023-06-20 14:07:42 +08:00
KnugiHK
55bc62cdc1 Add fallback to Android contact name just like commit 5f6b764 2023-06-20 13:49:22 +08:00
KnugiHK
d430c7bfba Update wording on progress 2023-06-19 20:40:10 +08:00
KnugiHK
672b85474e Bug fix on the wrong type of media_wa_type in old Android schema 2023-06-19 20:26:13 +08:00
KnugiHK
525d88f2c6 Fix < Python 3.11 compatibility 2023-06-19 20:09:45 +08:00
KnugiHK
b57087794a Bump version for preparing next release 2023-06-19 17:56:24 +08:00
KnugiHK
be316ebb89 Add aliases to extras_require 2023-06-18 15:15:39 +08:00
KnugiHK
1078f7e5f7 Bug fix on sticker not resizing 2023-06-18 15:10:00 +08:00
KnugiHK
ba2a88067a Update setup.py 2023-06-18 14:36:15 +08:00
KnugiHK
af53ba978b Merge branch 'main' into dev 2023-06-16 19:27:56 +08:00
Knugi
d55a42a549 Merge pull request #36 from nahoj/patch-1
Where to find the key file
2023-06-16 19:27:34 +08:00
KnugiHK
506f8e89f4 Implement import json and output HTML 2023-06-16 19:19:33 +08:00
KnugiHK
fff11b26a5 Store the device type 2023-06-16 19:12:06 +08:00
KnugiHK
fa66ef3a52 Output all attribute to JSON file 2023-06-16 19:11:55 +08:00
Johan Grande
0b93ae567e rephrase as suggested 2023-06-16 12:38:38 +02:00
Johan Grande
c62e07cb0a Merge branch 'main' into patch-1 2023-06-16 12:33:17 +02:00
KnugiHK
317d785d50 Bug fix on raise of exception when "media_folder" does not exists on the filesystem 2023-06-16 18:27:49 +08:00
KnugiHK
38c1e47be9 Add iphone-backup-decrypt to setup.py as extra dependency 2023-06-16 17:57:33 +08:00
KnugiHK
ff95625edf Dropping Python 3.7 and adding Python 3.11 support 2023-06-16 17:50:08 +08:00
KnugiHK
14854193bd Merge branch 'main' into dev 2023-06-16 14:26:34 +08:00
KnugiHK
67c1b43669 Update README.md 2023-06-16 14:07:04 +08:00
KnugiHK
23046e01ba Add links to README 2023-06-16 13:48:36 +08:00
KnugiHK
c366e656af Further reduce the maximum length of vcard file name #51 2023-06-16 01:46:27 +08:00
KnugiHK
41f45fb07c PEP8 2023-06-16 01:43:43 +08:00
KnugiHK
be9e790b12 Better handling of binary message #44 2023-06-16 01:25:51 +08:00
KnugiHK
bfdc68cd6a Add autoescape to rendering 2023-06-16 01:10:24 +08:00
KnugiHK
594a04adbc Update the way to handle encrypted iOS backup file
Since this commit, iphone_backup_decrypt must be re-installed
2023-06-15 21:32:57 +08:00
KnugiHK
20b2eec047 Support Android call logs 2023-06-15 18:25:29 +08:00
KnugiHK
011c8ff1e7 Support missed call (PM) metadata for Android 2023-06-15 17:44:38 +08:00
KnugiHK
e4c47ea41f Update whatsapp.html 2023-06-15 17:01:13 +08:00
KnugiHK
c344f05b05 Bug fix on wrong alias 2023-06-15 17:00:55 +08:00
KnugiHK
88ef4989fc Fix wrong error message 2023-06-15 17:00:34 +08:00
KnugiHK
f7f6b01c86 Resize sticker 2023-06-15 16:59:54 +08:00
KnugiHK
a49a911e03 Replace image rendered in the HTML to thumbnail if possible 2023-06-15 16:19:35 +08:00
KnugiHK
3443143744 Restore code for downloading media from whatsapp server 2023-06-15 02:16:53 +08:00
KnugiHK
5f6b764bb9 Add fallback to iOS contact name
Fallback = the name set by the contact
2023-06-14 21:53:23 +08:00
KnugiHK
3940b2991f Try to reduce the size of the template 2023-06-13 21:23:44 +08:00
KnugiHK
dc1df8a03e Just coding style 2023-06-13 19:48:31 +08:00
KnugiHK
dd5ec2219c Sync changes 2023-06-13 19:46:09 +08:00
KnugiHK
e0c2cf5f66 Implement iOS avatar #48 2023-06-13 19:44:16 +08:00
KnugiHK
8cdb694a16 Modify the root directory name of iOS media 2023-06-13 17:00:40 +08:00
KnugiHK
8294f06587 Extract whole WhatsApp directory instead of wanted files only 2023-06-13 16:29:27 +08:00
KnugiHK
200dea218f Update help message 2023-06-13 14:01:10 +08:00
Knugi
df93033c6c Update README.md 2023-06-13 05:54:14 +00:00
KnugiHK
8f90733da2 Change "Gathering" to "Processing" 2023-06-11 01:35:40 +08:00
KnugiHK
3fdf6d0818 Update LICENSE 2023-06-11 01:33:09 +08:00
KnugiHK
2fa5c4268e Rewrite a bit 2023-06-11 01:22:21 +08:00
KnugiHK
ed658d78dc Fix incorrect media path on iOS #49 2023-06-11 01:21:07 +08:00
KnugiHK
0280325b4a Bug fix 2023-06-11 00:47:11 +08:00
KnugiHK
a42ec5d762 Beautify 2023-06-10 19:58:14 +08:00
KnugiHK
c419dd5d39 Raise error if time is not str and int 2023-06-10 19:54:09 +08:00
KnugiHK
8d036a6d87 Rewrite a bit 2023-06-10 19:45:29 +08:00
KnugiHK
42435c38cc Add <br> to newline 2023-06-10 19:32:48 +08:00
KnugiHK
32caab7c40 Distinguish between media missing and media omitted 2023-06-10 19:32:38 +08:00
KnugiHK
0897dc2897 Implement export TXT chat #22 2023-06-10 19:24:39 +08:00
KnugiHK
f63b180500 Implement splitted outputs #23 2023-06-08 18:16:47 +08:00
KnugiHK
dbdfdaedcf Refine code to use the data model 2023-06-08 17:51:57 +08:00
KnugiHK
0e802f4554 Remove old file 2023-06-08 16:50:33 +08:00
KnugiHK
41dd5e545f Make the not supported note looks less intimidating
#39
2023-06-08 15:46:12 +08:00
KnugiHK
8750315e8e Update __main__.py 2023-06-02 02:41:06 +08:00
KnugiHK
e9499c3bb7 Add highlighting when navigate to replied message 2023-06-02 02:41:01 +08:00
KnugiHK
80c3ed11f6 Partially implement reply feature in iOS
#28
2023-06-02 01:27:05 +08:00
KnugiHK
7c78cce221 Update link for reporting offsets 2023-06-01 22:26:32 +08:00
KnugiHK
32a312d332 Add offset mentioned in #46 2023-06-01 22:21:58 +08:00
KnugiHK
328f34e632 Merge branch 'main' into dev 2023-05-19 13:25:21 +08:00
KnugiHK
9ac8839ecc Workaround for non-UTF8 message
#44
2023-05-19 13:23:51 +08:00
Knugi
e7113d72d7 Update README.md 2023-05-18 09:34:59 +00:00
KnugiHK
0a0ae8cf15 Update python-publish.yml 2023-05-18 16:37:35 +08:00
KnugiHK
b1d8d173a2 Bug fix on too long vCard file name 2023-05-17 01:06:31 +08:00
KnugiHK
3bd6f288ea Bug fix file exist exception 2023-05-17 00:48:33 +08:00
KnugiHK
bf06795962 Add checksum on compiled binaries 2023-05-17 00:19:45 +08:00
KnugiHK
20d8e1384a Bump version 2023-05-17 00:08:52 +08:00
KnugiHK
6fd0e61b64 Update help test 2023-05-16 23:29:10 +08:00
KnugiHK
bbb47cd839 Bug fix 2023-05-16 23:25:12 +08:00
Knugi
c155064ae1 Update README.md 2023-05-16 13:38:16 +00:00
Knugi
d4efd919f9 Update python-publish.yml 2023-05-16 11:49:57 +00:00
Knugi
13d761286e Update compile-binary.yml 2023-05-16 11:48:25 +00:00
Knugi
a943808734 Update compile-binary.yml 2023-05-16 11:48:00 +00:00
KnugiHK
0495970c38 Bump version 2023-05-16 19:33:29 +08:00
KnugiHK
32b14dc392 Update README.md 2023-05-16 19:31:45 +08:00
KnugiHK
6bea8d07f4 Update setup.py 2023-05-05 14:03:40 +08:00
KnugiHK
69fdb61bae Merge branch 'dev' 2023-05-05 13:55:41 +08:00
KnugiHK
e1f160fc7c Update setup.py 2023-05-05 13:53:11 +08:00
KnugiHK
3b7e02ba31 Add update checking 2023-05-05 13:53:00 +08:00
KnugiHK
bdb7d80831 Merge branch 'dev' 2023-05-05 10:42:48 +08:00
Knugi
ea7e019adc Update README.md
Make the command compatible with both Linux and Windows
2023-04-25 05:41:27 +00:00
KnugiHK
c7a01bb9c0 Handle deleted message in new schema
Related to #39 and #9
2023-04-25 13:02:20 +08:00
KnugiHK
7c0b90d458 Another attempt to fix the previous bug 2023-04-09 02:38:26 +08:00
KnugiHK
84383e1d9d Revert "Bug fix on empty vcf contact name"
This reverts commit 06a1d34567.
2023-04-09 02:37:04 +08:00
KnugiHK
06a1d34567 Bug fix on empty vcf contact name 2023-04-09 02:23:50 +08:00
KnugiHK
b371587d65 Bug fix on ChatStore initialization 2023-04-09 02:05:17 +08:00
KnugiHK
3e7d7916a7 PEP8 2023-03-28 12:23:46 +08:00
Knugi
17997e840f Update README.md 2023-03-28 04:21:09 +00:00
KnugiHK
640acb3f86 Move some utility functions to a separated python file 2023-03-25 18:33:22 +08:00
KnugiHK
cdfaf69f7a Add an option to disable html output 2023-03-25 18:26:03 +08:00
KnugiHK
ee5f8b82be Add suggestion to CryptX 2023-03-25 12:11:00 +08:00
KnugiHK
b9fa36acb4 Raise error if the error is not expected 2023-03-25 11:55:18 +08:00
KnugiHK
fb88124d21 Merge old and new schema processing logic 2023-03-25 11:51:27 +08:00
KnugiHK
b5effbd512 Change autobuffer to preload for video and audio tag 2023-03-25 11:46:52 +08:00
KnugiHK
430a5eccb8 Merge branch 'pr/26' into dev 2023-03-25 10:37:47 +08:00
GoComputing
8f0511a6e2 Re-enabled the HTML generation 2023-03-24 19:43:27 +01:00
Johan Grande
8380487e44 Where to find the key file 2023-03-17 12:04:40 +01:00
Knugi
45666d8878 Add Python 3.10 to classifiers 2023-02-26 07:43:36 +00:00
KnugiHK
2ba55719f1 Add #32 to common offset 2023-02-26 15:31:03 +08:00
KnugiHK
2d23052758 Improve argument parser 2023-02-13 16:31:35 +08:00
KnugiHK
10875060c9 Deprecate --iphone and replace with --ios 2023-02-13 16:15:54 +08:00
KnugiHK
0bb99d59e0 Prepare for size control of output file 2023-02-13 16:08:21 +08:00
KnugiHK
9178e5326b Transit from optparse to argparse 2023-02-13 12:53:20 +08:00
KnugiHK
26320413e8 Add offline availability of w3css 2023-02-13 12:23:43 +08:00
KnugiHK
a275a0f40c Why is this line not in last commit... 2023-02-13 00:27:55 +08:00
KnugiHK
4cb4ac3e7b Bug fix for sender name in group chat
#9
2023-02-13 00:25:31 +08:00
Knugi
4139cab00f Fix binary 2023-02-12 11:51:43 +00:00
Knugi
c1964bc2cd Prepare for standalone binary
https://github.com/KnugiHK/Whatsapp-Chat-Exporter/issues/29
2023-02-12 18:11:38 +08:00
KnugiHK
dab0493354 Revert "Prepare for standalone binary"
This reverts commit 4d6c80b561.
2023-02-12 18:11:00 +08:00
Knugi
e0c464c8d8 Update compile-binary.yml 2023-02-12 18:01:29 +08:00
KnugiHK
92d339d1c0 Implement standalone binary compilation 2023-02-12 18:01:29 +08:00
Knugi
4d6c80b561 Prepare for standalone binary
https://github.com/KnugiHK/Whatsapp-Chat-Exporter/issues/29
2023-02-12 18:01:29 +08:00
Knugi
d46a42a097 Update compile-binary.yml 2023-02-12 09:56:55 +00:00
KnugiHK
7cd259143a Implement standalone binary compilation 2023-02-12 17:54:32 +08:00
Knugi
726812a5f7 Prepare for standalone binary
https://github.com/KnugiHK/Whatsapp-Chat-Exporter/issues/29
2023-02-12 08:15:40 +00:00
KnugiHK
6fddc1c23a Merge branch 'main' into dev 2023-01-31 17:56:53 +08:00
KnugiHK
77ceaa25dd Bump version 2023-01-31 17:52:34 +08:00
KnugiHK
e09f18e2f2 Minor fix 2023-01-31 17:52:12 +08:00
KnugiHK
23114572bd Forgot to change the variable lol 2023-01-31 17:52:12 +08:00
KnugiHK
2f04b69f38 A more concrete way to determine database offset 2023-01-31 17:52:12 +08:00
KnugiHK
e7c246822b Link to the file intead of showing the path directly
Not tested
Ref: https://github.com/KnugiHK/Whatsapp-Chat-Exporter/issues/15
2023-01-31 17:52:12 +08:00
KnugiHK
2a215d024f Bug fix
Duplicated folder creation
https://github.com/KnugiHK/Whatsapp-Chat-Exporter/issues/14
2023-01-31 17:52:12 +08:00
KnugiHK
f267f53007 Remove unused dependencies 2023-01-31 17:52:12 +08:00
KnugiHK
3a30dfc800 Bump version 2023-01-31 17:52:12 +08:00
KnugiHK
9600da59ae Correct the default in #25 2023-01-31 17:25:59 +08:00
KnugiHK
26b58843fb Add message 2023-01-31 16:46:22 +08:00
KnugiHK
60575c7989 Implement #25
Copying media folder to the output directory will be the default starting from this commit.
2023-01-31 16:34:34 +08:00
KnugiHK
14b1cb7fde Minor fix 2023-01-30 18:53:01 +08:00
GoComputing
92b8903521 Fixed JSON export
Added serialization to the classes 'ChatStore' and 'Messages' so that they can be
JSON serialized.
2023-01-28 20:51:40 +01:00
KnugiHK
d3892a4e4f Fix caption part 2022-12-23 17:28:23 +08:00
KnugiHK
b37c13434e Merge branch 'dev' into message_table 2022-12-23 16:49:37 +08:00
KnugiHK
4b357d5ea9 Update the import of Crypt to latest one for message 2022-12-23 16:49:28 +08:00
KnugiHK
6407ba2136 Adopt the latest version 2022-12-21 21:45:20 +08:00
KnugiHK
f87108dadc Some left-over 2022-12-21 21:42:54 +08:00
KnugiHK
6ca7e81484 Support new WhatsApp database schema
https://github.com/KnugiHK/Whatsapp-Chat-Exporter/issues/9
2022-12-21 21:28:54 +08:00
KnugiHK
41d3659269 Prepare for porting 2022-12-21 20:16:37 +08:00
Knugi
580eaddb24 Delete old_README.md 2022-10-24 03:59:42 +00:00
Knugi
77b4b784d3 Update README.md 2022-10-24 03:59:24 +00:00
KnugiHK
d9a77e0eec Forgot to change the variable lol 2022-09-05 12:49:12 +08:00
KnugiHK
876729eb81 A more concrete way to determine database offset 2022-09-05 12:48:36 +08:00
KnugiHK
48f667d02b Implement exporting 64-digit crypt15 encryption key
https://github.com/KnugiHK/Whatsapp-Chat-Exporter/issues/20
2022-09-05 12:16:07 +08:00
KnugiHK
422ab2f784 Link to the file intead of showing the path directly
Not tested
Ref: https://github.com/KnugiHK/Whatsapp-Chat-Exporter/issues/15
2022-07-03 12:30:01 +08:00
KnugiHK
996ee65525 Bug fix
Duplicated folder creation
https://github.com/KnugiHK/Whatsapp-Chat-Exporter/issues/14
2022-05-25 18:28:07 +08:00
KnugiHK
042f6f9024 Remove unused dependencies 2022-05-23 20:12:50 +08:00
KnugiHK
507e88d9c3 Merge branch 'dev' 2022-05-09 18:31:03 +08:00
KnugiHK
60e1e7d3eb Bump version 2022-05-09 18:28:13 +08:00
Knugi
774fb6d781 Merge pull request #11 from asla9709/dev
Fixed bug where blank VCard media_name would crash the program.
2022-05-09 10:03:04 +00:00
Aakif Aslam
3ef3b02230 Fixed bug where blank VCard media_name would crash the program. 2022-04-24 18:01:32 -04:00
Knugi
07cc0f3571 Update README.md 2022-04-04 07:57:12 +00:00
KnugiHK
a1319eb835 Exclude default conversation from results 2022-04-01 17:17:43 +08:00
KnugiHK
8cbb0af43a Oh. I missed this change 2022-03-04 14:06:19 +08:00
KnugiHK
28c4a7b99f Prepare for new function 2022-03-04 14:03:15 +08:00
KnugiHK
e4c9d42927 Update __init__.py 2022-03-04 13:48:12 +08:00
KnugiHK
c274b6b1c0 Merge branch 'dev' 2022-03-04 13:47:54 +08:00
KnugiHK
eec739d7cf Add crypt15 dependency to android_backup 2022-03-04 13:47:17 +08:00
Knugi
3d7dca0682 Delete _config.yml 2022-02-25 08:57:31 +00:00
Knugi
24f7837171 Create python-publish.yml 2022-02-22 14:01:36 +00:00
KnugiHK
15201acbe6 Bump version 2022-02-22 21:53:13 +08:00
Knugi
6fd290efd8 typo 2022-02-22 13:52:06 +00:00
Knugi
691bfe31c8 Update README.md 2022-02-22 13:51:00 +00:00
KnugiHK
64eb2bcb9d Add some aliases 2022-02-22 21:42:58 +08:00
KnugiHK
1bc4a8c5b9 Create android_structure_backup_crypt15.png 2022-02-22 21:42:50 +08:00
Knugi
8a621827ff Update README.md 2022-02-22 13:42:16 +00:00
KnugiHK
227f438404 Add one more offset 2022-02-22 21:22:07 +08:00
KnugiHK
3e71817778 Make the brute-force more sensitive and bug fix 2022-02-22 21:19:19 +08:00
KnugiHK
08c5979eed Support Hex key for Crypt15 2022-02-22 18:59:04 +08:00
KnugiHK
0e6319eb4e Support crypt15 2022-02-22 18:33:54 +08:00
KnugiHK
734bb78cd8 Implement offset brute forcing
Not tested yet
2022-02-22 02:08:44 +08:00
KnugiHK
a522eb2034 PEP8 2022-01-17 13:01:10 +08:00
KnugiHK
9fe6a0d2a8 Refactoring 2022-01-17 12:06:15 +08:00
KnugiHK
c73eabe2a4 Clearer error message in decompressing decrypted backup 2022-01-17 10:46:02 +08:00
KnugiHK
1faf111e64 Handle case that the database file does not exist and clearer exit code 2021-12-30 11:59:30 +08:00
KnugiHK
9140c07feb Prevent PermissionError from raising 2021-12-28 20:04:40 +08:00
KnugiHK
abf4b20bc6 Merge branch 'main' of https://github.com/KnugiHK/Whatsapp-Chat-Exporter 2021-12-28 19:44:00 +08:00
KnugiHK
f2f6258960 Bump version number 2021-12-28 19:38:01 +08:00
Knugi
62af48c78e Update README.md 2021-12-28 11:34:23 +00:00
Knugi
c9158d202d Update To do 2021-12-28 11:27:29 +00:00
KnugiHK
fb5e4d5421 Merge branch 'dev' 2021-12-28 19:26:02 +08:00
Knugi
d85c91fbdc Update README.md 2021-11-18 03:09:03 +00:00
KnugiHK
0dddb63c5e Merge branch 'main' into dev 2021-08-13 20:28:49 +08:00
KnugiHK
dd36960ecb Implement CSS for metadata 2021-08-13 20:25:59 +08:00
Knugi
620a1bcdb7 Update README.md 2021-07-13 10:54:07 +00:00
Knugi
896a6d2ddd Update README.md 2021-07-13 10:53:41 +00:00
KnugiHK
4ee92e7efc Bug fix
The directory cannot be created if the parent directory is not present
2021-07-11 18:24:59 +08:00
KnugiHK
f91aac1e11 Implementing newline to <br> 2021-07-11 18:17:06 +08:00
KnugiHK
27a6ff98b3 Merge branch 'main' into dev 2021-07-11 11:05:41 +08:00
Knugi
1952c0835c Update README.md 2021-07-11 03:05:15 +00:00
KnugiHK
ab42cad166 Some PEP8 2021-07-10 22:01:04 +08:00
KnugiHK
3ed59ee051 Bug fix 2021-07-10 21:53:28 +08:00
KnugiHK
f9358ded14 Update README.md 2021-07-10 21:50:30 +08:00
KnugiHK
790f4ec5e0 Support custom template 2021-07-10 21:46:45 +08:00
KnugiHK
35ef4031fc Support crypt12 2021-07-10 21:28:49 +08:00
KnugiHK
b9f343cf2f Update README and setup.py 2021-07-10 21:11:49 +08:00
KnugiHK
18ee152688 Support crypt14 WhatsApp Backup 2021-07-10 21:08:52 +08:00
Knugi
620e89a185 Update README.md 2021-07-06 03:17:57 +00:00
KnugiHK
3ada8916f9 Merge branch 'main' of https://github.com/KnugiHK/Whatsapp-Chat-Exporter into main 2021-05-31 16:13:07 +08:00
Knugi
07ebb692e5 Create CNAME 2021-05-31 08:05:44 +00:00
Knugi
7255f0fe2b Set theme jekyll-theme-cayman 2021-05-31 08:04:21 +00:00
KnugiHK
684badb9a6 Update the string appear in wtsexporter --version 2021-05-31 11:09:47 +08:00
Knugi
e1221d9f59 Update README.md 2021-05-31 03:04:07 +00:00
KnugiHK
fc84e430ed Update setup.py 2021-05-31 10:59:29 +08:00
KnugiHK
0b0af518c3 Update setup.py 2021-05-31 10:54:35 +08:00
KnugiHK
7e84595074 Update setup.py 2021-05-31 10:49:48 +08:00
KnugiHK
1bdd5fe6df Update extract_iphone_media.py 2021-05-31 10:47:40 +08:00
KnugiHK
0ac7eecb47 Update version number for Pypi 2021-05-31 10:44:04 +08:00
KnugiHK
b0a469509c Update README.md 2021-05-31 10:43:49 +08:00
Knugi
cec68dd3a0 Update README.md 2021-05-12 06:11:24 +00:00
Knugi
46e12daa6a Update README.md 2021-05-12 05:51:27 +00:00
Knugi
4dd7f4e753 Remove a to-do task 2021-05-12 05:48:43 +00:00
Knugi
6cf6e50db9 Update README.md 2021-05-12 05:47:02 +00:00
KnugiHK
366d18b678 Create android_structure.png 2021-05-12 13:44:46 +08:00
Knugi
e8a8546a13 Update README.md 2021-05-12 05:44:32 +00:00
KnugiHK
baa733df5f Create old_README.md 2021-05-12 13:24:54 +08:00
KnugiHK
d9f38fc714 Create console script 2021-05-10 15:51:43 +08:00
KnugiHK
322281a8ec Prepare for publishing in PyPi 2021-05-10 15:51:30 +08:00
KnugiHK
3ee40ecda4 Move images to a folder 2021-05-10 15:49:41 +08:00
Knugi
f591c7a57f Update README.md 2021-05-10 05:48:25 +00:00
KnugiHK
e1e49261aa Merge branch 'main' of https://github.com/KnugiHK/Whatsapp-Chat-Exporter into main 2021-05-10 13:45:37 +08:00
KnugiHK
b58aaa8f73 Resize images 2021-05-10 13:45:30 +08:00
Knugi
c62f08340e Update README.md 2021-05-10 05:40:28 +00:00
Knugi
4fb759f974 Update README.md 2021-05-10 05:39:59 +00:00
KnugiHK
cb2e83721e Merge branch 'main' of https://github.com/KnugiHK/Whatsapp-Chat-Exporter into main 2021-05-10 13:36:56 +08:00
KnugiHK
be7e20317d Add support for encrypted iPhone backup 2021-05-10 13:33:13 +08:00
Knugi
cf2e69b594 Update README.md 2021-05-10 04:54:04 +00:00
Knugi
fb33a883e6 Update README.md 2021-05-10 04:47:52 +00:00
Knugi
58bc8634f7 Update README.md 2021-05-09 05:40:24 +00:00
28 changed files with 2954 additions and 898 deletions

82
.github/workflows/compile-binary.yml vendored Normal file
View File

@@ -0,0 +1,82 @@
name: Compile standalone binary
on:
release:
types: [published]
workflow_dispatch:
permissions:
contents: read
jobs:
linux:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.10'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pycryptodome javaobj-py3 ordered-set zstandard nuitka
pip install .
- name: Build binary with Nuitka
run: |
python -m nuitka --onefile --include-data-file=./Whatsapp_Chat_Exporter/whatsapp.html=./Whatsapp_Chat_Exporter/whatsapp.html --follow-imports Whatsapp_Chat_Exporter/__main__.py
cp __main__.bin wtsexporter_linux_x64
sha256sum wtsexporter_linux_x64
- uses: actions/upload-artifact@v3
with:
name: binary-linux
path: |
./wtsexporter_linux_x64
windows:
runs-on: windows-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.10'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pycryptodome javaobj-py3 ordered-set zstandard nuitka
pip install .
- name: Build binary with Nuitka
run: |
python -m nuitka --onefile --include-data-file=./Whatsapp_Chat_Exporter/whatsapp.html=./Whatsapp_Chat_Exporter/whatsapp.html --assume-yes-for-downloads --follow-imports Whatsapp_Chat_Exporter\__main__.py
copy __main__.exe wtsexporter_x64.exe
Get-FileHash wtsexporter_x64.exe
- uses: actions/upload-artifact@v3
with:
name: binary-windows
path: |
.\wtsexporter_x64.exe
macos:
runs-on: macos-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.10'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pycryptodome javaobj-py3 ordered-set zstandard nuitka
pip install .
- name: Build binary with Nuitka
run: |
python -m nuitka --onefile --include-data-file=./Whatsapp_Chat_Exporter/whatsapp.html=./Whatsapp_Chat_Exporter/whatsapp.html --follow-imports Whatsapp_Chat_Exporter/__main__.py
cp __main__.bin wtsexporter_macos_x64
shasum -a 256 wtsexporter_macos_x64
- uses: actions/upload-artifact@v3
with:
name: binary-macos
path: |
./wtsexporter_macos_x64

36
.github/workflows/python-publish.yml vendored Normal file
View File

@@ -0,0 +1,36 @@
# This workflow will upload a Python Package using Twine when a release is created
# For more information see: https://help.github.com/en/actions/language-and-framework-guides/using-python-with-github-actions#publishing-to-package-registries
# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.
name: Upload Python Package
on:
release:
types: [published]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.x'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build
- name: Build package
run: python -m build
- name: Publish package
uses: pypa/gh-action-pypi-publish@release/v1
with:
user: __token__
password: ${{ secrets.PYPI_API_TOKEN }}

1
CNAME Normal file
View File

@@ -0,0 +1 @@
wts.knugi.com

View File

@@ -1,6 +1,6 @@
MIT License
Copyright (c) 2021 Knugi
Copyright (c) 2021-2023 Knugi
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

168
README.md
View File

@@ -1,47 +1,161 @@
# Whatsapp-Chat-Exporter
A Whatsapp database parser that will give you the history of your Whatsapp conversation in HTML and JSON
[![Latest in Pypi](https://img.shields.io/pypi/v/whatsapp-chat-exporter?label=Latest%20in%20Pypi)](https://pypi.org/project/whatsapp-chat-exporter/)
![License MIT](https://img.shields.io/pypi/l/whatsapp-chat-exporter)
[![Python](https://img.shields.io/pypi/pyversions/Whatsapp-Chat-Exporter)](https://pypi.org/project/Whatsapp-Chat-Exporter/)
A customizable Android and iPhone Whatsapp database parser that will give you the history of your Whatsapp conversations in HTML and JSON. Inspired by [Telegram Chat Export Tool](https://telegram.org/blog/export-and-more).
**If you plan to uninstall WhatsApp or delete your WhatsApp account, please make a backup of your WhatsApp database. You may want to use this exporter again on the same database in the future as the exporter develops**
# Usage
First, clone this repo, and copy all py and html files to a working directory if you want to do so.
**Usage in README may be removed in the future. Check the usage in [Wiki](https://github.com/KnugiHK/Whatsapp-Chat-Exporter/wiki)**.
**If you want to use the old release (< 0.5) of the exporter, please follow the [old usage guide](https://github.com/KnugiHK/Whatsapp-Chat-Exporter/wiki/Old-Usage#usage)**.
First, install the exporter by:
```shell
git clone https://github.com/KnugiHK/Whatsapp-Chat-Exporter.git
pip install whatsapp-chat-exporter
pip install whatsapp-chat-exporter[android_backup] :; # Optional, if you want it to support decrypting Android WhatsApp backup.
```
Then, ready your WhatsApp database, place them in the root of working directory.
* For Android, it is called msgstore.db. If you want name of your contacts, get the contact database, which is called wa.db.
* For iPhone, it is called 7c7fba66680ef796b916b067077cc246adacf01d (YES, a hash).
Then, create a working directory in somewhere you want
```shell
mkdir working_wts
cd working_wts
```
## Working with Android
### Unencrypted WhatsApp database
Extract the WhatsApp database with whatever means, one possible means is to use the [WhatsApp-Key-DB-Extractor](https://github.com/KnugiHK/WhatsApp-Key-DB-Extractor)
Next, ready your media folder, place it in the root of working directory.
* For Android, copy the WhatsApp directory from your phone directly.
* For iPhone, run the extract_iphone_media.py, and you will get a folder called Message. Please note that, this script does not support encrypted backup.
```
python extract_iphone_media.py "C:\Users\[Username]\AppData\Roaming\Apple Computer\MobileSync\Backup\[device id]"
```
And now, you should have something like this:
After you obtain your WhatsApp database, copy the WhatsApp database and media folder to the working directory. The database is called msgstore.db. If you also want the name of your contacts, get the contact database, which is called wa.db. And copy the WhatsApp (Media) directory from your phone directly.
![Folder structure](structure.png)
And now, you should have something like this in the working directory.
Last, run the script regarding the type of phone.
![Android folder structure](imgs/android_structure.png)
#### Extracting
Simply invoke the following command from shell.
```sh
wtsexporter -a
```
python extract.py & :: Android
python extract_iphone.py & :: iPhone
### Encrypted Android WhatsApp Backup
In order to support the decryption, install pycryptodome if it is not installed
```sh
pip install pycryptodome # Or
pip install whatsapp-chat-exporter["android_backup"] # install along with this software
```
And you will get these:
### Crypt15 is now the easiest way to decrypt a backup. If you have the 32 bytes hex key generated when you enable End-to-End encrypted backup, you can use it to decrypt the backup. If you do not have the 32 bytes hex key, you can still use the key file extracted just like extacting key file for Crypt12 and Crypt14 to decrypt the backup.
#### Crypt12 or Crypt14
You will need the decryption key file from your phone. If you have root access, you can find it as `/data/data/com.whatsapp/files/key`. Otherwise, if you used WhatsApp-Key-DB-Extractor before, it will appear in the WhatsApp backup directory as `WhatsApp/Databases/.nomedia`.
Place the decryption key file (key) and the encrypted WhatsApp Backup (msgstore.db.crypt14) in the working directory. If you also want the name of your contacts, get the contact database, which is called wa.db. And copy the WhatsApp (Media) directory from your phone directly.
And now, you should have something like this in the working directory.
![Android folder structure with WhatsApp Backup](imgs/android_structure_backup.png)
#### Extracting
Simply invoke the following command from shell.
```sh
wtsexporter -a -k key -b msgstore.db.crypt14
```
#### Crypt15 (End-to-End Encrypted Backup)
To support Crypt15 backup, install javaobj-py3 if it is not installed
```sh
pip install javaobj-py3 # Or
pip install whatsapp-chat-exporter["crypt15"] # install along with this software
```
Place the encrypted WhatsApp Backup (msgstore.db.crypt15) in the working directory. If you also want the name of your contacts, get the contact database, which is called wa.db. And copy the WhatsApp (Media) directory from your phone directly.
If you do not have the 32 bytes hex key (64 hexdigits), place the decryption key file (encrypted_backup.key) extracted from Android. If you gave the 32 bytes hex key, simply put the key in the shell.
Now, you should have something like this in the working directory (if you do not have 32 bytes hex key).
![Android folder structure with WhatsApp Crypt15 Backup](imgs/android_structure_backup_crypt15.png)
##### Extracting
If you do not have 32 bytes hex key but have the key file available, simply invoke the following command from shell.
```sh
wtsexporter -a -k encrypted_backup.key -b msgstore.db.crypt15
```
If you have the 32 bytes hex key, simply put the hex key in the -k option and invoke the command from shell like this:
```sh
wtsexporter -a -k 432435053b5204b08e5c3823423399aa30ff061435ab89bc4e6713969cdaa5a8 -b msgstore.db.crypt15
```
## Working with iOS/iPadOS (iPhone or iPad)
Do an iPhone/iPad Backup with iTunes first.
* iPhone backup on Mac: https://support.apple.com/HT211229
* iPhone backup on Windows: https://support.apple.com/HT212156
* iPad backup: https://support.apple.com/guide/ipad/ipad9a74df05xx/ipados
### Encrypted iOS/iPadOS Backup
**If you are working on unencrypted iOS/iPadOS backup, skip this**
If you want to work on an encrypted iOS/iPadOS Backup, you should install iphone_backup_decrypt from [KnugiHK/iphone_backup_decrypt](https://github.com/KnugiHK/iphone_backup_decrypt) before you run the extract_iphone_media.py.
```sh
pip install git+https://github.com/KnugiHK/iphone_backup_decrypt
```
### Extracting
Simply invoke the following command from shell, remember to replace the username and device id correspondingly in the command.
#### Windows
```sh
wtsexporter -i -b "C:\Users\[Username]\AppData\Roaming\Apple Computer\MobileSync\Backup\[device id]"
```
#### Mac
```sh
wtsexporter -i -b ~/Library/Application\ Support/MobileSync/Backup/[device id]
```
## Results
After extracting, you will get these:
#### Private Message
![Private Message](pm.png)
![Private Message](imgs/pm.png)
#### Group Message
![Group Message](group.png)
![Group Message](imgs/group.png)
# Encrypted iPhone Backup
To do
## More options
Invoke the wtsexporter with --help option will show you all options available.
```sh
> wtsexporter --help
usage: wtsexporter [-h] [-a] [-i] [-e EXPORTED] [-w WA] [-m MEDIA] [-b BACKUP] [-o OUTPUT] [-j [JSON]] [-d DB] [-k KEY] [-t TEMPLATE] [-s] [-c] [--offline OFFLINE] [--size [SIZE]]
[--no-html] [--check-update] [--assume-first-as-me]
options:
-h, --help show this help message and exit
-a, --android Define the target as Android
-i, --iphone, --ios Define the target as iPhone/iPad
-e EXPORTED, --exported EXPORTED
Define the target as exported chat file and specify the path to the file
-w WA, --wa WA Path to contact database (default: wa.db/ContactsV2.sqlite)
-m MEDIA, --media MEDIA
Path to WhatsApp media folder (default: WhatsApp)
-b BACKUP, --backup BACKUP
Path to Android (must be used together with -k)/iPhone WhatsApp backup
-o OUTPUT, --output OUTPUT
Output to specific directory (default: result)
-j [JSON], --json [JSON]
Save the result to a single JSON file (default if present: result.json)
-d DB, --db DB Path to database file (default: msgstore.db/7c7fba66680ef796b916b067077cc246adacf01d)
-k KEY, --key KEY Path to key file
-t TEMPLATE, --template TEMPLATE
Path to custom HTML template
-s, --showkey Show the HEX key used to decrypt the database
-c, --move-media Move the media directory to output directory if the flag is set, otherwise copy it
--offline OFFLINE Relative path to offline static files
--size [SIZE], --output-size [SIZE], --split [SIZE]
Maximum (rough) size of a single output file in bytes, 0 for auto
--no-html Do not output html files
--check-update Check for updates (require Internet access)
--assume-first-as-me Assume the first message in a chat as sent by me (must be used together with -e)
WhatsApp Chat Exporter: 0.9.7 Licensed with MIT
```
# To do
1. Convert ```\r\n``` to ```<br>```
2. Reply in iPhone
3. The CSS for metadata (e.g. {Message Deleted})
4. Handle encrypted iPhone Backup
See [issues](https://github.com/KnugiHK/Whatsapp-Chat-Exporter/issues).
# Copyright
This is a MIT licensed project.
The Telegram Desktop's export is the reference for whatsapp.html in this repo
The Telegram Desktop's export is the reference for whatsapp.html in this repo.
bplist.py was released by Vladimir "Farcaller" Pouzanov under MIT license.
WhatsApp Chat Exporter is not affiliated, associated, authorized, endorsed by, or in any way officially connected with the WhatsApp LLC, or any of its subsidiaries or its affiliates. The official WhatsApp LLC website can be found at https://www.whatsapp.com/.

View File

@@ -0,0 +1,3 @@
#!/usr/bin/python3
__version__ = "0.9.7"

View File

@@ -0,0 +1,389 @@
#!/usr/bin/python3
import os
import sqlite3
import shutil
import json
import string
import glob
from Whatsapp_Chat_Exporter import extract_exported, extract_iphone
from Whatsapp_Chat_Exporter import extract, extract_iphone_media
from Whatsapp_Chat_Exporter.data_model import ChatStore
from Whatsapp_Chat_Exporter.utility import Crypt, check_update, import_from_json
from argparse import ArgumentParser, SUPPRESS
from sys import exit
try:
from .__init__ import __version__
except ImportError:
from Whatsapp_Chat_Exporter.__init__ import __version__
def main():
parser = ArgumentParser(
description = 'A customizable Android and iPhone WhatsApp database parser that '
'will give you the history of your WhatsApp conversations in HTML '
'and JSON. Android Backup Crypt12, Crypt14 and Crypt15 supported.',
epilog = f'WhatsApp Chat Exporter: {__version__} Licensed with MIT'
)
parser.add_argument(
'-a',
'--android',
dest='android',
default=False,
action='store_true',
help="Define the target as Android")
parser.add_argument(
'-i',
'--iphone',
'--ios',
dest='iphone',
default=False,
action='store_true',
help="Define the target as iPhone/iPad")
parser.add_argument(
"-e",
"--exported",
dest="exported",
default=None,
help="Define the target as exported chat file and specify the path to the file"
)
parser.add_argument(
"-w",
"--wa",
dest="wa",
default=None,
help="Path to contact database (default: wa.db/ContactsV2.sqlite)")
parser.add_argument(
"-m",
"--media",
dest="media",
default=None,
help="Path to WhatsApp media folder (default: WhatsApp)")
parser.add_argument(
"-b",
"--backup",
dest="backup",
default=None,
help="Path to Android (must be used together "
"with -k)/iPhone WhatsApp backup")
parser.add_argument(
"-o",
"--output",
dest="output",
default="result",
help="Output to specific directory (default: result)")
parser.add_argument(
'-j',
'--json',
dest='json',
nargs='?',
default=None,
type=str,
const="result.json",
help="Save the result to a single JSON file (default if present: result.json)")
parser.add_argument(
'-d',
'--db',
dest='db',
default=None,
help="Path to database file (default: msgstore.db/"
"7c7fba66680ef796b916b067077cc246adacf01d)")
parser.add_argument(
'-k',
'--key',
dest='key',
default=None,
help="Path to key file"
)
parser.add_argument(
"-t",
"--template",
dest="template",
default=None,
help="Path to custom HTML template"
)
parser.add_argument(
"--embedded",
dest="embedded",
default=False,
action='store_true',
help=SUPPRESS or "Embed media into HTML file (not yet implemented)"
)
parser.add_argument(
"-s",
"--showkey",
dest="showkey",
default=False,
action='store_true',
help="Show the HEX key used to decrypt the database"
)
parser.add_argument(
"-c",
"--move-media",
dest="move_media",
default=False,
action='store_true',
help="Move the media directory to output directory if the flag is set, otherwise copy it"
)
parser.add_argument(
"--offline",
dest="offline",
default=None,
help="Relative path to offline static files"
)
parser.add_argument(
"--size",
"--output-size",
"--split",
dest="size",
nargs='?',
type=int,
const=0,
default=None,
help="Maximum (rough) size of a single output file in bytes, 0 for auto"
)
parser.add_argument(
"--no-html",
dest="no_html",
default=False,
action='store_true',
help="Do not output html files"
)
parser.add_argument(
"--check-update",
dest="check_update",
default=False,
action='store_true',
help="Check for updates (require Internet access)"
)
parser.add_argument(
"--assume-first-as-me",
dest="assume_first_as_me",
default=False,
action='store_true',
help="Assume the first message in a chat as sent by me (must be used together with -e)"
)
parser.add_argument(
"--no-avatar",
dest="no_avatar",
default=False,
action='store_true',
help="Do not render avatar in HTML output"
)
parser.add_argument(
"--import",
dest="import_json",
default=False,
action='store_true',
help="Import JSON file and convert to HTML output"
)
parser.add_argument(
"--business",
dest="business",
default=False,
action='store_true',
help="Use Whatsapp Business default files (iOS only)"
)
parser.add_argument(
"--preserve-timestamp",
dest="preserve_timestamp",
default=False,
action='store_true',
help="Preserve the modification timestamp of the extracted files (iOS only)"
)
args = parser.parse_args()
# Check for updates
if args.check_update:
exit(check_update())
# Sanity checks
if args.android and args.iphone and args.exported and args.import_json:
print("You must define only one device type.")
exit(1)
if not args.android and not args.iphone and not args.exported and not args.import_json:
print("You must define the device type.")
exit(1)
if args.no_html and not args.json:
print("You must either specify a JSON output file or enable HTML output.")
exit(1)
if args.import_json and (args.android or args.iphone or args.exported or args.no_html):
print("You can only use --import with -j and without --no-html.")
exit(1)
elif args.import_json and not os.path.isfile(args.json):
print("JSON file not found.")
exit(1)
if args.android and args.business:
print("WhatsApp Business is only available on iOS for now.")
exit(1)
data = {}
if args.android:
contacts = extract.contacts
messages = extract.messages
media = extract.media
vcard = extract.vcard
create_html = extract.create_html
if args.db is None:
msg_db = "msgstore.db"
else:
msg_db = args.db
if args.key is not None:
if args.backup is None:
print("You must specify the backup file with -b")
exit(1)
print("Decryption key specified, decrypting WhatsApp backup...")
if "crypt12" in args.backup:
crypt = Crypt.CRYPT12
elif "crypt14" in args.backup:
crypt = Crypt.CRYPT14
elif "crypt15" in args.backup:
crypt = Crypt.CRYPT15
if os.path.isfile(args.key):
key = open(args.key, "rb")
elif all(char in string.hexdigits for char in args.key):
key = bytes.fromhex(args.key)
db = open(args.backup, "rb").read()
error = extract.decrypt_backup(db, key, msg_db, crypt, args.showkey)
if error != 0:
if error == 1:
print("Dependencies of decrypt_backup and/or extract_encrypted_key"
" are not present. For details, see README.md.")
exit(3)
elif error == 2:
print("Failed when decompressing the decrypted backup. "
"Possibly incorrect offsets used in decryption.")
exit(4)
else:
print("Unknown error occurred.", error)
exit(5)
if args.wa is None:
contact_db = "wa.db"
else:
contact_db = args.wa
if args.media is None:
args.media = "WhatsApp"
if os.path.isfile(contact_db):
with sqlite3.connect(contact_db) as db:
db.row_factory = sqlite3.Row
contacts(db, data)
elif args.iphone:
import sys
if "--iphone" in sys.argv:
print(
"WARNING: The --iphone flag is deprecated and will"
"be removed in the future. Use --ios instead."
)
contacts = extract_iphone.contacts
messages = extract_iphone.messages
media = extract_iphone.media
vcard = extract_iphone.vcard
create_html = extract.create_html
if args.business:
from Whatsapp_Chat_Exporter.utility import WhatsAppBusinessIdentifier as identifiers
else:
from Whatsapp_Chat_Exporter.utility import WhatsAppIdentifier as identifiers
if args.media is None:
args.media = identifiers.DOMAIN
if args.backup is not None:
if not os.path.isdir(args.media):
extract_iphone_media.extract_media(args.backup, identifiers, args.preserve_timestamp)
else:
print("WhatsApp directory already exists, skipping WhatsApp file extraction.")
if args.db is None:
msg_db = identifiers.MESSAGE
else:
msg_db = args.db
if args.wa is None:
contact_db = "ContactsV2.sqlite"
else:
contact_db = args.wa
if os.path.isfile(contact_db):
with sqlite3.connect(contact_db) as db:
db.row_factory = sqlite3.Row
contacts(db, data)
if not args.exported and not args.import_json:
if os.path.isfile(msg_db):
with sqlite3.connect(msg_db) as db:
db.row_factory = sqlite3.Row
messages(db, data, args.media)
media(db, data, args.media)
vcard(db, data, args.media)
if args.android:
extract.calls(db, data)
if not args.no_html:
create_html(
data,
args.output,
args.template,
args.embedded,
args.offline,
args.size,
args.no_avatar
)
else:
print(
"The message database does not exist. You may specify the path "
"to database file with option -d or check your provided path."
)
exit(2)
if os.path.isdir(args.media):
media_path = os.path.join(args.output, args.media)
if os.path.isdir(media_path):
print("\nWhatsApp directory already exists in output directory. Skipping...", end="\n")
else:
if not args.move_media:
if os.path.isdir(media_path):
print("\nWhatsApp directory already exists in output directory. Skipping...", end="\n")
else:
print("\nCopying media directory...", end="\n")
shutil.copytree(args.media, media_path)
else:
try:
shutil.move(args.media, f"{args.output}/")
except PermissionError:
print("\nCannot remove original WhatsApp directory. "
"Perhaps the directory is opened?", end="\n")
elif args.exported:
extract_exported.messages(args.exported, data, args.assume_first_as_me)
if not args.no_html:
extract.create_html(
data,
args.output,
args.template,
args.embedded,
args.offline,
args.size
)
for file in glob.glob(r'*.*'):
shutil.copy(file, args.output)
elif args.import_json:
import_from_json(args.json, data)
extract.create_html(
data,
args.output,
args.template,
args.embedded,
args.offline,
args.size
)
if args.json and not args.import_json:
if isinstance(data[next(iter(data))], ChatStore):
data = {jik: chat.to_json() for jik, chat in data.items()}
with open(args.json, "w") as f:
data = json.dumps(data)
print(f"\nWriting JSON file...({int(len(data)/1024/1024)}MB)")
f.write(data)
else:
print()
print("Everything is done!")
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,292 @@
#################################################################################
# Copyright (C) 2009-2011 Vladimir "Farcaller" Pouzanov <farcaller@gmail.com> #
# #
# Permission is hereby granted, free of charge, to any person obtaining a copy #
# of this software and associated documentation files (the "Software"), to deal #
# in the Software without restriction, including without limitation the rights #
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell #
# copies of the Software, and to permit persons to whom the Software is #
# furnished to do so, subject to the following conditions: #
# #
# The above copyright notice and this permission notice shall be included in #
# all copies or substantial portions of the Software. #
# #
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR #
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, #
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE #
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER #
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, #
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN #
# THE SOFTWARE. #
#################################################################################
import struct
import codecs
from datetime import datetime, timedelta
class BPListWriter(object):
def __init__(self, objects):
self.bplist = ""
self.objects = objects
def binary(self):
'''binary -> string
Generates bplist
'''
self.data = 'bplist00'
# TODO: flatten objects and count max length size
# TODO: write objects and save offsets
# TODO: write offsets
# TODO: write metadata
return self.data
def write(self, filename):
'''
Writes bplist to file
'''
if self.bplist != "":
pass
# TODO: save self.bplist to file
else:
raise Exception('BPlist not yet generated')
class BPListReader(object):
def __init__(self, s):
self.data = s
self.objects = []
self.resolved = {}
def __unpackIntStruct(self, sz, s):
'''__unpackIntStruct(size, string) -> int
Unpacks the integer of given size (1, 2 or 4 bytes) from string
'''
if sz == 1:
ot = '!B'
elif sz == 2:
ot = '!H'
elif sz == 4:
ot = '!I'
elif sz == 8:
ot = '!Q'
else:
raise Exception('int unpack size '+str(sz)+' unsupported')
return struct.unpack(ot, s)[0]
def __unpackInt(self, offset):
'''__unpackInt(offset) -> int
Unpacks int field from plist at given offset
'''
return self.__unpackIntMeta(offset)[1]
def __unpackIntMeta(self, offset):
'''__unpackIntMeta(offset) -> (size, int)
Unpacks int field from plist at given offset and returns its size and value
'''
obj_header = self.data[offset]
obj_type, obj_info = (obj_header & 0xF0), (obj_header & 0x0F)
int_sz = 2**obj_info
return int_sz, self.__unpackIntStruct(int_sz, self.data[offset+1:offset+1+int_sz])
def __resolveIntSize(self, obj_info, offset):
'''__resolveIntSize(obj_info, offset) -> (count, offset)
Calculates count of objref* array entries and returns count and offset to first element
'''
if obj_info == 0x0F:
ofs, obj_count = self.__unpackIntMeta(offset+1)
objref = offset+2+ofs
else:
obj_count = obj_info
objref = offset+1
return obj_count, objref
def __unpackFloatStruct(self, sz, s):
'''__unpackFloatStruct(size, string) -> float
Unpacks the float of given size (4 or 8 bytes) from string
'''
if sz == 4:
ot = '!f'
elif sz == 8:
ot = '!d'
else:
raise Exception('float unpack size '+str(sz)+' unsupported')
return struct.unpack(ot, s)[0]
def __unpackFloat(self, offset):
'''__unpackFloat(offset) -> float
Unpacks float field from plist at given offset
'''
obj_header = self.data[offset]
obj_type, obj_info = (obj_header & 0xF0), (obj_header & 0x0F)
int_sz = 2**obj_info
return int_sz, self.__unpackFloatStruct(int_sz, self.data[offset+1:offset+1+int_sz])
def __unpackDate(self, offset):
td = int(struct.unpack(">d", self.data[offset+1:offset+9])[0])
return datetime(year=2001,month=1,day=1) + timedelta(seconds=td)
def __unpackItem(self, offset):
'''__unpackItem(offset)
Unpacks and returns an item from plist
'''
obj_header = self.data[offset]
obj_type, obj_info = (obj_header & 0xF0), (obj_header & 0x0F)
if obj_type == 0x00:
if obj_info == 0x00: # null 0000 0000
return None
elif obj_info == 0x08: # bool 0000 1000 // false
return False
elif obj_info == 0x09: # bool 0000 1001 // true
return True
elif obj_info == 0x0F: # fill 0000 1111 // fill byte
raise Exception("0x0F Not Implemented") # this is really pad byte, FIXME
else:
raise Exception('unpack item type '+str(obj_header)+' at '+str(offset)+ 'failed')
elif obj_type == 0x10: # int 0001 nnnn ... // # of bytes is 2^nnnn, big-endian bytes
return self.__unpackInt(offset)
elif obj_type == 0x20: # real 0010 nnnn ... // # of bytes is 2^nnnn, big-endian bytes
return self.__unpackFloat(offset)
elif obj_type == 0x30: # date 0011 0011 ... // 8 byte float follows, big-endian bytes
return self.__unpackDate(offset)
elif obj_type == 0x40: # data 0100 nnnn [int] ... // nnnn is number of bytes unless 1111 then int count follows, followed by bytes
obj_count, objref = self.__resolveIntSize(obj_info, offset)
return self.data[objref:objref+obj_count] # XXX: we return data as str
elif obj_type == 0x50: # string 0101 nnnn [int] ... // ASCII string, nnnn is # of chars, else 1111 then int count, then bytes
obj_count, objref = self.__resolveIntSize(obj_info, offset)
return self.data[objref:objref+obj_count]
elif obj_type == 0x60: # string 0110 nnnn [int] ... // Unicode string, nnnn is # of chars, else 1111 then int count, then big-endian 2-byte uint16_t
obj_count, objref = self.__resolveIntSize(obj_info, offset)
return self.data[objref:objref+obj_count*2].decode('utf-16be')
elif obj_type == 0x80: # uid 1000 nnnn ... // nnnn+1 is # of bytes
# FIXME: Accept as a string for now
obj_count, objref = self.__resolveIntSize(obj_info, offset)
return self.data[objref:objref+obj_count]
elif obj_type == 0xA0: # array 1010 nnnn [int] objref* // nnnn is count, unless '1111', then int count follows
obj_count, objref = self.__resolveIntSize(obj_info, offset)
arr = []
for i in range(obj_count):
arr.append(self.__unpackIntStruct(self.object_ref_size, self.data[objref+i*self.object_ref_size:objref+i*self.object_ref_size+self.object_ref_size]))
return arr
elif obj_type == 0xC0: # set 1100 nnnn [int] objref* // nnnn is count, unless '1111', then int count follows
# XXX: not serializable via apple implementation
raise Exception("0xC0 Not Implemented") # FIXME: implement
elif obj_type == 0xD0: # dict 1101 nnnn [int] keyref* objref* // nnnn is count, unless '1111', then int count follows
obj_count, objref = self.__resolveIntSize(obj_info, offset)
keys = []
for i in range(obj_count):
keys.append(self.__unpackIntStruct(self.object_ref_size, self.data[objref+i*self.object_ref_size:objref+i*self.object_ref_size+self.object_ref_size]))
values = []
objref += obj_count*self.object_ref_size
for i in range(obj_count):
values.append(self.__unpackIntStruct(self.object_ref_size, self.data[objref+i*self.object_ref_size:objref+i*self.object_ref_size+self.object_ref_size]))
dic = {}
for i in range(obj_count):
dic[keys[i]] = values[i]
return dic
else:
raise Exception('don\'t know how to unpack obj type '+hex(obj_type)+' at '+str(offset))
def __resolveObject(self, idx):
try:
return self.resolved[idx]
except KeyError:
obj = self.objects[idx]
if type(obj) == list:
newArr = []
for i in obj:
newArr.append(self.__resolveObject(i))
self.resolved[idx] = newArr
return newArr
if type(obj) == dict:
newDic = {}
for k,v in obj.items():
key_resolved = self.__resolveObject(k)
if isinstance(key_resolved, str):
rk = key_resolved
else:
rk = codecs.decode(key_resolved, "utf-8")
rv = self.__resolveObject(v)
newDic[rk] = rv
self.resolved[idx] = newDic
return newDic
else:
self.resolved[idx] = obj
return obj
def parse(self):
# read header
if self.data[:8] != b'bplist00':
raise Exception('Bad magic')
# read trailer
self.offset_size, self.object_ref_size, self.number_of_objects, self.top_object, self.table_offset = struct.unpack('!6xBB4xI4xI4xI', self.data[-32:])
#print "** plist offset_size:",self.offset_size,"objref_size:",self.object_ref_size,"num_objs:",self.number_of_objects,"top:",self.top_object,"table_ofs:",self.table_offset
# read offset table
self.offset_table = self.data[self.table_offset:-32]
self.offsets = []
ot = self.offset_table
for i in range(self.number_of_objects):
offset_entry = ot[:self.offset_size]
ot = ot[self.offset_size:]
self.offsets.append(self.__unpackIntStruct(self.offset_size, offset_entry))
#print "** plist offsets:",self.offsets
# read object table
self.objects = []
k = 0
for i in self.offsets:
obj = self.__unpackItem(i)
#print "** plist unpacked",k,type(obj),obj,"at",i
k += 1
self.objects.append(obj)
# rebuild object tree
#for i in range(len(self.objects)):
# self.__resolveObject(i)
# return root object
return self.__resolveObject(self.top_object)
@classmethod
def plistWithString(cls, s):
parser = cls(s)
return parser.parse()
# helpers for testing
def plist(obj):
from Foundation import NSPropertyListSerialization, NSPropertyListBinaryFormat_v1_0
b = NSPropertyListSerialization.dataWithPropertyList_format_options_error_(obj, NSPropertyListBinaryFormat_v1_0, 0, None)
return str(b.bytes())
def unplist(s):
from Foundation import NSData, NSPropertyListSerialization
d = NSData.dataWithBytes_length_(s, len(s))
return NSPropertyListSerialization.propertyListWithData_options_format_error_(d, 0, None, None)
if __name__ == "__main__":
import os
import sys
import json
file_path = sys.argv[1]
with open(file_path, "rb") as fp:
data = fp.read()
out = BPListReader(data).parse()
with open(file_path + ".json", "w") as fp:
json.dump(out, indent=4)

View File

@@ -0,0 +1,98 @@
#!/usr/bin/python3
import os
from datetime import datetime
from typing import Union
class ChatStore():
def __init__(self, type, name=None, media=None):
if name is not None and not isinstance(name, str):
raise TypeError("Name must be a string or None")
self.name = name
self.messages = {}
self.type = type
if media is not None:
from Whatsapp_Chat_Exporter.utility import Device
if self.type == Device.IOS:
self.my_avatar = os.path.join(media, "Media/Profile/Photo.jpg")
elif self.type == Device.ANDROID:
self.my_avatar = None # TODO: Add Android support
else:
self.my_avatar = None
else:
self.my_avatar = None
self.their_avatar = None
self.their_avatar_thumb = None
self.status = None
def add_message(self, id, message):
if not isinstance(message, Message):
raise TypeError("message must be a Message object")
self.messages[id] = message
def delete_message(self, id):
if id in self.messages:
del self.messages[id]
def to_json(self):
serialized_msgs = {id: msg.to_json() for id, msg in self.messages.items()}
return {
'name': self.name,
'type': self.type,
'my_avatar': self.my_avatar,
'their_avatar': self.their_avatar,
'their_avatar_thumb': self.their_avatar_thumb,
'status': self.status,
'messages': serialized_msgs
}
def get_last_message(self):
return tuple(self.messages.values())[-1]
def get_messages(self):
return self.messages.values()
class Message():
def __init__(self, from_me: Union[bool,int], timestamp: int, time: Union[int,float,str], key_id: int):
self.from_me = bool(from_me)
self.timestamp = timestamp / 1000 if timestamp > 9999999999 else timestamp
if isinstance(time, int) or isinstance(time, float):
self.time = datetime.fromtimestamp(time/1000).strftime("%H:%M")
elif isinstance(time, str):
self.time = time
else:
raise TypeError("Time must be a string or integer")
self.media = False
self.key_id = key_id
self.meta = False
self.data = None
self.sender = None
self.safe = False
self.mime = None
# Extra
self.reply = None
self.quoted_data = None
self.caption = None
self.thumb = None # Android specific
self.sticker = False
def to_json(self):
return {
'from_me' : self.from_me,
'timestamp' : self.timestamp,
'time' : self.time,
'media' : self.media,
'key_id' : self.key_id,
'meta' : self.meta,
'data' : self.data,
'sender' : self.sender,
'safe' : self.safe,
'mime' : self.mime,
'reply' : self.reply,
'quoted_data' : self.quoted_data,
'caption' : self.caption,
'thumb' : self.thumb,
'sticker' : self.sticker
}

View File

@@ -0,0 +1,742 @@
#!/usr/bin/python3
import sqlite3
import json
import jinja2
import os
import shutil
import re
import io
import hmac
from pathlib import Path
from mimetypes import MimeTypes
from hashlib import sha256
from base64 import b64decode, b64encode
from Whatsapp_Chat_Exporter.data_model import ChatStore, Message
from Whatsapp_Chat_Exporter.utility import MAX_SIZE, ROW_SIZE, determine_metadata, get_status_location
from Whatsapp_Chat_Exporter.utility import rendering, Crypt, Device, get_file_name, setup_template
from Whatsapp_Chat_Exporter.utility import brute_force_offset, CRYPT14_OFFSETS, JidType
try:
import zlib
from Crypto.Cipher import AES
except ModuleNotFoundError:
support_backup = False
else:
support_backup = True
try:
import javaobj
except ModuleNotFoundError:
support_crypt15 = False
else:
support_crypt15 = True
def _generate_hmac_of_hmac(key_stream):
key = hmac.new(
hmac.new(
b'\x00' * 32,
key_stream,
sha256
).digest(),
b"backup encryption\x01",
sha256
)
return key.digest(), key_stream
def _extract_encrypted_key(keyfile):
key_stream = b""
for byte in javaobj.loads(keyfile):
key_stream += byte.to_bytes(1, "big", signed=True)
return _generate_hmac_of_hmac(key_stream)
def decrypt_backup(database, key, output, crypt=Crypt.CRYPT14, show_crypt15=False):
if not support_backup:
return 1
if isinstance(key, io.IOBase):
key = key.read()
if crypt is not Crypt.CRYPT15:
t1 = key[30:62]
if crypt is not Crypt.CRYPT15 and len(key) != 158:
raise ValueError("The key file must be 158 bytes")
# Determine the IV and database offsets
if crypt == Crypt.CRYPT14:
if len(database) < 191:
raise ValueError("The crypt14 file must be at least 191 bytes")
current_try = 0
offsets = CRYPT14_OFFSETS[current_try]
t2 = database[15:47]
iv = database[offsets["iv"]:offsets["iv"] + 16]
db_ciphertext = database[offsets["db"]:]
elif crypt == Crypt.CRYPT12:
if len(database) < 67:
raise ValueError("The crypt12 file must be at least 67 bytes")
t2 = database[3:35]
iv = database[51:67]
db_ciphertext = database[67:-20]
elif crypt == Crypt.CRYPT15:
if not support_crypt15:
return 1
if len(database) < 131:
raise ValueError("The crypt15 file must be at least 131 bytes")
t1 = t2 = None
iv = database[8:24]
db_offset = database[0] + 2 # Skip protobuf + protobuf size and backup type
db_ciphertext = database[db_offset:]
if t1 != t2:
raise ValueError("The signature of key file and backup file mismatch")
if crypt == Crypt.CRYPT15:
if len(key) == 32:
main_key, hex_key = _generate_hmac_of_hmac(key)
else:
main_key, hex_key = _extract_encrypted_key(key)
if show_crypt15:
hex_key = [hex_key.hex()[c:c+4] for c in range(0, len(hex_key.hex()), 4)]
print("The HEX key of the crypt15 backup is: " + ' '.join(hex_key))
else:
main_key = key[126:]
decompressed = False
while not decompressed:
cipher = AES.new(main_key, AES.MODE_GCM, iv)
db_compressed = cipher.decrypt(db_ciphertext)
try:
db = zlib.decompress(db_compressed)
except zlib.error:
if crypt == Crypt.CRYPT14:
current_try += 1
if current_try < len(CRYPT14_OFFSETS):
offsets = CRYPT14_OFFSETS[current_try]
iv = database[offsets["iv"]:offsets["iv"] + 16]
db_ciphertext = database[offsets["db"]:]
continue
else:
print("Common offsets are not applicable to "
"your backup. Trying to brute force it...")
for start_iv, end_iv, start_db in brute_force_offset():
iv = database[start_iv:end_iv]
db_ciphertext = database[start_db:]
cipher = AES.new(main_key, AES.MODE_GCM, iv)
db_compressed = cipher.decrypt(db_ciphertext)
try:
db = zlib.decompress(db_compressed)
except zlib.error:
continue
else:
decompressed = True
print(
f"The offsets of your IV and database are {start_iv} and "
f"{start_db}, respectively. To include your offsets in the "
"program, please report it by creating an issue on GitHub: "
"https://github.com/KnugiHK/Whatsapp-Chat-Exporter/discussions/47"
)
break
if not decompressed:
return 2
else:
return 3
else:
decompressed = True
if db[0:6].upper() == b"SQLITE":
with open(output, "wb") as f:
f.write(db)
return 0
else:
raise ValueError("The plaintext is not a SQLite database. Did you use the key to encrypt something...")
def contacts(db, data):
# Get contacts
c = db.cursor()
c.execute("""SELECT count() FROM wa_contacts""")
total_row_number = c.fetchone()[0]
print(f"Processing contacts...({total_row_number})")
c.execute("""SELECT jid, COALESCE(display_name, wa_name) as display_name, status FROM wa_contacts; """)
row = c.fetchone()
while row is not None:
data[row["jid"]] = ChatStore(Device.ANDROID, row["display_name"])
if row["status"] is not None:
data[row["jid"]].status = row["status"]
row = c.fetchone()
def messages(db, data, media_folder):
# Get message history
c = db.cursor()
try:
c.execute("""SELECT count() FROM messages""")
except sqlite3.OperationalError:
c.execute("""SELECT count() FROM message""")
total_row_number = c.fetchone()[0]
print(f"Processing messages...(0/{total_row_number})", end="\r")
try:
c.execute("""SELECT messages.key_remote_jid,
messages._id,
messages.key_from_me,
messages.timestamp,
messages.data,
messages.status,
messages.edit_version,
messages.thumb_image,
messages.remote_resource,
CAST(messages.media_wa_type as INTEGER) as media_wa_type,
messages.latitude,
messages.longitude,
messages_quotes.key_id as quoted,
messages.key_id,
messages_quotes.data as quoted_data,
messages.media_caption,
missed_call_logs.video_call,
chat.subject as chat_subject,
message_system.action_type,
message_system_group.is_me_joined,
jid_old.raw_string as old_jid,
jid_new.raw_string as new_jid,
jid_global.type as jid_type,
group_concat(receipt_user.receipt_timestamp) as receipt_timestamp,
group_concat(message.received_timestamp) as received_timestamp,
group_concat(receipt_user.read_timestamp) as read_timestamp,
group_concat(receipt_user.played_timestamp) as played_timestamp,
group_concat(messages.read_device_timestamp) as read_device_timestamp
FROM messages
LEFT JOIN messages_quotes
ON messages.quoted_row_id = messages_quotes._id
LEFT JOIN missed_call_logs
ON messages._id = missed_call_logs.message_row_id
INNER JOIN jid jid_global
ON messages.key_remote_jid = jid_global.raw_string
LEFT JOIN chat
ON chat.jid_row_id = jid_global._id
LEFT JOIN message_system
ON message_system.message_row_id = messages._id
LEFT JOIN message_system_group
ON message_system_group.message_row_id = messages._id
LEFT JOIN message_system_number_change
ON message_system_number_change.message_row_id = messages._id
LEFT JOIN jid jid_old
ON jid_old._id = message_system_number_change.old_jid_row_id
LEFT JOIN jid jid_new
ON jid_new._id = message_system_number_change.new_jid_row_id
LEFT JOIN receipt_user
ON receipt_user.message_row_id = messages._id
WHERE messages.key_remote_jid <> '-1'
GROUP BY message._id;"""
)
except sqlite3.OperationalError:
try:
c.execute("""SELECT jid_global.raw_string as key_remote_jid,
message._id,
message.from_me as key_from_me,
message.timestamp,
message.text_data as data,
message.status,
message_future.version as edit_version,
message_thumbnail.thumbnail as thumb_image,
message_media.file_path as remote_resource,
message_location.latitude,
message_location.longitude,
message_quoted.key_id as quoted,
message.key_id,
message_quoted.text_data as quoted_data,
message.message_type as media_wa_type,
jid_group.raw_string as group_sender_jid,
chat.subject as chat_subject,
missed_call_logs.video_call,
message.sender_jid_row_id,
message_system.action_type,
message_system_group.is_me_joined,
jid_old.raw_string as old_jid,
jid_new.raw_string as new_jid,
jid_global.type as jid_type,
group_concat(receipt_user.receipt_timestamp) as receipt_timestamp,
group_concat(message.received_timestamp) as received_timestamp,
group_concat(receipt_user.read_timestamp) as read_timestamp,
group_concat(receipt_user.played_timestamp) as played_timestamp
FROM message
LEFT JOIN message_quoted
ON message_quoted.message_row_id = message._id
LEFT JOIN message_location
ON message_location.message_row_id = message._id
LEFT JOIN message_media
ON message_media.message_row_id = message._id
LEFT JOIN message_thumbnail
ON message_thumbnail.message_row_id = message._id
LEFT JOIN message_future
ON message_future.message_row_id = message._id
LEFT JOIN chat
ON chat._id = message.chat_row_id
INNER JOIN jid jid_global
ON jid_global._id = chat.jid_row_id
LEFT JOIN jid jid_group
ON jid_group._id = message.sender_jid_row_id
LEFT JOIN missed_call_logs
ON message._id = missed_call_logs.message_row_id
LEFT JOIN message_system
ON message_system.message_row_id = message._id
LEFT JOIN message_system_group
ON message_system_group.message_row_id = message._id
LEFT JOIN message_system_number_change
ON message_system_number_change.message_row_id = message._id
LEFT JOIN jid jid_old
ON jid_old._id = message_system_number_change.old_jid_row_id
LEFT JOIN jid jid_new
ON jid_new._id = message_system_number_change.new_jid_row_id
LEFT JOIN receipt_user
ON receipt_user.message_row_id = message._id
WHERE key_remote_jid <> '-1'
GROUP BY message._id;"""
)
except Exception as e:
raise e
else:
table_message = True
else:
table_message = False
i = 0
while True:
try:
content = c.fetchone()
except sqlite3.OperationalError:
continue
else:
break
while content is not None:
if content["key_remote_jid"] not in data:
data[content["key_remote_jid"]] = ChatStore(Device.ANDROID, content["chat_subject"])
if content["key_remote_jid"] is None:
continue # Not sure
if "sender_jid_row_id" in content:
sender_jid_row_id = content["sender_jid_row_id"]
else:
sender_jid_row_id = None
message = Message(
from_me=not sender_jid_row_id and content["key_from_me"],
timestamp=content["timestamp"],
time=content["timestamp"],
key_id=content["key_id"],
)
if isinstance(content["data"], bytes):
message.data = ("The message is binary data and its base64 is "
'<a href="https://gchq.github.io/CyberChef/#recipe=From_Base64'
"('A-Za-z0-9%2B/%3D',true,false)Text_Encoding_Brute_Force"
f"""('Decode')&input={b64encode(b64encode(content["data"])).decode()}">""")
message.data += b64encode(content["data"]).decode("utf-8") + "</a>"
message.safe = message.meta = True
data[content["key_remote_jid"]].add_message(content["_id"], message)
i += 1
content = c.fetchone()
continue
if content["jid_type"] == JidType.GROUP and content["key_from_me"] == 0:
name = fallback = None
if table_message:
if content["sender_jid_row_id"] > 0:
_jid = content["group_sender_jid"]
if _jid in data:
name = data[_jid].name
if "@" in _jid:
fallback = _jid.split('@')[0]
else:
if content["remote_resource"] is not None:
if content["remote_resource"] in data:
name = data[content["remote_resource"]].name
if "@" in content["remote_resource"]:
fallback = content["remote_resource"].split('@')[0]
message.sender = name or fallback
else:
message.sender = None
if content["quoted"] is not None:
message.reply = content["quoted"]
if content["quoted_data"] is not None and len(content["quoted_data"]) > 200:
message.quoted_data = content["quoted_data"][:201] + "..."
else:
message.quoted_data = content["quoted_data"]
else:
message.reply = None
if not table_message and content["media_caption"] is not None:
# Old schema
message.caption = content["media_caption"]
elif table_message and content["media_wa_type"] == 1 and content["data"] is not None:
# New schema
message.caption = content["data"]
else:
message.caption = None
if content["status"] == 6: # 6 = Metadata, otherwise assume a message
message.meta = True
name = fallback = None
if table_message:
if content["sender_jid_row_id"] > 0:
_jid = content["group_sender_jid"]
if _jid in data:
name = data[_jid].name
if "@" in _jid:
fallback = _jid.split('@')[0]
else:
name = "You"
else:
_jid = content["remote_resource"]
if _jid is not None:
if _jid in data:
name = data[_jid].name
if "@" in _jid:
fallback = _jid.split('@')[0]
else:
name = "You"
message.data = determine_metadata(content, name or fallback)
if isinstance(message.data, str) and "<br>" in message.data:
message.safe = True
if message.data is None:
if content["video_call"] is not None: # Missed call
message.meta = True
if content["video_call"] == 1:
message.data = "A video call was missed"
elif content["video_call"] == 0:
message.data = "A voice call was missed"
elif content["data"] is None and content["thumb_image"] is None:
message.meta = True
message.data = None
else:
# Real message
message.sticker = content["media_wa_type"] == 20 # Sticker is a message
if content["key_from_me"] == 1:
if content["status"] == 5 and content["edit_version"] == 7 or table_message and content["media_wa_type"] == 15:
msg = "Message deleted"
message.meta = True
else:
if content["media_wa_type"] == 5:
msg = f"Location shared: {content['latitude'], content['longitude']}"
message.meta = True
else:
msg = content["data"]
if msg is not None:
if "\r\n" in msg:
msg = msg.replace("\r\n", "<br>")
if "\n" in msg:
msg = msg.replace("\n", "<br>")
else:
if content["status"] == 0 and content["edit_version"] == 7 or table_message and content["media_wa_type"] == 15:
msg = "Message deleted"
message.meta = True
else:
if content["media_wa_type"] == 5:
msg = f"Location shared: {content['latitude'], content['longitude']}"
message.meta = True
else:
msg = content["data"]
if msg is not None:
if "\r\n" in msg:
msg = msg.replace("\r\n", "<br>")
if "\n" in msg:
msg = msg.replace("\n", "<br>")
message.data = msg
data[content["key_remote_jid"]].add_message(content["_id"], message)
i += 1
if i % 1000 == 0:
print(f"Processing messages...({i}/{total_row_number})", end="\r")
while True:
try:
content = c.fetchone()
except sqlite3.OperationalError:
continue
else:
break
print(f"Processing messages...({total_row_number}/{total_row_number})", end="\r")
def media(db, data, media_folder):
# Get media
c = db.cursor()
c.execute("""SELECT count() FROM message_media""")
total_row_number = c.fetchone()[0]
print(f"\nProcessing media...(0/{total_row_number})", end="\r")
i = 0
try:
c.execute("""SELECT messages.key_remote_jid,
message_row_id,
file_path,
message_url,
mime_type,
media_key,
file_hash,
thumbnail
FROM message_media
INNER JOIN messages
ON message_media.message_row_id = messages._id
LEFT JOIN media_hash_thumbnail
ON message_media.file_hash = media_hash_thumbnail.media_hash
WHERE jid.type <> 7
ORDER BY messages.key_remote_jid ASC"""
)
except sqlite3.OperationalError:
c.execute("""SELECT jid.raw_string as key_remote_jid,
message_row_id,
file_path,
message_url,
mime_type,
media_key,
file_hash,
thumbnail
FROM message_media
INNER JOIN message
ON message_media.message_row_id = message._id
LEFT JOIN chat
ON chat._id = message.chat_row_id
INNER JOIN jid
ON jid._id = chat.jid_row_id
LEFT JOIN media_hash_thumbnail
ON message_media.file_hash = media_hash_thumbnail.media_hash
WHERE jid.type <> 7
ORDER BY jid.raw_string ASC"""
)
content = c.fetchone()
mime = MimeTypes()
if not os.path.isdir(f"{media_folder}/thumbnails"):
Path(f"{media_folder}/thumbnails").mkdir(parents=True, exist_ok=True)
while content is not None:
file_path = f"{media_folder}/{content['file_path']}"
message = data[content["key_remote_jid"]].messages[content["message_row_id"]]
message.media = True
if os.path.isfile(file_path):
message.data = file_path
if content["mime_type"] is None:
guess = mime.guess_type(file_path)[0]
if guess is not None:
message.mime = guess
else:
message.mime = "application/octet-stream"
else:
message.mime = content["mime_type"]
else:
if False: # Block execution
try:
r = requests.get(content["message_url"])
if r.status_code != 200:
raise RuntimeError()
except:
message.data = "The media is missing"
message.mime = "media"
message.meta = True
else:
...
message.data = "The media is missing"
message.mime = "media"
message.meta = True
if content["thumbnail"] is not None:
thumb_path = f"{media_folder}/thumbnails/{b64decode(content['file_hash']).hex()}.png"
if not os.path.isfile(thumb_path):
with open(thumb_path, "wb") as f:
f.write(content["thumbnail"])
message.thumb = thumb_path
i += 1
if i % 100 == 0:
print(f"Processing media...({i}/{total_row_number})", end="\r")
content = c.fetchone()
print(
f"Processing media...({total_row_number}/{total_row_number})", end="\r")
def vcard(db, data, media_folder):
c = db.cursor()
try:
c.execute("""SELECT message_row_id,
messages.key_remote_jid,
vcard,
messages.media_name
FROM messages_vcards
INNER JOIN messages
ON messages_vcards.message_row_id = messages._id
ORDER BY messages.key_remote_jid ASC;"""
)
except sqlite3.OperationalError:
c.execute("""SELECT message_row_id,
jid.raw_string as key_remote_jid,
vcard,
message.text_data as media_name
FROM message_vcard
INNER JOIN message
ON message_vcard.message_row_id = message._id
LEFT JOIN chat
ON chat._id = message.chat_row_id
INNER JOIN jid
ON jid._id = chat.jid_row_id
ORDER BY message.chat_row_id ASC;"""
)
rows = c.fetchall()
total_row_number = len(rows)
print(f"\nProcessing vCards...(0/{total_row_number})", end="\r")
path = f"{media_folder}/vCards"
if not os.path.isdir(path):
Path(path).mkdir(parents=True, exist_ok=True)
for index, row in enumerate(rows):
media_name = row["media_name"] if row["media_name"] is not None else ""
file_name = "".join(x for x in media_name if x.isalnum())
file_name = file_name.encode('utf-8')[:230].decode('utf-8', 'ignore')
file_path = os.path.join(path, f"{file_name}.vcf")
if not os.path.isfile(file_path):
with open(file_path, "w", encoding="utf-8") as f:
f.write(row["vcard"])
message = data[row["key_remote_jid"]].messages[row["message_row_id"]]
message.data = media_name + \
"The vCard file cannot be displayed here, " \
f"however it should be located at {file_path}"
message.mime = "text/x-vcard"
message.meta = True
print(f"Processing vCards...({index + 1}/{total_row_number})", end="\r")
def calls(db, data):
c = db.cursor()
c.execute("""SELECT count() FROM call_log""")
total_row_number = c.fetchone()[0]
if total_row_number == 0:
return
print(f"\nProcessing calls...({total_row_number})", end="\r")
c.execute("""SELECT call_log._id,
jid.raw_string,
from_me,
call_id,
timestamp,
video_call,
duration,
call_result,
bytes_transferred,
chat.subject as chat_subject
FROM call_log
INNER JOIN jid
ON call_log.jid_row_id = jid._id
LEFT JOIN chat
ON call_log.jid_row_id = chat.jid_row_id"""
)
chat = ChatStore(Device.ANDROID, "WhatsApp Calls")
content = c.fetchone()
while content is not None:
call = Message(
from_me=content["from_me"],
timestamp=content["timestamp"],
time=content["timestamp"],
key_id=content["call_id"],
)
_jid = content["raw_string"]
name = data[_jid].name if _jid in data else content["chat_subject"] or None
if _jid is not None and "@" in _jid:
fallback = _jid.split('@')[0]
else:
fallback = None
call.sender = name or fallback
call.meta = True
call.data = (
f"A {'video' if content['video_call'] else 'voice'} "
f"call {'to' if call.from_me else 'from'} "
f"{call.sender} was "
)
if content['call_result'] == 2:
call.data += "not answered." if call.from_me else "missed."
elif content['call_result'] == 3:
call.data += "unavailable."
elif content['call_result'] == 5:
call.data += (
f"initiated and lasted for {content['duration']} second(s) "
f"with {content['bytes_transferred']} bytes transferred."
)
chat.add_message(content["_id"], call)
content = c.fetchone()
data["000000000000000"] = chat
def create_html(
data,
output_folder,
template=None,
embedded=False,
offline_static=False,
maximum_size=None,
no_avatar=False
):
template = setup_template(template, no_avatar)
total_row_number = len(data)
print(f"\nGenerating chats...(0/{total_row_number})", end="\r")
if not os.path.isdir(output_folder):
os.mkdir(output_folder)
w3css = get_status_location(output_folder, offline_static)
for current, contact in enumerate(data):
chat = data[contact]
if len(chat.messages) == 0:
continue
safe_file_name, name = get_file_name(contact, chat)
if maximum_size is not None:
current_size = 0
current_page = 1
render_box = []
if maximum_size == 0:
maximum_size = MAX_SIZE
last_msg = chat.get_last_message().key_id
for message in chat.get_messages():
if message.data is not None and not message.meta and not message.media:
current_size += len(message.data) + ROW_SIZE
else:
current_size += ROW_SIZE + 100 # Assume media and meta HTML are 100 bytes
if current_size > maximum_size:
output_file_name = f"{output_folder}/{safe_file_name}-{current_page}.html"
rendering(
output_file_name,
template,
name,
render_box,
contact,
w3css,
f"{safe_file_name}-{current_page + 1}.html",
chat
)
render_box = [message]
current_size = 0
current_page += 1
else:
if message.key_id == last_msg:
if current_page == 1:
output_file_name = f"{output_folder}/{safe_file_name}.html"
else:
output_file_name = f"{output_folder}/{safe_file_name}-{current_page}.html"
rendering(
output_file_name,
template,
name,
render_box,
contact,
w3css,
False,
chat
)
else:
render_box.append(message)
else:
output_file_name = f"{output_folder}/{safe_file_name}.html"
rendering(
output_file_name,
template,
name,
chat.get_messages(),
contact,
w3css,
False,
chat
)
if current % 10 == 0:
print(f"Generating chats...({current}/{total_row_number})", end="\r")
print(f"Generating chats...({total_row_number}/{total_row_number})", end="\r")

View File

@@ -0,0 +1,92 @@
#!/usr/bin/python3
import os
from datetime import datetime
from mimetypes import MimeTypes
from Whatsapp_Chat_Exporter.data_model import ChatStore, Message
from Whatsapp_Chat_Exporter.utility import Device
def messages(path, data, assume_first_as_me=False):
"""Extracts messages from the exported file"""
with open(path, "r", encoding="utf8") as file:
you = ""
data["ExportedChat"] = ChatStore(Device.EXPORTED)
chat = data["ExportedChat"]
total_row_number = len(file.readlines())
file.seek(0)
for index, line in enumerate(file):
if len(line.split(" - ")) > 1:
time = line.split(" - ")[0]
if ":" not in line.split(time)[1]:
msg.data = line.split(time)[1][3:]
msg.meta = True
else:
name = line.split(time)[1].split(":")[0]
message = line.split(time)[1].split(name + ":")[1].strip()
name = name[3:]
if you == "":
if chat.name is None:
if not assume_first_as_me:
while True:
ans = input(f"Is '{name}' you? (Y/N)").lower()
if ans == "y":
you = name
break
elif ans == "n":
chat.name = name
break
else:
you = name
else:
if name != chat.name:
you = name
elif chat.name is None:
if name != you:
chat.name = name
msg = Message(
you == name,
datetime.strptime(time, "%d/%m/%Y, %H:%M").timestamp(),
time.split(", ")[1].strip(),
index
)
if "<Media omitted>" in message:
msg.data = "The media is omitted in the chat"
msg.mime = "media"
msg.meta = True
elif "(file attached)" in message:
mime = MimeTypes()
msg.media = True
file_path = os.path.join(os.path.dirname(path), message.split("(file attached)")[0].strip())
if os.path.isfile(file_path):
msg.data = file_path
guess = mime.guess_type(file_path)[0]
if guess is not None:
msg.mime = guess
else:
msg.mime = "application/octet-stream"
else:
msg.data = "The media is missing"
msg.mime = "media"
msg.meta = True
else:
msg.data = message
if "\r\n" in message:
msg.data = message.replace("\r\n", "<br>")
if "\n" in message:
msg.data = message.replace("\n", "<br>")
chat.add_message(index, msg)
else:
lookback = index - 1
while lookback not in chat.messages:
lookback -= 1
msg = chat.messages[lookback]
if msg.media:
msg.caption = line.strip()
else:
msg.data += "<br>" + line.strip()
if index % 1000 == 0:
print(f"Processing messages & media...({index}/{total_row_number})", end="\r")
print(f"Processing messages & media...({total_row_number}/{total_row_number})", end="\r")
return data

View File

@@ -0,0 +1,280 @@
#!/usr/bin/python3
import os
from glob import glob
from pathlib import Path
from mimetypes import MimeTypes
from Whatsapp_Chat_Exporter.data_model import ChatStore, Message
from Whatsapp_Chat_Exporter.utility import APPLE_TIME, Device
def contacts(db, data):
c = db.cursor()
# Get status only lol
c.execute("""SELECT count() FROM ZWAADDRESSBOOKCONTACT WHERE ZABOUTTEXT IS NOT NULL""")
total_row_number = c.fetchone()[0]
print(f"Pre-processing contacts...({total_row_number})")
c.execute("""SELECT ZWHATSAPPID, ZABOUTTEXT FROM ZWAADDRESSBOOKCONTACT WHERE ZABOUTTEXT IS NOT NULL""")
content = c.fetchone()
while content is not None:
if not content["ZWHATSAPPID"].endswith("@s.whatsapp.net"):
_id = content["ZWHATSAPPID"] + "@s.whatsapp.net"
data[_id] = ChatStore(Device.IOS)
data[_id].status = content["ZABOUTTEXT"]
content = c.fetchone()
def messages(db, data, media_folder):
c = db.cursor()
# Get contacts
c.execute("""SELECT count() FROM ZWACHATSESSION""")
total_row_number = c.fetchone()[0]
print(f"Processing contacts...({total_row_number})")
c.execute(
"""SELECT ZCONTACTJID,
ZPARTNERNAME,
ZPUSHNAME
FROM ZWACHATSESSION
LEFT JOIN ZWAPROFILEPUSHNAME
ON ZWACHATSESSION.ZCONTACTJID = ZWAPROFILEPUSHNAME.ZJID;"""
)
content = c.fetchone()
while content is not None:
is_phone = content["ZPARTNERNAME"].replace("+", "").replace(" ", "").isdigit()
if content["ZPUSHNAME"] is None or (content["ZPUSHNAME"] and not is_phone):
contact_name = content["ZPARTNERNAME"]
else:
contact_name = content["ZPUSHNAME"]
contact_id = content["ZCONTACTJID"]
if contact_id not in data:
data[contact_id] = ChatStore(Device.IOS, contact_name, media_folder)
else:
data[contact_id].name = contact_name
data[contact_id].my_avatar = os.path.join(media_folder, "Media/Profile/Photo.jpg")
path = f'{media_folder}/Media/Profile/{contact_id.split("@")[0]}'
avatars = glob(f"{path}*")
if 0 < len(avatars) <= 1:
data[contact_id].their_avatar = avatars[0]
else:
for avatar in avatars:
if avatar.endswith(".thumb") and data[content["ZCONTACTJID"]].their_avatar_thumb is None:
data[contact_id].their_avatar_thumb = avatar
elif avatar.endswith(".jpg") and data[content["ZCONTACTJID"]].their_avatar is None:
data[contact_id].their_avatar = avatar
content = c.fetchone()
# Get message history
c.execute("""SELECT count() FROM ZWAMESSAGE""")
total_row_number = c.fetchone()[0]
print(f"Processing messages...(0/{total_row_number})", end="\r")
c.execute("""SELECT COALESCE(ZFROMJID, ZTOJID) as _id,
ZWAMESSAGE.Z_PK,
ZISFROMME,
ZMESSAGEDATE,
ZTEXT,
ZMESSAGETYPE,
ZWAGROUPMEMBER.ZMEMBERJID,
ZMETADATA,
ZSTANZAID
FROM ZWAMESSAGE
LEFT JOIN ZWAGROUPMEMBER
ON ZWAMESSAGE.ZGROUPMEMBER = ZWAGROUPMEMBER.Z_PK
LEFT JOIN ZWAMEDIAITEM
ON ZWAMESSAGE.Z_PK = ZWAMEDIAITEM.ZMESSAGE;""")
i = 0
content = c.fetchone()
while content is not None:
_id = content["_id"]
Z_PK = content["Z_PK"]
if _id not in data:
data[_id] = ChatStore(Device.IOS)
path = f'{media_folder}/Media/Profile/{_id.split("@")[0]}'
avatars = glob(f"{path}*")
if 0 < len(avatars) <= 1:
data[_id].their_avatar = avatars[0]
else:
for avatar in avatars:
if avatar.endswith(".thumb"):
data[_id].their_avatar_thumb = avatar
elif avatar.endswith(".jpg"):
data[_id].their_avatar = avatar
ts = APPLE_TIME + content["ZMESSAGEDATE"]
message = Message(
from_me=content["ZISFROMME"],
timestamp=ts,
time=ts, # TODO: Could be bug
key_id=content["ZSTANZAID"][:17],
)
invalid = False
if "-" in _id and content["ZISFROMME"] == 0:
name = None
if content["ZMEMBERJID"] is not None:
if content["ZMEMBERJID"] in data:
name = data[content["ZMEMBERJID"]].name
if "@" in content["ZMEMBERJID"]:
fallback = content["ZMEMBERJID"].split('@')[0]
else:
fallback = None
else:
fallback = None
message.sender = name or fallback
else:
message.sender = None
if content["ZMESSAGETYPE"] == 6:
# Metadata
if "-" in _id:
# Group
if content["ZTEXT"] is not None:
# Chnaged name
try:
int(content["ZTEXT"])
except ValueError:
msg = f"The group name changed to {content['ZTEXT']}"
message.data = msg
message.meta = True
else:
invalid = True
else:
message.data = None
else:
message.data = None
else:
# real message
if content["ZMETADATA"] is not None and content["ZMETADATA"].startswith(b"\x2a\x14"):
quoted = content["ZMETADATA"][2:19]
message.reply = quoted.decode()
message.quoted_data = None # TODO
if content["ZMESSAGETYPE"] == 15: # Sticker
message.sticker = True
if content["ZISFROMME"] == 1:
if content["ZMESSAGETYPE"] == 14:
msg = "Message deleted"
message.meta = True
else:
msg = content["ZTEXT"]
if msg is not None:
if "\r\n" in msg:
msg = msg.replace("\r\n", "<br>")
if "\n" in msg:
msg = msg.replace("\n", "<br>")
else:
if content["ZMESSAGETYPE"] == 14:
msg = "Message deleted"
message.meta = True
else:
msg = content["ZTEXT"]
if msg is not None:
if "\r\n" in msg:
msg = msg.replace("\r\n", "<br>")
if "\n" in msg:
msg = msg.replace("\n", "<br>")
message.data = msg
if not invalid:
data[_id].add_message(Z_PK, message)
i += 1
if i % 1000 == 0:
print(f"Processing messages...({i}/{total_row_number})", end="\r")
content = c.fetchone()
print(
f"Processing messages...({total_row_number}/{total_row_number})", end="\r")
def media(db, data, media_folder):
c = db.cursor()
# Get media
c.execute("""SELECT count() FROM ZWAMEDIAITEM""")
total_row_number = c.fetchone()[0]
print(f"\nProcessing media...(0/{total_row_number})", end="\r")
i = 0
c.execute("""SELECT COALESCE(ZWAMESSAGE.ZFROMJID, ZWAMESSAGE.ZTOJID) as _id,
ZMESSAGE,
ZMEDIALOCALPATH,
ZMEDIAURL,
ZVCARDSTRING,
ZMEDIAKEY,
ZTITLE
FROM ZWAMEDIAITEM
INNER JOIN ZWAMESSAGE
ON ZWAMEDIAITEM.ZMESSAGE = ZWAMESSAGE.Z_PK
WHERE ZMEDIALOCALPATH IS NOT NULL
ORDER BY _id ASC""")
content = c.fetchone()
mime = MimeTypes()
while content is not None:
file_path = f"{media_folder}/Message/{content['ZMEDIALOCALPATH']}"
_id = content["_id"]
ZMESSAGE = content["ZMESSAGE"]
message = data[_id].messages[ZMESSAGE]
message.media = True
if os.path.isfile(file_path):
message.data = file_path
if content["ZVCARDSTRING"] is None:
guess = mime.guess_type(file_path)[0]
if guess is not None:
message.mime = guess
else:
message.mime = "application/octet-stream"
else:
message.mime = content["ZVCARDSTRING"]
else:
if False: # Block execution
try:
r = requests.get(content["ZMEDIAURL"])
if r.status_code != 200:
raise RuntimeError()
except:
message.data = "The media is missing"
message.mime = "media"
message.meta = True
else:
...
message.data = "The media is missing"
message.mime = "media"
message.meta = True
if content["ZTITLE"] is not None:
message.caption = content["ZTITLE"]
i += 1
if i % 100 == 0:
print(f"Processing media...({i}/{total_row_number})", end="\r")
content = c.fetchone()
print(
f"Processing media...({total_row_number}/{total_row_number})", end="\r")
def vcard(db, data, media_folder):
c = db.cursor()
c.execute("""SELECT DISTINCT ZWAVCARDMENTION.ZMEDIAITEM,
ZWAMEDIAITEM.ZMESSAGE,
COALESCE(ZWAMESSAGE.ZFROMJID,
ZWAMESSAGE.ZTOJID) as _id,
ZVCARDNAME,
ZVCARDSTRING
FROM ZWAVCARDMENTION
INNER JOIN ZWAMEDIAITEM
ON ZWAVCARDMENTION.ZMEDIAITEM = ZWAMEDIAITEM.Z_PK
INNER JOIN ZWAMESSAGE
ON ZWAMEDIAITEM.ZMESSAGE = ZWAMESSAGE.Z_PK""")
contents = c.fetchall()
total_row_number = len(contents)
print(f"\nProcessing vCards...(0/{total_row_number})", end="\r")
path = f'{media_folder}/Message/vCards'
if not os.path.isdir(path):
Path(path).mkdir(parents=True, exist_ok=True)
for index, content in enumerate(contents):
file_name = "".join(x for x in content["ZVCARDNAME"] if x.isalnum())
file_name = file_name.encode('utf-8')[:230].decode('utf-8', 'ignore')
file_path = os.path.join(path, f"{file_name}.vcf")
if not os.path.isfile(file_path):
with open(file_path, "w", encoding="utf-8") as f:
f.write(content["ZVCARDSTRING"])
message = data[content["_id"]].messages[content["ZMESSAGE"]]
message.data = content["ZVCARDNAME"] + \
"The vCard file cannot be displayed here, " \
f"however it should be located at {file_path}"
message.mime = "text/x-vcard"
message.media = True
message.meta = True
print(f"Processing vCards...({index + 1}/{total_row_number})", end="\r")

View File

@@ -0,0 +1,147 @@
#!/usr/bin/python3
import shutil
import sqlite3
import os
import time
import getpass
import threading
from Whatsapp_Chat_Exporter.utility import WhatsAppIdentifier
try:
from iphone_backup_decrypt import EncryptedBackup, RelativePath
from iphone_backup_decrypt import FailedToDecryptError
except ModuleNotFoundError:
support_encrypted = False
else:
support_encrypted = True
def extract_encrypted(base_dir, password, identifiers, bplist_reader=None):
backup = EncryptedBackup(backup_directory=base_dir, passphrase=password, cleanup=False, check_same_thread=False)
print("Decrypting WhatsApp database...", end="")
try:
backup.extract_file(
relative_path=RelativePath.WHATSAPP_MESSAGES,
domain=identifiers.DOMAIN,
output_filename=identifiers.MESSAGE
)
backup.extract_file(
relative_path=RelativePath.WHATSAPP_CONTACTS,
domain=identifiers.DOMAIN,
output_filename=identifiers.CONTACT
)
except FailedToDecryptError:
print("Failed to decrypt backup: incorrect password?")
exit()
else:
print("Done")
extract_thread = threading.Thread(
target=backup.extract_files_by_domain,
args=(identifiers.DOMAIN, identifiers.DOMAIN, bplist_reader)
)
extract_thread.daemon = True
extract_thread.start()
dot = 0
while extract_thread.is_alive():
print(f"Decrypting and extracting files{'.' * dot}{' ' * (3 - dot)}", end="\r")
if dot < 3:
dot += 1
time.sleep(0.5)
else:
dot = 0
time.sleep(0.4)
print(f"All required files decrypted and extracted.", end="\n")
extract_thread.handled = True
return backup
def is_encrypted(base_dir):
with sqlite3.connect(os.path.join(base_dir, "Manifest.db")) as f:
c = f.cursor()
try:
c.execute("""SELECT count()
FROM Files
""")
except sqlite3.OperationalError as e:
raise e # These error cannot be used to determine if the backup is encrypted
except sqlite3.DatabaseError:
return True
else:
return False
def extract_media(base_dir, identifiers, preserve_timestamp=False):
if preserve_timestamp:
from Whatsapp_Chat_Exporter.bplist import BPListReader
preserve_timestamp = BPListReader
if is_encrypted(base_dir):
if not support_encrypted:
print("You don't have the dependencies to handle encrypted backup.")
print("Read more on how to deal with encrypted backup:")
print("https://github.com/KnugiHK/Whatsapp-Chat-Exporter/blob/main/README.md#usage")
return False
print("Encryption detected on the backup!")
password = getpass.getpass("Enter the password for the backup:")
extract_encrypted(base_dir, password, identifiers, preserve_timestamp)
else:
wts_db = os.path.join(base_dir, identifiers.MESSAGE[:2], identifiers.MESSAGE)
contact_db = os.path.join(base_dir, identifiers.CONTACT[:2], identifiers.CONTACT)
if not os.path.isfile(wts_db):
if identifiers is WhatsAppIdentifier:
print("WhatsApp database not found.")
else:
print("WhatsApp Business database not found.")
exit()
else:
shutil.copyfile(wts_db, identifiers.MESSAGE)
if not os.path.isfile(contact_db):
print("Contact database not found.")
exit()
else:
shutil.copyfile(contact_db, identifiers.CONTACT)
_wts_id = identifiers.DOMAIN
with sqlite3.connect(os.path.join(base_dir, "Manifest.db")) as manifest:
manifest.row_factory = sqlite3.Row
c = manifest.cursor()
c.execute(
f"""SELECT count()
FROM Files
WHERE domain = '{_wts_id}'"""
)
total_row_number = c.fetchone()[0]
print(f"Extracting WhatsApp files...(0/{total_row_number})", end="\r")
c.execute(f"""SELECT fileID,
relativePath,
flags,
file AS metadata,
ROW_NUMBER() OVER(ORDER BY relativePath) AS _index
FROM Files
WHERE domain = '{_wts_id}'
ORDER BY relativePath""")
if not os.path.isdir(_wts_id):
os.mkdir(_wts_id)
row = c.fetchone()
while row is not None:
if row["relativePath"] == "":
row = c.fetchone()
continue
destination = os.path.join(_wts_id, row["relativePath"])
hashes = row["fileID"]
folder = hashes[:2]
flags = row["flags"]
if flags == 2:
try:
os.mkdir(destination)
except FileExistsError:
pass
elif flags == 1:
shutil.copyfile(os.path.join(base_dir, folder, hashes), destination)
if preserve_timestamp:
metadata = BPListReader(row["metadata"]).parse()
creation = metadata["$objects"][1]["Birth"]
modification = metadata["$objects"][1]["LastModified"]
os.utime(destination, (modification, modification))
if row["_index"] % 100 == 0:
print(f"Extracting WhatsApp files...({row['_index']}/{total_row_number})", end="\r")
row = c.fetchone()
print(f"Extracting WhatsApp files...({total_row_number}/{total_row_number})", end="\n")

View File

@@ -0,0 +1,312 @@
import jinja2
import json
import os
from bleach import clean as sanitize
from markupsafe import Markup
from datetime import datetime
from enum import IntEnum
from Whatsapp_Chat_Exporter.data_model import ChatStore
try:
from enum import StrEnum, IntEnum
except ImportError:
# < Python 3.11
from enum import Enum
class StrEnum(str, Enum):
pass
class IntEnum(int, Enum):
pass
MAX_SIZE = 4 * 1024 * 1024 # Default 4MB
ROW_SIZE = 0x3D0
def sanitize_except(html):
return Markup(sanitize(html, tags=["br"]))
def determine_day(last, current):
last = datetime.fromtimestamp(last).date()
current = datetime.fromtimestamp(current).date()
if last == current:
return None
else:
return current
def check_update():
import urllib.request
import json
from sys import platform
from .__init__ import __version__
package_url_json = "https://pypi.org/pypi/whatsapp-chat-exporter/json"
try:
raw = urllib.request.urlopen(package_url_json)
except Exception:
print("Failed to check for updates.")
return 1
else:
with raw:
package_info = json.load(raw)
latest_version = tuple(map(int, package_info["info"]["version"].split(".")))
current_version = tuple(map(int, __version__.split(".")))
if current_version < latest_version:
print("===============Update===============")
print("A newer version of WhatsApp Chat Exporter is available.")
print("Current version: " + __version__)
print("Latest version: " + package_info["info"]["version"])
if platform == "win32":
print("Update with: pip install --upgrade whatsapp-chat-exporter")
else:
print("Update with: pip3 install --upgrade whatsapp-chat-exporter")
print("====================================")
else:
print("You are using the latest version of WhatsApp Chat Exporter.")
return 0
def rendering(
output_file_name,
template,
name,
msgs,
contact,
w3css,
next,
chat,
):
if chat.their_avatar_thumb is None and chat.their_avatar is not None:
their_avatar_thumb = chat.their_avatar
else:
their_avatar_thumb = chat.their_avatar_thumb
with open(output_file_name, "w", encoding="utf-8") as f:
f.write(
template.render(
name=name,
msgs=msgs,
my_avatar=chat.my_avatar,
their_avatar=chat.their_avatar,
their_avatar_thumb=their_avatar_thumb,
w3css=w3css,
next=next,
status=chat.status,
)
)
class Device(StrEnum):
IOS = "ios"
ANDROID = "android"
EXPORTED = "exported"
def import_from_json(json_file, data):
from Whatsapp_Chat_Exporter.data_model import ChatStore, Message
with open(json_file, "r") as f:
temp_data = json.loads(f.read())
total_row_number = len(tuple(temp_data.keys()))
print(f"Importing chats from JSON...(0/{total_row_number})", end="\r")
for index, (jid, chat_data) in enumerate(temp_data.items()):
chat = ChatStore(chat_data.get("type"), chat_data.get("name"))
chat.my_avatar = chat_data.get("my_avatar")
chat.their_avatar = chat_data.get("their_avatar")
chat.their_avatar_thumb = chat_data.get("their_avatar_thumb")
chat.status = chat_data.get("status")
for id, msg in chat_data.get("messages").items():
message = Message(
msg["from_me"],
msg["timestamp"],
msg["time"],
msg["key_id"],
)
message.media = msg.get("media")
message.meta = msg.get("meta")
message.data = msg.get("data")
message.sender = msg.get("sender")
message.safe = msg.get("safe")
message.mime = msg.get("mime")
message.reply = msg.get("reply")
message.quoted_data = msg.get("quoted_data")
message.caption = msg.get("caption")
message.thumb = msg.get("thumb")
message.sticker = msg.get("sticker")
chat.add_message(id, message)
data[jid] = chat
print(f"Importing chats from JSON...({index + 1}/{total_row_number})", end="\r")
def get_file_name(contact: str, chat: ChatStore):
if "@" not in contact and contact not in ("000000000000000", "000000000000001", "ExportedChat"):
raise ValueError("Unexpected contact format: " + contact)
phone_number = contact.split('@')[0]
if "-" in contact:
file_name = ""
else:
file_name = phone_number
if chat.name is not None:
if file_name != "":
file_name += "-"
file_name += chat.name.replace("/", "-")
name = chat.name
else:
name = phone_number
return "".join(x for x in file_name if x.isalnum() or x in "- "), name
# Android Specific
CRYPT14_OFFSETS = (
{"iv": 67, "db": 191},
{"iv": 67, "db": 190},
{"iv": 66, "db": 99},
{"iv": 67, "db": 193},
{"iv": 67, "db": 194},
)
class Crypt(IntEnum):
CRYPT15 = 15
CRYPT14 = 14
CRYPT12 = 12
def brute_force_offset(max_iv=200, max_db=200):
for iv in range(0, max_iv):
for db in range(0, max_db):
yield iv, iv + 16, db
def determine_metadata(content, init_msg):
msg = init_msg if init_msg else ""
if content["is_me_joined"] == 1: # Override
return f"You were added into the group by {msg}"
if content["action_type"] == 1:
msg += f''' changed the group name to "{content['data']}"'''
elif content["action_type"] == 4:
msg += " was added to the group"
elif content["action_type"] == 5:
msg += " left the group"
elif content["action_type"] == 6:
msg += f" changed the group icon"
elif content["action_type"] == 7:
msg = "You were removed"
elif content["action_type"] == 8:
msg += ("WhatsApp Internal Error Occurred: "
"you cannot send message to this group")
elif content["action_type"] == 9:
msg += " created a broadcast channel"
elif content["action_type"] == 10:
try:
old = content['old_jid'].split('@')[0]
new = content['new_jid'].split('@')[0]
except (AttributeError, IndexError):
return None
else:
msg = f"{old} changed their number to {new}"
elif content["action_type"] == 11:
msg += f''' created a group with name: "{content['data']}"'''
elif content["action_type"] == 12:
msg += f" added someone" # TODO: Find out who
elif content["action_type"] == 13:
return # Someone left the group
elif content["action_type"] == 14:
msg += f" removed someone" # TODO: Find out who
elif content["action_type"] == 15:
return # Someone promoted someone as an admin
elif content["action_type"] == 18:
if msg != "You":
msg = f"The security code between you and {msg} changed"
else:
msg = "The security code in this chat changed"
elif content["action_type"] == 19:
msg = "This chat is now end-to-end encrypted"
elif content["action_type"] == 20:
msg = "Someone joined this group by using a invite link" # TODO: Find out who
elif content["action_type"] == 27:
msg += " changed the group description to:<br>"
msg += content['data'].replace("\n", '<br>')
elif content["action_type"] == 28:
try:
old = content['old_jid'].split('@')[0]
new = content['new_jid'].split('@')[0]
except (AttributeError, IndexError):
return None
else:
msg = f"{old} changed their number to {new}"
elif content["action_type"] == 46:
return # Voice message in PM??? Seems no need to handle.
elif content["action_type"] == 47:
msg = "The contact is an official business account"
elif content["action_type"] == 50:
msg = "The contact's account type changed from business to standard"
elif content["action_type"] == 56:
msg = "Messgae timer was enabled/updated/disabled"
elif content["action_type"] == 57:
if msg != "You":
msg = f"The security code between you and {msg} changed"
else:
msg = "The security code in this chat changed"
elif content["action_type"] == 58:
msg = "You blocked this contact"
elif content["action_type"] == 67:
return # (PM) this contact use secure service from Facebook???
elif content["action_type"] == 69:
return # (PM) this contact use secure service from Facebook??? What's the difference with 67????
else:
return # Unsupported
return msg
def get_status_location(output_folder, offline_static):
w3css = "https://www.w3schools.com/w3css/4/w3.css"
if not offline_static:
return w3css
import urllib.request
static_folder = os.path.join(output_folder, offline_static)
if not os.path.isdir(static_folder):
os.mkdir(static_folder)
w3css_path = os.path.join(static_folder, "w3.css")
if not os.path.isfile(w3css_path):
with urllib.request.urlopen(w3css) as resp:
with open(w3css_path, "wb") as f: f.write(resp.read())
w3css = os.path.join(offline_static, "w3.css")
def setup_template(template, no_avatar):
if template is None:
template_dir = os.path.dirname(__file__)
template_file = "whatsapp.html"
else:
template_dir = os.path.dirname(template)
template_file = os.path.basename(template)
template_loader = jinja2.FileSystemLoader(searchpath=template_dir)
template_env = jinja2.Environment(loader=template_loader, autoescape=True)
template_env.globals.update(
determine_day=determine_day,
no_avatar=no_avatar
)
template_env.filters['sanitize_except'] = sanitize_except
return template_env.get_template(template_file)
# iOS Specific
APPLE_TIME = datetime.timestamp(datetime(2001, 1, 1))
class WhatsAppIdentifier(StrEnum):
MESSAGE = "7c7fba66680ef796b916b067077cc246adacf01d"
CONTACT = "b8548dc30aa1030df0ce18ef08b882cf7ab5212f"
DOMAIN = "AppDomainGroup-group.net.whatsapp.WhatsApp.shared"
class WhatsAppBusinessIdentifier(StrEnum):
MESSAGE = "724bd3b98b18518b455a87c1f3ac3a0d189c4466"
CONTACT = "d7246a707f51ddf8b17ee2dddabd9e0a4da5c552"
DOMAIN = "AppDomainGroup-group.net.whatsapp.WhatsAppSMB.shared"
class JidType(IntEnum):
PM = 0
GROUP = 1
SYSTEM_BROADCAST = 5
STATUS = 11

View File

@@ -0,0 +1,269 @@
<!DOCTYPE html>
<html>
<head>
<title>Whatsapp - {{ name }}</title>
<meta charset="UTF-8">
<link rel="stylesheet" href="{{w3css}}">
<style>
html, body {
font-size: 12px;
scroll-behavior: smooth;
}
header {
position: fixed;
z-index: 20;
border-bottom: 2px solid #e3e6e7;
font-size: 2em;
font-weight: bolder;
background-color: white;
padding: 20px 0 20px 0;
}
footer {
border-top: 2px solid #e3e6e7;
font-size: 2em;
padding: 20px 0 20px 0;
}
article {
width:500px;
margin:100px auto;
z-index:10;
font-size: 15px;
word-wrap: break-word;
}
img, video {
max-width:100%;
}
div.reply{
font-size: 13px;
text-decoration: none;
}
div:target::before {
content: '';
display: block;
height: 115px;
margin-top: -115px;
visibility: hidden;
}
div:target {
border-style: solid;
border-width: 2px;
animation: border-blink 0.5s steps(1) 5;
border-color: rgba(0,0,0,0)
}
table {
width: 100%;
}
@keyframes border-blink {
0% {
border-color: #2196F3;
}
50% {
border-color: rgba(0,0,0,0);
}
}
.avatar {
border-radius:50%;
overflow:hidden;
max-width: 64px;
max-height: 64px;
}
.name {
color: #3892da;
}
.pad-left-10 {
padding-left: 10px;
}
.pad-right-10 {
padding-right: 10px;
}
.reply_link {
color: #168acc;
}
.blue {
color: #70777a;
}
.sticker {
max-width: 100px !important;
max-height: 100px !important;
}
</style>
</head>
<body>
<header class="w3-center w3-top">
Chat history with {{ name }}
{% if status is not none %}
<br>
<span class="w3-small">{{ status }}</span>
{% endif %}
</header>
<article class="w3-container">
<div class="table">
{% set last = {'last': 946688461.001} %}
{% for msg in msgs -%}
<div class="w3-row w3-padding-small w3-margin-bottom" id="{{ msg.key_id }}">
{% if determine_day(last.last, msg.timestamp) is not none %}
<div class="w3-center w3-padding-16 blue">{{ determine_day(last.last, msg.timestamp) }}</div>
{% if last.update({'last': msg.timestamp}) %}{% endif %}
{% endif %}
{% if msg.from_me == true %}
<div class="w3-row">
<div class="w3-left blue">{{ msg.time }}</div>
<div class="name w3-right-align pad-left-10">You</div>
</div>
<div class="w3-row">
{% if not no_avatar and my_avatar is not none %}
<div class="w3-col m10 l10">
{% else %}
<div class="w3-col m12 l12">
{% endif %}
<div class="w3-right-align">
{% if msg.reply is not none %}
<div class="reply">
<span class="blue">Replying to </span>
<a href="#{{msg.reply}}" class="reply_link">
{% if msg.quoted_data is not none %}
"{{msg.quoted_data}}"
{% else %}
this message
{% endif %}
</a>
</div>
{% endif %}
{% if msg.meta == true or msg.media == false and msg.data is none %}
<div class="w3-panel w3-border-blue w3-pale-blue w3-rightbar w3-leftbar w3-threequarter w3-center">
{% if msg.safe %}
<p>{{ msg.data | safe or 'Not supported WhatsApp internal message' }}</p>
{% else %}
<p>{{ msg.data or 'Not supported WhatsApp internal message' }}</p>
{% endif %}
</div>
{% else %}
{% if msg.media == false %}
{{ msg.data | sanitize_except() | urlize(none, true, '_blank') }}
{% else %}
{% if "image/" in msg.mime %}
<a href="{{ msg.data }}">
<img src="{{ msg.thumb if msg.thumb is not none else msg.data }}" {{ 'class="sticker"' | safe if msg.sticker }} />
</a>
{% elif "audio/" in msg.mime %}
<audio controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
</audio>
{% elif "video/" in msg.mime %}
<video controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
</video>
{% elif "/" in msg.mime %}
<div class="w3-panel w3-border-blue w3-pale-blue w3-rightbar w3-leftbar w3-threequarter w3-center">
<p>The file cannot be displayed here, however it should be located at <a href="./{{ msg.data }}">here</a></p>
</div>
{% else %}
{% filter escape %}{{ msg.data }}{% endfilter %}
{% endif %}
{% if msg.caption is not none %}
<br>
{{ msg.caption | urlize(none, true, '_blank') }}
{% endif %}
{% endif %}
{% endif %}
</div>
</div>
{% if not no_avatar and my_avatar is not none %}
<div class="w3-col m2 l2 pad-left-10">
<a href="{{ my_avatar }}">
<img src="{{ my_avatar }}" onerror="this.style.display='none'" class="avatar">
</a>
</div>
{% endif %}
</div>
{% else %}
<div class="w3-row">
<div class="w3-left pad-right-10 name">
{% if msg.sender is not none %}
{{ msg.sender }}
{% else %}
{{ name }}
{% endif %}
</div>
<div class="w3-right-align blue">{{ msg.time }}</div>
</div>
<div class="w3-row">
{% if not no_avatar %}
<div class="w3-col m2 l2">
{% if their_avatar is not none %}
<a href="{{ their_avatar }}"><img src="{{ their_avatar_thumb or '' }}" onerror="this.style.display='none'" class="avatar"></a>
{% else %}
<img src="{{ their_avatar_thumb or '' }}" onerror="this.style.display='none'" class="avatar">
{% endif %}
</div>
<div class="w3-col m10 l10">
{% else %}
<div class="w3-col m12 l12">
{% endif %}
<div class="w3-left-align">
{% if msg.reply is not none %}
<div class="reply">
<span class="blue">Replying to </span>
<a href="#{{msg.reply}}" class="reply_link">
{% if msg.quoted_data is not none %}
"{{msg.quoted_data}}"
{% else %}
this message
{% endif %}
</a>
</div>
{% endif %}
{% if msg.meta == true or msg.media == false and msg.data is none %}
<div class="w3-panel w3-border-blue w3-pale-blue w3-rightbar w3-leftbar w3-threequarter w3-center">
{% if msg.safe %}
<p>{{ msg.data | safe or 'Not supported WhatsApp internal message' }}</p>
{% else %}
<p>{{ msg.data or 'Not supported WhatsApp internal message' }}</p>
{% endif %}
</div>
{% else %}
{% if msg.media == false %}
{{ msg.data | sanitize_except() | urlize(none, true, '_blank') }}
{% else %}
{% if "image/" in msg.mime %}
<a href="{{ msg.data }}">
<img src="{{ msg.thumb if msg.thumb is not none else msg.data }}" {{ 'class="sticker"' | safe if msg.sticker }} />
</a>
{% elif "audio/" in msg.mime %}
<audio controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
</audio>
{% elif "video/" in msg.mime %}
<video controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
</video>
{% elif "/" in msg.mime %}
<div class="w3-panel w3-border-blue w3-pale-blue w3-rightbar w3-leftbar w3-threequarter w3-center">
<p>The file cannot be displayed here, however it should be located at <a href="./{{ msg.data }}">here</a></p>
</div>
{% else %}
{% filter escape %}{{ msg.data }}{% endfilter %}
{% endif %}
{% if msg.caption is not none %}
<br>
{{ msg.caption | urlize(none, true, '_blank') }}
{% endif %}
{% endif %}
{% endif %}
</div>
</div>
</div>
{% endif %}
</div>
{% endfor %}
</div>
</article>
<footer class="w3-center">
{% if next %}
<a href="./{{ next }}">Next</a>
{% else %}
End of history
{% endif %}
</footer>
</body>
</html>

View File

@@ -1,356 +0,0 @@
#!/usr/bin/python3
import sqlite3
import json
import jinja2
import os
import requests
import shutil
import re
from datetime import datetime
from mimetypes import MimeTypes
def determine_day(last, current):
last = datetime.fromtimestamp(last).date()
current = datetime.fromtimestamp(current).date()
if last == current:
return None
else:
return current
def contacts(db, data):
# Get contacts
c = db.cursor()
c.execute("""SELECT count() FROM wa_contacts""")
total_row_number = c.fetchone()[0]
print(f"Gathering contacts...({total_row_number})")
c.execute("""SELECT jid, display_name FROM wa_contacts; """)
row = c.fetchone()
while row is not None:
data[row[0]] = {"name": row[1], "messages": {}}
row = c.fetchone()
def messages(db, data):
# Get message history
c = db.cursor()
c.execute("""SELECT count() FROM messages""")
total_row_number = c.fetchone()[0]
print(f"Gathering messages...(0/{total_row_number})", end="\r")
phone_number_re = re.compile(r"[0-9]+@s.whatsapp.net")
c.execute("""SELECT messages.key_remote_jid,
messages._id,
messages.key_from_me,
messages.timestamp,
messages.data,
messages.status,
messages.edit_version,
messages.thumb_image,
messages.remote_resource,
messages.media_wa_type,
messages.latitude,
messages.longitude,
messages_quotes.key_id as quoted,
messages.key_id,
messages_quotes.data,
messages.media_caption
FROM messages
LEFT JOIN messages_quotes
ON messages.quoted_row_id = messages_quotes._id;""")
i = 0
content = c.fetchone()
while content is not None:
if content[0] not in data:
data[content[0]] = {"name": None, "messages": {}}
data[content[0]]["messages"][content[1]] = {
"from_me": bool(content[2]),
"timestamp": content[3]/1000,
"time": datetime.fromtimestamp(content[3]/1000).strftime("%H:%M"),
"media": False,
"key_id": content[13]
}
if "-" in content[0] and content[2] == 0:
name = None
if content[8] in data:
name = data[content[8]]["name"]
if "@" in content[8]:
fallback = content[8].split('@')[0]
else:
fallback = None
else:
fallback = None
data[content[0]]["messages"][content[1]]["sender"] = name or fallback
else:
data[content[0]]["messages"][content[1]]["sender"] = None
if content[12] is not None:
data[content[0]]["messages"][content[1]]["reply"] = content[12]
data[content[0]]["messages"][content[1]]["quoted_data"] = content[14]
else:
data[content[0]]["messages"][content[1]]["reply"] = None
if content[15] is not None:
data[content[0]]["messages"][content[1]]["caption"] = content[15]
else:
data[content[0]]["messages"][content[1]]["caption"] = None
if content[5] == 6:
if "-" in content[0]:
# Is Group
if content[4] is not None:
try:
int(content[4])
except ValueError:
msg = "{The group name changed to "f"{content[4]}"" }"
data[content[0]]["messages"][content[1]]["data"] = msg
else:
del data[content[0]]["messages"][content[1]]
else:
thumb_image = content[7]
if thumb_image is not None:
if b"\x00\x00\x01\x74\x00\x1A" in thumb_image:
# Add user
added = phone_number_re.search(
thumb_image.decode("unicode_escape"))[0]
if added in data:
name_right = data[added]["name"]
else:
name_right = added.split('@')[0]
if content[8] is not None:
if content[8] in data:
name_left = data[content[8]]["name"]
else:
name_left = content[8].split('@')[0]
msg = "{"f"{name_left}"f" added {name_right or 'You'}""}"
else:
msg = "{"f"Added {name_right or 'You'}""}"
elif b"\xac\xed\x00\x05\x74\x00" in thumb_image:
# Changed number
original = content[8].split('@')[0]
changed = thumb_image[7:].decode().split('@')[0]
msg = "{"f"{original} changed to {changed}""}"
data[content[0]]["messages"][content[1]]["data"] = msg
else:
if content[4] is None:
del data[content[0]]["messages"][content[1]]
else:
# Private chat
if content[4] is None and content[7] is None:
del data[content[0]]["messages"][content[1]]
else:
if content[2] == 1:
if content[5] == 5 and content[6] == 7:
msg = "{Message deleted}"
else:
if content[9] == "5":
msg = "{ Location shared: "f"{content[10], content[11]}"" }"
else:
msg = content[4]
else:
if content[5] == 0 and content[6] == 7:
msg = "{Message deleted}"
else:
if content[9] == "5":
msg = "{ Location shared: "f"{content[10], content[11]}"" }"
else:
msg = content[4]
data[content[0]]["messages"][content[1]]["data"] = msg
i += 1
if i % 1000 == 0:
print(f"Gathering messages...({i}/{total_row_number})", end="\r")
content = c.fetchone()
print(
f"Gathering messages...({total_row_number}/{total_row_number})", end="\r")
def media(db, data, media_folder):
# Get media
c = db.cursor()
c.execute("""SELECT count() FROM message_media""")
total_row_number = c.fetchone()[0]
print(f"\nGathering media...(0/{total_row_number})", end="\r")
i = 0
c.execute("""SELECT messages.key_remote_jid,
message_row_id,
file_path,
message_url,
mime_type,
media_key
FROM message_media
INNER JOIN messages
ON message_media.message_row_id = messages._id
ORDER BY messages.key_remote_jid ASC""")
content = c.fetchone()
mime = MimeTypes()
while content is not None:
file_path = f"{media_folder}/{content[2]}"
data[content[0]]["messages"][content[1]]["media"] = True
if os.path.isfile(file_path):
data[content[0]]["messages"][content[1]]["data"] = file_path
if content[4] is None:
guess = mime.guess_type(file_path)[0]
if guess is not None:
data[content[0]]["messages"][content[1]]["mime"] = guess
else:
data[content[0]]["messages"][content[1]]["mime"] = "data/data"
else:
data[content[0]]["messages"][content[1]]["mime"] = content[4]
else:
# if "https://mmg" in content[4]:
# try:
# r = requests.get(content[3])
# if r.status_code != 200:
# raise RuntimeError()
# except:
# data[content[0]]["messages"][content[1]]["data"] = "{The media is missing}"
# data[content[0]]["messages"][content[1]]["media"] = True
# data[content[0]]["messages"][content[1]]["mime"] = "media"
# else:
data[content[0]]["messages"][content[1]]["data"] = "{The media is missing}"
data[content[0]]["messages"][content[1]]["mime"] = "media"
i += 1
if i % 100 == 0:
print(f"Gathering media...({i}/{total_row_number})", end="\r")
content = c.fetchone()
print(
f"Gathering media...({total_row_number}/{total_row_number})", end="\r")
def vcard(db, data):
c = db.cursor()
c.execute("""SELECT message_row_id,
messages.key_remote_jid,
vcard,
messages.media_name
FROM messages_vcards
INNER JOIN messages
ON messages_vcards.message_row_id = messages._id
ORDER BY messages.key_remote_jid ASC;""")
rows = c.fetchall()
total_row_number = len(rows)
print(f"\nGathering vCards...(0/{total_row_number})", end="\r")
base = "WhatsApp/vCards"
for index, row in enumerate(rows):
if not os.path.isdir(base):
os.mkdir(base)
file_name = "".join(x for x in row[3] if x.isalnum())
file_path = f"{base}/{file_name}.vcf"
if not os.path.isfile(file_path):
with open(file_path, "w", encoding="utf-8") as f:
f.write(row[2])
data[row[1]]["messages"][row[0]]["data"] = row[3] + \
"{ The vCard file cannot be displayed here, however it " \
"should be located at " + file_path + "}"
data[row[1]]["messages"][row[0]]["mime"] = "text/x-vcard"
print(f"Gathering vCards...({index + 1}/{total_row_number})", end="\r")
def create_html(data, output_folder):
templateLoader = jinja2.FileSystemLoader(searchpath="./")
templateEnv = jinja2.Environment(loader=templateLoader)
templateEnv.globals.update(determine_day=determine_day)
TEMPLATE_FILE = "whatsapp.html"
template = templateEnv.get_template(TEMPLATE_FILE)
total_row_number = len(data)
print(f"\nCreating HTML...(0/{total_row_number})", end="\r")
if not os.path.isdir(output_folder):
os.mkdir(output_folder)
for current, contact in enumerate(data):
if len(data[contact]["messages"]) == 0:
continue
phone_number = contact.split('@')[0]
if "-" in contact:
file_name = ""
else:
file_name = phone_number
if data[contact]["name"] is not None:
if file_name != "":
file_name += "-"
file_name += data[contact]["name"].replace("/", "-")
name = data[contact]["name"]
else:
name = phone_number
safe_file_name = ''
safe_file_name = "".join(x for x in file_name if x.isalnum() or x in "- ")
with open(f"{output_folder}/{safe_file_name}.html", "w", encoding="utf-8") as f:
f.write(
template.render(
name=name,
msgs=data[contact]["messages"].values(),
my_avatar=None,
their_avatar=f"WhatsApp/Avatars/{contact}.j"
)
)
if current % 10 == 0:
print(f"Creating HTML...({current}/{total_row_number})", end="\r")
print(f"Creating HTML...({total_row_number}/{total_row_number})", end="\r")
if __name__ == "__main__":
from optparse import OptionParser
parser = OptionParser()
parser.add_option(
"-w",
"--wa",
dest="wa",
default="wa.db",
help="Path to contact database")
parser.add_option(
"-m",
"--media",
dest="media",
default="WhatsApp",
help="Path to WhatsApp media folder"
)
# parser.add_option(
# "-t",
# "--template",
# dest="html",
# default="wa.db",
# help="Path to HTML template")
(options, args) = parser.parse_args()
msg_db = "msgstore.db"
output_folder = "temp"
contact_db = options.wa
media_folder = options.media
if len(args) == 1:
msg_db = args[0]
elif len(args) == 2:
msg_db = args[0]
output_folder = args[1]
data = {}
if os.path.isfile(contact_db):
with sqlite3.connect(contact_db) as db:
contacts(db, data)
if os.path.isfile(msg_db):
with sqlite3.connect(msg_db) as db:
messages(db, data)
media(db, data, media_folder)
vcard(db, data)
create_html(data, output_folder)
if not os.path.isdir(f"{output_folder}/WhatsApp"):
shutil.move(media_folder, f"{output_folder}/")
with open("result.json", "w") as f:
data = json.dumps(data)
print(f"\nWriting JSON file...({int(len(data)/1024/1024)}MB)")
f.write(data)
print("Everything is done!")

View File

@@ -1,306 +0,0 @@
#!/usr/bin/python3
import sqlite3
import json
import jinja2
import os
import requests
import shutil
from datetime import datetime
from mimetypes import MimeTypes
APPLE_TIME = datetime.timestamp(datetime(2001, 1, 1))
def determine_day(last, current):
last = datetime.fromtimestamp(last).date()
current = datetime.fromtimestamp(current).date()
if last == current:
return None
else:
return current
def messages(db, data):
c = db.cursor()
# Get contacts
c.execute("""SELECT count() FROM ZWACHATSESSION""")
total_row_number = c.fetchone()[0]
print(f"Gathering contacts...({total_row_number})")
c.execute("""SELECT ZCONTACTJID, ZPARTNERNAME FROM ZWACHATSESSION; """)
row = c.fetchone()
while row is not None:
data[row[0]] = {"name": row[1], "messages": {}}
row = c.fetchone()
# Get message history
c.execute("""SELECT count() FROM ZWAMESSAGE""")
total_row_number = c.fetchone()[0]
print(f"Gathering messages...(0/{total_row_number})", end="\r")
c.execute("""SELECT COALESCE(ZFROMJID, ZTOJID),
ZWAMESSAGE.Z_PK,
ZISFROMME,
ZMESSAGEDATE,
ZTEXT,
ZMESSAGETYPE,
ZWAGROUPMEMBER.ZMEMBERJID
FROM main.ZWAMESSAGE
LEFT JOIN main.ZWAGROUPMEMBER
ON main.ZWAMESSAGE.ZGROUPMEMBER = main.ZWAGROUPMEMBER.Z_PK;""")
i = 0
content = c.fetchone()
while content is not None:
if content[0] not in data:
data[content[0]] = {"name": None, "messages": {}}
ts = APPLE_TIME + content[3]
data[content[0]]["messages"][content[1]] = {
"from_me": bool(content[2]),
"timestamp": ts,
"time": datetime.fromtimestamp(ts).strftime("%H:%M"),
"media": False,
"reply": None,
"caption": None
}
if "-" in content[0] and content[2] == 0:
name = None
if content[6] is not None:
if content[6] in data:
name = data[content[6]]["name"]
if "@" in content[6]:
fallback = content[6].split('@')[0]
else:
fallback = None
else:
fallback = None
data[content[0]]["messages"][content[1]]["sender"] = name or fallback
else:
data[content[0]]["messages"][content[1]]["sender"] = None
if content[5] == 6:
# Metadata
if "-" in content[0]:
# Group
if content[4] is not None:
# Chnaged name
try:
int(content[4])
except ValueError:
msg = "{The group name changed to "f"{content[4]}"" }"
data[content[0]]["messages"][content[1]]["data"] = msg
else:
del data[content[0]]["messages"][content[1]]
else:
data[content[0]]["messages"][content[1]]["data"] = None
else:
data[content[0]]["messages"][content[1]]["data"] = None
else:
# real message
if content[2] == 1:
if content[5] == 14:
msg = "{Message deleted}"
else:
msg = content[4]
else:
if content[5] == 14:
msg = "{Message deleted}"
else:
msg = content[4]
data[content[0]]["messages"][content[1]]["data"] = msg
i += 1
if i % 1000 == 0:
print(f"Gathering messages...({i}/{total_row_number})", end="\r")
content = c.fetchone()
print(
f"Gathering messages...({total_row_number}/{total_row_number})", end="\r")
def media(db, data, media_folder):
c = db.cursor()
# Get media
c.execute("""SELECT count() FROM ZWAMEDIAITEM""")
total_row_number = c.fetchone()[0]
print(f"\nGathering media...(0/{total_row_number})", end="\r")
i = 0
c.execute("""SELECT COALESCE(ZWAMESSAGE.ZFROMJID, ZWAMESSAGE.ZTOJID) as _id,
ZMESSAGE,
ZMEDIALOCALPATH,
ZMEDIAURL,
ZVCARDSTRING,
ZMEDIAKEY,
ZTITLE
FROM ZWAMEDIAITEM
INNER JOIN ZWAMESSAGE
ON ZWAMEDIAITEM.ZMESSAGE = ZWAMESSAGE.Z_PK
WHERE ZMEDIALOCALPATH IS NOT NULL
ORDER BY _id ASC""")
content = c.fetchone()
mime = MimeTypes()
while content is not None:
file_path = f"Message/{content[2]}"
data[content[0]]["messages"][content[1]]["media"] = True
if os.path.isfile(file_path):
data[content[0]]["messages"][content[1]]["data"] = file_path
if content[4] is None:
guess = mime.guess_type(file_path)[0]
if guess is not None:
data[content[0]]["messages"][content[1]]["mime"] = guess
else:
data[content[0]]["messages"][content[1]]["mime"] = "data/data"
else:
data[content[0]]["messages"][content[1]]["mime"] = content[4]
else:
# if "https://mmg" in content[4]:
# try:
# r = requests.get(content[3])
# if r.status_code != 200:
# raise RuntimeError()
# except:
# data[content[0]]["messages"][content[1]]["data"] = "{The media is missing}"
# data[content[0]]["messages"][content[1]]["mime"] = "media"
# else:
data[content[0]]["messages"][content[1]]["data"] = "{The media is missing}"
data[content[0]]["messages"][content[1]]["mime"] = "media"
if content[6] is not None:
data[content[0]]["messages"][content[1]]["caption"] = content[6]
i += 1
if i % 100 == 0:
print(f"Gathering media...({i}/{total_row_number})", end="\r")
content = c.fetchone()
print(
f"Gathering media...({total_row_number}/{total_row_number})", end="\r")
def vcard(db, data):
c = db.cursor()
c.execute("""SELECT DISTINCT ZWAVCARDMENTION.ZMEDIAITEM,
ZWAMEDIAITEM.ZMESSAGE,
COALESCE(ZWAMESSAGE.ZFROMJID,
ZWAMESSAGE.ZTOJID) as _id,
ZVCARDNAME,
ZVCARDSTRING
FROM ZWAVCARDMENTION
INNER JOIN ZWAMEDIAITEM
ON ZWAVCARDMENTION.ZMEDIAITEM = ZWAMEDIAITEM.Z_PK
INNER JOIN ZWAMESSAGE
ON ZWAMEDIAITEM.ZMESSAGE = ZWAMESSAGE.Z_PK""")
rows = c.fetchall()
total_row_number = len(rows)
print(f"\nGathering vCards...(0/{total_row_number})", end="\r")
base = "Message/vCards"
for index, row in enumerate(rows):
if not os.path.isdir(base):
os.mkdir(base)
file_name = "".join(x for x in row[3] if x.isalnum())
file_path = f"{base}/{file_name[:200]}.vcf"
if not os.path.isfile(file_path):
with open(file_path, "w", encoding="utf-8") as f:
f.write(row[4])
data[row[2]]["messages"][row[1]]["data"] = row[3] + \
"{ The vCard file cannot be displayed here, however it " \
"should be located at " + file_path + "}"
data[row[2]]["messages"][row[1]]["mime"] = "text/x-vcard"
data[row[2]]["messages"][row[1]]["media"] = True
print(f"Gathering vCards...({index + 1}/{total_row_number})", end="\r")
def create_html(data, output_folder):
templateLoader = jinja2.FileSystemLoader(searchpath="./")
templateEnv = jinja2.Environment(loader=templateLoader)
templateEnv.globals.update(determine_day=determine_day)
TEMPLATE_FILE = "whatsapp.html"
template = templateEnv.get_template(TEMPLATE_FILE)
total_row_number = len(data)
print(f"\nCreating HTML...(0/{total_row_number})", end="\r")
if not os.path.isdir(output_folder):
os.mkdir(output_folder)
for current, contact in enumerate(data):
if len(data[contact]["messages"]) == 0:
continue
phone_number = contact.split('@')[0]
if "-" in contact:
file_name = ""
else:
file_name = phone_number
if data[contact]["name"] is not None:
if file_name != "":
file_name += "-"
file_name += data[contact]["name"].replace("/", "-")
name = data[contact]["name"]
else:
name = phone_number
safe_file_name = ''
safe_file_name = "".join(x for x in file_name if x.isalnum() or x in "- ")
with open(f"{output_folder}/{safe_file_name}.html", "w", encoding="utf-8") as f:
f.write(
template.render(
name=name,
msgs=data[contact]["messages"].values(),
my_avatar=None,
their_avatar=f"WhatsApp/Avatars/{contact}.j"
)
)
if current % 10 == 0:
print(f"Creating HTML...({current}/{total_row_number})", end="\r")
print(f"Creating HTML...({total_row_number}/{total_row_number})", end="\r")
if __name__ == "__main__":
from optparse import OptionParser
parser = OptionParser()
parser.add_option(
"-w",
"--wa",
dest="wa",
default="wa.db",
help="Path to contact database")
parser.add_option(
"-m",
"--media",
dest="media",
default="Message",
help="Path to WhatsApp media folder"
)
# parser.add_option(
# "-t",
# "--template",
# dest="html",
# default="wa.db",
# help="Path to HTML template")
(options, args) = parser.parse_args()
msg_db = "7c7fba66680ef796b916b067077cc246adacf01d"
output_folder = "temp"
contact_db = options.wa
media_folder = options.media
if len(args) == 1:
msg_db = args[0]
elif len(args) == 2:
msg_db = args[0]
output_folder = args[1]
data = {}
if os.path.isfile(msg_db):
with sqlite3.connect(msg_db) as db:
messages(db, data)
media(db, data, media_folder)
vcard(db, data)
create_html(data, output_folder)
if not os.path.isdir(f"{output_folder}/WhatsApp"):
shutil.move(media_folder, f"{output_folder}/")
with open("result.json", "w") as f:
data = json.dumps(data)
print(f"\nWriting JSON file...({int(len(data)/1024/1024)}MB)")
f.write(data)
print("Everything is done!")

View File

@@ -1,50 +0,0 @@
#!/usr/bin/python3
import shutil
import sqlite3
import os
def extract_media(base_dir):
with sqlite3.connect(f"{base_dir}/Manifest.db") as manifest:
c = manifest.cursor()
c.execute("""SELECT count()
FROM Files
WHERE relativePath
LIKE 'Message/Media/%'""")
total_row_number = c.fetchone()[0]
print(f"Gathering media...(0/{total_row_number})", end="\r")
c.execute("""SELECT fileID,
relativePath,
flags
FROM Files
WHERE relativePath
LIKE 'Message/Media/%'""")
row = c.fetchone()
if not os.path.isdir("Message"):
os.mkdir("Message")
if not os.path.isdir("Message/Media"):
os.mkdir("Message/Media")
i = 0
while row is not None:
destination = row[1]
hashes = row[0]
folder = hashes[:2]
flags = row[2]
if flags == 2:
os.mkdir(destination)
elif flags == 1:
shutil.copyfile(f"{base_dir}/{folder}/{hashes}", destination)
i += 1
if i % 100 == 0:
print(f"Gathering media...({i}/{total_row_number})", end="\r")
row = c.fetchone()
print(f"Gathering media...({total_row_number}/{total_row_number})", end="\r")
if __name__ == "__main__":
from optparse import OptionParser
parser = OptionParser()
(_, args) = parser.parse_args()
base_dir = args[0]
extract_media(base_dir)

BIN
group.png

Binary file not shown.

Before

Width:  |  Height:  |  Size: 29 KiB

BIN
imgs/android_structure.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

BIN
imgs/group.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

BIN
imgs/pm.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 126 KiB

View File

Before

Width:  |  Height:  |  Size: 7.8 KiB

After

Width:  |  Height:  |  Size: 7.8 KiB

BIN
pm.png

Binary file not shown.

Before

Width:  |  Height:  |  Size: 32 KiB

69
setup.py Normal file
View File

@@ -0,0 +1,69 @@
import setuptools
from re import search
with open("README.md", "r") as fh:
long_description = fh.read()
with open("Whatsapp_Chat_Exporter/__init__.py", encoding="utf8") as f:
version = search(r'__version__ = "(.*?)"', f.read()).group(1)
setuptools.setup(
name="whatsapp-chat-exporter",
version=version,
author="KnugiHK",
author_email="hello@knugi.com",
description=("A Whatsapp database parser that will give you the "
"history of your Whatsapp conversations in HTML and JSON. "
"Android, iOS, iPadOS, Crypt12, Crypt14, Crypt15 supported."),
long_description=long_description,
long_description_content_type="text/markdown",
license="MIT",
keywords=[
"android", "ios", "parsing", "history", "iphone", "message", "crypt15",
"customizable", "whatsapp", "android-backup", "messages", "crypt14",
"crypt12", "whatsapp-chat-exporter", "whatsapp-export", "iphone-backup",
"whatsapp-database", "whatsapp-database-parser", "whatsapp-conversations"
],
platforms=["any"],
url="https://github.com/KnugiHK/Whatsapp-Chat-Exporter",
packages=setuptools.find_packages(),
package_data={
'': ['whatsapp.html']
},
classifiers=[
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
"Development Status :: 4 - Beta",
"Environment :: Console",
"Intended Audience :: End Users/Desktop",
"Topic :: Communications :: Chat",
"Topic :: Utilities",
"Topic :: Database"
],
python_requires='>=3.8',
install_requires=[
'jinja2',
'bleach'
],
extras_require={
'android_backup': ["pycryptodome", "javaobj-py3"],
'crypt12': ["pycryptodome"],
'crypt14': ["pycryptodome"],
'crypt15': ["pycryptodome", "javaobj-py3"],
'all': ["pycryptodome", "javaobj-py3"],
'everything': ["pycryptodome", "javaobj-py3"],
'backup': ["pycryptodome", "javaobj-py3"]
},
entry_points={
"console_scripts": [
"wtsexporter = Whatsapp_Chat_Exporter.__main__:main",
"waexporter = Whatsapp_Chat_Exporter.__main__:main",
"whatsapp-chat-exporter = Whatsapp_Chat_Exporter.__main__:main"
]
}
)

View File

@@ -1,158 +0,0 @@
<!DOCTYPE html>
<html>
<head>
<title>Whatsapp - {{ name }}</title>
<link rel="stylesheet" href="https://www.w3schools.com/w3css/4/w3.css">
<style>
@import url('https://fonts.googleapis.com/css2?family=Noto+Sans+HK:wght@300;400&display=swap');
html {
font-family: 'Noto Sans HK', sans-serif;
font-size: 12px;
scroll-behavior: smooth;
}
header {
position: fixed;
z-index: 20;
border-bottom: 2px solid #e3e6e7;
font-size: 2em;
font-weight: bolder;
background-color: white;
padding: 20px 0 20px 0;
}
footer {
border-top: 2px solid #e3e6e7;
font-size: 2em;
padding: 20px 0 20px 0;
}
article {
width:500px;
margin:100px auto;
z-index:10;
font-size: 15px;
word-wrap: break-word;
}
img, video {
max-width:100%;
}
a.anchor {
display: block;
position: relative;
top: -100px;
visibility: hidden;
}
div.reply{
font-size: 13px;
text-decoration: none;
}
</style>
</head>
<body>
<header class="w3-center w3-top">Chat history with {{ name }}</header>
<article class="w3-container">
<div class="table" style="width:100%">
{% set last = {'last': 946688461.001} %}
{% for msg in msgs -%}
<div class="w3-row" style="padding-bottom: 10px">
<a class="anchor" id="{{ msg.key_id }}"></a>
{% if determine_day(last.last, msg.timestamp) is not none %}
<div class="w3-center" style="color:#70777c;padding: 10px 0 10px 0;">{{ determine_day(last.last, msg.timestamp) }}</div>
{% if last.update({'last': msg.timestamp}) %}{% endif %}
{% endif %}
{% if msg.from_me == true %}
<div class="w3-row">
<div style="float: left; color:#70777c;">{{ msg.time }}</div>
<div style="padding-left: 10px; text-align: right; color: #3892da;">You</div>
</div>
<div class="w3-row">
<div class="w3-col m10 l10">
<div style="text-align: right;">
{% if msg.reply is not none %}
<div class="reply">
<span style="color: #70777a;">Replying to </span>
<a href="#{{msg.reply}}" style="color: #168acc;">"{{ msg.quoted_data or 'media' }}"</a>
</div>
{% endif %}
{% if msg.media == false %}
{% filter escape %}{{ msg.data or "{This message is not supported yet}" | replace('\n', '<br>') }}{% endfilter %}
{% else %}
{% if "image/" in msg.mime %}
<a href="{{ msg.data }}"><img src="{{ msg.data }}" /></a>
{% elif "audio/" in msg.mime %}
<audio controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
</audio>
{% elif "video/" in msg.mime %}
<video controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
</video>
{% elif "/" in msg.mime %}
{The file cannot be displayed here, however it should be located at {{ msg.data }}}
{% else %}
{% filter escape %}{{ msg.data }}{% endfilter %}
{% endif %}
{% if msg.caption is not none %}
<br>
{{ msg.caption }}
{% endif %}
{% endif %}
</div>
</div>
<div class="w3-col m2 l2" style="padding-left: 10px"><img src="{{ my_avatar }}" onerror="this.style.display='none'"></div>
</div>
{% else %}
<div class="w3-row">
<div style="padding-right: 10px; float: left; color: #3892da;">
{% if msg.sender is not none %}
{{ msg.sender }}
{% else %}
{{ name }}
{% endif %}
</div>
<div style="text-align: right; color:#70777c;">{{ msg.time }}</div>
</div>
<div class="w3-row">
<div class="w3-col m2 l2"><img src="{{ their_avatar }}" onerror="this.style.display='none'"></div>
<div class="w3-col m10 l10">
<div style="text-align: left;">
{% if msg.reply is not none %}
<div class="reply">
<span style="color: #70777a;">Replying to </span>
<a href="#{{msg.reply}}" style="color: #168acc;">"{{ msg.quoted_data or 'media' }}"</a>
</div>
{% endif %}
{% if msg.media == false %}
{% filter escape %}{{ msg.data or "{This message is not supported yet}" }}{% endfilter %}
{% else %}
{% if "image/" in msg.mime %}
<a href="{{ msg.data }}"><img src="{{ msg.data }}" /></a>
{% elif "audio/" in msg.mime %}
<audio controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
</audio>
{% elif "video/" in msg.mime %}
<video controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
</video>
{% elif "/" in msg.mime %}
{The file cannot be displayed here, however it should be located at {{ msg.data }}}
{% else %}
{% filter escape %}{{ msg.data }}{% endfilter %}
{% endif %}
{% if msg.caption is not none %}
<br>
{{ msg.caption }}
{% endif %}
{% endif %}
</div>
</div>
</div>
{% endif %}
</div>
{% endfor %}
</div>
</article>
<footer class="w3-center">
End of history
</footer>
</body>
</html>