27 Commits
0.6 ... 0.7.0

Author SHA1 Message Date
KnugiHK
abf4b20bc6 Merge branch 'main' of https://github.com/KnugiHK/Whatsapp-Chat-Exporter 2021-12-28 19:44:00 +08:00
KnugiHK
f2f6258960 Bump version number 2021-12-28 19:38:01 +08:00
Knugi
62af48c78e Update README.md 2021-12-28 11:34:23 +00:00
Knugi
c9158d202d Update To do 2021-12-28 11:27:29 +00:00
KnugiHK
fb5e4d5421 Merge branch 'dev' 2021-12-28 19:26:02 +08:00
Knugi
d85c91fbdc Update README.md 2021-11-18 03:09:03 +00:00
KnugiHK
0dddb63c5e Merge branch 'main' into dev 2021-08-13 20:28:49 +08:00
KnugiHK
dd36960ecb Implement CSS for metadata 2021-08-13 20:25:59 +08:00
Knugi
620a1bcdb7 Update README.md 2021-07-13 10:54:07 +00:00
Knugi
896a6d2ddd Update README.md 2021-07-13 10:53:41 +00:00
KnugiHK
4ee92e7efc Bug fix
The directory cannot be created if the parent directory is not present
2021-07-11 18:24:59 +08:00
KnugiHK
f91aac1e11 Implementing newline to <br> 2021-07-11 18:17:06 +08:00
KnugiHK
27a6ff98b3 Merge branch 'main' into dev 2021-07-11 11:05:41 +08:00
Knugi
1952c0835c Update README.md 2021-07-11 03:05:15 +00:00
KnugiHK
ab42cad166 Some PEP8 2021-07-10 22:01:04 +08:00
KnugiHK
3ed59ee051 Bug fix 2021-07-10 21:53:28 +08:00
KnugiHK
f9358ded14 Update README.md 2021-07-10 21:50:30 +08:00
KnugiHK
790f4ec5e0 Support custom template 2021-07-10 21:46:45 +08:00
KnugiHK
35ef4031fc Support crypt12 2021-07-10 21:28:49 +08:00
KnugiHK
b9f343cf2f Update README and setup.py 2021-07-10 21:11:49 +08:00
KnugiHK
18ee152688 Support crypt14 WhatsApp Backup 2021-07-10 21:08:52 +08:00
Knugi
620e89a185 Update README.md 2021-07-06 03:17:57 +00:00
KnugiHK
3ada8916f9 Merge branch 'main' of https://github.com/KnugiHK/Whatsapp-Chat-Exporter into main 2021-05-31 16:13:07 +08:00
Knugi
07ebb692e5 Create CNAME 2021-05-31 08:05:44 +00:00
Knugi
7255f0fe2b Set theme jekyll-theme-cayman 2021-05-31 08:04:21 +00:00
KnugiHK
684badb9a6 Update the string appear in wtsexporter --version 2021-05-31 11:09:47 +08:00
Knugi
e1221d9f59 Update README.md 2021-05-31 03:04:07 +00:00
11 changed files with 280 additions and 100 deletions

1
CNAME Normal file
View File

@@ -0,0 +1 @@
wts.knugi.com

View File

@@ -1,13 +1,18 @@
# Whatsapp-Chat-Exporter # Whatsapp-Chat-Exporter
An Android and iPhone Whatsapp database parser that will give you the history of your Whatsapp conversations in HTML and JSON. [![Latest in Pypi](https://img.shields.io/pypi/v/whatsapp-chat-exporter?label=Latest%20in%20Pypi)](https://pypi.org/project/whatsapp-chat-exporter/)
![License MIT](https://img.shields.io/pypi/l/whatsapp-chat-exporter)
[![Python](https://img.shields.io/pypi/pyversions/Whatsapp-Chat-Exporter)](https://pypi.org/project/Whatsapp-Chat-Exporter/)
A customizable Android and iPhone Whatsapp database parser that will give you the history of your Whatsapp conversations in HTML and JSON.
**If you plan to uninstall WhatsApp or delete your WhatsApp account, please make a backup of your WhatsApp database. You may want to use this exporter again on the same database in the future as the exporter develops** **If you plan to uninstall WhatsApp or delete your WhatsApp account, please make a backup of your WhatsApp database. You may want to use this exporter again on the same database in the future as the exporter develops**
# Usage # Usage
**If you want to use the old release (< 0.5) of the exporter, please follow the [old usage guide](old_README.md#usage)** **If you want to use the old release (< 0.5) of the exporter, please follow the [old usage guide](https://github.com/KnugiHK/Whatsapp-Chat-Exporter/blob/main/old_README.md#usage)**
First, install the exporter by: First, install the exporter by:
```shell ```shell
pip install whatsapp-chat-exporter pip install whatsapp-chat-exporter
pip install whatsapp-chat-exporter[android_backup] & :: Optional, if you want it to support decrypting Android WhatsApp backup.
``` ```
Then, create a working directory in somewhere you want Then, create a working directory in somewhere you want
```shell ```shell
@@ -15,6 +20,7 @@ mkdir working_wts
cd working_wts cd working_wts
``` ```
## Working with Android ## Working with Android
### Unencrypted WhatsApp database
Extract the WhatsApp database with whatever means, one possible means is to use the [WhatsApp-Key-DB-Extractor](https://github.com/KnugiHK/WhatsApp-Key-DB-Extractor) Extract the WhatsApp database with whatever means, one possible means is to use the [WhatsApp-Key-DB-Extractor](https://github.com/KnugiHK/WhatsApp-Key-DB-Extractor)
After you obtain your WhatsApp databse, copy the WhatsApp database and media folder to the working directory. The database is called msgstore.db. If you also want the name of your contacts, get the contact database, which is called wa.db. And copy the WhatsApp (Media) directory from your phone directly. After you obtain your WhatsApp databse, copy the WhatsApp database and media folder to the working directory. The database is called msgstore.db. If you also want the name of your contacts, get the contact database, which is called wa.db. And copy the WhatsApp (Media) directory from your phone directly.
@@ -22,12 +28,28 @@ After you obtain your WhatsApp databse, copy the WhatsApp database and media fol
And now, you should have something like this in the working directory. And now, you should have something like this in the working directory.
![Android folder structure](imgs/android_structure.png) ![Android folder structure](imgs/android_structure.png)
### Extracting #### Extracting
Simply invoke the following command from shell, remember to replace the username and device id correspondingly in the command. Simply invoke the following command from shell.
```sh ```sh
wtsexporter -a wtsexporter -a
``` ```
### Encrypted Android WhatsApp Backup
In order to support the decryption, install pycryptodome if it is not installed
```sh
pip install pycryptodome
```
Place the decryption key file (key) and the encrypted WhatsApp Backup (msgstore.db.crypt14) in the working directory. If you also want the name of your contacts, get the contact database, which is called wa.db. And copy the WhatsApp (Media) directory from your phone directly.
And now, you should have something like this in the working directory.
![Android folder structure with WhatsApp Backup](imgs/android_structure_backup.png)
#### Extracting
Simply invoke the following command from shell.
```sh
wtsexporter -a -k key -b msgstore.db.crypt14
```
## Working with iPhone ## Working with iPhone
Do an iPhone Backup with iTunes first. Do an iPhone Backup with iTunes first.
### Encrypted iPhone Backup ### Encrypted iPhone Backup
@@ -35,7 +57,6 @@ Do an iPhone Backup with iTunes first.
If you want to work on an encrypted iPhone Backup, you should install iphone_backup_decrypt from [KnugiHK/iphone_backup_decrypt](https://github.com/KnugiHK/iphone_backup_decrypt) before you run the extract_iphone_media.py. If you want to work on an encrypted iPhone Backup, you should install iphone_backup_decrypt from [KnugiHK/iphone_backup_decrypt](https://github.com/KnugiHK/iphone_backup_decrypt) before you run the extract_iphone_media.py.
```sh ```sh
pip install biplist pycryptodome & :: Optional, since the pip will install these dependencies automatically.
pip install git+https://github.com/KnugiHK/iphone_backup_decrypt pip install git+https://github.com/KnugiHK/iphone_backup_decrypt
``` ```
### Extracting ### Extracting
@@ -67,17 +88,19 @@ Options:
-m MEDIA, --media=MEDIA -m MEDIA, --media=MEDIA
Path to WhatsApp media folder Path to WhatsApp media folder
-b BACKUP, --backup=BACKUP -b BACKUP, --backup=BACKUP
Path to iPhone backup Path to Android (must be used together with -k)/iPhone
WhatsApp backup
-o OUTPUT, --output=OUTPUT -o OUTPUT, --output=OUTPUT
Output to specific directory Output to specific directory
-j, --json Save the result to a single JSON file -j, --json Save the result to a single JSON file
-d DB, --db=DB Path to database file -d DB, --db=DB Path to database file
-k KEY, --key=KEY Path to key file
-t TEMPLATE, --template=TEMPLATE
Path to custom HTML template
``` ```
# To do # To do
1. Convert ```\r\n``` to ```<br>``` 1. Reply in iPhone
2. Reply in iPhone
3. The CSS for metadata (e.g. {Message Deleted})
# Copyright # Copyright
This is a MIT licensed project. This is a MIT licensed project.

View File

@@ -1 +1 @@
__version__ = "0.6" __version__ = "0.7.0"

View File

@@ -1,5 +1,6 @@
from .__init__ import __version__ from .__init__ import __version__
from Whatsapp_Chat_Exporter import extract, extract_iphone, extract_iphone_media from Whatsapp_Chat_Exporter import extract, extract_iphone
from Whatsapp_Chat_Exporter import extract_iphone_media
from optparse import OptionParser from optparse import OptionParser
import os import os
import sqlite3 import sqlite3
@@ -8,7 +9,7 @@ import json
def main(): def main():
parser = OptionParser(version=__version__) parser = OptionParser(version=f"Whatsapp Chat Exporter: {__version__}")
parser.add_option( parser.add_option(
'-a', '-a',
'--android', '--android',
@@ -40,7 +41,8 @@ def main():
"--backup", "--backup",
dest="backup", dest="backup",
default=None, default=None,
help="Path to iPhone backup") help="Path to Android (must be used together "
"with -k)/iPhone WhatsApp backup")
parser.add_option( parser.add_option(
"-o", "-o",
"--output", "--output",
@@ -60,6 +62,19 @@ def main():
dest='db', dest='db',
default=None, default=None,
help="Path to database file") help="Path to database file")
parser.add_option(
'-k',
'--key',
dest='key',
default=None,
help="Path to key file"
)
parser.add_option(
"-t",
"--template",
dest="template",
default=None,
help="Path to custom HTML template")
(options, args) = parser.parse_args() (options, args) = parser.parse_args()
if options.android and options.iphone: if options.android and options.iphone:
@@ -80,6 +95,18 @@ def main():
msg_db = "msgstore.db" msg_db = "msgstore.db"
else: else:
msg_db = options.db msg_db = options.db
if options.key is not None:
if options.backup is None:
print("You must specify the backup file with -b")
return False
print("Decryption key specified, decrypting WhatsApp backup...")
key = open(options.key, "rb").read()
db = open(options.backup, "rb").read()
is_crypt14 = False if "crypt12" in options.backup else True
if not extract.decrypt_backup(db, key, msg_db, is_crypt14):
print("Dependencies of decrypt_backup are not "
"present. For details, see README.md")
return False
if options.wa is None: if options.wa is None:
contact_db = "wa.db" contact_db = "wa.db"
else: else:
@@ -120,7 +147,7 @@ def main():
messages(db, data) messages(db, data)
media(db, data, options.media) media(db, data, options.media)
vcard(db, data) vcard(db, data)
create_html(data, options.output) create_html(data, options.output, options.template)
if not os.path.isdir(f"{options.output}/{options.media}"): if not os.path.isdir(f"{options.output}/{options.media}"):
shutil.move(options.media, f"{options.output}/") shutil.move(options.media, f"{options.output}/")

View File

@@ -8,8 +8,22 @@ import requests
import shutil import shutil
import re import re
import pkgutil import pkgutil
from pathlib import Path
from bleach import clean as sanitize
from markupsafe import Markup
from datetime import datetime from datetime import datetime
from mimetypes import MimeTypes from mimetypes import MimeTypes
try:
import zlib
from Crypto.Cipher import AES
except ModuleNotFoundError:
support_backup = False
else:
support_backup = True
def sanitize_except(html):
return Markup(sanitize(html, tags=["br"]))
def determine_day(last, current): def determine_day(last, current):
@@ -21,6 +35,39 @@ def determine_day(last, current):
return current return current
def decrypt_backup(database, key, output, crypt14=True):
if not support_backup:
return False
if len(key) != 158:
raise ValueError("The key file must be 158 bytes")
t1 = key[30:62]
if crypt14:
if len(database) < 191:
raise ValueError("The crypt14 file must be at least 191 bytes")
t2 = database[15:47]
iv = database[67:83]
db_ciphertext = database[191:]
else:
if len(database) < 67:
raise ValueError("The crypt12 file must be at least 67 bytes")
t2 = database[3:35]
iv = database[51:67]
db_ciphertext = database[67:-20]
if t1 != t2:
raise ValueError("The signature of key file and backup file mismatch")
main_key = key[126:]
cipher = AES.new(main_key, AES.MODE_GCM, iv)
db_compressed = cipher.decrypt(db_ciphertext)
db = zlib.decompress(db_compressed)
if db[0:6].upper() == b"SQLITE":
with open(output, "wb") as f:
f.write(db)
return True
else:
raise ValueError("The plaintext is not a SQLite database. Did you use the key to encrypt something...")
def contacts(db, data): def contacts(db, data):
# Get contacts # Get contacts
c = db.cursor() c = db.cursor()
@@ -72,7 +119,9 @@ def messages(db, data):
"timestamp": content[3]/1000, "timestamp": content[3]/1000,
"time": datetime.fromtimestamp(content[3]/1000).strftime("%H:%M"), "time": datetime.fromtimestamp(content[3]/1000).strftime("%H:%M"),
"media": False, "media": False,
"key_id": content[13] "key_id": content[13],
"meta": False,
"data": None
} }
if "-" in content[0] and content[2] == 0: if "-" in content[0] and content[2] == 0:
name = None name = None
@@ -107,8 +156,9 @@ def messages(db, data):
try: try:
int(content[4]) int(content[4])
except ValueError: except ValueError:
msg = "{The group name changed to "f"{content[4]}"" }" msg = f"The group name changed to {content[4]}"
data[content[0]]["messages"][content[1]]["data"] = msg data[content[0]]["messages"][content[1]]["data"] = msg
data[content[0]]["messages"][content[1]]["meta"] = True
else: else:
del data[content[0]]["messages"][content[1]] del data[content[0]]["messages"][content[1]]
else: else:
@@ -127,15 +177,16 @@ def messages(db, data):
name_left = data[content[8]]["name"] name_left = data[content[8]]["name"]
else: else:
name_left = content[8].split('@')[0] name_left = content[8].split('@')[0]
msg = "{"f"{name_left}"f" added {name_right or 'You'}""}" msg = f"{name_left} added {name_right or 'You'}"
else: else:
msg = "{"f"Added {name_right or 'You'}""}" msg = f"Added {name_right or 'You'}"
elif b"\xac\xed\x00\x05\x74\x00" in thumb_image: elif b"\xac\xed\x00\x05\x74\x00" in thumb_image:
# Changed number # Changed number
original = content[8].split('@')[0] original = content[8].split('@')[0]
changed = thumb_image[7:].decode().split('@')[0] changed = thumb_image[7:].decode().split('@')[0]
msg = "{"f"{original} changed to {changed}""}" msg = f"{original} changed to {changed}"
data[content[0]]["messages"][content[1]]["data"] = msg data[content[0]]["messages"][content[1]]["data"] = msg
data[content[0]]["messages"][content[1]]["meta"] = True
else: else:
if content[4] is None: if content[4] is None:
del data[content[0]]["messages"][content[1]] del data[content[0]]["messages"][content[1]]
@@ -147,20 +198,34 @@ def messages(db, data):
else: else:
if content[2] == 1: if content[2] == 1:
if content[5] == 5 and content[6] == 7: if content[5] == 5 and content[6] == 7:
msg = "{Message deleted}" msg = "Message deleted"
data[content[0]]["messages"][content[1]]["meta"] = True
else: else:
if content[9] == "5": if content[9] == "5":
msg = "{ Location shared: "f"{content[10], content[11]}"" }" msg = f"Location shared: {content[10], content[11]}"
data[content[0]]["messages"][content[1]]["meta"] = True
else: else:
msg = content[4] msg = content[4]
if msg is not None:
if "\r\n" in msg:
msg = msg.replace("\r\n", "<br>")
if "\n" in msg:
msg = msg.replace("\n", "<br>")
else: else:
if content[5] == 0 and content[6] == 7: if content[5] == 0 and content[6] == 7:
msg = "{Message deleted}" msg = "Message deleted"
data[content[0]]["messages"][content[1]]["meta"] = True
else: else:
if content[9] == "5": if content[9] == "5":
msg = "{ Location shared: "f"{content[10], content[11]}"" }" msg = f"Location shared: {content[10], content[11]}"
data[content[0]]["messages"][content[1]]["meta"] = True
else: else:
msg = content[4] msg = content[4]
if msg is not None:
if "\r\n" in msg:
msg = msg.replace("\r\n", "<br>")
if "\n" in msg:
msg = msg.replace("\n", "<br>")
data[content[0]]["messages"][content[1]]["data"] = msg data[content[0]]["messages"][content[1]]["data"] = msg
@@ -168,8 +233,7 @@ def messages(db, data):
if i % 1000 == 0: if i % 1000 == 0:
print(f"Gathering messages...({i}/{total_row_number})", end="\r") print(f"Gathering messages...({i}/{total_row_number})", end="\r")
content = c.fetchone() content = c.fetchone()
print( print(f"Gathering messages...({total_row_number}/{total_row_number})", end="\r")
f"Gathering messages...({total_row_number}/{total_row_number})", end="\r")
def media(db, data, media_folder): def media(db, data, media_folder):
@@ -215,8 +279,9 @@ def media(db, data, media_folder):
# data[content[0]]["messages"][content[1]]["media"] = True # data[content[0]]["messages"][content[1]]["media"] = True
# data[content[0]]["messages"][content[1]]["mime"] = "media" # data[content[0]]["messages"][content[1]]["mime"] = "media"
# else: # else:
data[content[0]]["messages"][content[1]]["data"] = "{The media is missing}" data[content[0]]["messages"][content[1]]["data"] = "The media is missing"
data[content[0]]["messages"][content[1]]["mime"] = "media" data[content[0]]["messages"][content[1]]["mime"] = "media"
data[content[0]]["messages"][content[1]]["meta"] = True
i += 1 i += 1
if i % 100 == 0: if i % 100 == 0:
print(f"Gathering media...({i}/{total_row_number})", end="\r") print(f"Gathering media...({i}/{total_row_number})", end="\r")
@@ -239,27 +304,34 @@ def vcard(db, data):
total_row_number = len(rows) total_row_number = len(rows)
print(f"\nGathering vCards...(0/{total_row_number})", end="\r") print(f"\nGathering vCards...(0/{total_row_number})", end="\r")
base = "WhatsApp/vCards" base = "WhatsApp/vCards"
if not os.path.isdir(base):
Path(base).mkdir(parents=True, exist_ok=True)
for index, row in enumerate(rows): for index, row in enumerate(rows):
if not os.path.isdir(base):
os.mkdir(base)
file_name = "".join(x for x in row[3] if x.isalnum()) file_name = "".join(x for x in row[3] if x.isalnum())
file_path = f"{base}/{file_name}.vcf" file_path = f"{base}/{file_name}.vcf"
if not os.path.isfile(file_path): if not os.path.isfile(file_path):
with open(file_path, "w", encoding="utf-8") as f: with open(file_path, "w", encoding="utf-8") as f:
f.write(row[2]) f.write(row[2])
data[row[1]]["messages"][row[0]]["data"] = row[3] + \ data[row[1]]["messages"][row[0]]["data"] = row[3] + \
"{ The vCard file cannot be displayed here, however it " \ "The vCard file cannot be displayed here, " \
"should be located at " + file_path + "}" f"however it should be located at {file_path}"
data[row[1]]["messages"][row[0]]["mime"] = "text/x-vcard" data[row[1]]["messages"][row[0]]["mime"] = "text/x-vcard"
data[row[1]]["messages"][row[0]]["meta"] = True
print(f"Gathering vCards...({index + 1}/{total_row_number})", end="\r") print(f"Gathering vCards...({index + 1}/{total_row_number})", end="\r")
def create_html(data, output_folder): def create_html(data, output_folder, template=None):
templateLoader = jinja2.FileSystemLoader(searchpath=os.path.dirname(__file__)) if template is None:
template_dir = os.path.dirname(__file__)
template_file = "whatsapp.html"
else:
template_dir = os.path.dirname(template)
template_file = os.path.basename(template)
templateLoader = jinja2.FileSystemLoader(searchpath=template_dir)
templateEnv = jinja2.Environment(loader=templateLoader) templateEnv = jinja2.Environment(loader=templateLoader)
templateEnv.globals.update(determine_day=determine_day) templateEnv.globals.update(determine_day=determine_day)
TEMPLATE_FILE = "whatsapp.html" templateEnv.filters['sanitize_except'] = sanitize_except
template = templateEnv.get_template(TEMPLATE_FILE) template = templateEnv.get_template(template_file)
total_row_number = len(data) total_row_number = len(data)
print(f"\nCreating HTML...(0/{total_row_number})", end="\r") print(f"\nCreating HTML...(0/{total_row_number})", end="\r")

View File

@@ -7,12 +7,19 @@ import os
import requests import requests
import shutil import shutil
import pkgutil import pkgutil
from pathlib import Path
from bleach import clean as sanitize
from markupsafe import Markup
from datetime import datetime from datetime import datetime
from mimetypes import MimeTypes from mimetypes import MimeTypes
APPLE_TIME = datetime.timestamp(datetime(2001, 1, 1)) APPLE_TIME = datetime.timestamp(datetime(2001, 1, 1))
def sanitize_except(html):
return Markup(sanitize(html, tags=["br"]))
def determine_day(last, current): def determine_day(last, current):
last = datetime.fromtimestamp(last).date() last = datetime.fromtimestamp(last).date()
current = datetime.fromtimestamp(current).date() current = datetime.fromtimestamp(current).date()
@@ -62,7 +69,9 @@ def messages(db, data):
"time": datetime.fromtimestamp(ts).strftime("%H:%M"), "time": datetime.fromtimestamp(ts).strftime("%H:%M"),
"media": False, "media": False,
"reply": None, "reply": None,
"caption": None "caption": None,
"meta": False,
"data": None
} }
if "-" in content[0] and content[2] == 0: if "-" in content[0] and content[2] == 0:
name = None name = None
@@ -87,8 +96,9 @@ def messages(db, data):
try: try:
int(content[4]) int(content[4])
except ValueError: except ValueError:
msg = "{The group name changed to "f"{content[4]}"" }" msg = f"The group name changed to {content[4]}"
data[content[0]]["messages"][content[1]]["data"] = msg data[content[0]]["messages"][content[1]]["data"] = msg
data[content[0]]["messages"][content[1]]["meta"] = True
else: else:
del data[content[0]]["messages"][content[1]] del data[content[0]]["messages"][content[1]]
else: else:
@@ -99,14 +109,26 @@ def messages(db, data):
# real message # real message
if content[2] == 1: if content[2] == 1:
if content[5] == 14: if content[5] == 14:
msg = "{Message deleted}" msg = "Message deleted"
data[content[0]]["messages"][content[1]]["meta"] = True
else: else:
msg = content[4] msg = content[4]
if msg is not None:
if "\r\n" in msg:
msg = msg.replace("\r\n", "<br>")
if "\n" in msg:
msg = msg.replace("\n", "<br>")
else: else:
if content[5] == 14: if content[5] == 14:
msg = "{Message deleted}" msg = "Message deleted"
data[content[0]]["messages"][content[1]]["meta"] = True
else: else:
msg = content[4] msg = content[4]
if msg is not None:
if "\r\n" in msg:
msg = msg.replace("\r\n", "<br>")
if "\n" in msg:
msg = msg.replace("\n", "<br>")
data[content[0]]["messages"][content[1]]["data"] = msg data[content[0]]["messages"][content[1]]["data"] = msg
i += 1 i += 1
if i % 1000 == 0: if i % 1000 == 0:
@@ -138,7 +160,7 @@ def media(db, data, media_folder):
content = c.fetchone() content = c.fetchone()
mime = MimeTypes() mime = MimeTypes()
while content is not None: while content is not None:
file_path = f"Message/{content[2]}" file_path = f"{media_folder}/{content[2]}"
data[content[0]]["messages"][content[1]]["media"] = True data[content[0]]["messages"][content[1]]["media"] = True
if os.path.isfile(file_path): if os.path.isfile(file_path):
@@ -161,8 +183,9 @@ def media(db, data, media_folder):
# data[content[0]]["messages"][content[1]]["data"] = "{The media is missing}" # data[content[0]]["messages"][content[1]]["data"] = "{The media is missing}"
# data[content[0]]["messages"][content[1]]["mime"] = "media" # data[content[0]]["messages"][content[1]]["mime"] = "media"
# else: # else:
data[content[0]]["messages"][content[1]]["data"] = "{The media is missing}" data[content[0]]["messages"][content[1]]["data"] = "The media is missing"
data[content[0]]["messages"][content[1]]["mime"] = "media" data[content[0]]["messages"][content[1]]["mime"] = "media"
data[content[0]]["messages"][content[1]]["meta"] = True
if content[6] is not None: if content[6] is not None:
data[content[0]]["messages"][content[1]]["caption"] = content[6] data[content[0]]["messages"][content[1]]["caption"] = content[6]
i += 1 i += 1
@@ -190,28 +213,35 @@ def vcard(db, data):
total_row_number = len(rows) total_row_number = len(rows)
print(f"\nGathering vCards...(0/{total_row_number})", end="\r") print(f"\nGathering vCards...(0/{total_row_number})", end="\r")
base = "Message/vCards" base = "Message/vCards"
if not os.path.isdir(base):
Path(base).mkdir(parents=True, exist_ok=True)
for index, row in enumerate(rows): for index, row in enumerate(rows):
if not os.path.isdir(base):
os.mkdir(base)
file_name = "".join(x for x in row[3] if x.isalnum()) file_name = "".join(x for x in row[3] if x.isalnum())
file_path = f"{base}/{file_name[:200]}.vcf" file_path = f"{base}/{file_name[:200]}.vcf"
if not os.path.isfile(file_path): if not os.path.isfile(file_path):
with open(file_path, "w", encoding="utf-8") as f: with open(file_path, "w", encoding="utf-8") as f:
f.write(row[4]) f.write(row[4])
data[row[2]]["messages"][row[1]]["data"] = row[3] + \ data[row[2]]["messages"][row[1]]["data"] = row[3] + \
"{ The vCard file cannot be displayed here, however it " \ "The vCard file cannot be displayed here, " \
"should be located at " + file_path + "}" f"however it should be located at {file_path}"
data[row[2]]["messages"][row[1]]["mime"] = "text/x-vcard" data[row[2]]["messages"][row[1]]["mime"] = "text/x-vcard"
data[row[2]]["messages"][row[1]]["media"] = True data[row[2]]["messages"][row[1]]["media"] = True
data[row[2]]["messages"][row[1]]["meta"] = True
print(f"Gathering vCards...({index + 1}/{total_row_number})", end="\r") print(f"Gathering vCards...({index + 1}/{total_row_number})", end="\r")
def create_html(data, output_folder): def create_html(data, output_folder, template=None):
templateLoader = jinja2.FileSystemLoader(searchpath=os.path.dirname(__file__)) if template is None:
template_dir = os.path.dirname(__file__)
template_file = "whatsapp.html"
else:
template_dir = os.path.dirname(template)
template_file = os.path.basename(template)
templateLoader = jinja2.FileSystemLoader(searchpath=template_dir)
templateEnv = jinja2.Environment(loader=templateLoader) templateEnv = jinja2.Environment(loader=templateLoader)
templateEnv.globals.update(determine_day=determine_day) templateEnv.globals.update(determine_day=determine_day)
TEMPLATE_FILE = "whatsapp.html" templateEnv.filters['sanitize_except'] = sanitize_except
template = templateEnv.get_template(TEMPLATE_FILE) template = templateEnv.get_template(template_file)
total_row_number = len(data) total_row_number = len(data)
print(f"\nCreating HTML...(0/{total_row_number})", end="\r") print(f"\nCreating HTML...(0/{total_row_number})", end="\r")

View File

@@ -6,21 +6,24 @@ import os
import getpass import getpass
try: try:
from iphone_backup_decrypt import EncryptedBackup, RelativePath from iphone_backup_decrypt import EncryptedBackup, RelativePath
except: except ModuleNotFoundError:
support_encrypted = False support_encrypted = False
else: else:
support_encrypted = True support_encrypted = True
def extract_encrypted(base_dir, password): def extract_encrypted(base_dir, password):
backup = EncryptedBackup(backup_directory=base_dir, passphrase=password) backup = EncryptedBackup(backup_directory=base_dir, passphrase=password)
print("Decrypting WhatsApp database...") print("Decrypting WhatsApp database...")
backup.extract_file(relative_path=RelativePath.WHATSAPP_MESSAGES, output_filename="7c7fba66680ef796b916b067077cc246adacf01d") backup.extract_file(relative_path=RelativePath.WHATSAPP_MESSAGES,
backup.extract_file(relative_path=RelativePath.WHATSAPP_CONTACTS, output_filename="ContactsV2.sqlite") output_filename="7c7fba66680ef796b916b067077cc246adacf01d")
backup.extract_file(relative_path=RelativePath.WHATSAPP_CONTACTS,
output_filename="ContactsV2.sqlite")
data = backup.execute_sql("""SELECT count() data = backup.execute_sql("""SELECT count()
FROM Files FROM Files
WHERE relativePath WHERE relativePath
LIKE 'Message/Media/%'""" LIKE 'Message/Media/%'"""
) )
total_row_number = data[0][0] total_row_number = data[0][0]
print(f"Gathering media...(0/{total_row_number})", end="\r") print(f"Gathering media...(0/{total_row_number})", end="\r")
data = backup.execute_sql("""SELECT fileID, data = backup.execute_sql("""SELECT fileID,
@@ -30,7 +33,7 @@ def extract_encrypted(base_dir, password):
FROM Files FROM Files
WHERE relativePath WHERE relativePath
LIKE 'Message/Media/%'""" LIKE 'Message/Media/%'"""
) )
if not os.path.isdir("Message"): if not os.path.isdir("Message"):
os.mkdir("Message") os.mkdir("Message")
if not os.path.isdir("Message/Media"): if not os.path.isdir("Message/Media"):
@@ -56,6 +59,7 @@ def extract_encrypted(base_dir, password):
print(f"Gathering media...({i}/{total_row_number})", end="\r") print(f"Gathering media...({i}/{total_row_number})", end="\r")
print(f"Gathering media...({total_row_number}/{total_row_number})", end="\r") print(f"Gathering media...({total_row_number}/{total_row_number})", end="\r")
def is_encrypted(base_dir): def is_encrypted(base_dir):
with sqlite3.connect(f"{base_dir}/Manifest.db") as f: with sqlite3.connect(f"{base_dir}/Manifest.db") as f:
c = f.cursor() c = f.cursor()
@@ -68,6 +72,7 @@ def is_encrypted(base_dir):
else: else:
return False return False
def extract_media(base_dir): def extract_media(base_dir):
if is_encrypted(base_dir): if is_encrypted(base_dir):
if not support_encrypted: if not support_encrypted:
@@ -81,7 +86,7 @@ def extract_media(base_dir):
wts_db = os.path.join(base_dir, "7c/7c7fba66680ef796b916b067077cc246adacf01d") wts_db = os.path.join(base_dir, "7c/7c7fba66680ef796b916b067077cc246adacf01d")
if not os.path.isfile(wts_db): if not os.path.isfile(wts_db):
print("WhatsApp database not found.") print("WhatsApp database not found.")
sys.exit(1) exit()
else: else:
shutil.copyfile(wts_db, "7c7fba66680ef796b916b067077cc246adacf01d") shutil.copyfile(wts_db, "7c7fba66680ef796b916b067077cc246adacf01d")
with sqlite3.connect(f"{base_dir}/Manifest.db") as manifest: with sqlite3.connect(f"{base_dir}/Manifest.db") as manifest:

View File

@@ -72,29 +72,37 @@
<a href="#{{msg.reply}}" style="color: #168acc;">"{{ msg.quoted_data or 'media' }}"</a> <a href="#{{msg.reply}}" style="color: #168acc;">"{{ msg.quoted_data or 'media' }}"</a>
</div> </div>
{% endif %} {% endif %}
{% if msg.media == false %} {% if msg.meta == true or msg.media == false and msg.data is none %}
{% filter escape %}{{ msg.data or "{This message is not supported yet}" | replace('\n', '<br>') }}{% endfilter %} <div style="text-align: center;" class="w3-panel w3-border-blue w3-pale-blue w3-rightbar w3-leftbar">
<p>{{ msg.data or 'This message is not supported' }}</p>
</div>
{% else %} {% else %}
{% if "image/" in msg.mime %} {% if msg.media == false %}
<a href="{{ msg.data }}"><img src="{{ msg.data }}" /></a> {{ msg.data | sanitize_except() }}
{% elif "audio/" in msg.mime %}
<audio controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
</audio>
{% elif "video/" in msg.mime %}
<video controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
</video>
{% elif "/" in msg.mime %}
{The file cannot be displayed here, however it should be located at {{ msg.data }}}
{% else %} {% else %}
{% filter escape %}{{ msg.data }}{% endfilter %} {% if "image/" in msg.mime %}
<a href="{{ msg.data }}"><img src="{{ msg.data }}" /></a>
{% elif "audio/" in msg.mime %}
<audio controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
</audio>
{% elif "video/" in msg.mime %}
<video controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
</video>
{% elif "/" in msg.mime %}
<div style="text-align: center;" class="w3-panel w3-border-blue w3-pale-blue w3-rightbar w3-leftbar">
<p>The file cannot be displayed here, however it should be located at {{ msg.data }}</p>
</div>
{% else %}
{% filter escape %}{{ msg.data }}{% endfilter %}
{% endif %}
{% if msg.caption is not none %}
<br>
{{ msg.caption }}
{% endif %}
{% endif %} {% endif %}
{% if msg.caption is not none %}
<br>
{{ msg.caption }}
{% endif %} {% endif %}
{% endif %}
</div> </div>
</div> </div>
<div class="w3-col m2 l2" style="padding-left: 10px"><img src="{{ my_avatar }}" onerror="this.style.display='none'"></div> <div class="w3-col m2 l2" style="padding-left: 10px"><img src="{{ my_avatar }}" onerror="this.style.display='none'"></div>
@@ -120,27 +128,35 @@
<a href="#{{msg.reply}}" style="color: #168acc;">"{{ msg.quoted_data or 'media' }}"</a> <a href="#{{msg.reply}}" style="color: #168acc;">"{{ msg.quoted_data or 'media' }}"</a>
</div> </div>
{% endif %} {% endif %}
{% if msg.media == false %} {% if msg.meta == true or msg.media == false and msg.data is none %}
{% filter escape %}{{ msg.data or "{This message is not supported yet}" }}{% endfilter %} <div style="text-align: center;" class="w3-panel w3-border-blue w3-pale-blue w3-rightbar w3-leftbar">
<p>{{ msg.data or 'This message is not supported' }}</p>
</div>
{% else %} {% else %}
{% if "image/" in msg.mime %} {% if msg.media == false %}
<a href="{{ msg.data }}"><img src="{{ msg.data }}" /></a> {{ msg.data | sanitize_except() }}
{% elif "audio/" in msg.mime %}
<audio controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
</audio>
{% elif "video/" in msg.mime %}
<video controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
</video>
{% elif "/" in msg.mime %}
{The file cannot be displayed here, however it should be located at {{ msg.data }}}
{% else %} {% else %}
{% filter escape %}{{ msg.data }}{% endfilter %} {% if "image/" in msg.mime %}
{% endif %} <a href="{{ msg.data }}"><img src="{{ msg.data }}" /></a>
{% if msg.caption is not none %} {% elif "audio/" in msg.mime %}
<br> <audio controls="controls" autobuffer="autobuffer">
{{ msg.caption }} <source src="{{ msg.data }}" />
</audio>
{% elif "video/" in msg.mime %}
<video controls="controls" autobuffer="autobuffer">
<source src="{{ msg.data }}" />
</video>
{% elif "/" in msg.mime %}
<div style="text-align: center;" class="w3-panel w3-border-blue w3-pale-blue w3-rightbar w3-leftbar">
<p>The file cannot be displayed here, however it should be located at {{ msg.data }}</p>
</div>
{% else %}
{% filter escape %}{{ msg.data }}{% endfilter %}
{% endif %}
{% if msg.caption is not none %}
<br>
{{ msg.caption }}
{% endif %}
{% endif %} {% endif %}
{% endif %} {% endif %}
</div> </div>

1
_config.yml Normal file
View File

@@ -0,0 +1 @@
theme: jekyll-theme-cayman

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

View File

@@ -12,12 +12,13 @@ setuptools.setup(
version=version, version=version,
author="KnugiHK", author="KnugiHK",
author_email="info@knugi.com", author_email="info@knugi.com",
description="A Whatsapp database parser that will give you the history of your Whatsapp conversations in HTML and JSON.", description="A Whatsapp database parser that will give you the "
"history of your Whatsapp conversations in HTML and JSON.",
long_description=long_description, long_description=long_description,
long_description_content_type="text/markdown", long_description_content_type="text/markdown",
url="https://github.com/KnugiHK/Whatsapp-Chat-Exporter", url="https://github.com/KnugiHK/Whatsapp-Chat-Exporter",
packages=setuptools.find_packages(), packages=setuptools.find_packages(),
package_data = { package_data={
'': ['whatsapp.html'] '': ['whatsapp.html']
}, },
classifiers=[ classifiers=[
@@ -36,8 +37,12 @@ setuptools.setup(
], ],
python_requires='>=3.7', python_requires='>=3.7',
install_requires=[ install_requires=[
'jinja2' 'jinja2',
'bleach'
], ],
extras_require={
'android_backup': ["pycryptodome"]
},
entry_points={ entry_points={
"console_scripts": [ "console_scripts": [
"wtsexporter = Whatsapp_Chat_Exporter.__main__:main" "wtsexporter = Whatsapp_Chat_Exporter.__main__:main"