Remove a to-do task

README.md

@@ -1,45 +1,85 @@
 # Whatsapp-Chat-Exporter
-A Whatsapp database parser that will give you the history of your Whatsapp conversation in HTML and JSON
+A Whatsapp database parser that will give you the history of your Whatsapp conversations in HTML and JSON  
+**If you plan to uninstall WhatsApp or delete your WhatsApp account, please make a backup of your WhatsApp database. You may want to use this exporter again on the same database in the future as the exporter develops**
 
 # Usage
-First, clone this repo, and copy all py and html files to a working directory if you want to do so.
+**If you want to use the old release of the exporter, please follow the [old usage guide](old_README.md#usage)**
+
+First, clone this repo and install the exporter.
 ```shell
 git clone https://github.com/KnugiHK/Whatsapp-Chat-Exporter.git
+cd Whatsapp-Chat-Exporter
+python setup.py install
 ```
-Then, ready your WhatsApp database, place them in the root of working directory.
-* For Android, it is called msgstore.db. If you want name of your contacts, get the contact database, which is called wa.db.
-* For iPhone, it is called 7c7fba66680ef796b916b067077cc246adacf01d (YES, a hash).
+Then, create a working directory in somewhere you want
+```shell
+mkdir working_wts
+cd working_wts
+```
+## Working with Android
+Extract the WhatsApp database with whatever means, one possible means is to use the [WhatsApp-Key-DB-Extractor](https://github.com/KnugiHK/WhatsApp-Key-DB-Extractor)
 
-Next, ready your media folder, place it in the root of working directory.
-* For Android, copy the WhatsApp directory from your phone directly.
-* For iPhone, run the extract_iphone_media.py, and you will get a folder called Message. Please note that, this script does not support encrypted backup.
-```
-python extract_iphone_media.py "C:\Users\[Username]\AppData\Roaming\Apple Computer\MobileSync\Backup\[device id]"
-```
-And now, you should have something like this:
+After you obtain your WhatsApp databse, copy the WatsApp database and media folder to the working directory. The database is called msgstore.db. If you also want the name of your contacts, get the contact database, which is called wa.db. And copy the WhatsApp (Media) directory from your phone directly.
 
-![Folder structure](structure.png)
+And now, you should have something like this in the working directory.
 
-Last, run the script regarding the type of phone.
+![Android folder structure](imgs/android_structure.png)
+### Extracting
+Simply invoke the following command from shell, remember to replace the username and device id correspondingly in the command.
+```sh
+wtsexporter -a
 ```
-python extract.py & :: Android
-python extract_iphone.py & :: iPhone
+
+## Working with iPhone
+Do an iPhone Backup with iTunes first.
+### Encrypted iPhone Backup
+**If you are working on unencrypted iPhone backup, skip this**
+
+If you want to work on an encrypted iPhone Backup, you should install iphone_backup_decrypt from [KnugiHK/iphone_backup_decrypt](https://github.com/KnugiHK/iphone_backup_decrypt) before you run the extract_iphone_media.py.
+```sh
+pip install biplist pycryptodome & :: Optional, since the pip will install these dependencies automatically.
+pip install git+https://github.com/KnugiHK/iphone_backup_decrypt
 ```
-And you will get these:
+### Extracting
+Simply invoke the following command from shell, remember to replace the username and device id correspondingly in the command.
+```sh
+wtsexporter -i -b "C:\Users\[Username]\AppData\Roaming\Apple Computer\MobileSync\Backup\[device id]"
+```
+
+## Results
+After extracting, you will get these:
 #### Private Message
-![Private Message](pm.png)
+![Private Message](imgs/pm.png)
 
 #### Group Message
-![Group Message](group.png)
+![Group Message](imgs/group.png)
 
-# Encrypted iPhone Backup
-To do
+## More options
+Invoke the wtsexporter with --help option will show you all options available.
+```sh
+> wtsexporter --help
+Usage: wtsexporter [options]
+
+Options:
+  --version             show program's version number and exit
+  -h, --help            show this help message and exit
+  -a, --android         Define the target as Android
+  -i, --iphone          Define the target as iPhone
+  -w WA, --wa=WA        Path to contact database
+  -m MEDIA, --media=MEDIA
+                        Path to WhatsApp media folder
+  -b BACKUP, --backup=BACKUP
+                        Path to iPhone backup
+  -o OUTPUT, --output=OUTPUT
+                        Output to specific directory
+  -j, --json            Save the result to a single JSON file
+  -d DB, --db=DB        Path to database file
+```
 
 # To do
 1. Convert ```\r\n``` to ```<br>```
 2. Reply in iPhone
 3. The CSS for metadata (e.g. {Message Deleted})
-4. Handle encrypted iPhone Backup
 
 # Copyright
 This is a MIT licensed project.

Whatsapp_Chat_Exporter/__init__.py

@@ -0,0 +1 @@
+__version__ = "0.5"

Whatsapp_Chat_Exporter/__main__.py

@@ -0,0 +1,136 @@
+from .__init__ import __version__
+from Whatsapp_Chat_Exporter import extract, extract_iphone, extract_iphone_media
+from optparse import OptionParser
+import os
+import sqlite3
+import shutil
+import json
+
+
+def main():
+    parser = OptionParser(version=__version__)
+    parser.add_option(
+        '-a',
+        '--android',
+        dest='android',
+        default=False,
+        action='store_true',
+        help="Define the target as Android")
+    parser.add_option(
+        '-i',
+        '--iphone',
+        dest='iphone',
+        default=False,
+        action='store_true',
+        help="Define the target as iPhone")
+    parser.add_option(
+        "-w",
+        "--wa",
+        dest="wa",
+        default=None,
+        help="Path to contact database")
+    parser.add_option(
+        "-m",
+        "--media",
+        dest="media",
+        default=None,
+        help="Path to WhatsApp media folder")
+    parser.add_option(
+        "-b",
+        "--backup",
+        dest="backup",
+        default=None,
+        help="Path to iPhone backup")
+    parser.add_option(
+        "-o",
+        "--output",
+        dest="output",
+        default="result",
+        help="Output to specific directory")
+    parser.add_option(
+        '-j',
+        '--json',
+        dest='json',
+        default=False,
+        action='store_true',
+        help="Save the result to a single JSON file")
+    parser.add_option(
+        '-d',
+        '--db',
+        dest='db',
+        default=None,
+        help="Path to database file")
+    (options, args) = parser.parse_args()
+
+    if options.android and options.iphone:
+        print("You must define only one device type.")
+        exit()
+    if not options.android and not options.iphone:
+        print("You must define the device type.")
+        exit()
+    data = {}
+
+    if options.android:
+        contacts = extract.contacts
+        messages = extract.messages
+        media = extract.media
+        vcard = extract.vcard
+        create_html = extract.create_html
+        if options.db is None:
+            msg_db = "msgstore.db"
+        else:
+            msg_db = options.db
+        if options.wa is None:
+            contact_db = "wa.db"
+        else:
+            contact_db = options.wa
+        if options.media is None:
+            options.media = "WhatsApp"
+        
+        if len(args) == 1:
+            msg_db = args[0]
+
+        if os.path.isfile(contact_db):
+            with sqlite3.connect(contact_db) as db:
+                contacts(db, data)
+        
+    elif options.iphone:
+        messages = extract_iphone.messages
+        media = extract_iphone.media
+        vcard = extract_iphone.vcard
+        create_html = extract_iphone.create_html
+        if options.backup is not None:
+            extract_iphone_media.extract_media(options.backup)
+        if options.db is None:
+            msg_db = "7c7fba66680ef796b916b067077cc246adacf01d"
+        else:
+            msg_db = options.db
+        if options.wa is None:
+            contact_db = "ContactsV2.sqlite"
+        else:
+            contact_db = options.wa
+        if options.media is None:
+            options.media = "Message"
+
+        if len(args) == 1:
+            msg_db = args[0]
+
+    if os.path.isfile(msg_db):
+        with sqlite3.connect(msg_db) as db:
+            messages(db, data)
+            media(db, data, options.media)
+            vcard(db, data)
+        create_html(data, options.output)
+
+    if not os.path.isdir(f"{options.output}/{options.media}"):
+        shutil.move(options.media, f"{options.output}/")
+
+    if options.json:
+        with open("result.json", "w") as f:
+            data = json.dumps(data)
+            print(f"\nWriting JSON file...({int(len(data)/1024/1024)}MB)")
+            f.write(data)
+    else:
+        print()
+
+    print("Everything is done!")

Whatsapp_Chat_Exporter/extract.py

@@ -7,6 +7,7 @@ import os
 import requests
 import shutil
 import re
+import pkgutil
 from datetime import datetime
 from mimetypes import MimeTypes
 
@@ -254,7 +255,7 @@ def vcard(db, data):
 
 
 def create_html(data, output_folder):
-    templateLoader = jinja2.FileSystemLoader(searchpath="./")
+    templateLoader = jinja2.FileSystemLoader(searchpath=os.path.dirname(__file__))
     templateEnv = jinja2.Environment(loader=templateLoader)
     templateEnv.globals.update(determine_day=determine_day)
     TEMPLATE_FILE = "whatsapp.html"

Whatsapp_Chat_Exporter/extract_iphone.py

@@ -6,6 +6,7 @@ import jinja2
 import os
 import requests
 import shutil
+import pkgutil
 from datetime import datetime
 from mimetypes import MimeTypes
 
@@ -206,12 +207,12 @@ def vcard(db, data):
 
 
 def create_html(data, output_folder):
-    templateLoader = jinja2.FileSystemLoader(searchpath="./")
+    templateLoader = jinja2.FileSystemLoader(searchpath=os.path.dirname(__file__))
     templateEnv = jinja2.Environment(loader=templateLoader)
     templateEnv.globals.update(determine_day=determine_day)
     TEMPLATE_FILE = "whatsapp.html"
     template = templateEnv.get_template(TEMPLATE_FILE)
-
+    
     total_row_number = len(data)
     print(f"\nCreating HTML...(0/{total_row_number})", end="\r")
 

Whatsapp_Chat_Exporter/extract_iphone_media.py

@@ -0,0 +1,128 @@
+#!/usr/bin/python3
+
+import shutil
+import sqlite3
+import os
+import getpass
+try:
+    from iphone_backup_decrypt import EncryptedBackup, RelativePath
+except:
+    support_encrypted = False
+else:
+    support_encrypted = True
+
+def extract_encrypted(base_dir, password):
+    backup = EncryptedBackup(backup_directory=base_dir, passphrase=password)
+    print("Decrypting WhatsApp database...")
+    backup.extract_file(relative_path=RelativePath.WHATSAPP_MESSAGES, output_filename="7c7fba66680ef796b916b067077cc246adacf01d")
+    backup.extract_file(relative_path=RelativePath.WHATSAPP_CONTACTS, output_filename="ContactsV2.sqlite")
+    data = backup.execute_sql("""SELECT count()
+                                FROM Files
+                                WHERE relativePath
+                                    LIKE 'Message/Media/%'"""
+    )
+    total_row_number = data[0][0]
+    print(f"Gathering media...(0/{total_row_number})", end="\r")
+    data = backup.execute_sql("""SELECT fileID,
+                                        relativePath,
+                                        flags,
+                                        file
+                                FROM Files
+                                WHERE relativePath
+                                    LIKE 'Message/Media/%'"""
+    )
+    if not os.path.isdir("Message"):
+        os.mkdir("Message")
+    if not os.path.isdir("Message/Media"):
+        os.mkdir("Message/Media")
+    i = 0
+    for row in data:
+        destination = row[1]
+        hashes = row[0]
+        folder = hashes[:2]
+        flags = row[2]
+        file = row[3]
+        if flags == 2:
+            try: 
+                os.mkdir(destination)
+            except FileExistsError:
+                pass
+        elif flags == 1:
+            decrypted = backup.decrypt_inner_file(file_id=hashes, file_bplist=file)
+            with open(destination, "wb") as f:
+                f.write(decrypted)
+        i += 1
+        if i % 100 == 0:
+            print(f"Gathering media...({i}/{total_row_number})", end="\r")
+    print(f"Gathering media...({total_row_number}/{total_row_number})", end="\r")
+
+def is_encrypted(base_dir):
+    with sqlite3.connect(f"{base_dir}/Manifest.db") as f:
+        c = f.cursor()
+        try:
+            c.execute("""SELECT count()
+                    FROM Files
+                    """)
+        except sqlite3.DatabaseError:
+            return True
+        else:
+            return False
+
+def extract_media(base_dir):
+    if is_encrypted(base_dir):
+        if not support_encrypted:
+            print("You don't have the dependencies to handle encrypted backup.")
+            print("Read more about how to deal with encrypted backup:")
+            print("https://github.com/KnugiHK/Whatsapp-Chat-Exporter/blob/main/README.md#usage")
+            return False
+        password = getpass.getpass("Enter the password:")
+        extract_encrypted(base_dir, password)
+    else:
+        wts_db = os.path.join(base_dir, "7c/7c7fba66680ef796b916b067077cc246adacf01d")
+        if not os.path.isfile(wts_db):
+            print("WhatsApp database not found.")
+            sys.exit(1)
+        else:
+            shutil.copyfile(wts_db, "7c7fba66680ef796b916b067077cc246adacf01d")
+        with sqlite3.connect(f"{base_dir}/Manifest.db") as manifest:
+            c = manifest.cursor()
+            c.execute("""SELECT count()
+                        FROM Files
+                        WHERE relativePath
+                            LIKE 'Message/Media/%'""")
+            total_row_number = c.fetchone()[0]
+            print(f"Gathering media...(0/{total_row_number})", end="\r")
+            c.execute("""SELECT fileID,
+                                relativePath,
+                                flags
+                        FROM Files
+                        WHERE relativePath
+                            LIKE 'Message/Media/%'""")
+            row = c.fetchone()
+            if not os.path.isdir("Message"):
+                os.mkdir("Message")
+            if not os.path.isdir("Message/Media"):
+                os.mkdir("Message/Media")
+            i = 0
+            while row is not None:
+                destination = row[1]
+                hashes = row[0]
+                folder = hashes[:2]
+                flags = row[2]
+                if flags == 2:
+                    os.mkdir(destination)
+                elif flags == 1:
+                    shutil.copyfile(f"{base_dir}/{folder}/{hashes}", destination)
+                i += 1
+                if i % 100 == 0:
+                    print(f"Gathering media...({i}/{total_row_number})", end="\r")
+                row = c.fetchone()
+            print(f"Gathering media...({total_row_number}/{total_row_number})", end="\r")
+
+
+if __name__ == "__main__":
+    from optparse import OptionParser
+    parser = OptionParser()
+    (_, args) = parser.parse_args()
+    base_dir = args[0]
+    extract_media(base_dir)

Whatsapp_Chat_Exporter/whatsapp.html

@@ -1,158 +1,158 @@
-<!DOCTYPE html>
-<html>
-	<head>
-		<title>Whatsapp - {{ name }}</title> 
-		<link rel="stylesheet" href="https://www.w3schools.com/w3css/4/w3.css"> 
-		<style>
-		@import url('https://fonts.googleapis.com/css2?family=Noto+Sans+HK:wght@300;400&display=swap');
-		html {
-			font-family: 'Noto Sans HK', sans-serif;
-			font-size: 12px;
-			scroll-behavior: smooth;
-		}
-		header {
-			position: fixed;
-			z-index: 20;
-			border-bottom: 2px solid #e3e6e7;
-			font-size: 2em;
-			font-weight: bolder;
-			background-color: white;
-			padding: 20px 0 20px 0;
-		}
-		footer {
-			border-top: 2px solid #e3e6e7;
-			font-size: 2em;
-			padding: 20px 0 20px 0;
-		}
-		article {
-			width:500px;
-			margin:100px auto;
-			z-index:10;
-			font-size: 15px;
-			word-wrap: break-word;
-		}
-		img, video {
-			max-width:100%;
-		}
-		a.anchor {
-			display: block;
-			position: relative;
-			top: -100px;
-			visibility: hidden;
-		}
-		div.reply{
-			font-size: 13px;
-			text-decoration: none;
-		}
-		</style>
-	</head>
-	<body>
-		<header class="w3-center w3-top">Chat history with {{ name }}</header>
-		<article class="w3-container">
-			<div class="table" style="width:100%">
-			{% set last = {'last': 946688461.001} %}
-			{% for msg in msgs -%}
-				<div class="w3-row" style="padding-bottom: 10px">
-					<a class="anchor" id="{{ msg.key_id }}"></a>
-					{% if determine_day(last.last, msg.timestamp) is not none  %}
-					<div class="w3-center" style="color:#70777c;padding: 10px 0 10px 0;">{{ determine_day(last.last, msg.timestamp) }}</div>
-					{% if last.update({'last': msg.timestamp}) %}{% endif %}
-					{% endif %}
-					{% if msg.from_me == true %}
-					<div class="w3-row">
-						<div style="float: left; color:#70777c;">{{ msg.time }}</div>
-						<div style="padding-left: 10px; text-align: right; color: #3892da;">You</div>
-					</div>
-					<div class="w3-row">
-						<div class="w3-col m10 l10">
-							<div style="text-align: right;">
-							{% if msg.reply is not none %}
-								<div class="reply">
-									<span style="color: #70777a;">Replying to </span>
-									<a href="#{{msg.reply}}" style="color: #168acc;">"{{ msg.quoted_data or 'media' }}"</a>
-								</div>
-							{% endif %}
-							{% if msg.media == false %}
-								{% filter escape %}{{ msg.data or "{This message is not supported yet}" | replace('\n', '<br>') }}{% endfilter %}
-							{% else %}
-								{% if "image/" in msg.mime %}
-								<a href="{{ msg.data }}"><img src="{{ msg.data }}" /></a>
-								{% elif "audio/" in msg.mime %}
-								<audio controls="controls" autobuffer="autobuffer">
-									<source src="{{ msg.data }}" />
-								</audio>
-								{% elif "video/" in msg.mime %}
-								<video controls="controls" autobuffer="autobuffer">
-									<source src="{{ msg.data }}" />
-								</video>
-								{% elif "/" in msg.mime %}
-									{The file cannot be displayed here, however it should be located at {{ msg.data }}}
-								{% else %}
-									{% filter escape %}{{ msg.data }}{% endfilter %}
-								{% endif %}
-								{% if msg.caption is not none %}
-								<br>
-								{{ msg.caption }}
-								{% endif %}
-							{% endif %}
-							</div>
-						</div>
-						<div class="w3-col m2 l2" style="padding-left: 10px"><img src="{{ my_avatar }}" onerror="this.style.display='none'"></div>
-					</div>
-					{% else %}
-					<div class="w3-row">
-						<div style="padding-right: 10px; float: left; color: #3892da;">
-							{% if msg.sender is not none %}
-							{{ msg.sender }}
-							{% else %}
-							{{ name }}
-							{% endif %}
-						</div>
-						<div style="text-align: right; color:#70777c;">{{ msg.time }}</div>
-					</div>
-					<div class="w3-row">
-						<div class="w3-col m2 l2"><img src="{{ their_avatar }}" onerror="this.style.display='none'"></div>
-						<div class="w3-col m10 l10">
-							<div style="text-align: left;">
-							{% if msg.reply is not none %}
-								<div class="reply">
-									<span style="color: #70777a;">Replying to </span>
-									<a href="#{{msg.reply}}" style="color: #168acc;">"{{ msg.quoted_data or 'media' }}"</a>
-								</div>
-							{% endif %}
-							{% if msg.media == false %}
-								{% filter escape %}{{ msg.data or "{This message is not supported yet}" }}{% endfilter %}
-							{% else %}
-								{% if "image/" in msg.mime %}
-								<a href="{{ msg.data }}"><img src="{{ msg.data }}" /></a>
-								{% elif "audio/" in msg.mime %}
-								<audio controls="controls" autobuffer="autobuffer">
-									<source src="{{ msg.data }}" />
-								</audio>
-								{% elif "video/" in msg.mime %}
-								<video controls="controls" autobuffer="autobuffer">
-									<source src="{{ msg.data }}" />
-								</video>
-								{% elif "/" in msg.mime %}
-									{The file cannot be displayed here, however it should be located at {{ msg.data }}}
-								{% else %}
-									{% filter escape %}{{ msg.data }}{% endfilter %}
-								{% endif %}
-								{% if msg.caption is not none %}
-								<br>
-								{{ msg.caption }}
-								{% endif %}
-							{% endif %}
-							</div>
-						</div>
-					</div>
-					{% endif %}
-				</div>
-			{% endfor %}
-			</div>
-		</article>
-		<footer class="w3-center">
-			End of history
-		</footer>
-	</body>
+<!DOCTYPE html>
+<html>
+	<head>
+		<title>Whatsapp - {{ name }}</title> 
+		<link rel="stylesheet" href="https://www.w3schools.com/w3css/4/w3.css"> 
+		<style>
+		@import url('https://fonts.googleapis.com/css2?family=Noto+Sans+HK:wght@300;400&display=swap');
+		html {
+			font-family: 'Noto Sans HK', sans-serif;
+			font-size: 12px;
+			scroll-behavior: smooth;
+		}
+		header {
+			position: fixed;
+			z-index: 20;
+			border-bottom: 2px solid #e3e6e7;
+			font-size: 2em;
+			font-weight: bolder;
+			background-color: white;
+			padding: 20px 0 20px 0;
+		}
+		footer {
+			border-top: 2px solid #e3e6e7;
+			font-size: 2em;
+			padding: 20px 0 20px 0;
+		}
+		article {
+			width:500px;
+			margin:100px auto;
+			z-index:10;
+			font-size: 15px;
+			word-wrap: break-word;
+		}
+		img, video {
+			max-width:100%;
+		}
+		a.anchor {
+			display: block;
+			position: relative;
+			top: -100px;
+			visibility: hidden;
+		}
+		div.reply{
+			font-size: 13px;
+			text-decoration: none;
+		}
+		</style>
+	</head>
+	<body>
+		<header class="w3-center w3-top">Chat history with {{ name }}</header>
+		<article class="w3-container">
+			<div class="table" style="width:100%">
+			{% set last = {'last': 946688461.001} %}
+			{% for msg in msgs -%}
+				<div class="w3-row" style="padding-bottom: 10px">
+					<a class="anchor" id="{{ msg.key_id }}"></a>
+					{% if determine_day(last.last, msg.timestamp) is not none  %}
+					<div class="w3-center" style="color:#70777c;padding: 10px 0 10px 0;">{{ determine_day(last.last, msg.timestamp) }}</div>
+					{% if last.update({'last': msg.timestamp}) %}{% endif %}
+					{% endif %}
+					{% if msg.from_me == true %}
+					<div class="w3-row">
+						<div style="float: left; color:#70777c;">{{ msg.time }}</div>
+						<div style="padding-left: 10px; text-align: right; color: #3892da;">You</div>
+					</div>
+					<div class="w3-row">
+						<div class="w3-col m10 l10">
+							<div style="text-align: right;">
+							{% if msg.reply is not none %}
+								<div class="reply">
+									<span style="color: #70777a;">Replying to </span>
+									<a href="#{{msg.reply}}" style="color: #168acc;">"{{ msg.quoted_data or 'media' }}"</a>
+								</div>
+							{% endif %}
+							{% if msg.media == false %}
+								{% filter escape %}{{ msg.data or "{This message is not supported yet}" | replace('\n', '<br>') }}{% endfilter %}
+							{% else %}
+								{% if "image/" in msg.mime %}
+								<a href="{{ msg.data }}"><img src="{{ msg.data }}" /></a>
+								{% elif "audio/" in msg.mime %}
+								<audio controls="controls" autobuffer="autobuffer">
+									<source src="{{ msg.data }}" />
+								</audio>
+								{% elif "video/" in msg.mime %}
+								<video controls="controls" autobuffer="autobuffer">
+									<source src="{{ msg.data }}" />
+								</video>
+								{% elif "/" in msg.mime %}
+									{The file cannot be displayed here, however it should be located at {{ msg.data }}}
+								{% else %}
+									{% filter escape %}{{ msg.data }}{% endfilter %}
+								{% endif %}
+								{% if msg.caption is not none %}
+								<br>
+								{{ msg.caption }}
+								{% endif %}
+							{% endif %}
+							</div>
+						</div>
+						<div class="w3-col m2 l2" style="padding-left: 10px"><img src="{{ my_avatar }}" onerror="this.style.display='none'"></div>
+					</div>
+					{% else %}
+					<div class="w3-row">
+						<div style="padding-right: 10px; float: left; color: #3892da;">
+							{% if msg.sender is not none %}
+							{{ msg.sender }}
+							{% else %}
+							{{ name }}
+							{% endif %}
+						</div>
+						<div style="text-align: right; color:#70777c;">{{ msg.time }}</div>
+					</div>
+					<div class="w3-row">
+						<div class="w3-col m2 l2"><img src="{{ their_avatar }}" onerror="this.style.display='none'"></div>
+						<div class="w3-col m10 l10">
+							<div style="text-align: left;">
+							{% if msg.reply is not none %}
+								<div class="reply">
+									<span style="color: #70777a;">Replying to </span>
+									<a href="#{{msg.reply}}" style="color: #168acc;">"{{ msg.quoted_data or 'media' }}"</a>
+								</div>
+							{% endif %}
+							{% if msg.media == false %}
+								{% filter escape %}{{ msg.data or "{This message is not supported yet}" }}{% endfilter %}
+							{% else %}
+								{% if "image/" in msg.mime %}
+								<a href="{{ msg.data }}"><img src="{{ msg.data }}" /></a>
+								{% elif "audio/" in msg.mime %}
+								<audio controls="controls" autobuffer="autobuffer">
+									<source src="{{ msg.data }}" />
+								</audio>
+								{% elif "video/" in msg.mime %}
+								<video controls="controls" autobuffer="autobuffer">
+									<source src="{{ msg.data }}" />
+								</video>
+								{% elif "/" in msg.mime %}
+									{The file cannot be displayed here, however it should be located at {{ msg.data }}}
+								{% else %}
+									{% filter escape %}{{ msg.data }}{% endfilter %}
+								{% endif %}
+								{% if msg.caption is not none %}
+								<br>
+								{{ msg.caption }}
+								{% endif %}
+							{% endif %}
+							</div>
+						</div>
+					</div>
+					{% endif %}
+				</div>
+			{% endfor %}
+			</div>
+		</article>
+		<footer class="w3-center">
+			End of history
+		</footer>
+	</body>
 </html>

extract_iphone_media.py

@@ -1,50 +0,0 @@
-#!/usr/bin/python3
-
-import shutil
-import sqlite3
-import os
-
-
-def extract_media(base_dir):
-    with sqlite3.connect(f"{base_dir}/Manifest.db") as manifest:
-        c = manifest.cursor()
-        c.execute("""SELECT count()
-                    FROM Files
-                    WHERE relativePath
-                        LIKE 'Message/Media/%'""")
-        total_row_number = c.fetchone()[0]
-        print(f"Gathering media...(0/{total_row_number})", end="\r")
-        c.execute("""SELECT fileID,
-                            relativePath,
-                            flags
-                    FROM Files
-                    WHERE relativePath
-                        LIKE 'Message/Media/%'""")
-        row = c.fetchone()
-        if not os.path.isdir("Message"):
-            os.mkdir("Message")
-        if not os.path.isdir("Message/Media"):
-            os.mkdir("Message/Media")
-        i = 0
-        while row is not None:
-            destination = row[1]
-            hashes = row[0]
-            folder = hashes[:2]
-            flags = row[2]
-            if flags == 2:
-                os.mkdir(destination)
-            elif flags == 1:
-                shutil.copyfile(f"{base_dir}/{folder}/{hashes}", destination)
-            i += 1
-            if i % 100 == 0:
-                print(f"Gathering media...({i}/{total_row_number})", end="\r")
-            row = c.fetchone()
-        print(f"Gathering media...({total_row_number}/{total_row_number})", end="\r")
-
-
-if __name__ == "__main__":
-    from optparse import OptionParser
-    parser = OptionParser()
-    (_, args) = parser.parse_args()
-    base_dir = args[0]
-    extract_media(base_dir)

old_README.md

@@ -0,0 +1,34 @@
+# Whatsapp-Chat-Exporter
+A Whatsapp database parser that will give you the history of your Whatsapp conversations in HTML and JSON  
+**If you plan to uninstall WhatsApp or delete your WhatsApp account, please make a backup of your WhatsApp database. You may want to use this exporter again on the same database in the future as the exporter develops**
+
+# Usage
+First, clone this repo, and copy all py and html files to a working directory if you want to do so.
+```shell
+git clone https://github.com/KnugiHK/Whatsapp-Chat-Exporter.git
+```
+Then, ready your WhatsApp database, place them in the root of working directory.
+* For Android, it is called msgstore.db. If you want name of your contacts, get the contact database, which is called wa.db.
+* For iPhone, it is called 7c7fba66680ef796b916b067077cc246adacf01d (YES, a hash).
+
+Next, ready your media folder, place it in the root of working directory.
+* For Android, copy the WhatsApp directory from your phone directly.
+* For iPhone, run the extract_iphone_media.py, and you will get a folder called Message.
+```
+python extract_iphone_media.py "C:\Users\[Username]\AppData\Roaming\Apple Computer\MobileSync\Backup\[device id]"
+```
+And now, you should have something like this:
+
+![Folder structure](imgs/structure.png)
+
+Last, run the script regarding the type of phone.
+```
+python extract.py & :: Android
+python extract_iphone.py & :: iPhone
+```
+And you will get these:
+#### Private Message
+![Private Message](imgs/pm.png)
+
+#### Group Message
+![Group Message](imgs/group.png)

setup.py

@@ -0,0 +1,46 @@
+import setuptools
+from re import search
+
+with open("README.md", "r") as fh:
+    long_description = fh.read()
+
+with open("Whatsapp_Chat_Exporter/__init__.py", encoding="utf8") as f:
+    version = search(r'__version__ = "(.*?)"', f.read()).group(1)
+
+setuptools.setup(
+    name="whatsapp-chat-exporter",
+    version=version,
+    author="KnugiHK",
+    author_email="info@knugi.com",
+    description="A Whatsapp database parser that will give you the history of your Whatsapp conversations in HTML and JSON.",
+    long_description=long_description,
+    long_description_content_type="text/markdown",
+    url="https://github.com/KnugiHK/Whatsapp-Chat-Exporter",
+    packages=setuptools.find_packages(),
+    package_data = {
+        '': ['whatsapp.html']
+    },
+    classifiers=[
+        "Programming Language :: Python :: 3 :: Only",
+        "Programming Language :: Python :: 3.7",
+        "Programming Language :: Python :: 3.8",
+        "Programming Language :: Python :: 3.8",
+        "License :: OSI Approved :: MIT License",
+        "Operating System :: OS Independent",
+        "Development Status :: Beta",
+        "Environment :: Console",
+        "Intended Audience :: End Users/Desktop"
+        "Topic :: Communications :: Chat",
+        "Topic :: Utilities",
+        "Topic :: Database"
+    ],
+    python_requires='>=3.7',
+    install_requires=[
+       'jinja2'
+    ],
+    entry_points={
+        "console_scripts": [
+            "wtsexporter = Whatsapp_Chat_Exporter.__main__:main"
+        ]
+    }
+)