Question:
I purchased an http proxy and saw that the login / password from ip proxy is transmitted in its pure form. Tech support said they only work with http.
When I connect to the site, I first see an http connection with a proxy, and then TLS packets come.
The main question is: how is traffic transmitted with different types of proxy connections? At what stages does encryption and decryption take place under different protocols?
I tried to figure it out myself, but there is a difference on the sites: some write that https proxy does not exist, but there is a connection using the connect method between the server and the client that communicate via https; others, which act like MITM, establishes two https client-proxy/proxy-server connections and decrypts and encrypts the traffic for each.
Answer:
Officially, the concept of HTTPS proxy does not exist. This term can fall under 2 concepts:
- You connect to the proxy using TLS and then use the HTTP CONNECT method to connect. The third party does not see anything but TLS packets.
- You are connecting to a proxy without TLS. The third party sees all HTTP headers: the CONNECT method, the host/port you are connecting to, the authorization headers.
On various resources where they provide lists of free HTTPS proxies, they usually mean the second option. I can't say anything about paid proxies.
There are also special types of HTTP proxies that allow you to connect only to HTTP servers. At the same time, they see all the traffic, they can cache something on their own, change the HTTP headers of your requests / responses. Commonly used (used?) in corporate networks.
Let's consider in detail the HTTPS proxy of the 2nd type. Although they are called HTTPS and you connect to them via HTTP, you can use them for any protocol that runs over TCP: SSH, FTP, POP3, IMAP, SMTP, etc. You can also connect to other SOCKS/HTTP- proxy and arrange a chain of proxies of any length. The implementation of each specific proxy may vary, it may have white / black lists for hosts, or ports (this may limit communication over protocols that hang on these ports).
On the question of when encryption occurs in the HTTPS proxy of the 2nd type. It only happens when the protocol you are connecting with switches to encryption. If you are connecting to an HTTPS server, then the TLS handshake is where the data exchange begins. The proxy server/your ISP sees the same TLS packets that your ISP sees without using a proxy. When using TLS version less than or equal to 1.2, they see the hostname and certificate in its purest form. Thus, they can find out which site you are visiting (even if you have hidden the IP address of the visited resource using a proxy).
About MITM when using HTTPS proxies of type 1 and 2. A proxy server can slip you a left SSL certificate, and thus be able to read and change all traffic in its pure form. To prevent this from happening, there are root certificates. Here the rule is simple, you cannot ignore SSL certificate verification errors. In browsers, the procedure for ignoring an SSL certificate error has only become more complicated lately, in order to create problems for dim-witted users who click "OK, continue" in all windows.
An example of traffic connecting to a proxy without authorization. You are connecting via TCP to the proxy host:port. Further, according to the HTTP specification, a request is sent:
CONNECT example.com:443 HTTP/1.1\r\n
Host: example.com\r\n
\r\n
According to the rules of the HTTP protocol, you are waiting for a response, it looks like this:
HTTP/1.1 200 Connection established\r\n
\r\n
At this time, the proxy server has opened a TCP connection to example.com . Then everything works very simply. Anything you send to the proxy, it forwards to example.com . Everything that example.com sends to the proxy is forwarded to you.
The proxy server does not replace anything (unless it is a malicious proxy). IP does not need to be changed anywhere, this is handled at the level of TCP / IP packets. example.com sees the IP of the proxy server in the TCP/IP header.
Next, you send TLS packets to the proxy, and the proxy sends them to example.com . Everything happens transparently. If you have code that can do a TLS handshake, you can pass it a socket after the proxy's HTTP response (or a high-level wrapper over sockets), and the code below will work as usual. The code does not know that it is communicating through a proxy. Just like example.com does not know that it is communicating with you through a proxy.
An example of the simplest C# code with a demo (it is greatly simplified, but works with real HTTP proxies):
var client = new TcpClient("прокси-сервер", 12345 /*порт*/);
var stream = client.GetStream();
// создаем HTTP запрос, конвертируем в байты, отправляем по TCP
string request =
"CONNECT ru.stackoverflow.com:443 HTTP/1.1\r\n" +
"Host: ru.stackoverflow.com\r\n" +
"\r\n";
byte[] bytes = Encoding.ASCII.GetBytes(request);
stream.Write(bytes);
// получаем ответ, конвертируем в строку
// тут нужен парсинг ответа от сервера, предположим сервер вернул 200 код
byte[] buffer = new byte[1024];
int read = stream.Read(buffer, 0, buffer.Length);
string response = Encoding.ASCII.GetString(buffer, 0, read);
Console.WriteLine("Proxy response: " + response);
// создаем TLS поверх нашего TCP соединения
var tls = new SslStream(stream);
// проходим рукопожатие, указываем хост что бы проверить сертификат
// класс SslStream "не догадывается" что он работает через прокси
// он работает в обычном режиме
tls.AuthenticateAsClient("ru.stackoverflow.com");
// создаем HTTP запрос к ru.stackoverflow.com
request =
"GET / HTTP/1.1\r\n" +
"Host: ru.stackoverflow.com\r\n" +
"Connection: Close\r\n" +
"\r\n";
bytes = Encoding.ASCII.GetBytes(request);
// отправляем его через TLS
tls.Write(bytes);
// принимаем ответ через TLS
using var reader = new StreamReader(tls);
response = reader.ReadToEnd();
Console.WriteLine(response);
Result of work:
Proxy response: HTTP/1.1 200 Connection established
HTTP/1.1 200 OK
Connection: close
cache-control: private
content-type: text/html; charset=utf-8
strict-transport-security: max-age=15552000
x-frame-options: SAMEORIGIN
x-request-guid: 4024b045-1c35-4d53-a827-6f58f89c7c79
content-security-policy: upgrade-insecure-requests; frame-ancestors 'self' https://stackexchange.com
Accept-Ranges: bytes
Date: Wed, 20 Oct 2021 22:25:14 GMT
Via: 1.1 varnish
X-Served-By: cache-hhn4052-HHN
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1634768715.812812,VS0,VE98
Vary: Fastly-SSL
X-DNS-Prefetch-Control: off
Set-Cookie: prov=ad256a6c-746c-ebe9-7e57-d3ba0f3e1928; domain=.stackoverflow.com; expires=Fri, 01-Jan-2055 00:00:00 GMT; path=/; HttpOnly
transfer-encoding: chunked
3f8
<!DOCTYPE html>
...
And here is an example of a proxy that can work with the code above. As you can see, there is much less code, and there is no magic here:
// слушаем порт 12345
var listener = new TcpListener(IPAddress.Any, 12345);
listener.Start();
// принимаем сокет от прокси-клиента
var socket = listener.AcceptSocket();
var clientToProxyStream = new NetworkStream(socket);
// читаем HTTP запрос от клиента
byte[] bytes = new byte[1024];
int read = clientToProxyStream.Read(bytes, 0, bytes.Length);
string request = Encoding.ASCII.GetString(bytes, 0, read);
// тут должен быть парсинг запроса
// предположим нас попросили подключится к ru.stackoverflow.com
// подключаемся к ru.stackoverflow.com
var client = new TcpClient("ru.stackoverflow.com", 443);
var proxyToStackoverflowStream = client.GetStream();
// отправляем HTTP ответ обратно клиенту
string response = "HTTP/1.1 200 Connection established\r\n\r\n";
bytes = Encoding.ASCII.GetBytes(response);
clientToProxyStream.Write(bytes);
// перенаправляем все данные между потоками
// всё что пришло в clientToProxyStream отправляем в proxyToStackoverflowStream
// всё что пришло в proxyToStackoverflowStream отправляем в clientToProxyStream
var task1 = clientToProxyStream.CopyToAsync(proxyToStackoverflowStream);
var task2 = proxyToStackoverflowStream.CopyToAsync(clientToProxyStream);
// ждем пока задачи копирования завершатся, тоесть закрытия соединения
await Task.WhenAll(task1, task2);