Question:
I purchased an http proxy and saw that the login / password from the ip proxy is transmitted in its pure form. The tech support said that they only work with http.
When I connect to the site, I see first a connection via http with a proxy, and then there are TLS packets.
The main question is: how is traffic transmitted with different types of proxy connections? At what stages does encryption and decryption occur under different protocols?
I tried to figure it out myself, but there is a difference on the websites: some write that the https proxy does not exist, but there is a connection using the connect method between a server and a client that communicate via https; others, which act as MITM, establish two https client-proxy / proxy connections and for each decrypts and encrypts the traffic.
Answer:
There is no official concept of HTTPS proxy. 2 concepts can fall under this term:
- You connect to the proxy using TLS, and then use the HTTP CONNECT method to connect. At the same time, the third party does not see anything except TLS packets.
- You are connecting to a proxy without TLS. The third party sees all HTTP headers: CONNECT method, host / port where you are connecting, authorization headers.
On various resources where they provide lists of free HTTPS proxies, they usually mean the second option. I can't say anything about paid proxies.
There are also special types of HTTP proxies that only allow connections to HTTP servers. At the same time, they see all traffic, can cache something on themselves, change the HTTP headers of your requests / responses. Usually used (are they?) In corporate networks.
Let's consider in detail the HTTPS proxy of the 2nd type. Although they are called HTTPS, and you connect to them via HTTP, you can use them for any protocol over TCP: SSH, FTP, POP3, IMAP, SMTP, etc. You can also connect to other SOCKS / HTTP proxy and arrange a chain of proxies of any length. The implementation of each specific proxy may differ, it may contain white / black lists for hosts, or ports (this may restrict communication over the protocols that hang on these ports).
On the question of when encryption occurs in the HTTPS proxy of the 2nd type. It only happens when the protocol you are connecting to goes to encryption. If you are connecting to an HTTPS server, then the TLS handshake is where the data exchange begins. The proxy server / your ISP sees the same TLS packets that your ISP sees without using a proxy. When using TLS version less than or equal to 1.2, they see the hostname and certificate in their pure form. Thus, they can find out which site you are visiting (even if you hid the IP address of the visited resource using a proxy).
About MITM when using HTTPS proxies of types 1 and 2. The proxy server can slip the left SSL certificate to you, and thus be able to read and modify all traffic in its pure form. To prevent this from happening, there are root certificates. The rule here is simple, you cannot ignore SSL certificate verification errors. In browsers, the procedure for ignoring SSL certificate errors has only become more complicated lately, in order to create problems for near users who click "OK, continue" in all windows.
An example of traffic connecting to a proxy without authorization. You are connecting via TCP to host: proxy port. Further, according to the HTTP specification, a request is sent:
CONNECT example.com:443 HTTP/1.1\r\n
Host: example.com\r\n
\r\n
According to the rules of the HTTP protocol, you are waiting for a response, it looks like this:
HTTP/1.1 200 Connection established\r\n
\r\n
At this time, the proxy server opened a TCP connection to example.com . Then everything works very simply. Anything you send to the proxy will be forwarded to example.com . Everything that example.com sent to the proxy server is forwarded to you.
The proxy server does not override anything (unless it is a malicious proxy). IP does not need to be substituted anywhere, it is handled at the TCP / IP packet level. example.com sees the IP of the proxy server in the TCP / IP header.
Next, you send TLS packets to the proxy, and the proxy sends them to example.com . Everything happens transparently. If you have code that knows how to do a TLS handshake, you can pass a socket to it after the HTTP response from the proxy server (or a high-level wrapper over sockets), and the code below will work as usual. The code does not know that it is communicating through a proxy. Like example.com doesn't know that it is communicating with you through a proxy.
An example of the simplest C # code with a demo (it is greatly simplified, but works with real HTTP proxies):
var client = new TcpClient("прокси-сервер", 12345 /*порт*/);
var stream = client.GetStream();
// создаем HTTP запрос, конвертируем в байты, отправляем по TCP
string request =
"CONNECT ru.stackoverflow.com:443 HTTP/1.1\r\n" +
"Host: ru.stackoverflow.com\r\n" +
"\r\n";
byte[] bytes = Encoding.ASCII.GetBytes(request);
stream.Write(bytes);
// получаем ответ, конвертируем в строку
// тут нужен парсинг ответа от сервера, предположим сервер вернул 200 код
byte[] buffer = new byte[1024];
int read = stream.Read(buffer, 0, buffer.Length);
string response = Encoding.ASCII.GetString(buffer, 0, read);
Console.WriteLine("Proxy response: " + response);
// создаем TLS поверх нашего TCP соединения
var tls = new SslStream(stream);
// проходим рукопожатие, указываем хост что бы проверить сертификат
// класс SslStream "не догадывается" что он работает через прокси
// он работает в обычном режиме
tls.AuthenticateAsClient("ru.stackoverflow.com");
// создаем HTTP запрос к ru.stackoverflow.com
request =
"GET / HTTP/1.1\r\n" +
"Host: ru.stackoverflow.com\r\n" +
"Connection: Close\r\n" +
"\r\n";
bytes = Encoding.ASCII.GetBytes(request);
// отправляем его через TLS
tls.Write(bytes);
// принимаем ответ через TLS
using var reader = new StreamReader(tls);
response = reader.ReadToEnd();
Console.WriteLine(response);
Result of work:
Proxy response: HTTP/1.1 200 Connection established
HTTP/1.1 200 OK
Connection: close
cache-control: private
content-type: text/html; charset=utf-8
strict-transport-security: max-age=15552000
x-frame-options: SAMEORIGIN
x-request-guid: 4024b045-1c35-4d53-a827-6f58f89c7c79
content-security-policy: upgrade-insecure-requests; frame-ancestors 'self' https://stackexchange.com
Accept-Ranges: bytes
Date: Wed, 20 Oct 2021 22:25:14 GMT
Via: 1.1 varnish
X-Served-By: cache-hhn4052-HHN
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1634768715.812812,VS0,VE98
Vary: Fastly-SSL
X-DNS-Prefetch-Control: off
Set-Cookie: prov=ad256a6c-746c-ebe9-7e57-d3ba0f3e1928; domain=.stackoverflow.com; expires=Fri, 01-Jan-2055 00:00:00 GMT; path=/; HttpOnly
transfer-encoding: chunked
3f8
<!DOCTYPE html>
...
And here is an example of a proxy server that can work with the code above. As you can see, there is much less code, and there is no magic here:
// слушаем порт 12345
var listener = new TcpListener(IPAddress.Any, 12345);
listener.Start();
// принимаем сокет от прокси-клиента
var socket = listener.AcceptSocket();
var clientToProxyStream = new NetworkStream(socket);
// читаем HTTP запрос от клиента
byte[] bytes = new byte[1024];
int read = clientToProxyStream.Read(bytes, 0, bytes.Length);
string request = Encoding.ASCII.GetString(bytes, 0, read);
// тут должен быть парсинг запроса
// предположим нас попросили подключится к ru.stackoverflow.com
// подключаемся к ru.stackoverflow.com
var client = new TcpClient("ru.stackoverflow.com", 443);
var proxyToStackoverflowStream = client.GetStream();
// отправляем HTTP ответ обратно клиенту
string response = "HTTP/1.1 200 Connection established\r\n\r\n";
bytes = Encoding.ASCII.GetBytes(response);
clientToProxyStream.Write(bytes);
// перенаправляем все данные между потоками
// всё что пришло в clientToProxyStream отправляем в proxyToStackoverflowStream
// всё что пришло в proxyToStackoverflowStream отправляем в clientToProxyStream
var task1 = clientToProxyStream.CopyToAsync(proxyToStackoverflowStream);
var task2 = proxyToStackoverflowStream.CopyToAsync(clientToProxyStream);
// ждем пока задачи копирования завершатся, тоесть закрытия соединения
await Task.WhenAll(task1, task2);