Question:
This is not a courseworker and not a diploma! I have not been a student for a long time. This is an attempt to write another free program for "home" car diagnostics. And I'm not very strong in Python. Therefore, I ask for help.
Initial data:
There is a device that is connected to the computer via a USB port. The computer and the device exchange data in symbolic form. In fact, this is a classic connection: the device plays the role of a server, and the computer plays the role of a terminal. If someone says something, then this device is an OBD-II adapter (On-Board Diagnostics – a device for diagnosing cars) ELM327.
The system works according to a very simple principle – to a request (command) of a client (terminal), the server returns a response. Here is an example request-response:
> ati
ELM327 v1.5
> atdp
AUTO, ISO 15765-4 (CAN 11/250)
This is how the response line "ELM327 v1.5" looks like in hex form:
45 4C 4D 33 32 37 20 76 31 2E 35 0D 0D 3E
And so the response line "AUTO, ISO 15765-4 (CAN 11/250)":
41 55 54 4F 2C 20 49 53 4F 20 31 35 37 36 35 2D 34 20 28 43 41 4E 20 31 31 2F 32 35 30 29 0D 0D 3E
First, in the response line, instead of the '\n' character, the character '\r' is used. (There is no problem here. This is easily fixed.) Second, the response ends with a '>' character (hex code = 0x3E), which makes the response processing code more complicated. Here, too, there is no unsolvable problem.
The problem I can't get over is that in Python 3 I can't really get a character string from a port. In Python-2.7, the request-response works without problems.
Here is the code for the second Python:
def show_response(port):
while True:
resp = ''
while True:
ch = port.read(1)
if len(ch) == 1:
# print ch
if ch == '\r':
resp += '\n'
else:
resp += ch
if resp[-2:] == '\n>':
print resp[:-2],
break
For the third Python, this code (of course!) Does not fit. I don't mean replacing the Python 2 print statement with the Python 3 print() function. It's all easy to fix.
The problem is in receiving data from the serial port. If in the second Python it is enough to do (simplified) like this:
ch = port.read(1)
print ch
, then in the third Python, the read() function returns not a string of characters, but a string of bytes.
I am Si-shnik. And for me, a byte is from and in Africa a byte. From my point of view, what a byte is, what an ASCII character (not multibyte characters like utf-8 !!!, namely a one-byte ASCII character from the first half of the code table) is the same set of bits. Therefore, the read() function in the second Python's serial module, which returns a character string, and the read() function in the third Python's serial module, which returns a byte string, should produce the same result.
But in practice, instead of the byte string "45 4C 4D 33 32 …" (the response to the ati command in Python 2), I get only three bytes "7F BF ED" in Python 3. And I can't get anything else. The exchange between the ELM327 and the computer takes place in bytes (characters?) in the range 0..0x7F. It doesn't smell like Cyrillic! Why such strange codes — 0xBF and 0xED ?
What am I doing wrong?
Here is the code for the third Python:
def show_response(port):
#print('show_response')
while True:
resp = ''
while True:
b = port.read(1)
if len(b) != 0:
print('0x{0:02X}'.format(b[0]))
#if ch == '\r':
# resp += '\n'
#else:
# resp += ch
if resp[-2:] == '\n>':
print(resp[:-2], end='')
break
The port initialization for the second and third Python is almost the same:
#!/usr/bin/env python3
#coding:utf8
import serial
import time
from multiprocessing import Process
...
if __name__ == '__main__':
try:
port = serial.Serial("/dev/ttyUSB0", 38400, timeout=0.2)
except serial.SerialException:
print('Соединение не удалось')
exit(1)
port.flushOutput()
port.flushInput()
p1 = Process(target=show_response, args=(port,))
p1.daemon = True
p1.start()
command(port)
print('Пока-пока!')
The serial package was installed from the standard repositories — on one computer (Ubuntu-10.04) it was the package installer for the second Python ($ sudo apt-get install python-serial), on another computer (Debian-8) it was for the third Python (# apt-get install python3-serial).
I do not understand where to dig? Someone write me a magic pendel in the right direction, otherwise I’ll die without understanding the essence of Python’s machinations with bytes.
UPDATE 08/08/2015 – 00:53
I changed the program, since it is generally microscopic. Now the port is stupidly opened, the devas command is stupidly given – 'ati\r' and everything that flows from the port in response to this command is stupidly displayed on the console – no processes, no tricks, nothing superfluous! And yet the output to the console is exactly the same.
Here is the text of this program:
#!/usr/bin/env python3
#coding:utf8
import serial
import time
if __name__ == '__main__':
try:
port = serial.Serial("/dev/ttyUSB0", 38400, timeout=1)
except serial.SerialException:
print('Соединение не удалось')
exit(1)
port.flushOutput()
port.flushInput()
# Послать запрос
cmd = bytes('ati\r', 'utf-8')
port.write(cmd)
time.sleep(0.1)
# Принять ответ и вывести его на консоль
while True:
print('.')
resp = port.readline()
# Вариант вывода 1
if len(resp) > 0:
print('[{0:d}] = '.format(len(resp)), end = '')
for b in resp:
print('0x{0:02X} '.format(b), end='')
'''
# Вариант вывода 2
string = str(port.readline())
if len(string) > 0:
print('[{0:d}] = '.format(len(string)), end = '')
for ch in string:
print('0x{0:02X} '.format(ord(ch)), end='')
'''
'''
# Вариант вывода 3
string = str(port.readline())
if len(string) > 0:
print('[{0:d}] = '.format(len(string)), end = '')
print(string)
'''
The output for the first option is this (screenshot):
$ ./myOBDm2.py
.
[2] = 0x7F 0xBF .
.
.
.
The output for the second option is:
$ ./myOBDm2.py
.
[3] = 0x62 0x27 0x27 .
[3] = 0x62 0x27 0x27 .
[3] = 0x62 0x27 0x27 .
[3] = 0x62 0x27 0x27 .
[3] = 0x62 0x27 0x27 .
[3] = 0x62 0x27 0x27 .
The output for the third option is:
$ ./myOBDm2.py
.
[3] = b''
.
[3] = b''
.
[3] = b''
.
A closer look at the output of the third option hints that the readline() function returns an empty byte string – after all, three b'' characters is nothing more than a string of bytes in Python. I wonder what that means? Why is serial in Python 3 so weird?
I also tried to iterate over encoding options when sending a request:
# Послать запрос
cmd = bytes('ati\r', 'utf-8') # 'cp866', 'cp1251', 'ascii'
port.write(cmd)
Nothing has changed, the encoding has no effect.
The device has LEDs that blink when receiving and transmitting data via the USB interface. Judging by the blinking, the exchange is underway.
UPDATE 08/08/2015 – 16:47
I made two changes in the program:
...
# Послать запрос
cmd = bytes('ati\r\n', 'ascii') # -1- Добавил '\n'
port.write(cmd)
time.sleep(1)
# Принять ответ и вывести его на консоль
while True:
print('.')
resp = port.read() # -2- Изменил функцию (была readline)
# Вариант вывода 1
if len(resp) > 0:
print('[{0:d}] = '.format(len(resp)), end = '')
for b in resp:
print('0x{0:02X} '.format(b), end='')
...
As a result, the console received the correct result:
$ ./myOBDm2.py
.
[1] = 0x61 .
[1] = 0x74 .
[1] = 0x69 .
[1] = 0x0D .
[1] = 0x45 .
[1] = 0x4C .
[1] = 0x4D .
[1] = 0x33 .
[1] = 0x32 .
[1] = 0x37 .
[1] = 0x20 .
[1] = 0x76 .
[1] = 0x31 .
[1] = 0x2E .
[1] = 0x35 .
[1] = 0x0D .
[1] = 0x0D .
[1] = 0x3E .
.
.
.
.
.
.
.
But it is too early to report on the solution of the problem, since the problem disappeared only for the first (after turning on the computer) launch of the program. Repeated and all subsequent launches of the program gave the same incorrect results:
$ ./myOBDm2.py
.
[1] = 0x7F .
[1] = 0xBF .
[1] = 0xFE .
.
.
.
.
It should be noted that when returning a response from the device, the lines end with the character '\r', and not '\n' and not their combination '\r\n'.
According to my ideas, the device ignores the character '\n'. The description actually says that the device also ignores other "white" characters (space, tab, ..). For example, the device understands the commands 'ati\r' and 'at i\r' equally correctly.
Thus, the presence or absence of the '\n' character at the end of the command does not affect the performance of the device, which is confirmed in practice.
UPDATE 08/08/2015 – 19:31
Okay. I reboot the computer and look at the port settings. The port settings before starting the program are as follows:
$ stty -F /dev/ttyUSB0
speed 57600 baud; line = 0;
eof = ^A; min = 1; time = 0;
-brkint -icrnl -imaxbel
-opost -onlcr
-icanon -echo -echoe
Then I start the program. She works fine. I exit the program, and again I look at the port settings:
$ stty -F /dev/ttyUSB0
speed 38400 baud; line = 0;
eof = ^A; min = 0; time = 0;
-brkint -icrnl -imaxbel
-opost -onlcr
-isig -icanon -iexten -echo -echoe -echok -echoctl -echoke
Op-pa! What is called – "Find seven differences!"
Using the method of enumeration of enabling/disabling port settings (more precisely, terminal_port), we managed to find out that the -iexten parameter "interferes".
If you turn off this option before starting the program
$ stty -F /dev/ttyUSB0 iexten
$ stty -F /dev/ttyUSB0
speed 38400 baud; line = 0;
eof = ^A; min = 0; time = 0;
-brkint -icrnl -imaxbel
-opost -onlcr
-isig -icanon -echo -echoe -echok -echoctl -echoke
, then the program works normally again.
In other words, the serial module in Python-3, when a port is opened, changes its settings. The serial module in Python-2 changes the port settings in the same way, but here (in the second Python) these settings do not lead to fatal consequences.
Now it remains to find a way, when opening a port in the serial module for Python-3, to indicate that the IEXTEN parameter does not need to be set.
The problem is still unresolved. I keep digging. As I "treat" I will publish my steps.
UPDATE from 08/28/2015-02:01
As a result of unhurried attempts to make the ELM327 device work under Python-3, I came to the understanding that perhaps it is not Python that is not able to receive a response from the device, but vice versa – Python is not able to send the device command. More precisely: Python sends a garbled command, the girl gets who-knows-what and gives Python an answer like "I don't understand". Since Python does not work correctly, it also understands the answer in a distorted form. An indirect confirmation of this is that after 30 skunds, the devas sends some short "burp" to the computer. I assumed that it could be something like "unfinished command is removed by timeout". At least such a phenomenon ("belching") is not observed when working under Python-2. In other words, Python-2 sends the correct command to the device, and Python-3 sends a distorted one. Accordingly, the devas react in this way.
To check that Python-3 sends nonsense, I set up a simple (for electronics engineers, and I am an electronics engineer) experiment. I took two Chinese USB-UART type CH340G converters and connected them in a null modem. Then, I connected one to a working computer, on which the program runs under Python-3, and the other to a laptop, on which the exact same (well, except for print and some other differences) program runs under Python-2.
Then, if Python-3 distorts commands, I will see these distortions on the second computer.
Yes! A significant addition is that the ELM-327 device uses a CH340T converter chip, this is a CH340G clone.
As a result, I saw that when transmitting data in both directions, there are no distortions at all! Tried at different baud rates. Everything works clearly. Hence the conclusion – problems in the connection of the device and Python-3.
The ELM327 device is non-separable, somewhat reminiscent of power supplies for notes. I had to break the body with a hammer. Opened more or less normally.
And then the second series of the detective begins.
I connected an oscilloscope to the UART output of the CH340T to see what it was sending directly to the microcontroller.
Hold on to the chair! It turns out that the chip is transmitting the correct data, but not at the speed we expect. More precisely, in a Python program, when initializing a serial port, the speed of operation is indicated. A program running under Python-2 sets the CH340T chip to a given speed. But a program running under Python-3, for some unknown reason, cannot configure this microcircuit. As a result, it turns out that the program from under Python-2 accesses the microcontroller at a speed of 38400, and the program from under Python-3 – at a speed of 9600 Baud.
The microcontroller expects that they will "knock" at 38400, and therefore, of course, it does not correctly understand the command from the computer. Next, the microcontroller gives a response like "Are you there, completely fucked up?" at 38400, but the CH340T takes it at 9600 and sends garbage to the computer. And after 30 seconds, the microcontroller sends another packet to the computer like "Fuck you! I'm canceling this command. Enter the next one!".
Now I need to understand why Python-3 is not able to properly initialize the CH340T converter chip.
UPDATE from 08/28/2015-04:05
I will say right away – it was not possible to defeat the problem, but we managed to find a "workaround". The recipe is simple – in Python-3, don't initialize the port and set the speed at the same time.
You don't have to do this:
try:
port = serial.Serial("/dev/ttyUSB0", 38400, timeout=0.2)
except serial.SerialException:
print('Соединение не удалось')
exit(1)
You should do this:
try:
port = serial.Serial("/dev/ttyUSB0")
port.baudrate = 38400
port.timeout = 0.2
except serial.SerialException:
print('Соединение не удалось')
exit(1)
I understand that this is messed up somewhere in the constructor of the Serial class. But something no longer rushes me to repair this madhouse.
Answer:
A similar situation on Pyton2 when receiving information from the UT60D tester, which only transmits signals to the port.
When the port is opened in the recommended settings, +10 volts at the port input and signal bursts disappear.
Helped:
ser.rtsToggle=True