How can I convert 8 bit array to 16 bit array in Python?

Question

I'm transferring 16 bit numbers from a STM32 (from an ADC) over SPI to raspberry pi 4. On the pi side, I have a script that runs in a loop, waits for a GPIO pin to go high to act as a "detect" for my system to then enable the raspberry pi to initiate the SPI transfer. Unfortunately the raspberry pi hardware only supports 8 bit SPI and so the numbers I'm getting back are split across 2 x 8 bits. The data I'm transferring:

uint16_t spi_test_16[64] = {1023, 19, 38, 47, 51, 52, 53, 59, 76, 91, 99, 119, 172, 174,
    179, 205, 215, 218, 225, 235, 242, 264, 284, 308, 316, 334, 404, 431, 434,
    442, 442, 457, 468, 479, 490, 496, 508, 509, 512, 523, 524, 524, 525, 589, 598,
    642, 674, 711, 716, 749, 761, 767, 771, 788, 790, 818, 854, 928, 935, 947, 964,
    976, 987, 996}

and the data I'm receiving:

(255, 3, 19, 0, 38, 0, 47, 0, 51, 0, 52, 0, 53, 0, 59, 0, 76, 0, 91, 0, 99, 0, 119, 0, 172, 0  174, 0, 179, 0, 205, 0, 215, 0, 218, 0, 225, 0, 235, 0, 242, 0, 8, 1, 28, 1, 52, 1, 60, 1, 78, 1, 148, 1, 175, 1, 178, 1, 186, 1, 186, 1, 201, 1, 212, 1, 223, 1, 234, 1, 240, 1, 252, 1, 253, 1, 0, 2, 11, 2, 12, 2, 12, 2, 13, 2, 77, 2, 86, 2, 130, 2, 162, 2, 199, 2, 204, 2, 237, 2, 249, 2, 255, 2, 3, 3, 20, 3, 22, 3, 50, 3, 86, 3, 160, 3, 167, 3, 179, 3, 196, 3, 208, 3, 219, 3, 228, 3)

As you can see, it's getting the right numbers but they're being formatted incorrectly because it's in an 2 x 8 bit format. I want help in order to get the values back in to the original format using Python. My code:

    #a simple test for spi
    
    import spidev
    import time
    import RPi.GPIO as GPIO
    import numpy
    
    GPIO.setmode(GPIO.BOARD)
    GPIO.setup(26, GPIO.IN)
    GPIO.setup(40, GPIO.OUT)
    #using spi0
    bus = 0
    
    #device is chip select.
    device = 0
    
    #enable
    spi = spidev.SpiDev()
    
    #open a connection to a specific bus and device (CS pin)
    spi.open(bus, device)
    
    #set spi speed and mode
    spi.max_speed_hz = 100000
    spi.mode = 0
    spi.bits_per_word = 8
    #msg = ["Hello! "]
    msg = [72, 101, 108, 108, 111, 33, 32]
    buffer_size = 128
    spi_buffer = [0]*buffer_size
    msgRx = 0;

    flag = 0

print ("python spi test program with STM32")
#GPIO.output(7,0)
GPIO.output(40,1)
time.sleep (1)
while True :
    
    if GPIO.input(26) == 1:
        #print ("Message receiving")
        time.sleep(0.1)
        if GPIO.input(26) == 1:
            GPIO.output(40,0)
            #msgRx = spi.readbytes(160)
            msgRx = spi.xfer3(spi_buffer)
            GPIO.output(40,1)
            print (msgRx)
            ##listofzeroes = [0]*50000
            print("msgRx reset")
            msgRx = numpy.asarray(msgRx, dtype=numpy.uint16)
            for index, value in enumerate(msgRx):
                msgRx[index] >> 8

            print (msgRx)
            time.sleep(1)
            #flag = 1
    #if flag == 1:        
        #print("Message Complete")
        #flag = 0

I've attempted a couple of things, but currently in the main While loop I'm converting the received list into a 16bit unsigned integer array and then bit shifting to the left. But then how do I add the second cell of the array to the first etc?

TDLR; how can I convert a list of 2 x 8 bit integers into a list of 1 x 16 bit integers?

juanpa.arrivillaga · Accepted Answer · 2021-12-15 17:45:21Z

Use numpy:

In [9]: data = (255, 3, 19, 0, 38, 0, 47, 0, 51, 0, 52, 0, 53, 0, 59, 0, 76, 0, 91, 0, 99, 0, 119, 0, 172, 0,  174, 0, 179, 0, 205, 0, 215, 0, 218, 0, 225, 0, 235, 0, 242, 0, 8, 1, 28, 1, 52, 1, 60, 1, 78, 1, 148, 1, 175, 1, 178, 1,
   ...: 186, 1, 186, 1, 201, 1, 212, 1, 223, 1, 234, 1, 240, 1, 252, 1, 253, 1, 0, 2, 11, 2, 12, 2, 12, 2, 13, 2, 77, 2, 86, 2, 130, 2, 162, 2, 199, 2, 204, 2, 237, 2, 249, 2, 255, 2, 3, 3, 20, 3, 22, 3, 50, 3, 86, 3, 160, 3, 167, 3,
   ...:  179, 3, 196, 3, 208, 3, 219, 3, 228, 3)
   ...:

In [10]: arr = np.array(data, np.uint8)

It looks like in your tuple, you have unsigned 8-bit integers. So then just create a view of that as unsigned 16-bit integers:

In [11]: arr = arr.view(np.uint16)

In [12]: arr
Out[12]:
array([1023,   19,   38,   47,   51,   52,   53,   59,   76,   91,   99,
        119,  172,  174,  179,  205,  215,  218,  225,  235,  242,  264,
        284,  308,  316,  334,  404,  431,  434,  442,  442,  457,  468,
        479,  490,  496,  508,  509,  512,  523,  524,  524,  525,  589,
        598,  642,  674,  711,  716,  749,  761,  767,  771,  788,  790,
        818,  854,  928,  935,  947,  964,  976,  987,  996], dtype=uint16)

As noted in the comment, to make the above portable, you can specify the endianess using the following notation:

arr = arr.view('<H').

The < means 'little-endian', and H is the character code for unsigned 16 bit integers. See the docs for more on specifying dtypes.

Note, I don't know the library you are using, but you have a commented out line:

msgRx = spi.readbytes(160)

Probably, you just want to use that, but read 128 bytes (for the 64 unsigned 16-bit integers) then use np.frombuffer:

msgRx = spi.readbytes(128)
msgRx = numpy.frombuffer(msgRx, dtype=numpy.uint16)

It would be a bit safer to specify the endianess when the view is created, in case this code is ever run on a big-endian platform: arr = arr.view('<H'). (< means 'little-endian', i.e. low byte first, and H is the character code for unsigned 16 bit integers.)
I'm trying to learn Python but in C I'd have used a for loop to iterate through each cell of the array, bit-shifted the value and then added the index+1 to the the cell etc. to make the value of two 8 bit cells, one 16 bit cell. Python seems to have a library for every occasion but I'm used to doing things on a lower level. I keep overthinking it. Thanks for the answer!
@ChrisD19 yeah don't loop over numpy arrays. They support vectorized operations. For example, the for loop you had above should have just been msgRx >>= 8 (if you want it to work in-place, to create a new array you can use msgRx >> 8)

Vincent Bénet · Accepted Answer · 2021-12-15 16:49:08Z

This is working:

def to_bin(int_num):
    return '{0:08b}'.format(int_num)


def to_int(bina):
    return int(bina, 2)


def transform(int_num_1, int_num_2):
    print(f"{int_num_1 = }")
    print(f"{int_num_2 = }")
    bina_1 = to_bin(int_num_1)
    bina_2 = to_bin(int_num_2)
    print(f"{bina_1 = }")
    print(f"{bina_2 = }")
    trame = bina_2 + bina_1
    print(f"{trame = }")
    result = to_int(trame)
    print(f"{result = }")
    return result


input_list = [255, 3, 19, 0, 38, 0, 47, 0, 51, 0, 52, 0, 53, 0, 59, 0, 76, 0, 91, 0, 99, 0, 119, 0, 172, 0,  174, 0, 179, 0, 205, 0, 215, 0, 218, 0, 225, 0, 235, 0, 242, 0, 8, 1, 28, 1, 52, 1, 60, 1, 78, 1, 148, 1, 175, 1, 178, 1, 186, 1, 186, 1, 201, 1, 212, 1, 223, 1, 234, 1, 240, 1, 252, 1, 253, 1, 0, 2, 11, 2, 12, 2, 12, 2, 13, 2, 77, 2, 86, 2, 130, 2, 162, 2, 199, 2, 204, 2, 237, 2, 249, 2, 255, 2, 3, 3, 20, 3, 22, 3, 50, 3, 86, 3, 160, 3, 167, 3, 179, 3, 196, 3, 208, 3, 219, 3, 228, 3]
for i in range(len(input_list)//2):
    transform(input_list[i*2], input_list[i*2+1])

The trick is to convert to binaries, concatenate them and reconvert into integers.

Results for first 3:

int_num_1 = 255
int_num_2 = 3
bina_1 = '11111111'
bina_2 = '00000011'
trame = '0000001111111111'
result = 1023

int_num_1 = 19
int_num_2 = 0
bina_1 = '00010011'
bina_2 = '00000000'
trame = '0000000000010011'
result = 19

int_num_1 = 38
int_num_2 = 0
bina_1 = '00100110'
bina_2 = '00000000'
trame = '0000000000100110'
result = 38

EDIT: Use the answer of @juanpa.arrivillaga (much faster). Take this answer to understand how it's work!

solid answer, if I knew Python syntax better I'd have taken this route

Collectives�� on Stack Overflow

How can I convert 8 bit array to 16 bit array in Python?

2 Answers 2

3 Comments

1 Comment

Hot Network Questions

Collectives��� on Stack Overflow

2 Answers 2

3 Comments

1 Comment

Related

Collectives�� on Stack Overflow