Spresense camera with GPS overlay
|1||×||Spresense Main Board||178-3376|
|1||×||Spresense Extension Board||178-3377|
|1||×||ST7735 TFT display|
This example uses Sony’s Spresense board with the new multicore microcontroller CXD5602GG featuring 6 ARM Cortex-M4 cores and an integrated GNSS receiver among other features. In this project the GNSS module is used to acquire the current position simultaneously as the camera is streaming video to the display. To view the video a 160×128 TFT display with a ST7735 SPI display driver is used. This example is written using the Spresense board package in the Arduino IDE.
The display driver used is initially the standard driver provided as a library in the Arduino IDE that has been ported for Spresense. It also got a new function enabling most of the functions to operate on a framebuffer instead of directly writing operation directly to the display. This method saves a lot of overhead writes over the SPI bus and in the end increases the frame rate from 6-7 fps to +20 fps. Using this framebuffer method removes all flickering that can occur when writing shapes and text to the display which are overwritten with a new image every frame.
Image: Display connection schematic.
The camera is initiated during the setup phase to use two buffers. Using two buffers instead of one has shown to increase the frame rate. The tradeoff is of course that twice amount of memory is used. Each time a frame has been captured by the camera a callback function will be called in the sketch. This callback is specified in the startStreaming() function. Pushing the raw captured buffer directly to the display have show a distorted picture so a few things have to be done to get the captured data in a compatible format.
First, the captured data is stored in YUV422, so this has to be converted to 16 bit RGB format. Function for this are provided by the camera library: convertPixFormat(CAM_IMAGE_PIX_FMT_RGB565);
Secondly, the captured image of 340×240 is too large for the 160×128 pixel display. Instead of rescaling the image I have taken a shortcut and just re arranged the pixel data for the new smaller size, this means that the exceeding pixels are not shown.
Thirdly, seems that dumping this data to the display still looks funny. Could this be an issue with endianness? Simply swapping the endianness for the uint16_t type does the job.
Fourthly, the initial display driver configures the display for RGB, changing this to BGR will give the result we’re after. Now the stream from the camera comes out just like it should!
The GNSS library is polled. It has to be initiated and started but after that it is up to the user to ask for information. There are a few points that are important to get good result.
During the setup phase, it is necessary to specify what satellite types to acquire signals from. This depends on where on the world you are at the moment. The table below lists where the supported systems are available.
|GPS||GLONASS||QZSS-L1C/A||WAAS||QZSS-L1S||95% Accuracy||Effective place|
|x||<5m||under open sky|
|x||x||<5m||in East Asia and Oceania|
|x||x||<7.8m||in the city|
|x||x||x||<7.8m||in the city of East Asia and Oceania|
|x||x||<2m||in North America|
Three libraries are used in this project.
GNSS and Camera are provided in the Spresense package so no installation is needed.
But the TFT drivers have to be downloaded and installed into the Arduino IDE build system. One way of doing this is simply to download the package directly from GitHub:
In the Arduino IDE click [Sketch->Include library->Add .zip library]. Select the zip file you just downloaded. This method will work, if you’re lucky. If you already have a TFT library installed you might run into problems since the currently installed drivers might be selected prior the downloaded ones by the Arduino IDE.
Renaming the folder where the new drivers are stored might help the Arduino IDE locating the files.
In Ubuntu renaming the folder could be done with the following command:
mv -v ~/Arduino/libraries/TE-KarlKomierowski-TFT-*/ ~/Arduino/libraries/TFT
Post author: KarlKomierowski