Skip to main content

OpenCV with VizionSDK

Introduction​

This article will introduce how to display with OpenCV and control the camera using VizionSDK.
You can download the Sample Code here.

OpenCV Version​

These examples use OpenCV version 4.9 or newer.

Load OpenCV and VizionSDK​

Import the OpenCV and VizionSDK libraries to support the following examples.

#include "VizionSDK.h"
#include <opencv2/highgui.hpp>
#include <opencv2/opencv.hpp>

Start the device​

Discover the device and initial it. Then, set the device format to the minimum size for MJPG format.

// initial the camera
int dev_idx = 0;
auto cam = VxInitialCameraDevice(dev_idx);
VxOpen(cam);

// set format to min size for mjpg format
std::vector<VxFormat> fmt_list;
VxGetFormatList(cam, fmt_list);
int min_width = 1920;
int min_height = 1080;
VxFormat min_fmt;
for (auto fmt : fmt_list) {
    // find MJPG smallest size format
    if (fmt.format == VX_IMAGE_FORMAT::MJPG &&
        fmt.width * fmt.height < min_width * min_height) {
        min_width = fmt.width;
        min_height = fmt.height;
        min_fmt = fmt;
    }
}
 
if (VxSetFormat(cam, min_fmt) == 0) {
    std::cout << "Set Capture Format Success!" << std::endl;
} else {
    std::cout << "Set Capture Format Failed!" << std::endl;
}

Display the image​

Use VizionSDK VxGetImage function to retrieve the image data and convert it into OpenCV mat to display in the window.

This example demonstrates how to retrieve the format, set it to a specific format, and display the image in a window.

// start streaming
VxStartStreaming(cam);

// get the min format image
uint8_t* raw_data = new uint8_t[3840 * 2160 * 2];
int raw_size = 0;
VxGetImage(cam, raw_data, &raw_size, 2500);

// retrieve the data into the mat array and display with cv2.imshow()
cv::Mat matImg;
matImg = cv::imdecode(cv::Mat(1, raw_size, CV_8UC1, raw_data), cv::IMREAD_COLOR);
cv::imshow("MJPG Image", matImg);
cv::waitKey(0);
cv::destroyAllWindows();

VxStopStreaming(cam);
VxClose(cam);

OpenCV VideoCapture with VizionSDK​

Adjust the image properties using VizionSDK and capture frames with OpenCV's VideoCapture.

This example demonstrates how to open a video window displaying a live camera stream. While the stream is active, you can press keys to increase or decrease the brightness.

  1. Initialize VideoCapture with opencv:

    // Capture image with opencv VideoCapture API
    cv::VideoCapture cap;
    #ifdef _WIN32
        cap = cv::VideoCapture(dev_idx, cv::CAP_DSHOW);
    #else
        cap = cv::VideoCapture(dev_idx);
    #endif
    cap.set(cv::CAP_PROP_FRAME_WIDTH, min_width);
    cap.set(cv::CAP_PROP_FRAME_HEIGHT, min_height);

  2. Retrieve the current brightness value:

    The code below shows how to handle brightness, and you can follow the same structure for other properties, such as:

    • ISP_EHDR_EXPOSURE_MAX_TIME

    • ISP_IMAGE_EXPOSURE_TIME

    • ISP_IMAGE_GAIN

    For more image property examples (e.g., eHDR), refer to the full example here

    // Get the brightness range
    int min_brightness, max_brightness, step, def;
    VxGetISPImageProcessingRange(cam, VX_ISP_IMAGE_PROPERTIES::ISP_IMAGE_BRIGHTNESS, min_brightness,
                                max_brightness, step, def);
    std::cout << "Brightness min: " << min_brightness << ", max: " << max_brightness
            << ", step: " << step << ", default: " << def << std::endl;

    // Get the brightness
    int brightness, flag;
    VxGetISPImageProcessing(cam, VX_ISP_IMAGE_PROPERTIES::ISP_IMAGE_BRIGHTNESS, brightness, flag);
    std::cout << "Intial ISP brightness: " << brightness << ", flag: " << flag << std::endl;

  3. Capture and display video in a window. Use the following keys during playback:

    • u: Increase brightness
    • d: Decrease brightness
    • q: Stop playing video and exit the program.

    The draw_label helper function (included in the sample code) is used to overlay text labels directly onto the video display window.

    cv::Mat frame;
    while (true) {
        bool ret = cap.read(frame);
        if (ret) {
            std::string brightness_label = "brightness: " + std::to_string(brightness);
            std::string label_str = "u: increase, d: decrease, q: exit";
            draw_label(frame, brightness_label, cv::Point(20, 20));
            draw_label(frame, label_str, cv::Point(20, 40));
            cv::imshow("Adjust brightness", frame);
            int key = cv::waitKey(1) & 0xFF;
            if (key == 'u') {
                if (brightness < max_brightness) {
                    brightness = brightness + step;
                }
                VxSetISPImageProcessing(cam, VX_ISP_IMAGE_PROPERTIES::ISP_IMAGE_BRIGHTNESS,
                                        brightness);
                std::cout << "Set brightness to: " << brightness << std::endl;
            }
            if (key == 'd') {
                if (brightness > min_brightness) {
                    brightness = brightness - step;
                }
                VxSetISPImageProcessing(cam, VX_ISP_IMAGE_PROPERTIES::ISP_IMAGE_BRIGHTNESS,
                                        brightness);
                std::cout << "Set brightness to: " << brightness << std::endl;
            }
            if (key == 'q') {
                break;
            }
        } else {
            std::cout << "Can't receive frame (stream end?). Exiting ..." << std::endl;
            break;
        }
    }