Skip to main content

OpenCV with VizionSDK

Introduction​

This article will introduce how to display with OpenCV and control the camera using VizionSDK.
You can download the Sample Code here.

OpenCV Version​

These examples use OpenCV version 4.9 or newer.

Load OpenCV and VizionSDK​

Import the OpenCV and VizionSDK libraries to support the following examples.

#include "VizionSDK.h"
#include <opencv2/highgui.hpp>
#include <opencv2/opencv.hpp>

Start the device​

Discover and initialize the device. Then, set the device format to the minimum resolution supported for either MJPG or UYVY format.
By default, MJPG is selected. You can choose UYVY if needed.

// initial the camera
int dev_idx = 0;
auto cam = VxInitialCameraDevice(dev_idx);
VxOpen(cam);

// set format to 640*480 for UYVY or MJPG format
VX_IMAGE_FORMAT selectFormat;
std::cout << "Choose a format: \n1. UYVY\n2. MJPG(Default)" << std::endl;
std::cout << "Please enter format(1 or 2): ";
int fmtId;
std::cin >> fmtId;

if (fmtId == 1) {
    selectFormat = VX_IMAGE_FORMAT::UYVY;
} else {
    selectFormat = VX_IMAGE_FORMAT::MJPG;
}

std::vector<VxFormat> fmt_list;
VxGetFormatList(cam, fmt_list);
int cap_width = 640;
int cap_height = 480;
VxFormat cap_fmt;
for (auto fmt : fmt_list) {
    // find UYVY 640*480 format
    if (fmt.format == selectFormat && fmt.width * fmt.height == cap_width * cap_height) {
        cap_width = fmt.width;
        cap_height = fmt.height;
        cap_fmt = fmt;
    }
}

if (VxSetFormat(cam, cap_fmt) == 0) {
    std::cout << "Set Capture Format Success!" << std::endl;
} else {
    std::cout << "Set Capture Format Failed!" << std::endl;
}

Display the image​

Use VizionSDK VxGetImage function to retrieve the image data and convert it into OpenCV mat to display in the window.

This example demonstrates how to retrieve the image data, and display it in a window.

// start streaming
VxStartStreaming(cam);

// get the min format image
uint8_t* raw_data = new uint8_t[3840 * 2160 * 2];
int raw_size = 0;
VxGetImage(cam, raw_data, &raw_size, 2500);

// retrieve the data into the mat array and display with cv2.imshow()
cv::Mat matImg;
if (selectFormat == VX_IMAGE_FORMAT::UYVY) {
    matImg = cv::Mat(min_height, min_width, CV_8UC2, raw_data);
    // convert from UYVY to BGR
    cv::Mat bgrImg;
    cv::cvtColor(matImg, bgrImg, cv::COLOR_YUV2BGR_UYVY);
    cv::imshow("UYVY Image", bgrImg);
} else if (selectFormat == VX_IMAGE_FORMAT::MJPG) {
    matImg = cv::imdecode(cv::Mat(1, raw_size, CV_8UC1, raw_data), cv::IMREAD_COLOR);
    cv::imshow("MJPG Image", matImg);
}

cv::waitKey(0);
cv::destroyAllWindows();

VxStopStreaming(cam);
VxClose(cam);

OpenCV VideoCapture with VizionSDK​

Adjust the image properties using VizionSDK and capture frames with OpenCV's VideoCapture.

This example demonstrates how to open a video window displaying a live camera stream. While the stream is active, you can press keys to increase or decrease the brightness.

  1. Initialize VideoCapture with opencv:

    // Capture image with opencv VideoCapture API
    cv::VideoCapture cap;
    #ifdef _WIN32
        cap = cv::VideoCapture(dev_idx, cv::CAP_DSHOW);
    #else
        cap = cv::VideoCapture(dev_idx);
    #endif
    cap.set(cv::CAP_PROP_FRAME_WIDTH, min_width);
    cap.set(cv::CAP_PROP_FRAME_HEIGHT, min_height);

  2. Retrieve the current brightness value:

    The code below shows how to handle brightness, and you can follow the same structure for other properties, such as:

    • ISP_EHDR_EXPOSURE_MAX_TIME

    • ISP_IMAGE_EXPOSURE_TIME

    • ISP_IMAGE_GAIN

    For more image property examples (e.g., eHDR), refer to the full example here

    // Get the brightness range
    int min_brightness, max_brightness, step, def;
    VxGetISPImageProcessingRange(cam, VX_ISP_IMAGE_PROPERTIES::ISP_IMAGE_BRIGHTNESS, min_brightness,
                                max_brightness, step, def);
    std::cout << "Brightness min: " << min_brightness << ", max: " << max_brightness
            << ", step: " << step << ", default: " << def << std::endl;

    // Get the brightness
    int brightness, flag;
    VxGetISPImageProcessing(cam, VX_ISP_IMAGE_PROPERTIES::ISP_IMAGE_BRIGHTNESS, brightness, flag);
    std::cout << "Intial ISP brightness: " << brightness << ", flag: " << flag << std::endl;

  3. Capture and display video in a window. Use the following keys during playback:

    • u: Increase brightness
    • d: Decrease brightness
    • q: Stop playing video and exit the program.

    The draw_label helper function (included in the sample code) is used to overlay text labels directly onto the video display window.

    cv::Mat frame;
    while (true) {
        bool ret = cap.read(frame);
        if (ret) {
            std::string brightness_label = "brightness: " + std::to_string(brightness);
            std::string label_str = "u: increase, d: decrease, q: exit";
            draw_label(frame, brightness_label, cv::Point(20, 20));
            draw_label(frame, label_str, cv::Point(20, 40));
            cv::imshow("Adjust brightness", frame);
            int key = cv::waitKey(1) & 0xFF;
            if (key == 'u') {
                if (brightness < max_brightness) {
                    brightness = brightness + step;
                }
                VxSetISPImageProcessing(cam, VX_ISP_IMAGE_PROPERTIES::ISP_IMAGE_BRIGHTNESS,
                                        brightness);
                std::cout << "Set brightness to: " << brightness << std::endl;
            }
            if (key == 'd') {
                if (brightness > min_brightness) {
                    brightness = brightness - step;
                }
                VxSetISPImageProcessing(cam, VX_ISP_IMAGE_PROPERTIES::ISP_IMAGE_BRIGHTNESS,
                                        brightness);
                std::cout << "Set brightness to: " << brightness << std::endl;
            }
            if (key == 'q') {
                break;
            }
        } else {
            std::cout << "Can't receive frame (stream end?). Exiting ..." << std::endl;
            break;
        }
    }