Table of Contents

Image Acquisition

Image acquisition is known as the process of taking images by means of the integrated image sensor and making the images accessible to the processor. This camera uses an FPGA to perform this action, i.e. apply necessary signals and timing to the image sensor, reading image data from the sensor and transferring data to the system processor's memory. Image transfers to system memory is always done using DMA to offload the processor.

Involved components

Besides the FPGA which carries out necessary hardware level tasks a Linux kernel driver module called camlink.ko plays an important role in the image acquisition process. This driver interfaces the FPGA and is responsible for image frame buffer handling and DMA programming. On top of this driver an image acquisition API is provided as convenient interface to user applications. This API encapsules sensor and camera hardware specific details and provides a high level abstraction for the process of image acquisition.

Buffer management

One important point in image acquisition is buffer management as data rates are very high in imaging. The main question is where image data is stored and who is responsible for allocating buffer memory. Basically, there are two possibilities. Either the driver is responsible for buffer allocation or the user application itself. This camera is able to support both methods although driver based buffer handling is strongly recommended for several reasons:

Buffer management

The image above illustrates driver level buffer handling. During image acquisition setup the user can specify the total amount of memory which should be allocated. The driver splits this memory into individual image buffers each capable of holding exactly one image. Depending on the image size more or less image buffers can be created. The user has to estimate how may image buffers will be necessary for a given application to work properly.

The driver initiates DMA transfers directly into it's image buffers without any additional copy. To make image data available for the user a memory mapping of image buffers into user address space is created. This is possible due to the virtual memory management concepts under Linux.

After successful image acquisition the user gets a pointer to the mapped image buffer. He can use this pointer for read and write operations, i.e. image data can be directly manipulated in-place. To guarantee that an image buffer doesn't get overwritten by subsequent acquisitions the driver locks this buffer and grants exclusive access for the user after passing over the pointer for this buffer. The lock is removed when the user frees the buffer by explicitly calling an API function to release the image buffer which makes it available again for subsequent acquisitions.

Acquisition modes

The camera supports different modes for image acquisition depending on the user application's needs:

Stop mode

In this mode the camera stops any image acquisition activities and goes to an idle state.

Single shot

In single shot mode every image has to be explicitly requested by setting a trigger signal. The trigger can be raised by active high or active low external input signals or by software using an appropriate API function.

Once a trigger is raised it doesn't immediately lead to an image acquisition on the sensor. Instead an adjustable trigger delay is introduced before exposure of the sensor is initiated. This is very useful in real world applications whenever external trigger signals (e.g. generated by PLC or light barrier) doesn't exactly match the required trigger moment (or position).

Illumination control is also derived from trigger signal. After an adjustable delay illumination (which may be an external light/flash or the camera internal LEDs) is turned on for a given illumination period which is adjustable, too. See the following figure for illustration.

Image processing may start immediately after the image was transferred to the processor's main memory.

Single shot acquisition mode

Free run

In free run mode the image sensor takes one image after the other in an endless loop. No trigger signal is used in this mode. The camera starts acquiring images immediately after this acquisition mode is set to ACQ_MODE_FREE_RUN.

The resulting frame rate is given by the sum of exposure time and readout time, hence, frame rate and exposure time cannot be adjusted independently from each other. Please, see the following image for an illustration of the free run mode.

Since image transfer via DMA is done by the kernel driver independently from application, image processing may happen concurrently with image acquisition. The buffer management of the kernel driver acts as a large FIFO buffer between acquisition and processing. The application has to guarantee that the mean processing time is less than or equal to the frame rate.

Free run acquisition mode

Select shot

Select shot mode is a combination of free run and single shot mode. Basically, the image sensor is operated in free run mode, i.e. images are permanently acquired from the sensor without having to apply any trigger signal. The main difference to free run mode is that images are not transferred to main memory unless the user application explicitly requests them by calling the API function GetImage(). Similar to single shot mode the user gets images on demand.

An advantage of using select shot mode is that the time it takes to get an image is less in average than for single shot mode because exposure time may be hidden. In single shot mode exposure time is always included in the total image acquisition time.

Free run slave

The most versatile mode for image acquisition is free run slave. In this mode the image sensor is completely under control of the FPGA, i.e. the FPGA generates all necessary timing signals to control image exposure and readout.

Free run slave is the only mode where pixel exposure and readout can happen in parallel. Hence, highest frame rates can be achieved in free run slave mode by completely overlapping exposure time with readout time of the previous frame. See the following illustration for this mode.

In free run slave mode exposure time and frame rate can be adjusted independently. This is very important in practical applications that have to guarantee a certain and exact frame rate (e.g. scanning systems). When overlapping exposure and read out the maximum frame rate is given by the read out time, which in turn depends on the selected camera window and the amount of data to read, respectively. Of course, exposure time is limited to the read out time when running at maximum frame rate. Higher exposure times require to reduce frame rate to the reciprocal value of exposure time.

Free run slave acquisition mode

Please keep in mind that not all image sensor variants available for the FESTO camera support concurrent exposure and readout. See the sensor documentation for details.

Example

A simple example acquiring images from the camera can be found → here