This class is intended to execute a tracking analysis on an image sequence. It is initialized with the path to the folder where images are stored. This class can be used inside an application by creating a new thread and calling the method startProcess. The tracking workflow can be changed by reimplementing the method startProcess and imageProcessing. This class can also be used as a library by constructing Tracking tracking("", "") to access the public class members and builds a new workflow. More...

Inheritance diagram for Tracking:

Public Slots
virtual void	startProcess ()
	Initializes a tracking analysis and triggers its execution. Constructs from the path to a folder where the image sequence is stored, detects the image format and processes the first image to detect objects. First, it computes the background by averaging images from the sequence, then it subtracts the background from the first image and then binarizes the resulting image. It detects the objects by contour analysis and extracts features by computing the object moments. It triggers the analysis of the second image of the sequence.

void	updatingParameters (const QMap< QString, QString > &)
	Updates the private members from the external parameters. This function links the tracking logic with the graphical user interface. More...

virtual void	imageProcessing ()
	Processes an image from an images sequence and tracks and matchs objects according to the previous image in the sequence. Takes a new image from the image sequence, substracts the background, binarises the image and crops according to the defined region of interest. Detects all the objects in the image and extracts the object features. Then matches detected objects with objects from the previous frame. This function emits a signal to display the images in the user interface.

Signals
void	progress (int) const
	Emitted when an image is processed. More...

void	backgroundProgress (int) const
	Emitted when an image to compute the background is processed. More...

void	finishedProcessFrame () const
	Emitted when the first image has been processed to trigger the starting of the analysis.

void	finished () const
	Emitted when all images have been processed.

void	forceFinished (QString message) const
	Emitted when a crash occurs during the analysis.

void	statistic (long long int time) const
	Emitted at the end of the analysis.

Public Member Functions
	Tracking (string path, string background, int startImage=0, int stopImage=-1)
	Constructs the tracking object from a path to an image sequence and an optional path to a background image. More...

	Tracking (string path, UMat background, int startImage=0, int stopImage=-1)
	Constructs the tracking object from a list of path, a background image and a range of image. More...

	Tracking (const Tracking &)=delete

Tracking &	operator= (const Tracking &)=delete

	~Tracking ()
	Destructs the tracking object.

Point2d	curvatureCenter (const Point3d &tail, const Point3d &head) const
	Computes the center of the curvature, defined as the intersection of the minor axis of the head ellipse with the minor axis of the tail ellipse of the object. More...

double	curvature (Point2d center, const Mat &image) const
	Computes the radius of curvature of the object defined as the inverse of the mean distance between each pixel of the object, and the center of the curvature. The center of curvature is defined as the intersection of the two minor axes of the head and tail ellipse. More...

double	divide (double a, double b) const
	Computes the float division and handle the division by 0 by returning 0. More...

bool	objectDirection (const UMat &image, vector< double > &information) const
	Computes the direction of the object from the object parameter (coordinate of the center of mass and orientation). To use this function, the object major axis has to be the horizontal axis of the image. Therefore, it is necessary to rotate the image before calling objectDirection. More...

vector< double >	objectInformation (const UMat &image) const
	Computes the equivalent ellipse of an object by computing the moments of the image. If the image is a circle, return nan as the orientation. More...

vector< Point3d >	reassignment (const vector< Point3d > &past, const vector< Point3d > &input, const vector< int > &assignment) const
	Sorts a vector accordingly to a new set of indexes. The sorted vector at index i is the input at index assignment[i]. More...

UMat	backgroundExtraction (VideoReader &video, int n, const int method, const int registrationMethod) const
	Computes the background of an image sequence by averaging n images. More...

void	registration (UMat imageReference, UMat &frame, int method) const
	Register two images. To speed-up, the registration is made in a pyramidal way: the images are downsampled then registered to have a an approximate transformation then upslampled to have the precise transformation. More...

void	binarisation (UMat &frame, char backgroundColor, int value) const
	Binarizes the image by thresholding. More...

vector< vector< Point3d > >	objectPosition (const UMat &frame, int minSize, int maxSize) const
	Computes the positions of the objects and extracts the object's features. More...

vector< int >	costFunc (const vector< vector< Point3d > > &prevPos, const vector< vector< Point3d > > &pos, double LENGHT, double ANGLE, double LO, double AREA, double PERIMETER) const
	Computes a cost function and use a global optimization association to associate targets between images. Method adapted from: "An effective and robust method for Tracking multiple fish in video image based on fish head detection" YQ Chen et al. Uses the Hungarian method implemented by Cong Ma, 2016 "https://github.com/mcximing/hungarian-algorithm-cpp" adapted from the Matlab implementation by Markus Buehren "https://fr.mathworks.com/matlabcentral/fileexchange/6543-functions-for-the-rectangular-assignment-problem". More...

void	cleaning (const vector< int > &occluded, vector< int > &lostCounter, vector< int > &id, vector< vector< Point3d > > &input, double param_maximalTime) const
	Cleans the data if an object is lost more than a certain time. More...

vector< Point3d >	prevision (vector< Point3d > past, vector< Point3d > present) const
	Predicts the next position of an object from the previous position. More...

vector< int >	findOcclusion (vector< int > assignment) const
	Finds the objects that are occluded during the tracking. More...

Static Public Member Functions
static double	modul (double angle)
	Computes the usual mathematical modulo 2*PI of an angle. More...

static double	angleDifference (double alpha, double beta)
	Computes the least difference between two angles, alpha - beta. The difference is oriented in the trigonometric convention. More...

static bool	exportTrackingResult (const QString path, QSqlDatabase db)
	Exports the tracking data from the database to a text file. More...

static bool	importTrackingResult (const QString path, QSqlDatabase db)
	Imports the tracking data from a text file to the database. More...

Public Attributes
UMat	m_binaryFrame

UMat	m_visuFrame

vector< vector< Point3d > >	m_out

vector< vector< Point3d > >	m_outPrev

Private Attributes
QElapsedTimer *	timer

bool	statusBinarisation

string	m_path

string	m_backgroundPath

UMat	m_background

int	m_displayTime

QString	m_savingPath

VideoReader *	video

int	m_im

QString	m_error

int	m_startImage

int	m_stopImage

Rect	m_ROI

QFile	m_logFile

vector< cv::String >	m_files

vector< int >	m_id

vector< int >	m_lost

int	m_idMax

int	param_n

int	param_maxArea

int	param_minArea

int	param_spot

double	param_len

double	param_angle

double	param_area

double	param_perimeter

double	param_lo

double	param_to

int	param_thresh

double	param_nBackground

int	param_methodBackground

int	param_methodRegistrationBackground

int	param_registration

int	param_x1

int	param_y1

int	param_x2

int	param_y2

int	param_kernelSize

int	param_kernelType

int	param_morphOperation

QMap< QString, QString >	parameters

Detailed Description

This class is intended to execute a tracking analysis on an image sequence. It is initialized with the path to the folder where images are stored. This class can be used inside an application by creating a new thread and calling the method startProcess. The tracking workflow can be changed by reimplementing the method startProcess and imageProcessing. This class can also be used as a library by constructing Tracking tracking("", "") to access the public class members and builds a new workflow.

Author: Benjamin Gallois

Version

Revision: 4.0

Contact: gallo.nosp@m.is.b.nosp@m.enjam.nosp@m.in08.nosp@m.@gmai.nosp@m.l.co.nosp@m.m

Constructor & Destructor Documentation

◆ Tracking() [1/2]

Tracking::Tracking	(	string	path,
		string	backgroundPath,
		int	startImage = `0`,
		int	stopImage = `-1`
	)

Constructs the tracking object from a path to an image sequence and an optional path to a background image.

Parameters

[in]	path	The path to a folder where images are stocked.
[in]	backgroundPath	The path to a background image.
[in]	startImage	Index of the beginning image.
[in]	stopImage	Index of the ending image.

◆ Tracking() [2/2]

Tracking::Tracking	(	string	path,
		UMat	background,
		int	startImage = `0`,
		int	stopImage = `-1`
	)

Constructs the tracking object from a list of path, a background image and a range of image.

Parameters

[in]	imagePath	List of path to the images.
[in]	background	A background image.
[in]	startImage	Index of the beginning image.
[in]	stopImage	Index of the ending image.

Member Function Documentation

◆ angleDifference()

double Tracking::angleDifference	(	double	alpha,
		double	beta
	)

static

Computes the least difference between two angles, alpha - beta. The difference is oriented in the trigonometric convention.

Parameters

[in]	alpha	Input angle.
[in]	beta	Input angle.

Returns: Least difference.

◆ backgroundExtraction()

UMat Tracking::backgroundExtraction	(	VideoReader &	video,
		int	n,
		const int	method,
		const int	registrationMethod
	)		const

Computes the background of an image sequence by averaging n images.

Parameters

[in]	files	List of paths to each image in the images sequence.
[in]	n	The number of images to average to computes the background.
[in]	Method	0: minimal projection, 1: maximal projection, 2: average projection.

Returns: The background image. TO DO: currently opening all the frames, to speed-up the process and if step is large can skip frames by replacing nextImage by getImage.

◆ backgroundProgress

void Tracking::backgroundProgress ( int ) const

signal

Emitted when an image to compute the background is processed.

Parameters

int	The number of processed image.

◆ binarisation()

void Tracking::binarisation	(	UMat &	frame,
		char	backgroundColor,
		int	value
	)		const

Binarizes the image by thresholding.

Parameters

[in,out]	frame	The image to binarize.
[in]	backgroundColor	If equals to 'w' the thresholded image will be inverted, if equal to 'b' it will not be inverted.
[in]	value	The value at which to threshold the image.

◆ cleaning()

void Tracking::cleaning	(	const vector< int > &	occluded,
		vector< int > &	lostCounter,
		vector< int > &	id,
		vector< vector< Point3d > > &	input,
		double	param_maximalTime
	)		const

Cleans the data if an object is lost more than a certain time.

Parameters

[in]	occluded	The vector with the index of object missing in the current image.
[in]	input	The vector at current image of size m <= n to be sorted.
[in]	lostCounter	The vector with the number of times each objects are lost consecutively.
[in]	id	The vector with the id of the objects.
[in]	param_maximalTime

Returns: The sorted vector.

◆ costFunc()

vector< int > Tracking::costFunc	(	const vector< vector< Point3d > > &	prevPos,
		const vector< vector< Point3d > > &	pos,
		double	LENGTH,
		double	ANGLE,
		double	LO,
		double	AREA,
		double	PERIMETER
	)		const

Computes a cost function and use a global optimization association to associate targets between images. Method adapted from: "An effective and robust method for Tracking multiple fish in video image based on fish head detection" YQ Chen et al. Uses the Hungarian method implemented by Cong Ma, 2016 "https://github.com/mcximing/hungarian-algorithm-cpp" adapted from the Matlab implementation by Markus Buehren "https://fr.mathworks.com/matlabcentral/fileexchange/6543-functions-for-the-rectangular-assignment-problem".

Parameters

[in]	prevPos	The vector of objects parameters at the previous image.
[in]	pos	The vector of objects parameters at the current image that we want to sort in order to conserve objects identity.
[in]	LENGTH	The typical displacement of an object in pixels.
[in]	ANGLE	The typical reorientation angle in radians.
[in]	LO	The maximal assignment distance in pixels.

Returns: The assignment vector containing the new index position to sort the pos vector.

◆ curvature()

double Tracking::curvature	(	Point2d	center,
		const Mat &	image
	)		const

Computes the radius of curvature of the object defined as the inverse of the mean distance between each pixel of the object, and the center of the curvature. The center of curvature is defined as the intersection of the two minor axes of the head and tail ellipse.

Parameters

[in]	center	Center of the curvature.
[in]	image	Binary image CV_8U.

Returns: Radius of curvature.

◆ curvatureCenter()

Point2d Tracking::curvatureCenter	(	const Point3d &	tail,
		const Point3d &	head
	)		const

Computes the center of the curvature, defined as the intersection of the minor axis of the head ellipse with the minor axis of the tail ellipse of the object.

Parameters

[in]	tail	The parameters of the tail ellipse: coordinate and direction of the major axis.
[in]	head	The parameters of the head ellipse: coordinate and direction of the major axis.

Returns: Coordinate of the curvature center.

◆ divide()

double Tracking::divide	(	double	a,
		double	b
	)		const

Computes the float division and handle the division by 0 by returning 0.

Parameters

[in]	a	Dividend.
[in]	a	Divisor.

Returns: Division result, 0 if b = 0.

◆ exportTrackingResult()

bool Tracking::exportTrackingResult	(	const QString	path,
		QSqlDatabase	db
	)

static

Exports the tracking data from the database to a text file.

Parameters

[in]	path	The path to a folder where to write the text file.
[in]	db	The database where tracking results are stored already opened.

◆ findOcclusion()

vector< int > Tracking::findOcclusion ( vector< int > assignment ) const

Finds the objects that are occluded during the tracking.

Parameters

[in] assignment The vector with the new indexes that will be used to sort the input vector.

Returns: The vector with the indexes of occluded objects.

◆ importTrackingResult()

bool Tracking::importTrackingResult	(	const QString	path,
		QSqlDatabase	db
	)

static

Imports the tracking data from a text file to the database.

Parameters

[in]	path	The path to a folder where to write the text file.
[in]	db	The database where tracking results are stored already opened.

◆ modul()

double Tracking::modul ( double angle )

static

Computes the usual mathematical modulo 2*PI of an angle.

Parameters

[in] angle Input angle.

Returns: Output angle.

◆ objectDirection()

bool Tracking::objectDirection	(	const UMat &	image,
		vector< double > &	information
	)		const

Computes the direction of the object from the object parameter (coordinate of the center of mass and orientation). To use this function, the object major axis has to be the horizontal axis of the image. Therefore, it is necessary to rotate the image before calling objectDirection.

Parameters

[in]	image	Binary image CV_8U.
[in,out]	information	The parameters of the object (x coordinate, y coordinate, orientation).

Returns: True if the direction angle is the orientation angle. False if the direction angle is the orientation angle plus pi.

◆ objectInformation()

vector< double > Tracking::objectInformation ( const UMat & image ) const

Computes the equivalent ellipse of an object by computing the moments of the image. If the image is a circle, return nan as the orientation.

Parameters

[in] image Binary image CV_8U.

Returns: The equivalent ellipse parameters: the object center of mass coordinate and its orientation.

Note: : This function computes the object orientation, not its direction.

◆ objectPosition()

vector< vector< Point3d > > Tracking::objectPosition	(	const UMat &	frame,
		int	minSize,
		int	maxSize
	)		const

Computes the positions of the objects and extracts the object's features.

Parameters

[in]	frame	Binary image CV_8U.
[in]	minSize	The minimal size of an object.
[in]	maxSize	The maximal size of an object.

Returns: All the parameters of all the objects formated as follows: one vector, inside of this vector, four vectors for parameters of the head, tail, body and features with number of object size. { { Point(xHead, yHead, thetaHead), ...}, Point({xTail, yTail, thetaHead), ...}, {Point(xBody, yBody, thetaBody), ...}, {Point(curvature, 0, 0), ...}}

◆ prevision()

vector< Point3d > Tracking::prevision	(	vector< Point3d >	past,
		vector< Point3d >	present
	)		const

Predicts the next position of an object from the previous position.

Parameters

past	The previous position parameters.
present	The current position parameters.

Returns: The predicted positions.

◆ progress

void Tracking::progress ( int ) const

signal

Emitted when an image is processed.

Parameters

int	Index of the processed image.

◆ reassignment()

vector< Point3d > Tracking::reassignment	(	const vector< Point3d > &	past,
		const vector< Point3d > &	input,
		const vector< int > &	assignment
	)		const

Sorts a vector accordingly to a new set of indexes. The sorted vector at index i is the input at index assignment[i].

Parameters

[in]	past	The vector at the previous image.
[in]	input	The vector at current image of size m <= n to be sorted.
[in]	assignment	The vector with the new indexes that will be used to sort the input vector.
[in]	lostCounter	The vector with the number of times each objects are lost consecutively.
[in]	id	The vector with the id of the objects.

Returns: The sorted vector.

◆ registration()

void Tracking::registration	(	UMat	imageReference,
		UMat &	frame,
		int	method
	)		const

Register two images. To speed-up, the registration is made in a pyramidal way: the images are downsampled then registered to have a an approximate transformation then upslampled to have the precise transformation.

Parameters

[in]	imageReference	The reference image for the registration.
[in,out]	frame	The image to register.
[in]	method	The method of registration: 0 = simple (phase correlation), 1 = ECC, 2 = Features based.

◆ updatingParameters

void Tracking::updatingParameters ( const QMap< QString, QString > & parameterList )

slot

Updates the private members from the external parameters. This function links the tracking logic with the graphical user interface.

Parameters

[in] parameterList The list of all the parameters used in the tracking.

Member Data Documentation

◆ m_background

UMat Tracking::m_background

private

Background image CV_8U.

◆ m_backgroundPath

string Tracking::m_backgroundPath

private

Path to an image background.

◆ m_binaryFrame

UMat Tracking::m_binaryFrame

Binary image CV_8U

◆ m_displayTime

int Tracking::m_displayTime

private

Binary image CV_8U.

◆ m_error

QString Tracking::m_error

private

QString containing unreadable images.

◆ m_files

vector<cv::String> Tracking::m_files

private

Vector containing the path for each image in the images sequence.

◆ m_id

vector<int> Tracking::m_id

private

Vector containing the objets Id.

◆ m_im

int Tracking::m_im

private

Index of the next image to process in the m_files list.

◆ m_logFile

QFile Tracking::m_logFile

private

Path to the file where to save logs.

◆ m_lost

vector<int> Tracking::m_lost

private

Vector containing the lost objects.

◆ m_out

vector<vector<Point3d> > Tracking::m_out

Objects information at iteration minus one

◆ m_outPrev

vector<vector<Point3d> > Tracking::m_outPrev

Objects information at current iteration

◆ m_path

string Tracking::m_path

private

Path to an image sequence.

◆ m_ROI

Rect Tracking::m_ROI

private

Rectangular region of interest.

◆ m_savingPath

QString Tracking::m_savingPath

private

Folder where to save files.

◆ m_startImage

int Tracking::m_startImage

private

Index of the next image to process in the m_files list.

◆ m_stopImage

int Tracking::m_stopImage

private

Index of the next image to process in the m_files list.

◆ m_visuFrame

UMat Tracking::m_visuFrame

Image 8 bit CV_8U

◆ param_angle

double Tracking::param_angle

private

Maximal change in direction of an object between two images.

◆ param_area

double Tracking::param_area

private

Normalization area.

◆ param_kernelSize

int Tracking::param_kernelSize

private

Size of the kernel of the morphological operation.

◆ param_kernelType

int Tracking::param_kernelType

private

Type of the kernel of the morphological operation.

◆ param_len

double Tracking::param_len

private

Maximal length travelled by an object between two images.

◆ param_lo

double Tracking::param_lo

private

Maximal distance allowed by an object to travel during an occlusion event.

◆ param_maxArea

int Tracking::param_maxArea

private

Maximal area of an object.

◆ param_methodBackground

int Tracking::param_methodBackground

private

The method used to compute the background.

◆ param_methodRegistrationBackground

int Tracking::param_methodRegistrationBackground

private

The method used to register the images for the background.

◆ param_minArea

int Tracking::param_minArea

private

Minimal area of an object.

◆ param_morphOperation

int Tracking::param_morphOperation

private

Type of the morphological operation.

◆ param_n

int Tracking::param_n

private

Number of objects.

◆ param_nBackground

double Tracking::param_nBackground

private

Number of images to average to compute the background.

◆ param_perimeter

double Tracking::param_perimeter

private

Normalization perimeter.

◆ param_registration

int Tracking::param_registration

private

Method of registration.

◆ param_spot

int Tracking::param_spot

private

Which spot parameters are used to computes the cost function. 0: head, 1: tail, 2: body.

◆ param_thresh

int Tracking::param_thresh

private

Value of the threshold to binarize the image.

◆ param_to

double Tracking::param_to

private

Maximal time.

◆ param_x1

int Tracking::param_x1

private

Top x corner of the region of interest.

◆ param_x2

int Tracking::param_x2

private

Bottom x corner of the region of interest.

◆ param_y1

int Tracking::param_y1

private

Top y corner of the region of interest.

◆ param_y2

int Tracking::param_y2

private

Bottom y corner of the region of interest.

◆ parameters

QMap<QString, QString> Tracking::parameters

private

map of all the parameters for the tracking.

◆ statusBinarisation

bool Tracking::statusBinarisation

private

True if wite objects on dark background, flase otherwise.

◆ timer

QElapsedTimer* Tracking::timer

private

Timer that measured the time during the analysis execution.

The documentation for this class was generated from the following files:

src/tracking.h
src/tracking.cpp

Public Slots

Signals

Public Member Functions

Static Public Member Functions

Public Attributes

Private Attributes

Detailed Description

Constructor & Destructor Documentation

◆ Tracking() [1/2]

◆ Tracking() [2/2]

Member Function Documentation

◆ angleDifference()

◆ backgroundExtraction()

◆ backgroundProgress

◆ binarisation()

◆ cleaning()

◆ costFunc()

◆ curvature()

◆ curvatureCenter()

◆ divide()

◆ exportTrackingResult()

◆ findOcclusion()

◆ importTrackingResult()

◆ modul()

◆ objectDirection()

◆ objectInformation()

◆ objectPosition()

◆ prevision()

◆ progress

◆ reassignment()

◆ registration()

◆ updatingParameters

Member Data Documentation

◆ m_background

◆ m_backgroundPath

◆ m_binaryFrame

◆ m_displayTime

◆ m_error

◆ m_files

◆ m_id

◆ m_im

◆ m_logFile

◆ m_lost

◆ m_out

◆ m_outPrev

◆ m_path

◆ m_ROI

◆ m_savingPath

◆ m_startImage

◆ m_stopImage

◆ m_visuFrame

◆ param_angle

◆ param_area

◆ param_kernelSize

◆ param_kernelType

◆ param_len

◆ param_lo

◆ param_maxArea

◆ param_methodBackground

◆ param_methodRegistrationBackground

◆ param_minArea

◆ param_morphOperation

◆ param_n

◆ param_nBackground

◆ param_perimeter

◆ param_registration

◆ param_spot

◆ param_thresh

◆ param_to

◆ param_x1

◆ param_x2

◆ param_y1

◆ param_y2

◆ parameters

◆ statusBinarisation

◆ timer