Student Researcher
University of Szeged, Research Group on Visual Computation
I am currently an Info-bionics Engineering MSc student at the University of Szeged in the Faculty of Science and Informatics and a student researcher in the Research Group on Visual Computation under the supervision of Prof. Zoltan Kato, at the Department of Image Processing and Computer Graphics.
I received my Bachelor of Science degree in Molecular Bionics Engineering from the University of Szeged in January, 2016. Between 2013-2016 I was working as a student researcher in the Biological Research Center of Szeged in the Biological Barriers Research Group under the supervision of Dr. Maria Deli, at the Institute of Biophysics.
University of Szeged, Research Group on Visual Computation
UMR 7039 CNRS and University of Lorraine, Research Center in Automatic Control of Nancy
Biological Research Center of Szeged, Hungarian Academy of Sciences, Biological Barriers Research Group
M.Sc. in Info-bionics Engineering
Institute of Informatics, University of Szeged
B.Sc. in Molecular Bionics Engineering
Institute of Chemistry, University of Szeged
Venue: Czech Technical University, Prague, Czech Republic
Venue: University of Szeged, Hungary
Venue: Eotvos Lorand University, Budapest, Hungary
Venue: University of Szeged, Hungary
The research project focuses on estimating the location and orientation of a moving stereo rig using extracted feature correspondences (e.g. points, lines, regions). The algorithm makes use of other sensors mounted on the same platform (e.g. GPS, IMU), but the final pose estimate is obtained via computer vision techniques. In general, pose estimation has 6 degree of freedom, but in our project we will develop algorithms for important special cases like pose estimation with known vertical direction, where the number of free parameters can be reduced to 4. The estimated camera pose can be used to implement viusal odometry.
Pose estimation is a fundamental building block of various vision applications, e.g. visual odometry, image based localization and navigation, fusion, and augmented reality. Herein, we are interested in absolute pose estimation, which consists of determining the position and orientation of a camera with respect to a 3D world coordinate frame. We propose a novel solution to the gPnL problem with known vertical direction. The vertical direction is typically obtained from an IMU (Inertial Measurement Unit). The only assumption about our generalized camera is that projection rays of a 3D line fall into coplanar subsets yielding a pencil of projection planes. Important special cases of such a camera include stereo and multiview perspective camera systems, perspective camera moving along a trajectory, as well as other non-perspective cameras with central omnidirectional or orthographic projection.
Video 1. Presentation of the extracted lines which we used for our algorithm as a least squares solver to estimate the position of the camera system. The corresponding 3D lines are shown with the same color as the camera. |
For the quantitative evaluation of our generalized pose estimation algorithm with line correspondences, we generated various benchmark datasets of 3D-2D line pairs. Each dataset has 1000 samples. Our method can be used as a least square solver as well as a minimal solver without reformulation. For the minimal case we only need 3 line pairs. The method was quantitatively evaluated on this large synthetic dataset as well as on a real dataset, which confirms its state of the art performance both in terms of quality and computational efficiency.
Video 2. Lidar laser scan for testing our least squares solver pose estimation algorithm with a 3-perspective-1-omnidirectional multi-view camera system. On the Lidar scan red dots are the estimated positions and green dots are the real location of the markers in metric 3D space. |
Video 3. Lidar laser scan for testing our minimal solver pose estimation algorithm solver with a 3-perspective camera system. On the Lidar scan, red dots are the estimated positions of our minimal solver, blue dots are the estimated positions of NP3L [G.H.Lee, ECCV 2016], and green dots are the real location of the markers in the 3D metric space. |
This work is partially supported by: the "Integrated program for training new generation of scientists in the fields of computer science", no EFOP-3.6.3-VEKOP-16-2017-0002; NKFI-6 fund through project K120366; the Agence Universitaire de la Francophonie (AUF) and the Romanian Institute for Atomic Physics (IFA), through the AUF-RO project NETASSIST; the Research & Development Operational Programme for the project "Modernization and Improvement of Technical Infrastructure for Research and Development of J. Selye University in the Fields of Nanotechnology and Intelligent Space", ITMS 26210120042, co-funded by the European Regional Development Fund.
She visited as a student researcher at Research Center in Automatic Control of Nancy, UMR 7039 CNRS and University of Lorraine in the framework of the NETASSIST project funded by AUF. This project develops methods for the networked control and sensing for a team of unmanned, assistive aerial vehicles that follows a group of vulnerable persons. On the control side, we consider multi-agent and consensus techniques, while on the vision side the focus is egomotion estimation of the UAVs and cooperative tracking of persons with filtering techniques. NETASSIST is an international cooperation project involving the Technical University of Cluj-Napoca in Romania, the University of Szeged in Hungary, and the University of Lorraine at Nancy, France.
The goal of this visit was to develop a drone control application based on sensor and visual information. For this we planned to process the stream of position and attitude measurements given by the vision algorithms, which she developed in her home institute, on a video taken from the drone, using a state estimator / filter and the dynamics of the drone. For the above-mentioned vision algorithm, she needed to work with 2D images and 3D dataset provided by OptiTrack and solve the question of feature extraction and matching in a video sequence for visual odometry application.
Glycocalyx which can be found in the surface of the endothalial cells is the part of the blood-brain barrier’s protection system and it has effect on the limitation of the material access. Research on the composition, rule and the change of the glycocalyx in illnesses recently became a significant research topic. However, we do not have data about the intestinal epithelial cell and particularly in the cerebral microvessels’ glycocalyx volume and a change in cell surface charge of direct measurements which have been performed. Our aim was therefore a direct measurement of surface charge of cultured brain endothelial cells and to examine how the surface charge and endothelial cell layers permeability change after the enzymatic digestion of the glycocalyx.
In our experiments D3 immortalized human brain endothelial cell line and primary rat brain endothelial cell cultures were employed. The amount of the glycocalyx was reduced by neuraminidase, trypsin and hyaluronidase enzymes which changes were validated using specific fluorescent labeled wheatgerm N-acetyl-D-glucosamine and sialic acid binding lectin. We took images from the binding of labeled lectin with confocal microscopy which fluorescence intensity was evaluated by Image J software. Cerebral endothelial cell surface charge was measured by the Malvern Zetasizer Nano ZS instrument. The brain endothelial cell lines were cultured into inserts to test their barrier function. We transferred dextran and Evans blue-albumin complex as a fluorescent reporter molecule to examine the cultures.
We successfully measured the surface charge of brain endothelial cells, and both cell types had strongly negative zeta potential which correlates to the literature. All three enzymes digested the glycocalyx and reduced the binding of the lectin. At the same time, the absolute value of the surface negative charge significantly reduced in both cell types. However, hyaluronidase and neuraminidase enzyme treatments did not affect the passage of the markers. The permeability increased only in the case of the trypsin digestion, although this enzyme cleavages the extracellular connections thus opens the paracellular transport route, which was visible in the morphological images.
We were the first one measuring and discribing the surface negative charge changes after the enzymatic digestion of the glycocalyx in human and rat brain endothelial cells. We found that the significant part of the surface charge origin from the glycocalyx in these cells, as after the enzymatic digestion the zeta potential of the cells became more positive. Reduction of the volume of the glycocalyx however did not affect the passage of the markers with neutral charge. In our future plans, we consider to use different materials with positive and negative charge for the surface modification of the glycocalyx, as well as to use other indicators with different charges in the permeability experiments.
Pose estimation is a fundamental building block of various vision applications, e.g. visual odometry, image-based localization and navigation, fusion, and augmented reality. Herein, we are interested in absolute pose estimation, which consists in determining the position and orientation of a camera with respect to a 3D world coordinate frame.
Modern applications, especially in vision-based localization and navigation for robotics and autonomous vehicles, it is often desirable to use multi-camera systems which covers large field of views. Not only classical image-based techniques, such as Structure from Motion (SfM) provide 3D measurements of a scene, but modern range sensors (e.g. Lidar, Kinect) record 3D structure directly. Thus the availability of 3D data is also becoming widespread, hence methods to estimate absolute pose of a set of cameras based on 2D measurements of the 3D scene received more attention.
Since modern cameras are frequently equipped with various location and orientation sensors, we assume that the vertical direction of the camera system (e.g. a gravity vector) is available.
In this work, we will discuss the problem of absolute pose estimation in case of a generalized camera using straight lines, which are common in urban environment. The only assumption about the imaging model is that 3D straight lines are projected via projection planes determined by the line and camera projection directions, i.e. correspondences are given as a 3D world line and its projection plane. Therefore we formulate the problem in terms of 4 unknowns using 3D line – projection plane correspondences which yields a closed form solution.
As an important special case, we address the problem of estimating the absolute pose of a multiview calibrated perspective camera system from 3D - 2D line correspondences. Herein, we propose two solutions: the first solution consists of a single linear system of equations, while the second solution yields a polynomial equation of degree three in one variable and one systems of linear equations which can be efficiently solved in closed- form.
The proposed algorithms have been evaluated on various synthetic datasets as well as on real data. All of the solutions can be used as a minimal solver as well as a least squares solver without reformulation. Experimental results confirm state of the art performance both in terms of quality and computing time.
Keywords: absolute pose estimation, vertical direction, line correspondences, gener- alized camera, multiview camera system
In this paper, we address the problem of estimating the absolute pose of a multiview calibrated perspective camera system from 3D - 2D line correspondences. We assume, that the vertical direction is known, which is often the case when the camera system is coupled with an IMU sensor, but it can also be obtained from vanishing points constructed in the images. Herein, we propose two solutions, both can be used as a minimal solver as well as a least squares solver without reformulation. The first solution consists of a single linear system of equations, while the second solution yields a polynomial equation of degree three in one variable and one systems of linear equations which can be efficiently solved in closed-form. The proposed algorithms have been evaluated on various synthetic datasets as well as on real data. Experimental results confirm state of the art performance both in terms of quality and computing time.
We propose a novel method to compute the absolute pose of a generalized camera based on straight lines, which are common in urban environment. The only assumption about the imaging model is that 3D straight lines are projected via projection planes determined by the line and camera projection directions, i.e. correspondences are given as a 3D world line and its projection plane. Since modern cameras are frequently equipped with various location and orientation sensors, we assume that the vertical direction (e.g. a gravity vector) is available. Therefore we formulate the problem in terms of 4 unknowns using 3D line - projection plane correspondences which yields a closed form solution. The solution can be used as a minimal solver as well as a least squares solver without reformulation. The proposed algorithm have been evaluated on various synthetic datasets as well as on real data. Experimental results confirm state of the art performance both in terms of quality and computing time.
My office location: 6720 Szeged, Arpad square 2, room 216.
Feel free to contact me by e-mail.