Photogrammetry is the science of performing measurements by making photos, especially used for determination of positions of surface points. For this experiment, a photogrammetric technique was used to determine the angle of repose for a specific type of sediment for both in air and in water. By projecting a dot on the sediment and recording an image of it, the light trajectories of the projector to the dot and from the dot to the camera can be determined. These trajectories intersect at the surface of the sediment on the position of the dot and they can be described in spatial coordinates (x, y, z) by
The angle of repose underwater versus dry conditioned experimental set-up
calibration of the camera and the projector. The determined trajectories can then be used to determine the local heights of the surface by calculating their intersection points. For the sediment in water, these trajectories change when they intersect with the water surface due to refraction and are then redirected by applying Snell’s law. By doing this, measurements can also be done for surfaces underwater. This method is used as well and described in more specific details by Wilting [12]. A short description of the used technique is given below.
Camera calibration
For this technique to work, the camera needed to be calibrated first, as the results were used in the projector calibration. By calibrating the camera, its pixel trajectories for this set-up were determined.
For the sake of simplicity, the calibration is described in a two-dimensional space. Whenever a surface with a circle on it at x = x0 is placed inside the tank at z = z0, the camera will see this circle with its center being on pixel coordinate xpixel,0 = x0, as shown in figure 4.4. Whenever this surface is lifted to spatial height z = z1, the circle will then be on the same spatial position x = x0, but it will then be seen by the camera on a different pixel coordinate xpixel,1. The only exception for this is for dots located directly in the center of view of the camera. If the spatial heights and positions of the circles are well defined, a two-dimensional spatial position can be connected to a pixel coordinate for both heights.
Figure 4.4: Schematic overview of the two dimensional principle of the pixel lines of the camera. With the gray bars representing the tank (6), the dots (4) representing circles on the surface and the crosses represent the circles seen by the camera and their respective trajectory (3), placed on the lower height for readability, since the pixel coordinates do not have a height and thus are all in the same plane. The camera (1) is presented with its outer boundaries of sight (2).
This can be done for n circles and m heights, which will lead to n times m spatial positions all being connected to a pixel coordinate. For each height, a third-order polynomial surface fit can be made to define the spatial coordinate for each pixel in that surface. After defining the m fits for the m heights, the same pixel coordinates can be substituted in the all of the fits to find m spatial coordinates for that pixel. With these spatial positions, a least-square fit can be used to determine the spatial trajectory of that pixel.
For the three-dimensional case, this would mean that there will not only be a shift in the xpixel-position, but also in the ypixel-position for a change in height. The spatial coordinates will then consist of the position in height, as well as the x, y-plane. Subsequently, pixel coordinates will be expressed as (xpixel, ypixel) and spatial coordinates in (x, y, z). This leads to three-dimensional trajectories for the pixels after fitting.
The angle of repose underwater versus dry conditioned experimental set-up
The calibration was done using a calibration plate. This plate is 12 mm thick. The plate has 23 columns of 23 rows of white dots on it, with their centers having a distance of 25mm in between. An example can be seen in figure 4.5. The plate was set on four different heights, defining the height by setting the bottom of the tank to z = 0, and making sure to have the plate in the exact same positions in the x, y-plane and orientation. After that, an image was recorded for every height.
Figure 4.5: A photo made by the camera of the calibration plate placed on the bottom of the tank z = 0.
MATLAB scripts were used to find the pixel coordinates of the center of the dots and the spatial coordinates of the dots were given, through the height and the dimensions of the plate. After defining the fits of the surface for each plane, an arbitrary grid was made to define the amount and the positions of the pixel trajectories to be determined. This grid was chosen to be equal in size to the dots found on the calibration plate. The result can be seen in figure 4.6.
The angle of repose underwater versus dry conditioned experimental set-up
Figure 4.6: The lines in space constructed by the camera calibration. Each line is connected to one pixel of the camera, whereas the line represents the spatial line which is seen by that pixel. The intersection point of all the lines is the middle of the lens of the camera. The black dots are the intersection points of the lines with the horizontal planes on the heights of the calibration. One line is plotted for each dot on the calibration plate.
Projector calibration
After the camera calibration, the projector calibration was done to determine its pixel trajectories as well. Both calibrations are needed to reconstruct a surface with this method. Instead of a circle being present at the surface, a predetermined projection is now visualized by a projector aiming at and being above the surface. For sake of simplicity, the projection is chosen to be completely black with a white dot in it, with the size of one pixel, on a random position. This dot is projected along its corresponding pixel trajectory. The pixel coordinate of this trajectory (xpixel,proj, ypixel,proj) is equal to the pixel coordinate of the dot and can be determined by analyzing the projection. The spatial height z of the surface at the position of the dot is known, as this is one of the chosen heights for the surface used for the calibration.
The camera pixel coordinate of the dot (xpixel, ypixel), and thus its corresponding trajectory, can be determined by capturing an image of the surface. With the pixel trajectory of the camera and the spatial height, the spatial coordinate of the dot (x, y, z) can be calculated by calculation the intersection point of the two. If projecting the same dot at different, well-known heights of a surface, the spatial position of the dot can be determined for each height. With these spatial positions a least-square fit is made to determine the spatial trajectory of the dot for that pixel of the projector. This is displayed in figure 4.7.
The angle of repose underwater versus dry conditioned experimental set-up
Figure 4.7: Schematic overview of the two dimensional principle of the pixel lines of the projector.
With the gray bars representing the tank (7), the green and red dots (5) representing dots projected by the projector and the crosses (6) represent the dots seen by the camera and their respective trajectory (4), placed on the lower height for readability, since the pixel coordinates do not have a height and thus are all in the same plane. The camera (1) is presented with its outer boundaries of sight (8). The projector (2) is presented with trajectory lines (3) of two dots.
For the calibration, the bottom plate was set at three different heights. For each height, a specific projection was used. This projection consisted of multiple pictures with a black background and white ellipses in it. More details on this projection are given later. The pictures were projected in the same order for every height. For each projected picture an image was recorded. Two examples of these images and the shift in the positions of the ellipses can be seen in figure 4.8.
(a) (b)
Figure 4.8: Two image made of the first picture projected. In figure (a) the first picture is projected at an height z = 8 mm. In figure (b) the first picture is projected at an height z = 188 mm.
By using the images, all the spatial positions of all the ellipses were determined. With these positions the trajectories of the projector pixels were determined as described before. The result can be seen in figure 4.9
The angle of repose underwater versus dry conditioned experimental set-up
Figure 4.9: The spatial lines of the projector calibration. Each line is connected to one pixel of the projector, whereas the line represents the spatial line which is projected by the projector. The intersection point of all the lines is the middle of the lens of the projector. The black crosses on the lines are the intersection points of the lines with the horizontal planes on the heights of the calibration. One line is plotted for each dot on the calibration plate.
Patterns
As mentioned, a specific projection was used for this experiment. This projection consisted of white ellipses with a black background. The reason to project ellipses is that the projector was aimed at an angle, and its projection, therefore, was stretched. The projection was thus programmed to project ellipses, leading to circles being recorded by the camera. This stretch was small due to the projector settings being optimized as much as possible for the set-up. Therefore, the ellipses were seen by the bare eye as circles. In the remainder of the report, one projection is called a pattern. An example of a pattern is shown in figure 4.10. The patterns had a resolution of 1280x780 pixels.
The angle of repose underwater versus dry conditioned experimental set-up
Figure 4.10: An example of one pattern.
Multiple patterns were projected during one measurement. These patterns were made by a MATLAB script. This script is based on the script made by De Zwart [3]. This script determines semi-random positions for the white ellipses, from now on referred to as dots, and places them in a picture. For this experiment, all the dots together had 20000 different positions, spread over 156 patterns. An example of the distribution of these positions can be seen in figure 4.11.
Figure 4.11: An example of all the projected dots placed in one figure to show the distribution of dots.
With each red dot representing one projected dot.
The density of positions per pattern, or resolution, affects the amount of images needed to reach enough positions for a reconstruction. As each image takes a substantial amount of time to record, the amount of images and thus the amount of patterns must be minimalized. The resolution is dependent on the dot size and the distance between each dot. The dots had a diameter of 20 pixels, as this was the minimal
The angle of repose underwater versus dry conditioned experimental set-up
size for the dots to be detected in the photos by MATLAB. The size of one pixel on the bottom plate was measured by projecting a pattern and measuring the size of the pattern, divided by its amount of pixels. The size of a pixel on a height of 9 mm from the bottom of the tank was determined to be approximately 0.6 × 0.6 mm. This led to a dot having a diameter of approximately 12 mm on the bottom plate.
The maximum measurable height gradient is determined by the distance between the dots. As the dots have a different position in the picture for a different height, the position might intersect with the trajectory of another dot if the height difference is too large. For a displacement of 100 mm in height, a dot will shift approximately 50 pixels in the photo, so the minimal distance between dots was set to be 50 pixels, which is approximately between 30 mm (horizontally or vertically) and 42,5 mm (diagonally).
Reconstruction
Once the calibrations are performed, the height distribution of a surface can be measured by using the trajectories determined in the calibrations. This is done by projecting the patterns again. This does not need to be the same pattern as used in the calibration, but the pixel coordinates of the projected dots must be known. This is due to the fact that the trajectories determined by the calibrations are trajectories for the pixels from the projector, they are not connected to the pattern used for the calibration.
After projecting patterns and recording the corresponding images, the pixels of the dot centers are determined for the patterns and for the images. These pixels from respectively the projector and the camera correspond both to a trajectory determined by the calibration. The intersection point between these trajectories then relates to a spatial coordinate and represents a position on the surface. For every projected dot a position on the surface can then be calculated. With these positions the surface could then be visualized and used for further calculations.
For every measurement, a reference object was placed in the tank within the range of the projected patterns to validate the measurement for having the proper dimensions. For this experiment, a cone was used as the reference object. The height of the cone was 56 mm and the diameter 103 mm. The cone was slightly tilted on one side. The tilt on this side was 3 mm. The side which was not tilted was marked with a black minus sign. A photo of the cone and a enlarged visualization of the cone acquired by a measurement can be seen in figure 4.13 .
Figure 4.12: The reference cone which was used every measurement to validate the measurement.
The angle of repose underwater versus dry conditioned experimental set-up
Figure 4.13: One example of the reconstruction of the reference cone. The reconstruction matches the actual dimensions of the reference cone.
The error in this photogrammetric method is related to the detection method of MATLAB for finding the centers of the dots, as well as the precision during the calibration in the place of the calibration plate and in the height of both the calibration and bottom plate. The error in height was determined by using the images made for the projector calibration as a measurement. Because all the dots are placed at the same height. The result of the reconstruction should have been a flat surface, but minor changes in height are present. The estimated error of this method is then determined by taking the average of the measurement error for the three heights used in the calibration. The measurement error is calculated by dividing the difference of the maximum and minimum value by 2. The measurement error in the height of the surface is ≈ 0.9 mm for this set-up. The average corrected sample standard deviation in the height for this set-up is σ = 0.3 mm.