Processing Oblique Imagery in Pix4D
Introduction
What is Image Annotation?
Image annotation is the process of removing unwanted parts of an image from the ray cloud. For example, in the first data set used for this lab we wanted to isolate the truck from the rest of the image. Using image annotation we are able to remove the pavement and other parts of the image to leave just the truck.
At what phase in the processing is this performed?
Image annotation is performed after the first step, Initial Processing, has been completed, but before the second step has been completed.
What types of Image Annotation are there?
There are three types of image annotation: Mask, Global Mask, and Carve.
- Mask annotation would be used for removing an obstacle that appears in a few images, such as scaffolding or tree branches.
- Global Mask annotation would be used when there is a consistent obstacle between all of the images, such as if the landing gear or propeller from the UAV was in view of the camera lens.
- Carve annotation is used for removal of large areas such as the sky or background of a photo.
Methods
The set-up for processing oblique images is similar to the process for nadir images. Typically I would ensure that the datum and coordinate system being selected are the same as what our metadata says they are. However, there was no metadata available for this project, so I allowed automatic detection to do its work. I kept automatic detection enabled for the camera options as well because I did not have access to any sensor data.
In the past we have often used the 3D maps processing template, but in this lab we will be utilizing the 3D models option. After that has been selected we proceeded with the Initial Processing only.
Joetruck Data Set
After the initial processing completed, it was time to annotate the images using the Carve method of annotation to remove everything from the images except for the truck. Figure 1 below shows the first annotated image.
 |
| Figure 1: Carve annotation on Joetruck |
I started off just removing parts of the background closest to the truck. The more images I annotated, the easier it got to ensure I only removed things I did not want in the point cloud. I ended up annotating over a dozen of the nearly 70 images, and did full annotations, such as in figure 2 below, on 3 images.
 |
| Figure 2: Full image annotation on Joetruck |
 |
| Figure 3: Point cloud Joetruck |
Light Pole Data Set
The light pole data set went through the initial processing much faster than the truck. I decided to spend a bit more time annotating the light pole set because of the faster processing to hopefully get a superior point cloud. I ended up fully annotating 8 images of the light pole, and did some annotation on about 25 of the 33 images, or 75% of them. Figure 4 and 5 show some of my annotations for the light pole. Once again, I utilized the carve annotation method.
 |
| Figure 4: Partial image annotation |
 |
| Figure 5: Full image annotation |
 |
| Figure 6: View of the full light pole |
 |
| Figure 7: Close up of the lights |
 |
| Figure 8: Close up of the electric box |
Results
The results for both data sets were fairly similar. They could use a lot of work. The parts of the objects that actually showed up in the point cloud actually look pretty good, however, there are gaps to be seen. On the truck set, part of the trim of the truck's side, and the bumpers are missing. As for the pole, there is a small section right under the lights that is missing, and the lights themselves are pretty limited. Viewing the full pole you can tell what the object is, but as you zoom in it gets harder to discern what exactly we're looking at.
A few things I have thought about in terms of making the final product look better is to utilize multiple types of image annotation. Though I thought carve would be the best option because I wanted to remove the entire background, maybe the mask annotation would have been useful as well for a few specific pictures. I did try reoptimizing the data, reannotating more images, and then processing again on the light pole data set, but the results were nearly identical to the first time processing the point cloud. Enabling 3D meshes would certainly bring back some of the quality, and I did enable it to see what it looked like, but there was a good bit of the background left over with the mesh enabled, and I decided that it looked better without that in.
Two things I think would make the biggest difference would be to have more images from more view points. The flight was all at roughly the same angle. What would happen if we utilized more nadir shots, as well as shots that look upwards at the light pole? I suspect the lights would get more recognizable, but processing time would likely double if we got the necessary pictures. The other big change I'd make would be to zoom in on the picture as I was annotating and get specific pixels either annotated in or out of the image. I kept the zoom at default while annotating because there were many images to annotate and time was a concern, but if I had unlimited time I would go back and assess every image to make it perfect. But, given the time constraints that a real life project would have, and how much time I want to dedicate to a specific project, these results work for our lab purposes.
In order to get a full view of each of the generated point clouds, I have created animated fly arounds of the project. Figures 9 and 10 show videos of these.
Conclusion
Utilizing oblique imagery is going to continue to grow in the UAS field. Nadir images can allow for a good bit of data, and some nice products can be generated. If you want a map, nadir is a great way to accomplish that. However, oblique imagery has the ability to create some very nice point clouds. Despite that, though, utilizing nadir images would have helped the data come out even better, as well as even more different angles of the objects. Despite some of the missing pieces of the truck in the point cloud, the parts of the truck that were generated have been done so very nicely. It is easy to identify the brand of truck, and you can almost even read the brand of tires. With larger data sets, better annotation, and higher quality sensors it is possible to create some truly amazing point cloud sets with oblique imagery. Being able to give that level of depth and detail to a product generated on the computer is huge for so many things. With high enough quality data, industrial inspections are going to be made easy with this technology. When inspections are done, in addition to the videos created, having 3D models of the object is going to help show companies exactly what the problems are, where they are, and even potentially how to prevent them.
Comments
Post a Comment