Space Apps Challenge 2018

Theme

Design an autonomous free-flyer to inspect a spacecraft for damage from Micro-Meteoroid and Orbital Debris (MMOD).

Our goal

Provide a decently enough devised applicable solution to the problem which should be scaled to lower proportions given limited resources and time.

Idea

The first subproblem we encountered , and probably the most important was how would we make the flyer learn how to scan for potential harm, and if found how will the mentioned harm get analyzed. Firstly, we found the need to define the question "What is considered a potential harm?" clearly, in order to start thinking of methods for achieving said detection. MMOD stands for Micrometeoroid and Orbital Debris according to Wikipedia. With proper analysis of data , we've come up with the approximation that MMOD sizes vary from a minimum of 1 mm to above 10 cm diameter. With that consideration, we could safely suppose that whatever damage is caused by MMOD has diameter size >=(greater or equal) 1 mm(for the smallest debris) . With this we cover every possible damage size caused by any of the said MMOD. Because the challenge is detection of damage strictly caused by MMOD we will ignore any detection of passive damage (spaceship decay and deformity proportioal to the ship's lifespan and detail of maintenance) . With this in mind, we knew that we had to develop a meticulous and precise method of detection in order to detect the minimalistic damages (which is also required to be achieved). After a mildly exhaustive brainstorming session and discussion we came up with our idea. We would use precise and dense laser grids in order to detect the damage. The idea is :

Instance example : We will suppose that we are overlooking the region R and we need to inspect it for damage. In order to do this, we must have already scanned the region R in a period when we know that there isn't any damage registered on it. The scan is done by a simple image of a projected laser grid on the surface. The grid consists of equally positioned vertical and horizontal lines (distance between each is the same). We will store that image in a folder and keep it close to our hands. Now, when time has come to actually scan for potential damage in the region R, we run a second scan by again, projecting a laser grid on the surface and taking a picture. Now, all we need to do in order to find out if there is damage on the surface is just compare the two laser grids from the initial (calibration scan - when the region R had no damage on it) scan and the actual first scan. Key concept : laser direction changes when there is an irregularity on the scanned surface. So every decently big enough dent, if picked by the laser would cause to laser to disrupt its natural vertical or horizontal flow. Now , the two images containing the laser grids are put through an image comparison algorithm that would also detect the location of the difference (if there is any). But... what if the damage caused on the ship's surface is by chance located in-between two laser beams. Then both pictures would be identical and we wouldn't detect anything in the region R even though there was damage. We solve this by doing not one, but multiple images of the same region, with translated laser grid in each picture. The translation of the grid (the shift) would be tightly connected to the size of the potential damage. Because we calculated that minimum damage diameter would be scaled in 1 mm, if we devise a laser grid so that each laser has a width of 1mm and distance between lasers of 1mm, we would only need two images to cover every scenario! One would be the normal laser grid, and the second one the laser grid shifted to one of the four directions by 1mm. But this would mean a very dense laser grid, which costs electricity. We leave the choice to NASA on this one.. would you rather use more electricity and less processing power (only 2 images) or less electricity but more processing power (more than 2 images, but less dense laser grid). The formula for the number of images needed is apparent.. It would be ceil(distance between lasers / width of lasers) +1.

Method

Let's formulate the process now. For simplicity in equations, we will suppose that we want to only use 2 pictures for each region max (for least processing power). Suppose we have a spacecraft with an area of P. If we use a camera that can cover a maximum of area P1 from distance R while still being able to detect object of 1mm (being able to detect will mean that the object will be displayed with more than 10px) then the number of regions in which we would split the spacecraft is P/P1. But ideally we want this number to be a whole number, so we would need to take a picture at a distance R from which we can cover some area P2, such that P2<=P1 and P/P2 is a whole number. Let's say that the number of regions now is n (where n=P/P2). In our calibration scan , we would need to perform 2*n images (n with a laser grid , and n without a laser grid - this will be explained later in the text). In order to take the images, we also need to determine every region center point. Now, when the first scan happens, we do the same process with one difference. Our flyer goes from region to region in the same order as the one in the calibration scan and in each of the regions it hovers exactly above the center point and takes a picture of the laser grid of the region. The difference between a calibration scan and a regular scan will be the next - in a calibration scan we do 2*n images, while in a normal scan we do n+p images, where p is the number of regions in which a harm is detected. We also know that p<=n. Theoretically , this aren't a lot of images, actually not at all. What are the problems that we need to address in this process? Firstly, we don't have a valid way to make the flyer move to the exact location of every region's center and stay there while it takes a photo and secondly we still don't have the comparison algorithm between images and the analyzation of those p images. We also need a way to update the calibration database. A mention : calibration is only done once on each spacecraft, and then never again.

Problems

At this point we started working towards solving problem 1 , the creation of the image comparison algorithm. For this problem, we decided to use Python and its libraries , mainly OpenCV for image processing. It wasn't very hard to write and assemble the code. The function takes two parameters - which are two images and as a result returns a number between -1 and 1 that maps to the level of similarity between both (1 being exactly the same, -1 being different in every pixel). The function also creates a new image for us, which is actually the 2nd image passed as a parameter but modified so that every difference between it and the first one is selected and focused by red rectangles. This allows us not only to confirm detection of damage, but also position of damage. A more efficient way of comparing images that we had thought of was first to isolate the laser grid from the image and then compare only laser grids, so less comparations will be made. This is done by another function that removes every color from a picture, other than a color from a certain range that we specify (according to the laser's color). For example, if we would to use a red laser, our function would remove every color besides ranges of red from the picture.

After solving this, we moved onto the 2nd problem, the automation of the flyer - its ability to move to a certain point and stay there as long as required (time to take pictures). This is a complex problem that requires some thought. In our instance of the experiment, we would be limited to using most of the known techniques for solving this problem , with the main cause being limited resources and time. Our potential solution to the scaled problem would be using an indoor positioning system in order to keep constant track of the flyer's current location. This method requires at least 3-4 antennas positioned in a region around the object which location is being calculated. Any WiFi / Bluetooth device can be counted as an antenna also. The antennas constantly send data to the device which actually is strength of the signal from each antenna to the device. In our experiment, we used Bluetooth devices to collect data. Overall, this is messy to handle and in order to visualize it better we designed a dynamic webpage that collects and presents said data (strength of signals and location of device). After that, triangulation methods are used to calculate the location of the device. But in, space proportions, positioning at least couple of antennas in every region would be considered inefficient. So our proposed solution would be mechanical minimal adjustments to the flyer using jet thrusters on gas and telemetry. Another consideration is that the flyer should move relative to the movement of the spacecraft and the distance between them in time of picture-taking must be constant. For this, thrusters enabling 3 dimensional movement must be implemented considering potential rotation of the spacecraft.

The third problem is the maintenance of the calibration database. As we mentioned in the first paragraph under "Method" we calculated that considering we need 2 images for each region and we have n regions, there would be 2*n images for the calibration scan and n+r images for every successive scan. Those r images are what's important for the maintenance of our calibration database. Every one of those images indicate a change in a separate region. Another potential worry for the maintenance of the calibration database would be a change in the rocket's design. Because we only want one calibration scan, if our flyer were to detect for example a new adjustment to the spacecraft's surface, it would need to determine that the change is not from а MMOD damage. But this is way too complex to design for now, so in an optimal situation we would want one operator inside of the spacecraft who would compare these types of images. If he determines that the image from the successive scan is only different because of a new design, he would update the calibration database meaning he would replace the image in database with the new image at the appropriate location. This would also apply to every damage that the crew decides not important, these images must be replaced also. This process defines the maintenance of the calibration database.

Design and cost efficiency

Next, we started thinking about the design of the flyer and the materials that we would use. We managed to do an initial schematics of the design and approximate a 3D version of it, but because of lack of resources, time and knowledge we didn't actually build it. The estimated materials of the flyer is light compact plastic, resistant to high temperatures.

It is important to mention that this flyer is designed to be a maintenance device, meaning it would be active for a certain time only and not all the time. The flyer includes a high-resolution camera, laser with the wanted grid (defraction) and some way of communication to send images to the operator. This costs power, so each time the flyer is deployed it will need to be recharged as required.

Summary

We believe that (maybe trivially) we have designed a consistent system that offers precise data about the certain subject while implementing known and existing methods. Although hard to realize in limited amount of time and resources, we'd managed to cover most of the essential aspects of the design in a scaled level.

Github repo: https://github.com/becutandavid/Trycer