Space Apps Challenge 2018

CHALLENGE

This project is solving the CAN YOU BUILD A... – Design by Nature challenge.

GitHub: https://github.com/b02902032/TabbyRangers_NASA2018

English slide: https://www.slideshare.net/ChiHsiYu/2018-nasa-hack...

Outline

• Abstract
• Introduction
• System Overview
• Methods
• Results
• Conclusion
• Reference and Resource

ABSTRACT

Inspired by nature, we’ve developed an autonomous damage inspection framework, CSMS, to improve current methods on spacecraft damages such as Micro-Meteoroid and Orbital Debris (MMOD). This system consists of multiple flyer robots, BeeBots, which are capable of collaborating with each other in harmony like a socialized bee colony. In this project we provide main algorithms and hardware investigations to discuss the feasibility of this method, and we hope this system can eventually benefit humankind in astro exploration.

INTRODUCTION

Damage inspection system nowadays typically use low-cost methods, such as a robotic manipulator arms containing 3D sensors, a long-distance camera, or directly inspecting by astronauts. The main defects of them are time-consuming, light source or angle limitation, and waste of extra-vehicular activity (EVA) resource, etc. Inspired by natural creatures, we found that bees are an outstanding basis for the development of a damage inspection system. Thus, the followings are three key features in our robot design:

• Behavior

We mainly adopted Artificial Bee Colony Algorithm to mimic bee's collaboration and task division. Our BeeBots can be grouped into employed bees, onlookers and scouts to exchange information and discover potential damage efficiently.

• Communication

Bees' dancing plays a crucial part in their cooperation and society. Unlike natural bees, we adopted the spirit but modified the implementation by using radio broadcasting system on our BeeBots to communicate with each other.

• Sensor

Compound eyes are one of the core sensors on bee which are able to detect visible light, ultraviolet light and polarized light. Some of the flowers emit ultraviolet marks to attract employed bees to land and collect nectar and pollen. Similarly, we implemented structured light scanning as BeeBot's "compound eyes" to observe features on the spacecraft surface.

IMPACT

• For space explorers
  

Space explorers can proceed to solve the problems after getting comprehensive information of the situation, always preparing themselves to respond to the fluctuating environment.

• For manipulators

Manipulators can get highly accurate results without spending much time and efforts, and also acquire quantified data to avoid limitation and errors.

• For NASA

Collected data can be provided to NASA as materials for spacecraft staff training, and the authority can establish supplementary measures for future missions based on experience.

• For the future

In the future, the system should play an important role in aerospace industry: BeeBots may be upgraded as astro-droids providing real-time hardware repairment; furthermore, they may also be used as pioneers or scouts to briefly report the surface situation of newly discovered planets, helping us explore the outer space more thoroughly.

SYSTEM OVERVIEW

The image below is the system structure of a BeeBot. Each BeeBot will switch among employed bee, onlooker and scout during the mission. Employed bees will either share obtained information or collect damage data. For collecting data, they will use structured light to scan damage areas based on location information, and store the data to database after using a machine learning model to predict the damage type. For onlookers, they wait for broadcasted information and proceed to assist the scanning. As for scouts, they are in charge of exploring new potential areas either randomly or from gathered information.

As for the system and implementation, the procedure can be divided into four steps:

A. Inspection

• Accurate and efficient scanning:
  With structured light projector on BeeBots, we can acquire quantified data to avoid limitation and errors from previous photographic scanning methods. Also, with autonomous data collecting system, astronauts can get highly accurate results without spending much time and efforts

B. Evaluation

• Data assistance and human-robot collaboration:
  Based on the depth information and predicted damage situation, astronauts can make reliable decisions either for repairment or targeting specific areas for more detailed investigation.

C. Response

• Risk evaluation and artificial intelligence aid:
  From quantified data analysis and the most feasible and secure suggestions provided by the system, related staff can proceed to solve the problems after getting comprehensive information of the situation, preparing themselves to respond to the fluctuating environment.

D. Integration

• Database establishing and staff training
  By accumulating data from different missions, the command center can build a systematic database. With the assistance of artificial intelligence, the authority can also accordingly design agile supplementary measures targeting long-distance spacecrafts for future missions. These data can be used as materials for spacecraft staff training as well.

METHODS

The methods are introduced in two aspects: hardware and software.

A. Hardware

(I) Kinematics - GITHUB LINK

When it comes to moving around the spacecraft, attitude control is a critical subject for mechanism design. We need to know what our current attitude is before we can spin our flyer to our desired angle or move to the designated area.A BeeBot contains three Hall sensors to observe the magnetic field generated from the device installed on the spacecraft. By reading the strength of the sensors, each BeeBot can determine its current attitude and figure out which direction it should spin. Installed in the center of the BeeBot, the control moment gyroscope (CMG) can thus manage the orientation of the flyer with respect to the spacecraft, followed by rotating the camera freely to the desired angle for inspection.

While executing damage inspection, each BeeBot needs to know its position relative to the spacecraft, therefore, we designed special landmark patterns painted on the surface of spacecraft with location information. Each pattern is unique, so when the flyer scans the pattern on the surface, it knows which part of the spacecraft component it is as well as its relative position. Besides, there are thrusters around the BeeBot, providing propulsion for moving around the spacecraft. Combining all the features above, the BeeBot has the ability to scan the surface smoothly and efficiently

(II) SCANNING (Demo using NiTE Middleware)

The inspection system is composed by a set of camera system and one structured light generator, which projects a specific pattern on the surface. Since the light pattern is projected by the system, the damage inspection system can be executed without sufficient light source. If there’s any damage on the surface of the spacecraft, the projected light pattern will be distorted by the shape of the damaged segment, and this information can be used for surface reconstruction. As a result, we can gather detail information about the depth of the surface of the spacecraft.

B. Software

(I) Artificial Bee Colony Algorithm (ABC) - GITHUB LINK

From this algorithm, bees can be divided into three groups: employed bees, onlookers and scouts, and the objective is to find the best nectar source. In short, employed bees share nectar sources information with the onlookers, and the latter can discover high-quality nectar sources based on the information. The scouts, on the other hand, randomly search for new nectar sources.

Adopting the ABC algorithm, our swarm of BeeBots can group up to search for the area with the most serious damage through social cooperation. With the power of dynamic programming, we can successfully reduce the dimension of the tasks to make damage inspection more efficient. Moreover, we modified the communication method of bees, dancing, into radio broadcasting, which transmits information more straight forward. For astronauts, they can either choose to let the BeeBots scan the whole spacecraft or designate a specific area if any prior knowledge is acquired.

(II) Damage Recognition and Suggestions - GITHUB LINK

Apart from the structured light, we want to leverage the potential of machine learning to further enhance the capability of our system. From the structured light scanning we can obtain depth information of an area, and on each BeeBot, a Convolutional Neural Network (CNN) model is adopted to give preliminary prediction to the spacecraft crew, and from the recognized damage type, The system is able to give suggestion to relevant staff for further repairment. The network uses bitmaps of depth values for training, and also in the future, active learning can be implemented on the system: the model selects potential damaged area images to the scientists and gets the correct labels from them. Through this method, the model can gradually raise the accuracy of prediction and lower the manpower required.

RESULTS

A. Appearance

These are the progress of designing out BeeBot model:

B. Structured Light Scanning Prototype

We used Microsoft Kinect to demonstrate the depth information scanning:

C. Control Moment Gyroscope Prototype

We use LEGO Mindstorm EV3 to build a control moment gyroscope model to demonstrate the effect:

D. CNN Model Performance

We used auto-generated depth data to simulate the damage condition of the surface, and used our CNN model to classify and predict the unseen observed damage. The following charts are the prediction results:

CONCLUSION

Overall, CSMS is a novel damage inspection system for spacecrafts aiming at improving the efficiency and accuracy of the past inspection methods, simultaneously providing repairment instructions and risk evaluation from systematic data analysis.

Every time an astronaut soars out of the atmosphere, what awaits is a difficult environment beyond our understanding on Earth. A subtle decision may bring an overwhelming impact on one’s life, and on the possibility of coming home safe and sound. What horrifies us is the uncertainty in the outer space, the silent and the most torturous suffering.

Thus, the spirit of our system design is to adopt the cutting-edge technology while humbly learning from Mother Nature, and create a system combining all the intelligence, strength and blessing from humankind. In the meantime, we aim to prevent the reoccurrence of any tragedies as those happened on Apollo 13 and Columbia, and to provide a safety net for all the admirable space heroes thousands of miles away. We sincerely look forward to the most valuable revolution BeeBots and CSMS can bring space safety assurance in the near future.

Who are behind the BeeBots project?

James Ke
National Taiwan University of Science and Technology
Department of Industrial Design

Lin Yu-Cheng
National Taiwan University of Science and Technology
Department of Electronic & Computer Engineering

Joey Yang
National Taiwan University
Graduate Institute of Networking & Multimedia

Ming Cue
National Taiwan University
Graduate Institute of Electrical Engineering

REFERENCE & Resources

• Microsoft Azure for cloud image recognition computing.
• Amazon AWS for Keyshot model rendering.
• Lego Mindstorm EV3 for control moment gyroscope prototype.
• Microsoft Kinect v1 for structured light scanning prototype.
• Pro/E and Keyshot for CSMS BeeBot free-flyer model.
• Hand-made prototype of CSMS BeeBot free-flyer.
• Artificial Bee Colony Algorithm
  Abu-Mouti, F.S., & El-Hawary, M.E. (2012). Overview of Artificial Bee Colony (ABC) algorithm and its applications. 2012 IEEE International Systems Conference SysCon 2012, 1-6..
• Programs are developed using Python.