Space Apps Challenge 2018

reface:

Wildfires regularly endanger both our lives and our ecosystem, our team has developed an idea to reduce this damage.  

We propose state subsidized kits of wireless sensors collecting telemetric data in real time and sending it to our cloud based IoT application via network of solar powered routers and Wi-Fi amplifiers, once the conditions on wildfire are met, our preconfigured autonomous drones with resident AI module, would be activated and survey the area for fires, AI module on board is capable of discerning wildfire from normal conditions.

On the onset of wildfires, authorities would then be contacted by our on cloud application with GPS information of the affected sensors along with other weather conditions that could predict the path of fire, allowing authorities to take corrective steps to neutralize it.

Motivation:

We are losing the fight against forest and wildland fires. Here’s why:

• Wildfires are increasing in frequency, duration and intensity worldwide. Drought and other factors have increased not only the susceptibility to wildfires, but on an actual basis, the duration of the wildfire season. In many regions, the wildfireseason has increased by 20% to 40% and the area burned by wildfire increasedup to four times.

• Nowhere is this more prevalent than in the United States. Over the last three full years according to the National Interagency Fire Center (NIFC), the average number of U.S. wildfires was 87,004 and the average area destroyed by fire was in excess of 8 million acres. This is an increase over the average of previous “worst years” by 13% and 8% respectively.

• The rise in frequency of wildfires has been directly attributed to climate change by leading geoscientists (Diffenbaugh), as warming winters promote the continued growth of vegetation, which when coupled with high temperatures during the summer, result in large amounts of dry tinder being formed to ignite and sustain wildfires. As climate change continues, regions that have previously been spared such phenomena are becoming more vulnerable, as the 2018 wildfire season in Sweden has demonstrated.

Current Interventions need to be augmented:

• Early Detection Can Be More Important Than Prevention

Unfortunately, most wildfires aren’t easily preventable. Although most wildfires are initiated by either human carelessness or arson, some of the most severe are the result of lightning and power line interference. To significantly mitigate the risk and cost of wildfires, they must be detected and suppressed prior to reaching an uncontrollable state. In other words, the faster a wildfire is suppressed, the lower the cost.

• Determining Fire Location, Size, Direction & Burn Rates

Current wildfire suppression approaches often hinge on trying to predict where the fires are most likely to occur and “pre-staging” interdiction resources accordingly. But even when these activities are paired with important fire prevention aspects of public education, fuel load control and eliminating ignition sources during high-risk periods, the need for rapid detection and suppression is still one of the most important tools for containing wildfire damage. This typically means:

1. Already having the right equipment in the right place
2. Access to the most capable tools to assist suppression
3. Having an accurate, up-to-date management view of the fire (e.g., wind direction, speed and other weather conditions, ongoing view of fire direction and growth rate)
4. Simulation and forecasting of the fire size, speed and direction
5. Making the correct resource allocation decisions (e.g., Do we send two firefighters in an SUV or a C-5 aerial flame suppression tanker?)

Although it’s certainly possible to gain this knowledge from a variety of “manual” sources, the first few minutes of fire inception are the most important, because they will determine fire deployment strategy.

Our Idea:

AUTOMATION of CRITICAL components.

We want to have a system in place that will automate most of the things that cause longest response time. Here is how we propose this solution.

• State subsidized, preconfigured kits. Ideally further incentivized by Insurance companies. (reduction in premium for kit holders)

• Kits would include : Water resistant temperature sensors, Solar powered WIFI Routers and amplifiers, solar panels to support the Routers / amplifiers, Camera Drones with AI module on them , instructions on installation of the software and support.

1. Our solution involves equipping residents of estates , ranches and areas that are prone to wildfires with the preconfigured kits. It’s a community driven approach. These communities , groups or individuals will then install these sensors (with help from a strategy team) along their estates (further ends of their estates, ranches and areas that are a few kilometers away from their residence areas) . The general advice would be keep the kits connected and ready to use during seasons that are prone to wildfires.
2. These sensors would be networked with the solar powered routers and WiFi amplifiers. Which are a part of the kit (they would be preconfigured as well)
3. The estate owners would have to install our software on their system (computer, phones) which would periodically collect the telemetric data over the MQTT, AMQP, and HTTPS protocols and send it to our IoT cloud application. (ingested via IoT Hub) See our architecture section for further details.
4. Our cloud application would keep a track of all the telemetric data on timeseries database.
5. Our application would also be consuming data coming from NASA satellites and platforms like MODIS (Moderate Resolution Imaging Spectroradiometer), VIIRS (Visible Infrared Imaging Radiometer Suite), LANDSAT , AVHRR (Advanced Very High Resolution Radiometer) , GOES (Geostationary Operational Environmental Satellite) and any other open APIs that feed weather information, that we think would help narrow the initial risk areas further.
6. Once an area is considered high risk, our application would alert users who have the kits within the risk area to start their kits (if they haven't already) .
7. The cloud application would have its own ML/AI module that checks for "wildfire conditions" on the telemetric data. Once that is met, our autonomous drones would take off to survey the area. The drones could be simple drones with just the camera and AI module installed to discern for fire vs normal conditions and send that data back to the application or drones could also have a small payload of CO2 / other fire retardant material "bomb" that can be dropped onto the source of fire, if fire is detected.
8. The data then send by the drone would be funneled back to the application on cloud, where its logic would recheck information for fires again (hence suppressing false alarms) .
9. The application would take the data coming from drone feed ( (pictorial / video data pulled from drones) and weather data from APIs to further analyze the path of the "wildfire".
10. The firefighting department can use the surveillance from the application and use it to curb the fire strategically. (rather than going in blind, or using wrong / ineffective methods)
11. The application would also alert the users in the area, so they can look into more effective escape plans. (we are using map APIs to show area around the users geographical location) .
12. The more number of kits that get deployed the richer the data would be , the faster the community can be alerted.

Demo:

Our Applications:

Here is the VISIO diagram representation of what it would look like:

• AI Camera Module on Autonomous Drone.

We have designed an Android-based wildfire recognition software module relying on trained image recognition. The system currently relies on a pre-trained neural-network based ImageNet network specified to differentiate between wildfires and normal flora through the embedded Android device camera. As a demonstration, a video of our current system, trained on only 250 images, can be found at GitHub. Upon commercial release, the system will be trained be over ten thousand images, in preparation to handle different field conditions.

• AI Navigation Module on Autonomous Drone.

An automated, beacon based navigation system will be designed for the drone upon commercial release. This may be done in-house or through collaboration with potential commercial partners currently developing autonomous drone systems for search and rescue applications.

• IoT Application that would have AI/ML components.

We have created the architecture for such an application, due to lack of time we didn’t create this application. This application would be installed on user premises, with fire department and state , NASA could potentially use this application as well to notify users who call them for information on wild fires along with their current capabilities. Please refer OneNote Link.

• Phone /computer client Application that takes data from the IoT application.

The idea is to make this application as user friendly as possible. Here are our wireframes for the same:

• Front page:

Simple start / stop

Once the user logins (application would have to pair user information with incoming telemetric data to allow for easier analysis.) the application would allow user to start the system or shut it down with one simple action.

• Once started the dashboard would show things like
  1. Profile information.
  2. Sensor Health
  3. Network Health
  4. Router Health
  5. Drone Health
  6. Drone data for last week, month, year..
  7. Live data feed of the area around a user (2 KM radius, 10 KM radius, 50 KM radius and 100 KM radius)
  8. Map - to show the area covered by individual drones.

Future Enhancements:

• Cellular capability on sensors:

For places that are not under estates / ranches and are state owned, state can consider adding cellular capable sensors that can connect to cloud directly (more remote areas but cellular capable) and have drones in predesignated places to intervene / surveil when necessary

• Drones

The automated drone observation system will rely on commercial, off-the-shelf products integrated into a single package. These components are the drone, AI-enabled camera system, sensors, and support components

a. Drone Technology

Initially, lighter lift drones featuring smaller camera devices would be used to trial the system. The lower resolution and zoom of their onboard cameras would naturally present challenges to fire identification, particularly at longer ranges, but their lower cost would make them much more affordable to the average consumer and exempt of many FAA regulations (<250 grams)

Larger, heavy-lift drones (<20kg) featuring quadcopter designs, may be provided as a possible future upgrade, featuring larger onboard DSLR cameras with higher resolutions and zoom capability. Such systems would allow the identification of wildfires at much safer ranges, while also possibly tracking multiple fires at once.

b. AI-enabled camera system

The initial system should be capable of running off commercial Android devices, these integrate routers, camera, and processor capabilities in a cost-effective package. Future iterations would feature upgraded components, and possibly the separation of camera, processor, and routers.

• Cloud Architecture

With the help of more data and data analysis, this can be scaled further to have predictive models. (things like information from power lines that have caused this and the conditions that created the power lines to cause the fires, if those conditions are met or power lines need repair to pre-emptively fix problems before they become bigger and harder to handle.

• Application:

This can be scaled to help with other disasters and disaster prone areas. (known variables that come together to cause known issues)

• Edge Gateway for more capabilities:

This would make the system more expensive on the onset, but reduce costs in terms of data telemetry (nuisance data, wrong sensor information etc)

Team:

Our team consists of a design expert, a cloud expert, and a ML/Mobile app developer.

Yijie Xu - Adrian

Research Assistant at A*STAR.

https://www.linkedin.com/in/yijie-xu-0174a325/

Vera

Designer

https://www.linkedin.com/in/vera-neo-6ba681150

Pooja Chandwani

Cloud Architect

https://www.linkedin.com/in/chandwanipooja/

References:

• Remote Monitoring solution accelerator overview https://docs.microsoft.com/en-us/azure/iot-acceler...
• Using Surveillance Systems for Wildfire Detection https://www.firefighternation.com/articles/2013/06...
• Background: Wildfires in California and the Western United States http://meteora.ucsd.edu/cap/fire_background.html#r...
• Artés, T., Cortés, A., & Margalef, T. (2016). Large Forest Fire Spread Prediction: Data and Computational Science. Procedia Computer Science, 80, 909-918.
• FIRESENSE, F. D. Management Through a Multi-Sensor Network for the Protection of Cultural Heritage Areas from the Risk of Fire and Extreme Weather Conditions. FP7-ENV-2009-1244088-FIRESENSE, 2009,/http://www. firesense. euS.
• Iot, Matters. Technology helps first responders fight California fires. Network World, 10/24/2017, www.networkworld.com/article/3234655/internet-of-t...
• Zhong, X., Duckham, M., Chong, D., & Tolhurst, K. (2016). Real-time estimation of wildfire perimeters from curated crowdsourcing. Scientific reports, 6, 24206.
• Yin, J., Lampert, A., Cameron, M., Robinson, B., & Power, R. (2012). Using social media to enhance emergency situation awareness. IEEE Intelligent Systems, 27(6), 52-59.
• Wes Siler. Now you can forecast fires like the weather, 2018, www.outsideonline.com/2328341/how-forecast-wildfir...
• Goodchild, M. F., & Glennon, J. A. (2010). Crowdsourcing geographic information for disaster response: a research frontier. International Journal of Digital Earth, 3(3), 231-241.