When it comes to utilizing drones on commercial endeavors of all types and sizes, figuring out what the “capture” phase of the project looks like is often the thing we talk about and focus on. Increasingly though, that phase isn’t the one most professionals are struggling with or even the most important thing that operators should be considering. After all, there’s a big difference between being able to capture data relevant to important decisions and being able to properly utilize that data to actually influence said decisions which will impact ROI.
The amount of data that can be captured by drones is astounding, and those capabilities continue to get bigger and better. As you can imagine, there are challenges associated with how all of this data can and should be handled and processed in real-time and after the capture phase is completed, and organizations such as Brainlike Inc. are directly speaking to these sorts of issues with their products and services. Brainlike focuses on improving decisions and saving money by converting raw data to valuable information in real time.
To find out the specifics of how and why such things are of critical importance to UAV operators of all types and sizes, I spoke with Bob Jannarone, CEO of Brainlike, Inc. We discussed the challenges various companies are facing when it comes to being able to properly utilize the data gathered by a drone, some of the mistakes he’s seen being made in this area, why automation is such an important consideration and plenty more. Jeremiah Karpowicz: Tell us a little bit about your company. Are you focused on a particular market or application?Bob Jannarone: We deliver computers with our patented Brainlike "triage software" inside. Our products reduce the effort of event detection analysis by up to 90%. For airborne marine mammals off Alaska's North slope, our products have detected bowhead whales with a hit rate of over 80%, while reducing average analysis effort from 3 minutes per image to less than one second per image. Brainlike currently focuses on wildlife detection and inspection (endangered mammals, tortoise, birds and defect detection, wind turbines, solar panels, etc.) How have and will those industries be impacted by drone technology?NOAA flew a Scan Eagle over the North Slope in 2015 to evaluate marine mammal survey potential. Their hopes include reducing manned flight cost and airborne observer danger while increasing coverage and maintaining detection accuracy. Their key challenge is reducing image processing effort.Advanced Aerial Inspection Resources flies Falcon 8 drones to inspect wind turbine blades affordably. Their products replace dangerous and time-consuming manual inspection with accurate and timely defect identification. They currently inspect images manually after delivering them to analysts, post-flight. They hope to reduce manual analysis time and effort in order to scale up their operations.Field analysts scan large field plots for desert tortoise to meet Clark County Nevada Desert Conservation requirements, at six-figure annual cost. Program managers are considering drones to increase coverage and decrease cost. A successful solution will be demonstrated by first gathering drone-based data, and then automatically triaging the data to reduce analysis effort while while maintaining or improving detection accuracy. What kind of challenges are companies and organizations in these industries facing when it comes to effectively utilizing video and photo gathered from drones?Their key challenge is justifying the use of drone-based sensing to reduce total operating cost. Cost components include: data gathering cost, which is often relatively minor; image analysis cost; detection precision (missed event and false alarm) cost; bandwidth reduction cost; and, for those who are considering automatic event detection, detector analysis and configuration cost.Bandwidth reduction is important in drone-based field inspection applications, where data must be sent to the cloud before events can be detected. Bandwidth reduction will be increasingly important in applications where high mega-byte photos are taken beyond line-of sight and analysts wish to see events on the ground in real-time, but only low mega-bit transmission is possible. Can you give us a specific example of a company using (or not using) drone-based sensing to impact their approach or bottom line?Shell Oil paid to get over 200,000 airborne images from manned aircraft in 2012. Marine mammal surveys based on these images influenced Shell's multi-billion decision to drill for oil. Shell paid LGL Alaska Research Associates to analyze the images. Manual image analysis took over three minutes per image--so long that the LGL team ran out of time and money. Product evaluation and refinement studies since then, commissioned by LGL and Shell, have shown that automated triage reduces analysis time to less than one second per image and yields detection rates up to 84%.In 2015 NOAA shot similar images from drones and will continue to do so. Drones could easily shoot twice as many images as aircraft at a much lower cost (as well as greatly reduced danger). Since analyst pay is at least $40/hour, and since automatic triage can reduce analyst time by 3 minutes per image, resulting analysis savings alone can be over $800,000. Savings after figuring in unmanned versus manned flight cost could be much more.In typical applications that we address, ranging from animal surveys to defect detection, most images contain no events at all and events that do occur are hard to find. The figure below (kindly supplied by Hi Def Aerial Surveying, Ltd) is typical.
One thing that seems to be a focus for you is around the immediacy users can see when utilizing a drone. But that immediacy is contingent on a number of factors, isn’t it?Yes, immediacy depends heavily on consumer needs. Many consumers only need real-time video looks or post-flight video files. Agricultural inspection, equipment inspection, wildlife survey, or mapping consumers may only need reports or maps several weeks, post-flight. However, oil companies may need to evaluate marine mammal presence before doing next-day seismic testing. Likewise, ranchers may want to identify stray livestock, predators, broken fences or poachers in time to react by the next day.The general immediacy challenges we address arise when drones are collecting so much data, so quickly that important events may be missed in time to satisfy customers. Users who only need to look at low-resolution, line-of-sight video streams don't face these challenges. Users who will want to see events within high-resolution photos beyond line-of-sight will eventually face the same challenges. How difficult is it for anyone using a drone to effectively take advantage of the opportunities presented by this immediacy? Users considering immediacy might best begin my monetizing potential advantages and costs. Along the way, they may find that the immediacy is worthwhile for meeting their most affordable needs but not worthwhile for meeting others.For example, a Brazilian rancher may be using streamed video from fixed-wing drones to find Kobe cattle that have strayed out of safe pastures. Ranch hands may view the video in-flight or post-flight. When they see stray cattle they can rescue them quickly, but seeing them all may be very difficult. It may take days of zoomed-in, slow motion scanning and take too long. The ranch could lose 100 cattle per year that are hidden in the video, at an annual cost of $400K. If the video could be triaged to highlight 75 of the missed cattle at an annual added cost of less than $100K, the rancher would save more that $100K per year. What is one of the most common mistakes you’ve seen made in this area?I wouldn't call them mistakes as much as milestones tied to sensing technology innovation in general. Companies first capitalized on drone excitement and publicity. They got themselves and their customers excited about all the cool vehicles they could make and all the neat things they could see. At that time, they were blissfully ignorant about data triage, processing time, bandwidth, and other costs but that's OK. Without this excitement and publicity, drones, like so many other sensing innovations, would never have gotten off the ground.Everybody has gotten excited about getting drone-based data, that's great, and that's how it should be. What we currently need are enough operators to think through the process and share their success -- both in choosing drone-based automation and choosing better alternatives -- so that others in the same boat will benefit from their experience. The timing is right for project managers to evaluate their specific data processing needs and carefully evaluate their vendors' capability to meet them. Have you seen operators or organizations essentially trick themselves when it comes to how quickly they’re able to do something useful with the data they gather from a UAV? That is to say, do they sometimes think they’re being more efficient than they actually are? Great question, underscored by our experience with marine mammal detection.Our customer first asked us if we could identify any variety of marine mammals under broad visibility and sea state conditions. Together, we found that narrowing focus was essential. Finding whale splashes in heavy seas or finding beluga whales within beluga-sized ice fields wouldn't work using affordable cameras. Together, we learned that detecting even the most straightforward events (whales in open water, walrus on ice) with high precision (manageable false alarms) wasn't easy. To their immense credit, they took time to set achievable requirements that we could meet. Together, we set and met achievable expectations, while carefully monetizing return on investment in order to justify the effort. Why is automation such an important factor whenever we talk about data?Automation is an important factor because huge streams of available data contain only a few nuggets of useful information. Automation will become used as affordable methods improve for identifying those nuggets precisely. Beyond drone-based sensing, five years have passed since the amount of data being produced by sensors exceeded the amount of data that could possibly be stored, let alone processed.To illustrate the kind of impact automation can make, Advanced Aerial Inspection Resources uses drones to take thousands of pictures of wind turbine blades on a daily basis, as shown below. (The wind turbine imagery in this article was kindly provided by AAIR.) Their inspections based on drone cameras identify more defects at lower costs than inspections based on visual observation. Since wind turbine inspection needs are rapidly expanding, AAIR has a great opportunity to grow rapidly.However, finding defects from images after their inspection flights takes more person-days than shooting the images in the field. If automatic image triage can reduce their detection cost by 90% while maintaining or improving their defect detection rates, AAIR will be able save lots of money and service many more customers. Automatic triage offers the added benefit of reducing many mega-bit photos to a smaller number of mega-bit chips. As a result, field-based triage could allow AAIR to transmit defect information from the field to the cloud for immediate validation by their staff and faster delivery to their customers.
Photo Courtesy of AAIR
Comments