The old saying that teaching someone to fish leads to a lifetime of food needs updating. With today’s machine vision technology, imaging the fishing catch can result in a lifetime of savings for commercial operations. The images and other data could also lead to more sustainable fishing. When federally licensed charter boats put to sea, they now have to document the numbers of fish they bring in. “This is the first year on the charter boat side that they have to report their catch — their landings of species and how many,” said Abby Webster, a captain with Cowboy Charters out of Freeport, Texas. Cameras capture images and video of fish on charter boats and commercial vessels. The images provide information about numbers and types of species landed. Courtesy of Integrated Monitoring and CoastIPC. Logging is done manually via a phone app, a process that takes up the captain’s time. The captain must input the species identification and keep track of whether a particular catch is kept or discarded. The limits vary by species and are given in terms of a certain number of each fish that can be hauled in per person on the boat, such as two red snapper for every paying member of a charter. Similar recordkeeping is required of commercial fisherman. In this case, however, some boats sail with a regulatory agency-approved observer on board — a third party who records how many fish and of which species are caught. Replacing human auditors These independent auditors are an expense that can run in the hundreds of dollars per day, up too as much $800. The auditors can be found on perhaps 15% to 20% of commercial fishing vessels, a sampling that serves to check the data that commercial captains report. In addition to the time spent by a crew member or the expense of an auditor, another problem with this approach for both charter boat and commercial fishing is the quality and quantity of the data. The information helps regulators to set limits on how many fish can be caught, as well as helping to determine when and where fishing should occur. Better data would help to ensure more sustainable fishing practices. Federally licensed charter boats must log their catch, and some are using vision technology to do so. Courtesy of Cowboy Charters. Seeking a solution to this problem on the charter boat side, Cowboy Charters turned to a technology package from Integrated Monitoring, which teamed up with CoastIPC to develop the system. Similar technology is also being deployed in commercial operations, according to Jeff Douglas, CEO of Integrated Monitoring. The system consists of several color cameras, enough to cover the fishing area. The resulting images have to be of high enough quality that an observer on land can later identify a fish’s species. Douglas said of the cameras in a system with typical resolution: “They’re similar to what you’d buy for a city-level surveillance project.” Solutions being implemented are 2160p, one of the 4K video standards. In addition to time-stamped video data, the system stores GPS information, making it possible to determine the location of a boat during a catch. Onboard industrial computers are designed with electronics that can withstand an environment in which salt spray, wide temperature swings, less than fully reliable power, and other issues are common. CoastIPC supplied the hardware, while Integrated Monitoring provided communication links and software. Depending on the application, the system can record at various frame rates and resolutions, store data for later analysis, and transmit the data in order to provide a by-the-minute view of the action at sea. The ongoing transmission of data makes it possible to determine whether a camera is working, instead of waiting until a ship returns to port and then finding that no images were captured. A vision camera images the deck on which fish are caught, eliminating the need for an onboard inspector. Courtesy of Integrated Monitoring and CoastIPC. Most of the time, the cameras record information of little interest to the task of identifying and counting fish species. Image analysis could then be used to zero in on the times when fishing is happening. Douglas said using machine learning and triggering algorithms that recognize certain activities of crew members shows promise as a way to segment out images of interest from the rest. This narrowing down process will help data transmission by reserving bandwidth for the most important video segments. It will also aid later review by trained observers on land, who will be able to quickly move from one catch event to the next. Finally, beyond the benefits of saving costs and acquiring better data on the total catch, the use of vision will bring new advertising opportunities for charter boat fishing. Once the machine vision elements of video capability and processing, along with a communication link, are in place, observers on land who may be interested in deep sea fishing can get closer to the action. “The ability for somebody to sit at their desk and watch someone catch a 30-lb king fish or a 20-lb red snapper — that’s going to be phenomenal,” Webster said.