12  Gaps

GFW’s AIS gaps dataset catalogues instances of long gaps in vessel’s AIS signal and identifies which gaps are likely due to intentional AIS disabling.

12.1 Key Tables

  • world-fishing-827.pipe_production_v20201001.proto_published_events_ais_gaps: Date-partitioned table of AIS gap events of 12+ hours in duration. Includes the boolean field intentional_disabling to indicate gaps that are likely due to suspected disabling. Presently, however, only intentional disabling events (intentional_disabling = 'true') are included in this table. This is the primary table for analyses of AIS gaps.

  • world-fishing-827.pipe_production_v20201001.proto_ais_gap_events: Date-partitioned table of AIS gap events of 6+ hours in duration. This table can be used to identify gap events that are currently ongoing (is_closed = False) but must be filtered to select AIS disabling events.

12.1.1 Secondary Tables

  • world-fishing-827.pipe_production_v20201001.proto_ais_off_events: Date-partitioned table of AIS positions that are followed by a 6+ hour gap in AIS signal

  • world-fishing-827.pipe_production_v20201001.proto_published_events_ais_off: Date-partitioned table of AIS off events, formatted to match the GFW events schema. Includes the boolean field intentional_disabling to indicate off events that are likely due to suspected disabling. This table can be used to identify AIS disabling events as soon as they occur.

  • world-fishing-827.pipe_production_v20201001.proto_ais_on_events: Date-partitioned table of AIS positions that are preceeded by a 6+ hour gap in AIS signal

  • gfw_research.sat_reception_smoothed_one_degree_v20210722: Monthly satellite AIS reception quality (modeled) for 2017-2020. This table is used to exclude AIS gaps from areas with unreliable AIS reception.

12.2 Data Description

AIS devices were designed to continually broadcast a vessel’s position in order to serve as a collision avoidance system. Indeed, it is not uncommon to observe hundreds or thousands of AIS positions for a vessel in a single day. Thus, when a vessel has an extended gap in AIS positions it can potentially indicate illegal behavior. However, for several reasons, not all AIS signals that are broadcast are received by satellite or terrestrial AIS receivers and it is not uncommon to observe transmission gaps in a vessel’s AIS signal that are many hours long. Satellites must be overhead and terrestrial receivers require line of sight to receive AIS messages. AIS messages may overlap in time, especially in areas of high vessel density, causing signal interference and preventing messages from being received by satellites. As a result, GFW records all AIS gap events over six hours (referred to as “naive gaps”) and then uses a set of filters to identify those gaps we believe to be caused by intentional AIS disabling.

12.2.1 AIS gap events (“naive gaps”)

The proto_ais_gap_events table contains all 6+ hour gaps in the AIS data. These gap events are created by combining all events in the two precursor tables - proto_ais_off_events and proto_ais_on_events. The proto_ais_gap_events table includes all fields in each precursor table, as well as the is_closed field to indicate whether an AIS gap is currently ongoing. The proto_ais_gap_events table is date-partitioned on the gap_start field, allowing it to be filtered for AIS gap events that started on a specific date. Gaps in this table are considered “naive gaps” because additional filtering is required to identify gaps due to intentional disabling.

12.2.1.1 AIS off/on events

AIS off events are defined as an AIS position that is followed by a six hour gap and AIS on events are AIS position preceded by a six hour gap. Each day, the proto_ais_off_events and proto_ais_on_events tables are updated with new events that occur on that date. Both tables are date partitioned and should be filtered using the _partitiontime field.

12.2.2 AIS disabling events

To identify AIS gaps that are most likely due to intentional disabling rather than technical issues, GFW developed a classification model based on the following rules, which are explained in more detail in the Caveats and Known Issues section:

  1. The gap event must be at least 12 hours
  2. The gap must start at least 50 nautical miles from shore
  3. The gap must start in an area with a satellite reception quality greater than 10 positions per day
  4. The vessel must have at least 14 satellite positions in the 12 hours prior to the gap

The proto_published_events_ais_gaps and proto_published_events_ais_off tables apply these filters and include a boolean intentional_disabling field to flag likely AIS disabling events. These filters are not perfect, however, and analysts may consider slightly relaxing certain filters (e.g. distance from shore) if they want to capture potential disabling events of lower confidence.

12.3 Caveats and Known Issues

12.3.1 Gap events less than 50 nautical miles from shore are unreliable due to differences in satellite and terrestrial AIS

Satellite AIS reception generally decreases closer to shore as high vessel densities lead to signal interference. At the same time, >99% of GFW’s terrestrial AIS messages are less than 50 nautical miles from shore - generally the upper range of terrestrial AIS receivers - and terrestrial AIS coverage varies considerably around the world. Through a combination of these factors, AIS gaps that start within 50 nautical miles could be due to numerous technical reasons, such as:

  • Transitioning from areas with terrestrial AIS coverage to poor satellite AIS reception
  • Poor satellite reception while approaching port followed by turning off AIS upon arrival. These situations are likely responsible for many of the very long AIS gaps (e.g. several months) in the data

12.3.2 Gaps shorter than 12 hours are unreliable due to satellite periodicity

The number of satellites over the horizon at different places on earth varies considerably hour to hour. At all latitudes under 60, the major peak is at 12 hours (one half a day) and the standard deviation of the number of satellites overhead is very high when considering time periods under 12 hours. For this reason, only gaps >= 12 hours can be considered intentional disabling events. The 12 hour threshold accounts for the approximate amount of time required for the swath of an individual AIS satellite in a sun-synchronous orbit to cover the same location.

12.4 Example Queries

  • ais_disabling_published_events.sql: Get all (completed) likely AIS disabling events during a certain period.

  • ais_disabling_published_events_fishing_vessels.sql: Get all (completed) likely AIS disabling events by fishing vessels during a certain period.

  • ais_disabling_ongoing_events.sql: Get all likely AIS disabling events that are currently ongoing (e.g. is_closed = false). This query demonstrates how to combine naive gap events with reception quality to identify likely disabling events in a way similar to how the intentional_disabling field is added to the published_events_ais_gaps table.