1–2 minutes

to read

Mapping Environment Agency data – expect the unexpected

I parked my current build project for a week or two while I did some QA work for a London-based client who had a flutter app built by a small team in Pakistan. Once this was complete I returned to the project only to find it no longer working. It’s a flutter app that uses Environment Agency river data from one of their data.gov.uk apis, and I was certain the only thing that could have changed since I parked the project was the data.

I wasn’t wrong – in my short absence some unexpected data had been added, breaking my mapping from json to model. One of the fields documented by the EA as a string had become in a single record amongst thousands, a list/array.

The documentation:

The data:

I also noticed that the api had become version 0.9 but there was no mention of this in the ‘Recent Updates’ documentation:

The moral of this story is to take the following EA warning seriously:

While they are intended to be stable and robust the open data APIs do not offer a guarantee of service level and should not be relied upon for safety critical applications.

I realised I had to code this more defensively, so in my factory method on the model a try/catch was dropped in, allowing anything consuming this to iterate over bad data but also log it so it can be investigated:

  factory Station.fromJson(Map<String, dynamic> json) {
    try {
      return Station(
        id: json['@id'],
        catchmentName: json['catchmentName'],
        lat: json['lat']?.toDouble(),
        long: json['long']?.toDouble(),
        //etc.....
      );
    } catch (e) {
      throw Exception('Failed to parse fromJson. Station Id:' + json['@id']);
    }
  }
}

Silicon

Hi I’m Ed, owner of Silicon Gorge Apps, building lovely iOS and Android apps for lovely clients.

I’m based in Bristol, South West England, but can build apps for anyone, anywhere.