{"id":2126,"date":"2024-02-15T08:50:32","date_gmt":"2024-02-15T07:50:32","guid":{"rendered":"https:\/\/www.pa3hcm.nl\/?p=2126"},"modified":"2024-02-15T09:05:27","modified_gmt":"2024-02-15T08:05:27","slug":"cli-for-dutch-water-data","status":"publish","type":"post","link":"https:\/\/www.pa3hcm.nl\/?p=2126","title":{"rendered":"CLI for dutch water data"},"content":{"rendered":"\n
\"\"<\/a><\/figure>\n\n\n\n

Here in the Netherlands we have a public organisation named Rijkswaterstaat<\/em>. This organization keeps track of all the water in and around our country. Think about sea levels, locks in rivers, etc. To do so they have an extensive network of sensors around the country, monitoring water levels, current speed, temperatures, etc. I wrote a command line interface to access this data.<\/p>\n\n\n\n\n\n\n\n

People living in The Netherlands are always curious about water. Although there is a high level of trust in all the waterworks, they are always just a bit more alert when a storm smashes the sea water against the dams and dunes, or when river tides gets too low to allow transport over water. They know how important these structures are, since a large part of the country is below sea level.<\/p>\n\n\n\n

The dutch government has a ministry and a couple of organisations that deal with all this. One of the most important organisations is Rijkswaterstaat<\/a>, which manages (according to Wikipedia) “public works and water management, including the construction and maintenance of waterways and roads, and flood protection and prevention”.<\/p>\n\n\n\n

One of the services of Rijkswaterstaat is running a large network of sensors to monitor the water in and around the country. All data of these sensors is publicly available, and Rijkswaterstaat also publishes this on their Waterinfo<\/a> website, with nice maps and graphs. <\/p>\n\n\n\n

Command line interface<\/h2>\n\n\n\n

The graphical representation is not really accessible for blind people, some of them prefer the “raw” data in text tables so see how water levels change in time. One of my friends is blind AND interested in this data, so I decided to build him a nice CLI (command line interface). This was rather easy to build, since the data is also available by means of a nice API<\/a>. I named the tool wtr<\/em>, it’s written in Python and the code is open source (GPL). You can find it on GitHub<\/a>. The code is tested on Linux, but will probably run on any operating system that can run Python.<\/p>\n\n\n\n

I’m actually quite happy with the code, since it is probably my most professionally written Python code so far. I used tools like pylint<\/code> and black<\/code> to ensure the code syntaxt and formatting. On GitHub I have configured GitHub Actions which run pylint<\/code> for miscellaneous Python versions and do some functional tests on the code as well. I used Visual Studio Code<\/em> to write the code, with some help of GitHub Copilot<\/em>. This is a gen-AI tool that acts like a second developer sitting next to you, helping you to suggest and improve code. This really speeds up the process.<\/p>\n\n\n\n

Caching metadata<\/h2>\n\n\n\n

The API provides two types of data: the actual measurements and metadata. The metadata includes location names, coordinates, units and descriptions of the measurements. When you run wtr<\/code> for the first time, it will download this metadata and cache it in a SQlite database. When this database gets lost, it will simply be recreated, since it is only for caching the metadata. No measurements are stored locally.<\/p>\n\n\n\n

Getting started<\/h2>\n\n\n\n

To list observations you need two things: the type of measure and the location.<\/p>\n\n\n\n

Run this command to obtain a list of available measures:<\/p>\n\n\n\n

$ wtr list -c measures<\/strong>\nCompartment_Code (Description): Measure (Description) - Note\nBS (Bodem\/Sediment): ACTVTCCTTE (Activiteitsconcentratie) - Activiteitsconcentratie americium 241 in Bodem\/Sediment t.o.v. drooggewicht in Bq\/kg\nBS (Bodem\/Sediment): D10 (maximale korreldiameter voor 10% van het monstervolume) - Maximale korreldiameter voor 10% van het monstervolume Bodem\/Sediment t.o.v. drooggewicht in um\n(...)<\/em><\/code><\/pre>\n\n\n\n
Run this command to see all locations, including the available measure(s):<\/p>\n\n\n\n
$ wtr list -c locations<\/strong>\nLocation (Name): Compartment_Code (Description) - Measure (Description)\nA12 (A12 platform): LT (Lucht) - WINDRTG (Windrichting)\nA12 (A12 platform): LT (Lucht) - WINDSTOOT (Maximale 3\" windstoot in de afgelopen 10 minuten)\n(...)<\/em><\/code><\/pre>\n\n\n\n
So let’s say you want information about the flow rate in one of our rivers, dutch stroomsnelheid<\/em>. First find the corresponding measure, for example:<\/p>\n\n\n\n
$ wtr list -c measures | grep -i stroom<\/strong>\nOW (Oppervlaktewater): STROOMRTG (Stroomrichting) - Stroomrichting Oppervlaktewater t.o.v. ware Noorden in graad\nOW (Oppervlaktewater): STROOMSHD (Stroomsnelheid) - Stroomsnelheid Oppervlaktewater m\/s<\/code><\/pre>\n\n\n\n
So the measure we need is named STROOMSHD<\/em><\/code>. Next find locations for this measure:<\/p>\n\n\n\n
$ wtr list -c locations | grep STROOMSHD\nALBL (Alblasserdam): OW (Oppervlaktewater) - STROOMSHD (Stroomsnelheid)\nALBSDRTOVR (Alblasserdam rechteroever (nabij brug)): OW (Oppervlaktewater) - STROOMSHD (Stroomsnelheid)\nBORNDP (Borndiep): OW (Oppervlaktewater) - STROOMSHD (Stroomsnelheid)\n(...)<\/em><\/code><\/pre>\n\n\n\n
We are interested in the location ALBL<\/em>, so let’s limit the list<\/p>\n\n\n\n
$ wtr observe -l ALBL -m STROOMSHD\n#\n# Description: Stroomsnelheid Oppervlaktewater m\/s\n# Compartment: OW (Oppervlaktewater)\n# Measure: STROOMSHD (Stroomsnelheid)\n# Location: ALBL (Alblasserdam)\n#\n# Timestamp                      Value\n  2023-12-29T06:20:00.000+01:00  -0.427\n  2023-12-29T06:30:00.000+01:00  -0.348\n  2023-12-29T06:40:00.000+01:00  -0.272\n  2023-12-29T06:50:00.000+01:00  -0.227\n  2023-12-29T07:00:00.000+01:00  -0.15\n  2023-12-29T07:10:00.000+01:00  -0.075\n  2023-12-29T07:20:00.000+01:00  0.021\n  2023-12-29T07:30:00.000+01:00  0.125\n  2023-12-29T07:40:00.000+01:00  0.218\n  2023-12-29T07:50:00.000+01:00  0.286\n  2023-12-29T08:00:00.000+01:00  0.377\n  2023-12-29T08:10:00.000+01:00  0.465\n  2023-12-29T08:20:00.000+01:00  0.525\n  2023-12-29T08:30:00.000+01:00  0.567\n  2023-12-29T08:40:00.000+01:00  0.616\n  2023-12-29T08:50:00.000+01:00  0.662\n  2023-12-29T09:00:00.000+01:00  0.717\n  2023-12-29T09:10:00.000+01:00  0.78<\/code><\/pre>\n\n\n\n
Default the most actual 18 rows are shown, which represents the past 3 hours (most measurements do have a 10 minute interval). If you want to go further back in time, use the --rows<\/code> (or -r<\/code>) option, for example:<\/p>\n\n\n\n
$ wtr observe -l ALBL -m STROOMSHD -r 144  # 24 hours<\/code><\/pre>\n\n\n\n
Notes:<\/p>\n\n\n\n