Automated Recording of the HamSCI WWV/WWVH Science Signal for Sunrise Festival

I'm planning on participating in the Sunrise Festival, which is a citizen science campaign run by the HamSCI group. Over the 30 April 2022 weekend, hams from all over North America are encouraged to record the WWV and WWVH Scientific Test Signal, transmitting at 8 and 48 minutes past the hour.

I previously used my KiwiSDR to receive these signals, and luckily there is an easy API for the KiwiSDR that will allow us to save recordings from the SDR.

kiwirecorder.py

kiwirecorder.py, written by John Seamons ZL/KF6VO, is a simple python program that remotely connects to a KiwiSDR and saves spectrum locally. The output file can be demodulated audio (SSB or AM) like you get from a traditional radio, or raw IQ files which are spectrum recordings. We are much more interested in these raw IQ files, as any type of demodulation (such as AM or FM) removes information from the recording.

To check out kiwirecorder in your home folder with git clone https://github.com/jks-prv/kiwiclient. I'm running this on a standard Ubuntu 20.04 installation, the server edition works well because there is no GUI. System requirements include Python3 and numpy, which is easily installed with sudo apt install python3-numpy.

Spectrum Recording

I mostly grabbed these arguments from wsprdaemon. Remember that you need to have at least one free audio channel per frequency on the KiwiSDR for kiwirecorder to connect and record spectrum.

Here's the command:

python3 -u /home/user/kiwiclient/kiwirecorder.py --freq=10000 --server-host=192.168.1.82 --server-port=8073 --OV --user=hamsci_sunrise --password=NULL --agc-gain=60 --quiet --no_compression --modulation=iq --fn=CALLSIGN_WWV_10MHz_$(date -u +%Y-%m-%d)_$(date -u +%H-%M)_iq --tlimit=60

Explanation of arguments:

  • --freq=10000 Frequency in kHz. WWV transmits on 2.5, 5, 10, 15, and 20 MHz.
  • --server-host=192.168.1.82 The IP address of the KiwiSDR. Please ask before using a public server!
  • --server-port=8073 Port of server.
  • --OV Print "ADC OV" message when Kiwi ADC is overloaded.
  • --user=hamsci_sunrise This "username" . Any name can be used, but hamsci_sunrise gives the administrators an idea of what you're doing.
  • --password=NULL Password for KiwiSDR if needed. If no password, use NULL.
  • --agc-gain=60 Max gain for the AGC.
  • --quiet Suppress any standard output.
  • --no_compression Don't use compression.
  • --modulation=iq Save IQ modulation format. Could also use am if you want to save an audio recording.
  • --fn=CALLSIGN_WWV_10MHz_$(date -u +%Y-%m-%d)_$(date -u +%H-%M)_iq This saves the filename in the appropriate Sunrise format. Callsign first, then the TX station, frequency, UTC YYYY-MM-DD_HH-MM timestamp, and _iq specifying this is an IQ file. Use _am for a regular AM audio file.
  • --tlimit=60 Record for 60 seconds, then stop.

The $(date -u +%Y-%m-%d) is a bash command substitution, which takes the output of the date command and puts it into the filename. The date command prints the current date/time, make sure to use the -u flag to get UTC time.

When you're testing this command, you can remove the --quiet flag to see what is actually happening, and also change the --tlimit argument to something shorter. I would also recommend setting --modulation to am so you can just play the audio file. If you can hear the metronomic tones of WWV through your headphones, you know you have all the settings correct.

Confusingly, even if you record an IQ file, the file will still have a .wav extension. It is not a real .wav audio file, and can't be played with an audio program.

One-off Recording

Since there's only one or two minutes per hour when WWV or WWVH actually transmits the science signal, instead of waiting around (then missing the time because you fell down a Wikipedia rabbit hole), use the at command to schedule the command at the appropriate time. at works best when passed an entire command via pipe.

echo "python3 -u /home/user/kiwiclient/kiwirecorder.py --freq=10000 --server-host=192.168.1.82 --server-port=8073 --OV --user=hamsci_sunrise --password=NULL --agc-gain=60 --quiet --no_compression --modulation=iq --fn=CALLSIGN_WWV_10MHz_$(date -u +%Y-%m-%d)_$(date -u +%H-%M)_iq --tlimit=60" | at 18:08 today

You can see what commands are scheduled with atq and delete mistakes with atrm 1.

Automated Recording with cron job

Since we want to set up these recordings to happen at 8 minutes after the hour for WWV and 48 minutes after the hour for WWVH, cron is the right tool to use.

Unfortunately, cron treats percent signs as newlines, so just stuffing the above command into a crontab won't work. The best way to do this is actually create a standalone bash script file, but in a pinch the percent signs can be escaped out with a backwards slash:

08 * * * * python3 -u /home/user/kiwiclient/kiwirecorder.py --freq=10000 --server-host=192.168.1.82 --server-port=8073 --OV --user=hamsci_sunrise --password=NULL --agc-gain=60 --quiet --no_compression --modulation=iq --fn=CALLSIGN_WWV_10MHz_$(date -u +\%Y-\%m-\%d)_$(date -u +\%H-\%M)_iq --tlimit=60

cron will put the files in your home folder. Each 1-minute IQ file is approximately 2.8 Megabytes.

Signal Analysis

The HamSCI group helpfully put together a Jupyter notebook for pulling some science data out of the noise. It's still under heavy development. If you don't have python and/or Jupyter notebook installed on your computer you can run it in a browser via Binder.

Here's a picture of what I received at my station. The top plot is amplitude (volume) of the signal, and the bottom is a spectrogram of the signal. Interestingly, you can also see the modified IRIG H markers at the bottom of the spectrogram.

Plot of received WWV test signal

Looking at the correlation between the transmitted signal, called the Template, and what I received at my station shows that the first white noise burst happened about 9 seconds after the start of the file. It's supposed to be at 10 seconds after the minute, so this shows that it takes about 1 second for the kiwirecorder cron job to begin collecting data.

Cross-correlation of white noise burst

The Jupyter notebook also looks at when the chirps happen, there's much more power in the chirps versus the white noise, so the correlation is much stronger.

Cross-correlation of white noise burst

I hope the HamSCI scientists will keep updating the analysis notebooks.

Bulk Signal Analysis

Unfortunately, the Juypter notebook is not a great tool for bulk signal analysis. Typing in every file name is way too slow, especially with 5 RF frequencies per hour, times 24 hours, times two transmit stations.

Luckily, it's all just python code, so with just some simple modifications we can accept filename input from the command line, then use a for loop to cycle through all the IQ files we have saved.

Changing the input file to sys.argv[1] allows an input from the command line. Since sys is automatically imported by Jupyter (but not by the regular python interpreter), we'll need to import it as well.

import sys
fname1 = sys.argv[1]

This allows us to input a file name on the command line. Python will still make and display the graphs, and pause execution until the graphs are closed. Commenting out all the plt. and fig. sections in both the sunrisefest.py and sunrisefest_module.py stops the display of graphs.

From the standard Jupyter notebook, we'll also need to comment out all the get_ipython() functions, as there is no ipython in the standard Python installation.

Creating a for loop to cycle through all the .wav IQ files in a particular folder is easy enough.

for file in CALLSIGN_WWV_*.wav; do python3 sunrisefest.py "$file"; done

This loop only runs through the WWV files; if you also recorded WWVH, open a second terminal and run the same command with CALLSIGN_WWVH_*.wav. Since a for loop is single-threaded, running two loops at the same time makes things go two times faster.

I'm hoping that sometime soon the HamSCI team will release a purpose-built decoder that makes it easier to bulk decode these files, but for now this is what I hacked together.

Sunrise Festival

As I write this, the Sunrise Festival is less than two weeks away. I've verified the cron job works, tested the analysis Jupyter notebook, and everything looks good. All systems go!

links