This month, freedv-gui/FreeDV Reporter got the following bug fixes and feature enhancements:

• A minor tweak to the main window to have all section boxes be the same width.
• 800XA mode was added to set of modes used for the multiple receive feature.
• Serial PTT can now be enabled along with OmniRig.
• A delay can now be added prior to transmitting audio and before bringing the radio out of transmit mode (to avoid damage to accessories such as noise cancellation devices).
• A new feature to FreeDV Reporter to allow up to 15 character long custom messages to be reported to other users. (These messages go out solely over the internet, not over radio.)
• Frequency units can now be displayed and entered in kHz if the user enables the appropriate setting.
• A bug was fixed where the space bar was inappropriately putting the FreeDV application into transmit mode.
• Compiler errors with certain versions of MinGW (used to build Windows versions of FreeDV) have been fixed.
• A Windows-only issue was fixed that prevented the entry of very high frequencies (such as anything above 10 GHz).
• Extra column space was removed in the FreeDV Reporter window to better enable use on smaller displays.
• A new 60 meter frequency (5368.5 kHz) was added to the default calling frequency list to avoid interference on the previously used 60 meter frequency.
• Validation of input was enhanced when customizing the frequency list.
• A bug was fixed that caused the incorrect sample rates to be used for filters (when enabled).

ezDV also got the following new features:

• Morse code beeping of the last octet of the IP address on connection to the configured router (to make it easier to find on the network).
• Allow connection to be forced to FreeDV Reporter even if using a radio without CAT control support (e.g. non-Flex or Icom radios).
• Minor tweaks to the web UI to improve ease of configurability for supported Wi-Fi enabled radios.

More information can be found in the commit history below:

• freedv-gui
• ezDV
• freedv-reporter

(Note that all commit logs above were generated with the following command line:)

git log --author="member@email" --after "Month 1, 2023" --before "Month 31, 2023" --all > commit.log

Codec 2 Algorithm Description

This month I finished the Codec 2 algorithm description document. It was quite a lot of work, and ended up being 30 pages long. Thanks jimt for proof reading and Mooneer for helping with the automation (we rebuild the doc as part of our automated tests).

There was some discussion on this Codec 2 mailing list thread around the need for a formal specification versus the documentation/reference code/test approach I have taken in explaining Codec 2. I haven’t had any comments or questions on the technical content yet, I guess the audience interested in the DSP is small.

Codec 2 Machine Learning and Male Speech

For a few months I’ve been exploring the use of Machine Learning (ML) with Codec 2. This has been something of a side project, as I don’t feel competent in ML. While the field shows a lot of promise, in the past I have struggled to build any ML systems that actually do something useful. However my side project appears to work and I have some meaningful improvements in speech quality of the core Codec 2 vocoder. As I’m on a learning curve, I’ve only billed a fraction of the hours spent on this work to the ARDC grant. Thanks Jean-Marc for your tips on ML and inspiring work.

The project involved building a filter that “narrows” the bandwidth of vocal tract resonances (or formants) for low pitched male speakers. It’s these peaks (known as formants) that convey the information in speech. It addresses the problem with energy distribution that I mentioned in the October report.

The following plot shows the ML “inference” in action. The aqua plot is the smoothed spectrum (ML input), and red is the ML output, which is pretty close to the ideal (green).

The smoothed spectrum (aqua) is an intermediate processing step that reduces the information (and hence bit rate) we need to transmit. Unfortunately it messes up low pitched, male speakers, while high pitched speakers such as females pass through OK. This is puzzling, as it seems reasonable to assume that male and female speech has the same amount of information. I have a theory that the formant bandwidth is important for male speech. So I designed a ML system to recover the narrow formant bandwidths for males from the smoothed spectrum. It’s a bit like un-blurring an image. This is tricky using traditional DSP that can only do linear transformations, but finding complex, non-linear relationships is something that ML is meant to be good at.

Here are some speech samples, with Codec 2 3200 as a reference (used for M17, and roughly the same quality as AMBE/MELP). Especially through headphones, the ML input sounds buzzy and muffled. Please note the ML output sample is not quantised (that’s the next step), but previous work suggests this could be quantised to about 1600 bits/s at the same quality using traditional DSP, and perhaps lower with ML based quantisation.

This addresses a long standing mystery (to me at least) of why low pitched male speakers sound poorer than females when the spectrum is coarsely represented. I had previously addressed this issue with a poorly understood “post filter” that used parameters based on educated guesses.

Next step is to see if we can use ML to help quantise the Codec 2 model parameters. If I can gain skills in this area we may be able to improve speech quality at a given bit rate and perhaps robustness to channel errors.

Merry Christmas! This release contains the following content:

1. Bugfixes:
   • Use SetSize/GetSize instead of SetClientSize/GetClientSize to work around startup sizing issue. (PR #611)
   • Check for RIGCAPS_NOT_CONST in Hamlib 4.6. (PR #615)
   • Make main screen gauges horizontal to work around sizing/layout issues. (PR #613)
   • Fix compiler issue with certain versions of MinGW. (PR #622)
   • Suppress use of space bar when in RX Only mode. (PR #623)
   • Fix Windows-specific issue preventing entry of very high frequencies. (PR #624)
2. Enhancements:
   • Add option to add a delay after starting TX and before ending TX. (PR #618)
   • Allow serial PTT to be enabled along with OmniRig. (PR #619)
   • Add 800XA to multi-RX list. (PR #617)
   • Add logic to report status message to FreeDV Reporter. (PR #620)
   • Allow display and entry of frequencies in KHz. (PR #621)
   • Add 5368.5 KHz to the default frequency list. (PR #626)

More information and download links can be found here.

This month, the freedv-gui project got several good feature improvements and bug fixes:

• macOS: include entitlements to allow usage of audio devices on Sonoma.
• Added the ability to force RX Only reporting for FreeDV Reporter.
• Operating system usage is now logged with FreeDV Reporter (to help guide future development).
• New calling frequencies for 160 and (for IARU Region 2) 80 and 40 meters: 1997 KHz, 3850 KHz, 7197 KHz.
• A visible “Help” button has been added to the main window of the application that goes directly to a list of resources for obtaining assistance.
• Fixed an issue preventing FreeDV from compiling when using Hamlib 4.6.
• Fixed issues with reloading previous window sizing.
• Fixed issue preventing the reload of the previous manually-entered frequency.

Additionally, ezDV got some significant improvements as well:

• Improvements to reduce sleep/off power usage due to new “charging status” mode.
• Fixed check for battery fuel gauge chip to allow operation without it being included on the board.
• Now reports as running “other” for the OS to FreeDV Reporter.
• Added ability to customize the hostname of ezDV (in case a user has multiple devices on the same network).
• Added ability to select Wi-Fi networks from a list (instead of needing to manually enter the SSID).

FreeDV Reporter updates:

• Added link between Discord and FreeDV Reporter.
• Fixed website formatting issues.
• Added escaping for HTML to prevent scripting attacks.

More information can be found in the commit history below:

• freedv-gui
• ezDV
• freedv-reporter

This month I’ve been busy documenting the Codec 2 algorithm. Codec 2 evolved from some code I developed in the 1990s when I studied speech coding. Around 2009 I pulled that code off the shelf and turned it into a practical speech codec, adding bits and pieces over the next decade. So now I’m making an effort to pull all the algorithms details together into one document. It’s a work in progress, currently located in this PR.

One mission of this project is to explain how speech codecs work, as they are often shrouded in mystery and indeed understanding is often discouraged by various closed source strategies.

I’m explaining Codec 2 at two levels, the first is aimed at the Radio Amateur, at a technical level that could be published in a Ham Radio magazine. Codec 2 was written by Hams for Hams so this is important. The second level is a deeper dive into the DSP, using math where appropriate, and assuming a familiarity with signal processing.

As I pull the various building blocks together and write about each one, I realise it is rather complicated. There are a lot of moving parts, and it’s been a while since I looked at it as a whole. So it’s important to document the algorithm to make it easier to understand for others, and as a baseline to help reach our ARDC project goals like improved speech quality and robustness over HF channels.

I feel the algorithm is best described by a combination of the source code, text description and math. Sometimes the source code does a better job of explaining than text or math, and want to avoid describing things twice (don’t repeat yourself principle).

Automated tests can be useful in explaining the algorithm, so I have made a list of tests and source code cleans ups I’d like to work on. I’m also including some GNU Octave simulations that let us peer into the algorithm and run it step by step – like a software oscilloscope.

Some of the code is 30 years old now, and still builds and runs cleanly. A testament to the longevity of C I guess. Signal processing, like math and physics, tends to remain relevant over time. Our human speech production hardware is somewhat older than 30 years and isn’t likely to change soon.

In early 2023, we were fortunate to receive a grant from the ARDC, to support a two year program of FreeDV development. This post in an excerpt from our grant application that describes the project plan we are now busy executing.

I have a background in project planning, so with the FreeDV Project Leadership team (PLT) designed the plan in work package form, much as I would plan a commercial project.

The work package breakdown is provided below, however we expect this will evolve over the course of the project. The grant funding will be directed by the FreeDV PLT to best accomplish our mission of creating great free software and hardware for digital voice communication over HF radio.

WP1000 Project Management. Budget at 10% of project. Coordinate resources, hiring a DSP engineer, and communication with stakeholders.  Evaluate progress against the plan, revise plan as project evolves.

WP2000 Codec 2 Improvement.  In this WP we will attempt to improve the speech quality of two Codec 2 modes, at a low (around 700 bit/s) and moderate (1200-2400) bit/s rate. Establish requirements for Codec 2 (bit rate, voice quality, background noise robustness, CPU load). The most promising areas are spectral quantisation and a better excitation model.  Develop algorithms for handling background noise. The moderate bit rate modes in Codec 2 (above 700 bits/s) have not been actively developed for many years, so there is likely some low hanging fruit here.  Progress in this WP will be measured by conducting tests using speech samples processed with the revised modes and SSB/commercial codecs as references.

WP3000 Rag chew mode.  A waveform for HF digital voice will be designed to use the new 1200-2400 bits/s mode from WP2000. Waveform design involves designing the modem, FEC code selection, and implementation.  The new waveform will be supported by automated tests, and integrated into libcodec2 and freedv-gui for use over the air.  A option for a high quality mode is to employ the neural speech coding (as per the experimental FreeDV 2020 modes). 

WP4000 Low SNR mode. Using our existing 700D/700E modes, carefully investigate modem performance over HF multipath channels (real and simulated), measure performance compared to theory, and look for potential optimisations.  Investigate the use of non-pilot symbol synchronization methods to lower waveform overheads. Investigate methods for Peak to Average Power Ratio (PAPR) reduction by Vector Quantiser (VQ) and interleaver optimisation.  With the new 700 bit/s codec from WP2000, develop a new low SNR waveform, integrate into libcodec2 and freedv-gui for use over the air.  Success will be measured by conducting controlled over the air experiments where we compare SSB to the low SNR mode using the same speech samples and Peak power.  The goal is to outperform SSB at low SNRs.

WP5000 Commercial radio integration.  When WP3000 and WP4000 are mature, reach out to commercial radio companies.  Promote the benefits of FreeDV: open standards, no license fees, and performance competitive to SSB.  Options for integration include linking libcodec2 into their radios DSP/Host CPU, or a plug-in OEM FreeDV module using a STM32 or ESP32 microcontroller.  Success will be measured against the goal of integrating FreeDV into at least 2 COTS HF radios serving the Amateur Radio market.

WP6000 HF Data Modes: Extend the current suite of HF data waveforms by developing and testing a high bit rate/high SNR QAM mode and sub 0dB SNR low SNR mode. Work with FreeDATA to integrate and test Over the Air (OTA).  Conduct an automated test campaign over many months that provides objective evidence of system performance over real world channels.

WP7000 ezDV Development: Extend ezDV with additional functionality and for ease of use. Develop a usable enclosure for the ezDV board and optimize the hardware for production, as well as manufacture a small run of devices for general ham use and testing. 

WP8000 Packaging Improvements: Improves the codebase and workflow to enable easier packaging of official releases by the major Linux distributions. Code signing (for platforms that heavily encourage/require it) will be put in place.

WP9000 CPU load optimizations: These are items intended to reduce the CPU required to run FreeDV. This activity is scheduled to start after the new modes have been developed.

WP10000: freedv-gui improvements: These are items intended to improve the general usability of the FreeDV application.

WP11000 Ongoing development and maintenance: libcodec2, freedv-gui, and embedded platforms (ezDV, the SM1000 successor). In addition to ongoing maintenance such as documentation of Codec 2 algorithm and FreeDV waveforms to a professional level.  This will encourage commercial adoption.

WP12000 Ongoing promotion: Provide a presence at major hamventions in the form of a FreeDV/Codec2 booth and/or talks given by the group. This presence would have live demos of FreeDV (both using the application and with embedded devices such as ezDV). There would be up to five hamventions targeted in this grant, with the ones selected based on historical attendance figures.

The project will also participate in various forms of other promotion (for example, on-air events, mailing lists, social media and discussion forums). Usage figures can be measured from online reporting during events (e.g. reports that are sent from the FreeDV PC application to the PSK Reporter service).

FreeDV version 1.9.5 has been released, which contains the following content:

1. Bugfixes:
   • Fix bug preventing frequency updates from being properly suppressed when frequency control is in focus. (PR #585)
   • Fix bug preventing 60 meter frequencies from using USB with DIGU/DIGL disabled. (PR #589)
   • Additional fix for PR #561 to parse/format frequencies using current locale. (PR #595)
   • Add entitlements to work around macOS Sonoma permissions bug. (PR #598)
   • Fix bug preventing FreeDV Reporter window from closing after resetting configuration to defaults. (PR #593)
   • Fix bug preventing reload of manually entered frequency on start. (PR #608)
2. Enhancements:
   • FreeDV Reporter: Add support for filtering the exact frequency. (PR #596)
   • Add confirmation dialog box before actually resetting configuration to defaults. (PR #593)
   • Add ability to double-click FreeDV Reporter entries to change the radio’s frequency. (PR #592)
   • FreeDV Reporter: Add ability to force RX Only reporting in Tools->Options. (PR #599)
   • Add new 160m/80m/40m calling frequencies for IARU R2. (PR #601)
   • Add Help button to allow users to get help more easily. (PR #607)
3. Build system:
   • Upgrade wxWidgets to 3.2.4. (PR #607)
4. Other:
   • Report OS usage to FreeDV Reporter. (PR #606)

More information and download links can be found here.

This post is summary of the work I have performed in October 2023 for our Enhancing HF Digital Voice With FreeDV ARDC grant.

Acquisition

My deep dive into the OFDM modem algorithms continues, this month focusing on the algorithms used by a FreeDV receiver to lock onto off air signals. The goal is fast sync with up to +/- 200 Hz frequency offset on low SNR multipath fading channels. In the past this has been quite a challenge, and was a source of problems with the earlier versions of FreeDV 700D.

This time around I’m using a little probability to describe the chances of successfully acquiring the FreeDV signal. A 90% chance means that 9 times out of 10 you will acquire the FreeDV signal in one frame (about 180ms). On poorer channels it may take 2 or 3 frames. Some other problems to watch out for are false acquisition – we don’t want to output speech when there is random noise on the channel, or acquire on a carrier when someone is tuning up. So there is a trade off in being sensitive enough to detect weak signals, but ignore those that aren’t valid FreeDV signals.

Anyway, I’ve been working through those issues one by one, doing a little math, running some simulations, and writing up the results. The acquisition work is documented in Section 6 of this report. It’s getting close to the point where we can update the OFDM modem code and try it over the air.

Codec 2

I’ve started working on Codec 2 again, revisiting some of the algorithms I prototyped in the ratek resampler study. In particular I’m looking at why low pitched speakers like males require a higher bit rate to quantise the spectrum than high pitched speakers (females or children). I have a theory that it is related to the distribution of energy over a pitch cycle.

Here’s a plot of some speech that shows the problem. At top we have the original (input) speech, at bottom after it’s passed through Codec 2. Notice how at the bottom the signal decays before the end of each cycle? The energy is confined to the start of each pitch cycle, rather than being more evenly distributed like the top plot. This appears to be related to a drop in speech quality with male speakers.

If we can understand the sources of distortion, we can improve Codec 2 speech quality at all bit rates.

Administration

On the admin side I’ve been writing up some Work Package descriptions – chunks of work we need to get done to reach the goals of our project. They contain a list of tasks to be done and a description of deliverables. The idea is to be very clear about the work we need done so it slots in neatly with the rest of the project. If you are interested in working on any of these (as a paid or volunteer team member), please contact us.

The development of various policies is continuing – we’ve put a freeze on maintenance for existing features that are likely to be superseded, in order to focus on the new and exciting work. We’re also working on a process for reviewing feature requests – we want to make sure any significant work we do is lined up with our project goals like improving Codec 2 and enhancing HF Digital voice. As our resources are finite we need to “filter” feature requests somehow.

If you want to contribute code or have a feature request for this project, it’s a very good idea to contact us before writing any code or raising a PR. We have many years of experience and very good idea of what needs to be done. We could really use your skills and enthusiasm, but would like to make sure we work together in ways that will most benefit this project.

The FreeDV project recently went to Pacificon in San Ramon, California to discuss the latest updates to the project as well as interact with its users. On the first day prior to the hamfest officially opening, we set up a FreeDV booth just down the hall from the ATV booth and across from Bioenno Power, which had several cool things on it:

(From left to right: DVB-T transceiver, Mel Whitten K0PFX, the latest SM1000, two separate ezDV setups with the Icom IC-705 and Elecraft KX3, and a display showing the latest FreeDV activity on FreeDV Reporter.)

Over the course of the weekend, the booth got significant traffic from hams from the San Francisco Bay Area and beyond interested in HF digital voice. One person (John AL7ID) even came from the Fairbanks (Alaska) area!

Mooneer Salem (K6AQ) also gave a talk introducing FreeDV and explaining how to get on the air using it, including discussing the software and hardware options available. As a result of the presentation and the presence at the booth, we were able to provide assistance to several users (some over email after the show concluded) who previously had difficulty getting FreeDV running:

As an additional surprise, the M17 project also had a booth at Pacificon this year, where they had their latest OpenHT project on display. Wojciech Kaczmarski SP5WWP, the person who started M17, also happened to be there and hung out with us at the booth for a bit:

All in all, it was definitely a fun and worthwhile experience. Hope to see you guys again next year!

FreeDV Update – Sep 2023