This month I’ve been busy documenting the Codec 2 algorithm. Codec 2 evolved from some code I developed in the 1990s when I studied speech coding. Around 2009 I pulled that code off the shelf and turned it into a practical speech codec, adding bits and pieces over the next decade. So now I’m making an effort to pull all the algorithms details together into one document. It’s a work in progress, currently located in this PR.
One mission of this project is to explain how speech codecs work, as they are often shrouded in mystery and indeed understanding is often discouraged by various closed source strategies.
I’m explaining Codec 2 at two levels, the first is aimed at the Radio Amateur, at a technical level that could be published in a Ham Radio magazine. Codec 2 was written by Hams for Hams so this is important. The second level is a deeper dive into the DSP, using math where appropriate, and assuming a familiarity with signal processing.
As I pull the various building blocks together and write about each one, I realise it is rather complicated. There are a lot of moving parts, and it’s been a while since I looked at it as a whole. So it’s important to document the algorithm to make it easier to understand for others, and as a baseline to help reach our ARDC project goals like improved speech quality and robustness over HF channels.
I feel the algorithm is best described by a combination of the source code, text description and math. Sometimes the source code does a better job of explaining than text or math, and want to avoid describing things twice (don’t repeat yourself principle).
Automated tests can be useful in explaining the algorithm, so I have made a list of tests and source code cleans ups I’d like to work on. I’m also including some GNU Octave simulations that let us peer into the algorithm and run it step by step – like a software oscilloscope.
Some of the code is 30 years old now, and still builds and runs cleanly. A testament to the longevity of C I guess. Signal processing, like math and physics, tends to remain relevant over time. Our human speech production hardware is somewhat older than 30 years and isn’t likely to change soon.
ARDC Grant Project Plan
In early 2023, we were fortunate to receive a grant from the ARDC, to support a two year program of FreeDV development. This post in an excerpt from our grant application that describes the project plan we are now busy executing.
I have a background in project planning, so with the FreeDV Project Leadership team (PLT) designed the plan in work package form, much as I would plan a commercial project.
The work package breakdown is provided below, however we expect this will evolve over the course of the project. The grant funding will be directed by the FreeDV PLT to best accomplish our mission of creating great free software and hardware for digital voice communication over HF radio.
WP1000 Project Management. Budget at 10% of project. Coordinate resources, hiring a DSP engineer, and communication with stakeholders. Evaluate progress against the plan, revise plan as project evolves.
WP2000 Codec 2 Improvement. In this WP we will attempt to improve the speech quality of two Codec 2 modes, at a low (around 700 bit/s) and moderate (1200-2400) bit/s rate. Establish requirements for Codec 2 (bit rate, voice quality, background noise robustness, CPU load). The most promising areas are spectral quantisation and a better excitation model. Develop algorithms for handling background noise. The moderate bit rate modes in Codec 2 (above 700 bits/s) have not been actively developed for many years, so there is likely some low hanging fruit here. Progress in this WP will be measured by conducting tests using speech samples processed with the revised modes and SSB/commercial codecs as references.
WP3000 Rag chew mode. A waveform for HF digital voice will be designed to use the new 1200-2400 bits/s mode from WP2000. Waveform design involves designing the modem, FEC code selection, and implementation. The new waveform will be supported by automated tests, and integrated into libcodec2 and freedv-gui for use over the air. A option for a high quality mode is to employ the neural speech coding (as per the experimental FreeDV 2020 modes).
WP4000 Low SNR mode. Using our existing 700D/700E modes, carefully investigate modem performance over HF multipath channels (real and simulated), measure performance compared to theory, and look for potential optimisations. Investigate the use of non-pilot symbol synchronization methods to lower waveform overheads. Investigate methods for Peak to Average Power Ratio (PAPR) reduction by Vector Quantiser (VQ) and interleaver optimisation. With the new 700 bit/s codec from WP2000, develop a new low SNR waveform, integrate into libcodec2 and freedv-gui for use over the air. Success will be measured by conducting controlled over the air experiments where we compare SSB to the low SNR mode using the same speech samples and Peak power. The goal is to outperform SSB at low SNRs.
WP5000 Commercial radio integration. When WP3000 and WP4000 are mature, reach out to commercial radio companies. Promote the benefits of FreeDV: open standards, no license fees, and performance competitive to SSB. Options for integration include linking libcodec2 into their radios DSP/Host CPU, or a plug-in OEM FreeDV module using a STM32 or ESP32 microcontroller. Success will be measured against the goal of integrating FreeDV into at least 2 COTS HF radios serving the Amateur Radio market.
WP6000 HF Data Modes: Extend the current suite of HF data waveforms by developing and testing a high bit rate/high SNR QAM mode and sub 0dB SNR low SNR mode. Work with FreeDATA to integrate and test Over the Air (OTA). Conduct an automated test campaign over many months that provides objective evidence of system performance over real world channels.
WP7000 ezDV Development: Extend ezDV with additional functionality and for ease of use. Develop a usable enclosure for the ezDV board and optimize the hardware for production, as well as manufacture a small run of devices for general ham use and testing.
WP8000 Packaging Improvements: Improves the codebase and workflow to enable easier packaging of official releases by the major Linux distributions. Code signing (for platforms that heavily encourage/require it) will be put in place.
WP9000 CPU load optimizations: These are items intended to reduce the CPU required to run FreeDV. This activity is scheduled to start after the new modes have been developed.
WP10000: freedv-gui improvements: These are items intended to improve the general usability of the FreeDV application.
WP11000 Ongoing development and maintenance: libcodec2, freedv-gui, and embedded platforms (ezDV, the SM1000 successor). In addition to ongoing maintenance such as documentation of Codec 2 algorithm and FreeDV waveforms to a professional level. This will encourage commercial adoption.
WP12000 Ongoing promotion: Provide a presence at major hamventions in the form of a FreeDV/Codec2 booth and/or talks given by the group. This presence would have live demos of FreeDV (both using the application and with embedded devices such as ezDV). There would be up to five hamventions targeted in this grant, with the ones selected based on historical attendance figures.
The project will also participate in various forms of other promotion (for example, on-air events, mailing lists, social media and discussion forums). Usage figures can be measured from online reporting during events (e.g. reports that are sent from the FreeDV PC application to the PSK Reporter service).
David’s FreeDV Update Oct 2023
This post is summary of the work I have performed in October 2023 for our Enhancing HF Digital Voice With FreeDV ARDC grant.
Acquisition
My deep dive into the OFDM modem algorithms continues, this month focusing on the algorithms used by a FreeDV receiver to lock onto off air signals. The goal is fast sync with up to +/- 200 Hz frequency offset on low SNR multipath fading channels. In the past this has been quite a challenge, and was a source of problems with the earlier versions of FreeDV 700D.
This time around I’m using a little probability to describe the chances of successfully acquiring the FreeDV signal. A 90% chance means that 9 times out of 10 you will acquire the FreeDV signal in one frame (about 180ms). On poorer channels it may take 2 or 3 frames. Some other problems to watch out for are false acquisition – we don’t want to output speech when there is random noise on the channel, or acquire on a carrier when someone is tuning up. So there is a trade off in being sensitive enough to detect weak signals, but ignore those that aren’t valid FreeDV signals.
Anyway, I’ve been working through those issues one by one, doing a little math, running some simulations, and writing up the results. The acquisition work is documented in Section 6 of this report. It’s getting close to the point where we can update the OFDM modem code and try it over the air.
Codec 2
I’ve started working on Codec 2 again, revisiting some of the algorithms I prototyped in the ratek resampler study. In particular I’m looking at why low pitched speakers like males require a higher bit rate to quantise the spectrum than high pitched speakers (females or children). I have a theory that it is related to the distribution of energy over a pitch cycle.
Here’s a plot of some speech that shows the problem. At top we have the original (input) speech, at bottom after it’s passed through Codec 2. Notice how at the bottom the signal decays before the end of each cycle? The energy is confined to the start of each pitch cycle, rather than being more evenly distributed like the top plot. This appears to be related to a drop in speech quality with male speakers.
If we can understand the sources of distortion, we can improve Codec 2 speech quality at all bit rates.
Administration
On the admin side I’ve been writing up some Work Package descriptions – chunks of work we need to get done to reach the goals of our project. They contain a list of tasks to be done and a description of deliverables. The idea is to be very clear about the work we need done so it slots in neatly with the rest of the project. If you are interested in working on any of these (as a paid or volunteer team member), please contact us.
The development of various policies is continuing – we’ve put a freeze on maintenance for existing features that are likely to be superseded, in order to focus on the new and exciting work. We’re also working on a process for reviewing feature requests – we want to make sure any significant work we do is lined up with our project goals like improving Codec 2 and enhancing HF Digital voice. As our resources are finite we need to “filter” feature requests somehow.
If you want to contribute code or have a feature request for this project, it’s a very good idea to contact us before writing any code or raising a PR. We have many years of experience and very good idea of what needs to be done. We could really use your skills and enthusiasm, but would like to make sure we work together in ways that will most benefit this project.