feed.xml

<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Quiet Project</title>
    <description>Transmit data with sound. Quiet Project offers the ability to build native binaries that work with your soundcard and a JS implementation that uses Web Audio
</description>
    <link>https://quiet.github.io/</link>
    <atom:link href="https://quiet.github.io/feed.xml" rel="self" type="application/rss+xml"/>
    <pubDate>Mon, 17 Sep 2018 02:33:59 -0700</pubDate>
    <lastBuildDate>Mon, 17 Sep 2018 02:33:59 -0700</lastBuildDate>
    <generator>Jekyll v3.8.3</generator>
    
      <item>
        <title>How Libcorrect Corrects Errors, Part I</title>
        <description>&lt;p&gt;&lt;a href=&quot;https://github.com/quiet/libcorrect&quot;&gt;Libcorrect&lt;/a&gt; is a BSD-licensed library for forward error correction. What this means is that it can be given a payload of data and apply specially chosen redundancy. The payload and redundancy are transmitted together and then sent along a medium that might add errors. Once received, the entire structure is decoded into the original payload. The redundancy allows libcorrect to decode the payload as long as the added errors do not exceed some limit. Because it is BSD-licensed, libcorrect can be used for personal and commercial applications.&lt;/p&gt;

&lt;p&gt;This blog post is the first in a series which will explain how libcorrect works and what techniques it uses to decode more quickly. This information will hopefully help others who wish to learn more about FEC, implement their own error correction, or contribute to libcorrect. When I first set out to write this library, I had only a passing familiarity with each of these algorithms and managed to piece together some idea of how they worked. It took me quite a while to research each algorithm and my hope is that this post can help others shortcut past some of this time spent.&lt;/p&gt;

&lt;p&gt;Libcorrect currently implements two kinds of error correction, &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolutional_code&quot;&gt;convolutional codes&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Reed%E2%80%93Solomon_error_correction&quot;&gt;Reed-Solomon error correction&lt;/a&gt;. It uses the &lt;a href=&quot;https://en.wikipedia.org/wiki/Viterbi_algorithm&quot;&gt;Viterbi algorithm&lt;/a&gt; to efficiently decode convolutional codes. Neither of these algorithms would be considered state of the art currently, but both were used extensively previously, with applications including dial-up modems, QR codes and long-range space communications. Convolutional codes are robust against Gaussian white noise – that is, transmission errors which occur randomly and according to a normal distribution. Reed-Solomon applies redundancy to a block of bytes and can repair errors that occur anywhere inside the block, even if all of the errors are contiguous to one another. Combining these techniques gives a powerful basis for transmitting in the presence of noise.&lt;/p&gt;

&lt;p&gt;This post will focus on the fundamentals of convolutional codes and the Viterbi algorithm. Part II will examine the techniques used by libcorrect to accelerate convolutional code decoding, which includes both portable optimizations as well as Intel SSE vectorizations. Later posts will cover libcorrect’s use of Reed-Solomon error correction.&lt;/p&gt;

&lt;h2 id=&quot;convolutional-codes&quot;&gt;Convolutional Codes&lt;/h2&gt;

&lt;p&gt;There are a wide range of options available for forward error correction. For example, we might choose to take each bit of our message and repeat it 3 times. On the receiving end, we would examine each grouping of 3 bits and pick whichever bit appears in majority. We’d like a scheme that has good resilience to errors while not adding too much overhead to our transmission.&lt;/p&gt;

&lt;p&gt;Convolutional codes offer a way to encode information about each bit of our message over multiple bits in the transmission. This method effectively “smears out” each bit with its neighboring bits so that no transmitted bit contains only information about just one message bit. The encoder pushes each bit through a shift register and then transmits the outputs of carefully chosen &lt;em&gt;XOR&lt;/em&gt; operations. The decoder recreates the message by simulating possible message bits and measuring errors against the received transmission, then choosing the sequence of message bits with the least error. This method can recover the message even when the transmitted bits are received incorrectly periodically. The following sections will examine in more detail how this technique is implemented.&lt;/p&gt;

&lt;h3 id=&quot;shift-register&quot;&gt;Shift Register&lt;/h3&gt;

&lt;p&gt;The shift register is a small piece of memory that can store &lt;em&gt;k&lt;/em&gt; bits. It is called a shift register because we add a new bit to one end and each bit after slides down to the next cell. We only write one bit in at a time, but we read all the bits concatenated together. If a 4-bit shift register has the contents &lt;code class=&quot;highlighter-rouge&quot;&gt;{0, 1, 0, 1}&lt;/code&gt;, we will write that as &lt;code class=&quot;highlighter-rouge&quot;&gt;0101&lt;/code&gt;. New bits shift in on the left hand side and are discarded on the right. This conceptual device is at the heart of the convolutional code algorithm.&lt;/p&gt;

&lt;figure class=&quot;figure&quot;&gt;
&lt;img src=&quot;/assets/sr_anim.gif&quot; class=&quot;figure-img img-fluid img-rounded&quot; alt=&quot;Shift Register&quot; /&gt;
  &lt;figcaption class=&quot;figure-caption&quot;&gt;Shift Register&lt;/figcaption&gt;
&lt;/figure&gt;

&lt;h3 id=&quot;polynomials&quot;&gt;Polynomials&lt;/h3&gt;

&lt;p&gt;A polynomial is a series of bitwise &lt;em&gt;XOR&lt;/em&gt; operations that computes a parity bit – the polynomial operates in &lt;a href=&quot;https://en.wikipedia.org/wiki/GF(2)&quot;&gt;GF(2)&lt;/a&gt;. These polynomials will return &lt;em&gt;1&lt;/em&gt; if an odd number of bits are set and &lt;em&gt;0&lt;/em&gt; if an even number of bits are set. The polynomials are run with the contents of the shift register as their input. For example, the polynomial &lt;em&gt;x&lt;sup&gt;2&lt;/sup&gt;&lt;/em&gt; + &lt;em&gt;1&lt;/em&gt; computes the bitwise &lt;em&gt;XOR&lt;/em&gt; of the third newest bit and newest bits in the shift register, while &lt;em&gt;x&lt;sup&gt;3&lt;/sup&gt;&lt;/em&gt; + &lt;em&gt;x&lt;/em&gt; + &lt;em&gt;1&lt;/em&gt; would do bitwise &lt;em&gt;XOR&lt;/em&gt; of the fourth, second, and first newest bits.&lt;/p&gt;

&lt;figure class=&quot;figure&quot;&gt;
&lt;img src=&quot;/assets/poly1_anim.gif&quot; class=&quot;figure-img img-fluid img-rounded&quot; alt=&quot;x&amp;lt;sup&amp;gt;2&amp;lt;/sup&amp;gt; + 1&quot; /&gt;
  &lt;figcaption class=&quot;figure-caption&quot;&gt;x&lt;sup&gt;2&lt;/sup&gt; + 1&lt;/figcaption&gt;
&lt;/figure&gt;

&lt;figure class=&quot;figure&quot;&gt;
&lt;img src=&quot;/assets/poly2_anim.gif&quot; class=&quot;figure-img img-fluid img-rounded&quot; alt=&quot;x&amp;lt;sup&amp;gt;3&amp;lt;/sup&amp;gt; + x + 1&quot; /&gt;
  &lt;figcaption class=&quot;figure-caption&quot;&gt;x&lt;sup&gt;3&lt;/sup&gt; + x + 1&lt;/figcaption&gt;
&lt;/figure&gt;

&lt;p&gt;It is these polynomials that form the basis of our redundancy. For each bit that we will load into the shift register, we will transmit the output of at least two different polynomials. The original message itself is not transmitted. Using more than two polynomials can increase tolerance to noise, but comes at the cost of reducing transmission throughput. The polynomials are chosen carefully to complement each other and increase tolerance to noise.&lt;/p&gt;

&lt;h3 id=&quot;encoding&quot;&gt;Encoding&lt;/h3&gt;

&lt;p&gt;Encoding convolutional codes involves taking the message we want to send and feeding it one bit at a time through our shift register. The shift register will initially start with all zeros. Each time we feed in a new bit, we will save the output of each polynomial that we have chosen. We continue this process until the entire message has been fed into the shift register, followed by a sequence of 0s to flush the shift register. Once we have completed this process, we send the interleaved polynomial outputs. The transmitted message does not contain the bits of the original message.&lt;/p&gt;

&lt;figure class=&quot;figure&quot;&gt;
&lt;img src=&quot;/assets/encoding.png&quot; class=&quot;figure-img img-fluid img-rounded&quot; alt=&quot;Encoding Message&quot; /&gt;
  &lt;figcaption class=&quot;figure-caption&quot;&gt;Encoding Message&lt;/figcaption&gt;
&lt;/figure&gt;

&lt;h3 id=&quot;decoding&quot;&gt;Decoding&lt;/h3&gt;

&lt;p&gt;As described in the section on encoding, we’ve been given the outputs of the polynomials, not of the shift register itself. What we want to do is to work backwards from these polynomials to determine the &lt;em&gt;most likely&lt;/em&gt; input bit to the shift register that produced the received bits. When the decoding algorithm has finished, we hope to recover the series of input bits that were used to generate the polynomial bits that we received.&lt;/p&gt;

&lt;p&gt;At first glance this might seem impossible as there are many bits in the shift register but only 1 bit from each polynomial. The only way we can determine which input bit produced the polynomial bits is by simulating the shift register for each group of polynomial bits that we receive. We measure the error between the expected polynomial outputs for each shift register state and the polynomial bits that we received and accumulate the error for each group of inputs. This works well because the shift register’s current state is closely associated with its state at the previous bit. A shift register with the contents &lt;code class=&quot;highlighter-rouge&quot;&gt;1001000&lt;/code&gt; will contain &lt;code class=&quot;highlighter-rouge&quot;&gt;B100100&lt;/code&gt; when the next bit &lt;em&gt;B&lt;/em&gt; is shifted in.&lt;/p&gt;

&lt;p&gt;Let’s use the example 4-bit shift register with polynomials &lt;em&gt;x&lt;sup&gt;2&lt;/sup&gt;&lt;/em&gt; + &lt;em&gt;1&lt;/em&gt; and &lt;em&gt;x&lt;sup&gt;3&lt;/sup&gt;&lt;/em&gt; + &lt;em&gt;x&lt;/em&gt; + &lt;em&gt;1&lt;/em&gt; mentioned previously. Our approach for decoding will be this: simulate a 4-bit shift register with contents all zero as we started with in the encoder. Next we simulate the first unknown bit being shifted in, which gives us two new possible shift register states, &lt;code class=&quot;highlighter-rouge&quot;&gt;1000&lt;/code&gt; and &lt;code class=&quot;highlighter-rouge&quot;&gt;0000&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;For each of these two states, we will evaluate both polynomial outputs. We then compare these outputs to the two bits we actually received. For both shift register states, we measure the error between what we’d expect our polynomials to generate and what we actually received. Finally, we record this error total for each state. It might turn out at this point that one state mismatched on both polynomials while the other mismatched on neither, yielding error counts of 2 and 0, respectively.&lt;/p&gt;

&lt;figure class=&quot;figure&quot;&gt;
&lt;img src=&quot;/assets/decoding.png&quot; class=&quot;figure-img img-fluid img-rounded&quot; alt=&quot;Decoding First Bit&quot; /&gt;
  &lt;figcaption class=&quot;figure-caption&quot;&gt;Decoding First Bit&lt;/figcaption&gt;
&lt;/figure&gt;

&lt;p&gt;We continue this process for every grouping of polynomial bits we receive. Every time we simulate new shift register states, we copy over the error total from the previous state. For example, if state &lt;code class=&quot;highlighter-rouge&quot;&gt;0000&lt;/code&gt; had an error of 2, then in the next bit, states &lt;code class=&quot;highlighter-rouge&quot;&gt;0000&lt;/code&gt; and &lt;code class=&quot;highlighter-rouge&quot;&gt;1000&lt;/code&gt; will start with an error of 2. Once we’re out of bits to decode, we choose the sequence of bits which has the smallest accumulated error. This sequence is declared to be the original message.&lt;/p&gt;

&lt;figure class=&quot;figure&quot;&gt;
&lt;img src=&quot;/assets/decoding_full.png&quot; class=&quot;figure-img img-fluid img-rounded&quot; alt=&quot;Decoding Second Bit&quot; /&gt;
  &lt;figcaption class=&quot;figure-caption&quot;&gt;Decoding Second Bit&lt;/figcaption&gt;
&lt;/figure&gt;

&lt;p&gt;This strategy has one major flaw. Every time we want to simulate another bit, we must store twice as many register states as we did in the previous bit. When we started, we needed only simulate &lt;code class=&quot;highlighter-rouge&quot;&gt;0000&lt;/code&gt; and &lt;code class=&quot;highlighter-rouge&quot;&gt;1000&lt;/code&gt;. For the following bit, we needed to simulate &lt;code class=&quot;highlighter-rouge&quot;&gt;0000&lt;/code&gt;, &lt;code class=&quot;highlighter-rouge&quot;&gt;0100&lt;/code&gt;, &lt;code class=&quot;highlighter-rouge&quot;&gt;1000&lt;/code&gt;, and &lt;code class=&quot;highlighter-rouge&quot;&gt;1100&lt;/code&gt;. Once the sequence is longer than the shift register, we still must track all possible states, which means that for a message of length &lt;em&gt;m&lt;/em&gt; we will need to track the error count for &lt;em&gt;2&lt;sup&gt;m&lt;/sup&gt;&lt;/em&gt; states. This adds up quickly! Thankfully there is a clever trick which reduces the need to track so many states.&lt;/p&gt;

&lt;h3 id=&quot;viterbi-algorithm&quot;&gt;Viterbi Algorithm&lt;/h3&gt;

&lt;p&gt;The Viterbi algorithm is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Dynamic_programming&quot;&gt;Dynamic programming&lt;/a&gt; approach to decoding convolutional codes. This algorithm makes one important observation about the sequence of decoded bits. Once our sequence of bits is longer than the length of the shift register, we can discard unlikely paths. Rather than storing error information about &lt;em&gt;2&lt;sup&gt;m&lt;/sup&gt;&lt;/em&gt; paths, we only need store &lt;em&gt;m&lt;/em&gt; * &lt;em&gt;2&lt;sup&gt;k&lt;/sup&gt;&lt;/em&gt; paths (for message length &lt;em&gt;m&lt;/em&gt; and shift register length &lt;em&gt;k&lt;/em&gt;).&lt;/p&gt;

&lt;p&gt;Let’s return to our previous 4-bit shift register. Suppose that we are decoding and have received our 4th set of bits and have calculated all of the error counts for the received bits. We have 16 possible sequences so far, and the next set of bits received will bring us to 32 sequences. Each sequence will track 5 bits, but only 4 bits will actually be used for error calculation. If we carefully compare and select sequences at this step, we can actually eliminate half of the sequences without any loss in ability to decode the message.&lt;/p&gt;

&lt;p&gt;Imagine that we have calculated the error for sequences &lt;code class=&quot;highlighter-rouge&quot;&gt;0110&lt;/code&gt; and &lt;code class=&quot;highlighter-rouge&quot;&gt;0111&lt;/code&gt;. In the next step, the rightmost bit will shift out of the shift register and will no longer make any contribution to the error count, but we still need to keep track of it to know which bit was transmitted when we finish decoding. Both of these sequences will appear to shift to &lt;code class=&quot;highlighter-rouge&quot;&gt;B011&lt;/code&gt; in the next step, where &lt;em&gt;B&lt;/em&gt; is the next bit transmitted. It turns out that we can simply discard whichever of these sequences has a larger error at this point and then record the rightmost bit from the “winning” sequence.&lt;/p&gt;

&lt;figure class=&quot;figure&quot;&gt;
&lt;img src=&quot;/assets/viterbi.png&quot; class=&quot;figure-img img-fluid img-rounded&quot; alt=&quot;Picking a sequence&quot; /&gt;
  &lt;figcaption class=&quot;figure-caption&quot;&gt;Picking a sequence&lt;/figcaption&gt;
&lt;/figure&gt;

&lt;p&gt;For every sequence at this step, there is a complementary sequence we can compare it to. Specifically, the sequences &lt;code class=&quot;highlighter-rouge&quot;&gt;XYZ0&lt;/code&gt; and &lt;code class=&quot;highlighter-rouge&quot;&gt;XYZ1&lt;/code&gt; will be compared and one chosen as a winner. We then store the rightmost bit of the sequence with the smaller error in a table. Once we have finished this step, we will have a table with one bit for each sequence of length &lt;em&gt;k - 1&lt;/em&gt;. We will repeat this step for every new group of inputs. Libcorrect calls this table a “history table.”&lt;/p&gt;

&lt;p&gt;Once we have finished decoding the message, we will actually work backwards to recover our message. We will start by inspecting which shift register state has the smallest total accumulated error and declare it to be our sequence. We will then shift this state backwards once and use this value to lookup the next bit in our history table. We will &lt;em&gt;OR&lt;/em&gt; this bit back into the register and then shift again, repeating the process until we have rewound the entire history table.&lt;/p&gt;

&lt;p&gt;You might be wondering whether we could simply choose the sequence with the smallest error amongst all sequences and store only that sequence, rather than storing an entire table. Although this is one possible strategy, it will not yield the same error correction robustness as storing all paths. If we encounter a short-lived burst of errors, it may adversely influence the error count for the correct sequence – remember that we are choosing the sequence which is most likely given the information we have. If we wait until the message has finished and then recover the bits, we are more likely to converge to the correct sequence.&lt;/p&gt;

&lt;p&gt;In practice, it is common to do some hybrid approach. The table can store some multiple of &lt;em&gt;k&lt;/em&gt; time shifts of sequences, with a periodic “rewind” operation clearing space for new sequences and decoding part of the message. For example, we might wait until we’ve decoded &lt;em&gt;20 * k&lt;/em&gt; groups of inputs, and then decode the oldest &lt;em&gt;15 * k&lt;/em&gt; bits, leaving the rest for more convergence. Letting the table grow larger uses more memory but requires less CPU time.&lt;/p&gt;

&lt;h3 id=&quot;demodulation&quot;&gt;Demodulation&lt;/h3&gt;

&lt;p&gt;When we send data across a network, we usually think of it as being received as a binary signal. The final part of the receiver might be a program that gets bytes of data from a socket. At a lower level, for certain kinds of lossy networks, the data might actually be transmitted as an analog signal. This implies that some part of the receive chain has to convert the received signal back to a digital signal. This component could be the software defined radio on a wireless chipset or even an analog-to-digital sampler on a soundcard.&lt;/p&gt;

&lt;p&gt;Often our receivers initially get this signal with some analog fidelity. The code that demodulates the signal can produce a ‘soft’ bit which encodes the demodulator’s confidence in the demodulation process. For example, the demodulator might produce an 8-bit confidence value for every single bit received, where &lt;code class=&quot;highlighter-rouge&quot;&gt;00000000&lt;/code&gt; implies a high level of confidence in a &lt;em&gt;0&lt;/em&gt; bit, &lt;code class=&quot;highlighter-rouge&quot;&gt;11111111&lt;/code&gt; implies a high level of confidence in a &lt;em&gt;1&lt;/em&gt; bit, and &lt;code class=&quot;highlighter-rouge&quot;&gt;10000000&lt;/code&gt; implies complete uncertainty in which bit was received. This confidence value is useful during error correction because we will want to aggregate our decoding process over a long sequence of bits. Receiving a high level of uncertainty about a single incorrect bit from the demodulator helps the decoder make the right decisions for the other nearby bits.&lt;/p&gt;

&lt;h3 id=&quot;decoding-soft-bits&quot;&gt;Decoding Soft Bits&lt;/h3&gt;

&lt;p&gt;We can make use of the demodulator’s soft bits during the convolutional code decoder process. If we run this process on soft bits rather than hard bits, we can continue to recover the message in the presence of more noise than we would have been able to otherwise.&lt;/p&gt;

&lt;p&gt;Decoding convolutional codes with soft bits is almost exactly the same process as with hard bits. Previously we calculated the error as the number of bits which differed between the received bits and the simulated polynomial outputs. With soft received bits, we will instead store the absolute value of the difference between the received soft value and the simulated polynomial value encoded as soft bits. For example, if the polynomial value is &lt;em&gt;1&lt;/em&gt; and the received soft value is &lt;code class=&quot;highlighter-rouge&quot;&gt;1110000&lt;/code&gt; then we store &lt;code class=&quot;highlighter-rouge&quot;&gt;11111111 - 11100000 = 11111&lt;/code&gt; as the error. If instead the polynomial value were &lt;em&gt;0&lt;/em&gt; and the received soft value were &lt;code class=&quot;highlighter-rouge&quot;&gt;10100000&lt;/code&gt; then the error would be &lt;code class=&quot;highlighter-rouge&quot;&gt;10100000&lt;/code&gt;. The error continues to accumulate as a sum just as it does when decoding hard bits.&lt;/p&gt;

&lt;h3 id=&quot;configuration&quot;&gt;Configuration&lt;/h3&gt;

&lt;p&gt;We can get more error correction resilience from convolutional codes both by increasing the length of the shift register and by adding more polynomials. Increasing the length of the shift register does not reduce the transmission speed but does cost more CPU resources to decode. Every time we lengthen the shift register by one bit, we will reduce the decoding CPU performance by about 50%. Adding more polynomials adds transmission overhead but has less CPU impact than adding another bit to the shift register.&lt;/p&gt;

&lt;p&gt;In practice, common lengths for the shift register range from 7 to 15 bits with 2 to 6 polynomials. Even a modern CPU cannot decode more than 1 million bits per second for a convolutional code with &lt;em&gt;k = 15&lt;/em&gt;. These configurations were used for a wide range of error correction including communications with the Voyager probe and the Mars Pathfinder.&lt;/p&gt;

&lt;hr /&gt;

&lt;p&gt;&lt;em&gt;Stay tuned for the next post which will examine how &lt;a href=&quot;https://github.com/quiet/libcorrect&quot;&gt;libcorrect&lt;/a&gt; accelerates the convolutional code decoding process. If you have questions or comments about this post, &lt;a href=&quot;https://discordapp.com/invite/eRw5UjF&quot;&gt;stop by and say hello&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
        <pubDate>Sun, 16 Sep 2018 14:25:13 -0700</pubDate>
        <link>https://quiet.github.io/quiet-blog/2018/09/16/How-Libcorrect-Does-Forward-Error-Correction.html</link>
        <guid isPermaLink="true">https://quiet.github.io/quiet-blog/2018/09/16/How-Libcorrect-Does-Forward-Error-Correction.html</guid>
        
        
        <category>quiet-blog</category>
        
      </item>
    
      <item>
        <title>Generating Swift Documentation From Objective-C</title>
        <description>&lt;p&gt;I’ve been working on building documentation for &lt;a href=&quot;https://github.com/quiet&quot;&gt;Quiet Modem Project&lt;/a&gt; and I recently came up against a snag in documenting my iOS library that contains Objective-C. Although it’s mostly advisable to stick to writing iOS libraries in Swift these days, I chose Objective-C because it felt nicer when wrapping C libraries. Since the core part of my project is a C library, being able to wrap it cleanly makes life a little easier for me.&lt;/p&gt;

&lt;p&gt;The standard I’ve set for my documentation is that I want to show both the code that declares the Class or Function (or some close simplification of it) as well as structured commentary written in plain English. If a function takes two parameters, I want to show the function’s header and a good thorough explanation of what it expects for parameters, what it returns, and any extra notes about how it behaves.&lt;/p&gt;

&lt;p&gt;It’s easy enough to programmatically generate API documentation for Objective-C. The wonderful &lt;a href=&quot;http://www.doxygen.org&quot;&gt;Doxygen&lt;/a&gt; is quite capable of parsing structured documentation out of comments. I’m not a fan of the actual pages and stylesheets it generates, but the XML output contains all the relevant documentation, and this output can be used to feed a separate documentation frontend like &lt;a href=&quot;https://www.mkdocs.org&quot;&gt;MkDocs&lt;/a&gt;. The structured output contains something like a structured parsing of the code itself as well as the comment strings I put alongside it. So is this the end of the story?&lt;/p&gt;

&lt;p&gt;This method can generate good documentation for Objective-C, but iOS developers are going to want Swift documentation. This means that what we really want to do is generate both the Objective-C docs and the corresponding docs for the translated Swift code. There’s good precedent for this, of course – Apple’s own documentation mostly does this. How would we translate Objective-C documentation into Swift documentation? There are major semantic and syntactic differences between the two languages.&lt;/p&gt;

&lt;p&gt;I thought about this for a while and started to feel despair. My project has bindings in JavaScript, Java (Android), Obj-C/Swift and then the original itself in C. One change in the core C library can require updates in documentation for 5 languages. Any solution I came up with would have to be automated, but there was surely no mode for Doxygen to generate Swift documentation from Objective-C.&lt;/p&gt;

&lt;p&gt;I had hoped to find that someone else had run into this problem before and had solved it. It was at this point I discovered the &lt;a href=&quot;https://github.com/realm/jazzy&quot;&gt;jazzy&lt;/a&gt; tool. Finding this gave me a lot of optimism that I was going to be able to automate the translation process. Although I couldn’t find a way to get jazzy to do the translation for me, I realized that it relied on a tool called SourceKit that’s supplied by Xcode and that can do some kinds of source code interactions, which I thought might include Objective-C to Swift translations. This made sense to me as I knew that Xcode must somehow have the ability to figure out a Swift header from an Objective-C .h file.&lt;/p&gt;

&lt;p&gt;I got &lt;a href=&quot;https://github.com/jpsim/SourceKitten&quot;&gt;SourceKitten&lt;/a&gt; running and started poking around SourceKit’s API. With a bit of Google searching, I found a request that would generate a Swift file from an Objective-C file. I was glad someone else had documented this since I would have never figured it out by myself. Yes, you really do need that UUID, which defines the SourceKit function you’re calling. It’s not localized or specific to your project.&lt;/p&gt;

&lt;noscript&gt;&lt;pre&gt;400: Invalid request
&lt;/pre&gt;&lt;/noscript&gt;
&lt;script src=&quot;https://gist.github.com/c02f20aeadc61ff02ac243bec6a864f5.js&quot;&gt; &lt;/script&gt;

&lt;p&gt;Running &lt;code class=&quot;highlighter-rouge&quot;&gt;sourcekitten request --yaml header.yaml | jq -r '.[&quot;key.sourcetext&quot;]' &amp;gt; Foo.swift&lt;/code&gt; transforms Foo.h into Foo.swift.&lt;/p&gt;

&lt;noscript&gt;&lt;pre&gt;400: Invalid request
&lt;/pre&gt;&lt;/noscript&gt;
&lt;script src=&quot;https://gist.github.com/39deba2bdfaa812f975b31527a6de5dc.js&quot;&gt; &lt;/script&gt;

&lt;noscript&gt;&lt;pre&gt;400: Invalid request
&lt;/pre&gt;&lt;/noscript&gt;
&lt;script src=&quot;https://gist.github.com/b0fa3a1435c1583049f2fd5c0f71a3f0.js&quot;&gt; &lt;/script&gt;

&lt;p&gt;This felt like a bit of magic the first time I saw it. The transformation isn’t always perfect, but it does a pretty good job considering it’s automated. Even our structured comments moved over.&lt;/p&gt;

&lt;p&gt;Code translation was an important step forward for what I wanted to do. This still wasn’t enough to build the documentation though, as what I need is the structured code listings and comments that Doxygen and similar tools build. If Doxygen had a Swift mode, this would have been the end of the story. I would just take these translated Swift files and send them through Doxygen. Unfortunately this isn’t the case.&lt;/p&gt;

&lt;p&gt;I did know that SourceKit must have something like this capability since tools like jazzy were using it. Generating documentation is thankfully a much more straightforward use of sourcekitten. &lt;code class=&quot;highlighter-rouge&quot;&gt;sourcekitten structure --file Foo.swift&lt;/code&gt; gets us the structured output in a format that’s not so different from Doxygen’s XML output. With some work this can be translated into a nice Markdown file that MkDocs will consume.&lt;/p&gt;

&lt;p&gt;Extracting the structured comments is trickier. In Objective-C files, SourceKit actually has some support for pulling structured comments that are in a Javadoc-like style, which is compatible with my Doxygen-style comments. It doesn’t offer the same support for Javadoc-style comments in Swift though as these comments are no longer considered relevant. Instead it is expected that Swift comments are written in Markdown. Even though SourceKit was able to maintain our comments, it can’t actually consume them in a useful manner.&lt;/p&gt;

&lt;p&gt;The best option I’ve found is using &lt;code class=&quot;highlighter-rouge&quot;&gt;sourcekitten doc --single-file Foo.swift&lt;/code&gt; to get the block of comments that is associated with each declaration. I believe the easiest option to turn these comment blocks into structured documentation is writing a parser specifically for this task. Thankfully this is considerably easier than extracting the structured code information. This allows me to get the plain English explanation of each function parameter displayed properly alongside the function declaration.&lt;/p&gt;

&lt;p&gt;With all of these tools in hand, it is finally possible to generate proper structured documentation for Swift users from Objective-C code. Although I’m sure there are many who would tell me to just give up Objective-C and wrap my libraries in Swift, I’m happy to say that a path to automated documentation does exist for stubborn programmers like myself.&lt;/p&gt;

</description>
        <pubDate>Mon, 13 Aug 2018 00:25:10 -0700</pubDate>
        <link>https://quiet.github.io/quiet-blog/2018/08/13/Objective-C-Swift-Documentation.html</link>
        <guid isPermaLink="true">https://quiet.github.io/quiet-blog/2018/08/13/Objective-C-Swift-Documentation.html</guid>
        
        
        <category>quiet-blog</category>
        
      </item>
    
      <item>
        <title>Quiet Profile Lab — Build a Modem, Learn Some DSP</title>
        <description>&lt;p&gt;&lt;a href=&quot;https://github.com/quiet/quiet-js&quot;&gt;Quiet.js&lt;/a&gt; and &lt;a href=&quot;https://github.com/quiet/quiet&quot;&gt;libquiet&lt;/a&gt; are capable of transmission via audible tones, ultrasonic tones, and through an audio cable at wide spectrum. Quiet provides a JSON file which provides parameters for each of these modes. A single set of parameters, a profile, sets the center frequency of the modem, the modulation, error correction modes used, and more. Creating a new profile in a way that’s robust to hardware limitations and yet provides good throughput can be difficult, which is why I’m now pleased to announce the &lt;a href=&quot;https://quiet.github.io/quiet-js/lab.html&quot;&gt;Quiet Profile Lab&lt;/a&gt;, a fully interactive workbench for creating and testing new profiles.&lt;/p&gt;

&lt;figure class=&quot;figure&quot;&gt;
&lt;img src=&quot;/images/qpl.png&quot; class=&quot;figure-img img-fluid img-rounded&quot; alt=&quot;Quiet Profile Lab in action&quot; /&gt;
  &lt;figcaption class=&quot;figure-caption&quot;&gt;Quiet Profile Lab in action&lt;/figcaption&gt;
&lt;/figure&gt;

&lt;p&gt;The &lt;a href=&quot;https://quiet.github.io/quiet-js/lab.html&quot;&gt;Quiet Profile Lab&lt;/a&gt; makes it easy to test out new ideas for profiles. It offers spectrum and constellation diagrams in real time, as well as statistics about throughput, performance, and error rate. This makes it suitable not only for creating new profiles but also learning about modem design. If you’re working from a laptop, you have the perfect testbench – the laptop’s mic will likely pick up the audio generated by its own speakers.&lt;/p&gt;

&lt;p&gt;If you are unfamiliar with the techniques used by modems, I welcome you to try out the Lab and experiment with different settings. Because it’s interactive, the Lab is a great way to examine the behavior of various modulation modes. While Wikipedia offers good explanations for many DSP terms, it can be beneficial to see them working in a live example. Even if many of the terms in the Lab are unfamiliar, you may be able to figure out what they do, just by experimenting with them.&lt;/p&gt;

&lt;p&gt;If you’re not sure where to start, here are some ideas. Try changing the center frequency. Do you notice changes in how the modem sounds? Reduce the gain and see how quiet you can get the modem while still receiving frames. Increase the interpolation factor (samples per symbol) to narrow the part of the frequency spectrum that your modem uses. Can you find a high frequency, narrow spectrum setting that transfers data but which you can’t hear?&lt;/p&gt;

&lt;p&gt;If you have the cable on hand, I highly recommend creating a loopback setup to test on, as well. Passing the audio over a cable greatly reduces noise and will allow you to pass a more delicate, higher throughput signal and will preserve the constellation more than using speakers/mic will. The Lab includes some presets to get you started in cable mode.&lt;/p&gt;

&lt;p&gt;Once you’re done, have a look at some of the standards used by the devices you own, like &lt;a href=&quot;http://electronicdesign.com/4g/introduction-lte-advanced-real-4g#%E2%80%9DFrequency%E2%80%9D&quot;&gt;4g LTE&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Orthogonal_frequency-division_multiplexing#OFDM_system_comparison_table&quot;&gt;802.11a&lt;/a&gt;. The Lab provides a good way to get familiar with some of the techniques used in common radios, which are remarkably similar despite being carried by electromagnetic waves rather than by sound.&lt;/p&gt;

&lt;p&gt;If you find a profile that you’d like to start using with &lt;a href=&quot;https://github.com/quiet/quiet-js&quot;&gt;Quiet.js&lt;/a&gt;, the Lab provides you with the properly formatted JSON text for the profile you’ve created. It’s as simple as copying the text and pasting it into quiet-profiles.json under a new key.&lt;/p&gt;

&lt;p&gt;If you’ve always wanted to learn about DSP but have not had the chance, I hope you’ll spend some time in the &lt;a href=&quot;https://quiet.github.io/quiet-js/lab.html&quot;&gt;Lab&lt;/a&gt;. I think you’ll really like it.&lt;/p&gt;

</description>
        <pubDate>Wed, 30 Mar 2016 09:30:20 -0700</pubDate>
        <link>https://quiet.github.io/quiet-blog/2016/03/30/quiet-profile-lab-build-modem-learn-dsp.html</link>
        <guid isPermaLink="true">https://quiet.github.io/quiet-blog/2016/03/30/quiet-profile-lab-build-modem-learn-dsp.html</guid>
        
        
        <category>quiet-blog</category>
        
      </item>
    
      <item>
        <title>Quiet</title>
        <description>&lt;p&gt;When I started working on &lt;a href=&quot;https://github.com/quiet/quiet&quot;&gt;libquiet&lt;/a&gt;, I was trying to answer a question for myself. I had seen projects which passed data through the headphone jack, which I thought was an interesting idea. I wanted to know how fast this method could send data. Many of these methods used Frequency-Shift Keying, which is easy to implement but typically does not achieve the maximum speed possible. I started researching, which lead me to &lt;a href=&quot;http://liquidsdr.org/&quot;&gt;liquid sdr&lt;/a&gt; which offers basic framing and all the modulation and error correction methods I would need to answer my question. And so, libquiet was born, creating a configurable modem engine which connects soundcard to liquid SDR.&lt;/p&gt;

&lt;p&gt;As the project continued, I realized it would also be possible to compile my library to JS using emscripten. I was surprised how well this works, and now, Quiet.js is compatible with the native binaries create with libquiet. My aim is to bring quiet to as many platforms as possible. It’s not just for your headphone jack, either. Quiet works quite well through your speakers.&lt;/p&gt;

&lt;p&gt;There’s a few reasons I chose the name quiet. For one, I was thinking about using the headphone jack, which wouldn’t emit any sound. As the project expanded, I realized that ultrasonic transmission would also be possible – another type of quiet modem. Additionally, the quiet modem uses SDR, but it’s “quiet” in the RF spectrum (mostly!). And finally, the best reason, because modems don’t work when they’re clipping! Turn down your volume before using quiet.&lt;/p&gt;

</description>
        <pubDate>Tue, 29 Mar 2016 11:30:45 -0700</pubDate>
        <link>https://quiet.github.io/quiet-blog/2016/03/29/quiet.html</link>
        <guid isPermaLink="true">https://quiet.github.io/quiet-blog/2016/03/29/quiet.html</guid>
        
        
        <category>quiet-blog</category>
        
      </item>
    
  </channel>
</rss>