1#+HUGO_BASE_DIR: ./
2#+HUGO_SECTION: posts
3#+HUGO_AUTO_SET_LASTMOD: t
4
5* Meta :@Meta:
6* Technology :@Technology:
7** TODO Audacity and the telemetry pull request
8:PROPERTIES:
9:EXPORT_FILE_NAME: audacity-and-the-telemetry-pull-request
10:EXPORT_HUGO_CUSTOM_FRONT_MATTER: :toc true
11:END:
12
13Five days ago at the time of writing, [[https://github.com/crsib][Dmitry Vedenko]] opened a Pull
14Request (PR) in [[https://github.com/audacity/audacity/pull/835][Audacity's GitHub repository]] entitled [[https://github.com/audacity/audacity/pull/835][/Basic telemetry
15for the Audacity/.]] About two days later, all hell broke loose. That PR
16now has over 3.3 thousand downvotes and more than one thousand comments
17from nearly 400 individuals. I started reading the posts shortly after
18they began and kept up with them over the following days, reading every
19single new post. I recognise that few people are going to feel like
20wading through over 1k comments so this is my attempt to provide a
21summary of the PR itself using the community's code reviews along with a
22summary of the various opinions conveyed in the comments.
23
24When I reference comments, I'll provide a footnote that includes a link
25to the comment and a link to a screenshot just in case it's removed or
26edited in the future.
27
28*** Audacity's acquisition
29
30I haven't been able to find /much/ information in this area so forgive me
31if I'm scant on details.
32
33On 30 April, a company called [[https://mu.se/][Muse Group]] acquired [[https://www.audacityteam.org/][Audacity]]. According to
34[[https://mu.se][their website]], Muse is the parent company behind many musical
35applications and tools. It was founded by Eugeny Naidenov just days
36before it acquired Audacity. Before all of this, Eugeny Naidenov founded
37[[https://www.ultimate-guitar.com/][Ultimate Guitar]] (UG) in 1998. The service grew rather quickly and now
38has over 300 million users. UG acquired [[https://deanzelinsky.com/][Dean Zelinsky Guitars]] in 2012,
39[[http://agilepartners.com/][Agile Partners]] in 2013, [[https://musescore.org/][MuseScore]] in 2017, and [[http://trycrescendo.com/][Crescendo]] in 2018. Muse
40Group was established in 2021 and it seems as if all of the services UG
41acquired were (or will be) transferred to Muse Group, as well as UG
42itself. Immediately following its establishment, Muse not only acquired
43Audacity but also [[https://www.staffpad.net/][StaffPad]].
44
45I say 30 April because that's when Muse published their [[https://mu.se/newsroom/tpost/6dhedma301-muse-group-acquires-audacity-expanding-c][press release]]
46and when Martin Keary (Tantacrul) published a video entitled [[https://www.youtube.com/watch?v=RMWNvwLiXIQ][/I’m now in
47charge of Audacity. Seriously./]] According to his comment,[fn:17] Martin
48will help with proposing Audacity's roadmap and many of its future
49features as well as working with the community. This has been his role
50with MuseScore since he joined that project and he will be continuing it
51here.
52
53~-----BEGIN PERSONAL OPINION-----~
54
55Looking at [[https://www.martinkeary.com/][his website,]] I also suspect he will play a large role in
56redesigning Audacity's interface. Considering that he was instrumental
57in designing [[https://www.martinkeary.com/#/ubuntu-touch-os/][the best mobile interface I've ever had the absolute
58pleasure of experiencing,]] I have high hopes that this is the case.
59
60~------END PERSONAL OPINION------~
61
62*** Telemetry implementation
63**** Implementation Basics
64
65A few days after the acquisition, a PR was opened that adds /Basic
66telemetry for the Audacity/. This implementation collects "application
67opened" events and sends those to Yandex to estimate the number of
68Audacity users. It also collects session start and end events, errors
69for debugging, file used for import and export, OS and Audacity
70versions, and the use of effects, generators, and analysis tools so they
71can prioritise future improvements. Sending this data would be optional
72and the user would be presented with a dialogue the first time they
73launch the application after installation or after they update to the
74including release. This description was mostly copied directly from [[https://github.com/audacity/audacity/pull/835#issue-629891447][the
75PR description itself.]]
76
77**** Frontend Implementation
78This is fairly straightforward and a pretty standard UI for prompting
79users to consent to analytics and crash logging. This section is
80included because the community has strong opinions regarding the
81language used and its design, but that will be discussed later. The
82screenshot below is copied directly from the PR.
83
84[[/assets/pngs/audacity-pr/consentdialogue.png]]
85
86**** Backend Implementation
87Many of the code reviews include the reviewer's personal opinion so I
88will summarise the comment, provide the code block in question, and link
89directly to the comment in a footnote.[fn:9]
90
91#+BEGIN_SRC c
92 if (!inputFile.Write (wxString::FromUTF8 (ClientID + "\n")))
93 return false;
94#+END_SRC
95[[https://github.com/crsib/audacity/blob/c9264d2478fe2af82aeb6e2a0295b00b3a27ce53/libraries/lib-telemetry/TelemetryManager.cpp#L199-L200][Lines 199-200 of TelemetryManager.cpp]] save the user's unique client ID
96to a file.[fn:8] This allows the analytics tool (in this case, Google
97Analytics) to aggregate data produced by a single user.
98
99#+BEGIN_SRC c
100 def_vars()
101
102 set( CURL_DIR "${_INTDIR}/libcurl" )
103 set( CURL_TAG "curl-7_76_0")
104#+END_SRC
105[[https://github.com/crsib/audacity/blob/c9264d2478fe2af82aeb6e2a0295b00b3a27ce53/cmake-proxies/libcurl/CMakeLists.txt#L3-L6][Lines 3-6 of CMakeLists.txt]] "vendor in" libcurl.[fn:10] This is when an
106application directly includes sources for a utility rather than making
107use utilities provided by the system itself.
108
109#+BEGIN_SRC c
110 ExternalProject_Add(curl
111 PREFIX "${CURL_DIR}"
112 INSTALL_DIR "${CURL_DIR}"
113 GIT_REPOSITORY https://github.com/curl/curl
114 GIT_TAG ${CURL_TAG}
115 GIT_SHALLOW Yes
116 CMAKE_CACHE_ARGS ${CURL_CMAKE_ARGS}
117 )
118#+END_SRC
119[[https://github.com/crsib/audacity/blob/c9264d2478fe2af82aeb6e2a0295b00b3a27ce53/cmake-proxies/libcurl/CMakeLists.txt#L29-L36][Lines 29-36 of CMakeLists.txt]] add curl as a remote dependency.[fn:11]
120This means that the machine building Audacity from its source code has
121to download curl during that build.
122
123#+BEGIN_SRC c
124 S.Id (wxID_NO).AddButton (rejectButtonTitle);
125 S.Id (wxID_YES).AddButton (acceptButtonTitle)->SetDefault ();
126#+END_SRC
127[[https://github.com/crsib/audacity/blob/c9264d2478fe2af82aeb6e2a0295b00b3a27ce53/src/telemetry/TelemetryDialog.cpp#L93-L94][Lines 93-94 of TelemetryDialog.cpp]] add buttons to the dialogue asking
128the user whether they consent to data collection.[fn:12] ~SetDefault~
129focuses the button indicating that the user does consent. This means
130that if the user doesn't really look at the dialogue and presses
131Spacebar or Enter, or if they do so accidentally by simply bumping the
132key, they unintentionally consent to data collection. If the user
133desires, this can later be changed in the settings menu. However, if
134they weren't aware what they were consenting to /or that they did
135consent/, they won't know to go back and opt out.
136
137There are other problems with the code that include [[https://github.com/audacity/audacity/pull/835#discussion_r628816050][simple mistakes,]]
138[[https://github.https://github.com/audacity/audacity/pull/835#discussion_r628774985][styling that's inconsistent with the rest of the project,]] [[https://github.com/audacity/audacity/pull/835#discussion_r628500849][unhandled
139return values resulting in skewed data,]] [[https://github.com/audacity/audacity/pull/835#discussion_r628792423][use of inappropriate functions,]]
140and [[https://github.com/audacity/audacity/pull/835#discussion_r628818054][spelling errors in the comments.]] I believe these are less important
141that those above so they won't be discussed.
142
143*** Community opinions
144There were many strong opinions regarding both the frontend and backend
145implementations of this PR, from the wording of the dialogue and
146highlighting the consent button to devices running something other than
147Windows and macOS not being able to send telemetry and thus skewing the
148data that /was/ collected.
149
150**** Opinions on the frontend
151
152Really, the only frontend here is the consent dialogue. However, there
153are /many/ comments about it, the most common of which is probably that
154the wording is not only too vague[fn:13] but also inaccurate[fn:14]. The
155assertion that Google Analytics are not anonymous and any data sent can
156be trivially de-anonymised (or de-pseudonymised) is repeated many times
157over. Below are a few links to comments stating such. I searched for the
158term /"anonymous"/, copied relevant links, and stopped when my scrollbar
159reached halfway down the page.
160
161- [[https://github.com/audacity/audacity/pull/835#discussion_r628156527][r628156527]]
162- [[https://github.com/audacity/audacity/pull/835#issuecomment-833969780][833969780]]
163- [[https://github.com/audacity/audacity/pull/835#issuecomment-833969933][833969933]]
164- [[https://github.com/audacity/audacity/pull/835#discussion_r627995927][r627995927]]
165- [[https://github.com/audacity/audacity/pull/835#issuecomment-834358022][834358022]]
166- [[https://github.com/audacity/audacity/pull/835#issuecomment-834377549][834377549]]
167- [[https://github.com/audacity/audacity/pull/835#issuecomment-834382007][834382007]]
168- [[https://github.com/audacity/audacity/pull/835#issuecomment-834385463][834385463]]
169- [[https://github.com/audacity/audacity/pull/835#issuecomment-834405825][834405825]]
170- [[https://github.com/audacity/audacity/pull/835#issuecomment-834531779][834531779]]
171- [[https://github.com/audacity/audacity/pull/835#issuecomment-834546874][834546874]]
172- [[https://github.com/audacity/audacity/pull/835#issuecomment-834638000][834638000]]
173
174The next most pervasive comment is regarding the consent buttons at the
175bottom of the dialogue where users opt in or out.[fn:15] Many individuals call
176this design a /dark pattern/. Harry Brignull, a UX specialist focusing on
177deceptive interface practises, describes dark patterns as [[https://www.darkpatterns.org/][/tricks used
178in websites and apps that make you do things that you didn't mean to/.]]
179The dark pattern in this situation is the opt-in button being
180highlighted. Many community members assert that users will see the big
181blue button and click it without actually reading the dialogue's
182contents. They just want to record their audio and this window is a
183distraction that prevents them from doing so; it needs to get out of the
184way and the quickest way to dismiss it is clicking that blue button.
185Below is a list of some comments criticising this design.
186
187 - [[https://github.com/audacity/audacity/pull/835#issuecomment-834286641][834286641]]
188 - [[https://github.com/audacity/audacity/pull/835#issuecomment-834358022][834358022]]
189 - [[https://github.com/audacity/audacity/pull/835#issuecomment-834399813][834399813]]
190 - [[https://github.com/audacity/audacity/pull/835#issuecomment-834479968][834479968]]
191 - [[https://github.com/audacity/audacity/pull/835#issuecomment-835250737][835250737]]
192 - [[https://github.com/audacity/audacity/pull/835#issuecomment-835253882][835253882]]
193 - [[https://github.com/audacity/audacity/pull/835#issuecomment-835291066][835291066]]
194 - [[https://github.com/audacity/audacity/pull/835#issuecomment-835445481][835445481]]
195
196Another issue that was brought up by a couple of individuals was the
197lack of a privacy policy.[fn:16] The consent dialogue links to one, but, at the
198time of writing, one does not exist at [[https://www.audacityteam.org/contact/privacy-policy/][the provided URL.]] I have [[https://web.archive.org/web/20210510012924/https://www.audacityteam.org/contact/privacy-policy/][archived
199the state of the page]] in case that changes in the future.
200
201**** Opinions on the backend
202
203#+BEGIN_SRC c
204 if (!inputFile.Write (wxString::FromUTF8 (ClientID + "\n")))
205 return false;
206#+END_SRC
207
208The issue many individuals take with this snippet is saving the
209~ClientID~. Say an individual has an odd file that causes Audacity to
210crash any time they try to open it. Say they attempt to open it a
211hundred times. Without giving the client a unique ID, it could look like
212there are 100 people having an issue opening a file instead of just the
213one. However, by virtue of each installation having an entirely unique
214ID, this telemetry /is not anonymous/. Anonymity would be sending
215statistics in such a way that connecting those failed attempts to a
216single user would be impossible. At best, this implementation is
217/pseudonymous/ because the client is given a random ID, you don't have to
218sign in with an account or something.
219
220#+BEGIN_SRC c
221 def_vars()
222
223 set( CURL_DIR "${_INTDIR}/libcurl" )
224 set( CURL_TAG "curl-7_76_0")
225#+END_SRC
226
227Timothe Litt's comment gives a good description of why "vendoring in"
228libcurl is a bad idea[fn:19] and Tyler True's comment gives a good
229overview of the pros and cons of doing so.[fn:18] Many people take issue
230with this /specifically/ because it's libcurl. Security flaws in it are
231/very/ common and Audacity's copy would need to be /manually/ kept up to
232date with every upstream release to ensure none of its vulnerabilities
233can be leveraged to compromise users. If the Audacity team was going to
234stay on top of all of the security fixes, they would need to release a
235new version every week or so.
236
237#+BEGIN_SRC c
238 ExternalProject_Add(curl
239 PREFIX "${CURL_DIR}"
240 INSTALL_DIR "${CURL_DIR}"
241 GIT_REPOSITORY https://github.com/curl/curl
242 GIT_TAG ${CURL_TAG}
243 GIT_SHALLOW Yes
244 CMAKE_CACHE_ARGS ${CURL_CMAKE_ARGS}
245 )
246#+END_SRC
247The problem with downloading curl at build-time is that it's simply
248disallowed for many Linux- and BSD-based operation systems. When a
249distribution builds an application from source, its build dependencies
250are often downloaded ahead of time and, as a security measure, the build
251machine is cut off from the internet to prevent any interference.
252Because this is disallowed, the build will fail and the application
253won't be available on those operation systems.
254
255Note, however, that these build machines would have the option to
256disable telemetry at build-time. This means the machine wouldn't attempt
257to download curl from GitHub and the build would succeed but, again,
258telemetry would be disabled for anyone not on Windows or macOS. This
259defeats the whole purpose of adding telemetry in the first place.
260
261#+BEGIN_SRC c
262 S.Id (wxID_NO).AddButton (rejectButtonTitle);
263 S.Id (wxID_YES).AddButton (acceptButtonTitle)->SetDefault ();
264#+END_SRC
265
266There was a lot of feedback about the decision to highlight the consent
267button but that was mentioned up in the frontend section; I won't rehash
268it here.
269
270**** Broader and particularly well-structured comments
271These are simply some comments I feel deserve particular attention.
272
273From SndChaser...
274- [[https://github.com/audacity/audacity/pull/835#issuecomment-834037351][834037351]]
275-
276
277*** The Audacity team's response
278
279*** My opinions
280Can't decide whether to include this section or not. If you make it all
281the way down here, let me know what you think.
282
283** TODO Catchy title about Supernote being "the new paper" :Supernote:Writing:Productivity:Organisation:
284:PROPERTIES:
285:EXPORT_FILE_NAME: something-about-supernote
286:EXPORT_HUGO_CUSTOM_FRONT_MATTER: :toc true
287:END:
288
289I like writing things down. I like the feel of the pen (preferably a
290fountain pen) glide smoothly over the paper, that nice solid feeling of
291the tip against the table, seeing the ink dry as it flows from the nib,
292accidentally swiping my hand through it before it's finished and
293smearing a bit of ink across the page, cursing under my breath as I dab
294it up with a handkerchief or a napkin or something else nearby. I also
295love that writing things by hand [[https://journals.sagepub.com/doi/abs/10.1177/0956797614524581][has an impact on memory and improves
296retention]].
297
298*** The problem
299Unfortunately, I don't love keeping up with that paper. Across many
300different classes, even with dedicated folders for each one, something
301important inevitably gets lost. Notebooks are also bulky and can take up
302a lot of space. I tried [[https://bulletjournal.com/][bullet journalling]] for about a month earlier
303this year and, while the process was enjoyable, the maintenance was not.
304My brain moves faster than my pen (even though I have terrible
305handwriting) and I inevitably forget letters or even whole words. This
306is a problem while writing in pen because white-out looks ugly and I
307dislike wasting whole pages because of a couple mistakes.
308
309The obvious solution here is to get an iPad with an Apple Pen, right?
310Right??
311
312Wrong because Apple bad[fn:2].
313
314*** The solution
315Enter the world of ... what are they even called? E-ink notebooks? Paper
316tablets? E-R/W[fn:1]? Do they even have a "device category" yet? I don't
317know but they solve my problem in a wonderful way.
318
319As the names suggest, these are devices that can /usually/ open and read
320e-books (EPUBs, PDFs, etc.), annotate them, and create standalone pages
321of notes as if they were full notebooks. The most well-known of these
322devices is likely the [[https://remarkable.com/][reMarkable]]. They had a [[https://venturebeat.com/2019/10/08/remarkable-raises-15-million-to-bring-its-e-paper-tablets-to-more-scribblers/][hugely successful
323crowdfunding campaign]] and produced the reMarkable 1, followed by [[https://blog.remarkable.com/remarkable-2-the-next-generation-paper-tablet-91b47d0080cb][the
324reMarkable 2 in 2020]]. There are a few others by now but we'll look at
325the reMarkable first.
326
327*** The reMarkable
328This device boasts all of the features I was looking for. It renders
329digital content, from books and manuals to comics and light novels,
330allows you to mark those documents up as you would if it were physical
331media, create full notebooks of hand written text, organise them,
332search, and, if your handwriting is legible enough (mine certainly is
333not), perform OCR on your notes and email a transcription to yourself.
334It even runs Linux and the developers have opened SSH up so you can
335remote in and tinker with it as much as you like. Because of this,
336there's a pretty awesome [[https://github.com/reHackable/awesome-reMarkable][community of people creating third-party tools
337and integrations]] that add even further functionality. My favourite is
338probably [[https://github.com/bordaigorl/rmview][rMview]], a really fast VNC client for the reMarkable that allows
339you to view your device's screen on any computer.
340
341After watching all of [[https://www.youtube.com/c/MyDeepGuide][MyDeepGuide's]] [[https://www.youtube.com/playlist?list=PLsSI9-gaSSmiXwb7Vjk5Vb-nB41UTnrXd][extensive playlist on the
342reMarkable]], however, I decided to go with a different product.
343
344*** Enter the Supernote A5X
345The [[https://www.supernote.com/#/product?type=SN-A5-X][Supernote A5X]] has all of the basic features the reMarkable has:
346reading documents, writing notes, and organising your content. Its
347implementation, on the other hand, seems to be much more polished. It
348also lacks some features from the reMarkable while adding others.
349
350*** Operating System
351While the reMarkable runs Codex[fn:3], a /"custom Linux-based OS
352optimised for low-latency e-paper"/, the Supernote just runs Android.
353There are both benefits and detriments to this; on one hand, they're
354running all of Android, bloated that it is, on a very lightweight
355tablet. On the other, they don't have to develop and maintain a custom
356operating system. This allows them to focus on other aspects that are
357arguably more important so I don't actually mind that it runs Android.
358
359The only place that Android stands out is in system operations; file
360transfer uses MTP and, when you swipe down from the top of the device, a
361small bar appears similar to what was in early Android. This lets you
362change WiFi networks, sync with the companion app on your LAN, the
363remote servers, take a screenshot, search, and access the system
364settings. Nothing else about the device really makes me think of
365Android.
366
367*** Community
368I don't usually browse Reddit but [[https://old.reddit.com/r/Supernote/][the Supernote community]] there is
369fascinating. I haven't looked around enough to know exactly what his
370relationship is with the company, but one of the members, [[https://old.reddit.com/user/hex2asc][u/hex2asc]],
371seems to represent Supernote in something of an official capacity. He's
372incredibly active and usually responds to posts and questions within a
373couple of days.
374
375Before I purchased one, [[https://old.reddit.com/r/Supernote/comments/lhffyd/sync_targets_open_document_formats_and_crossnote/][I wrote a post]] asking about a couple of things
376that concerned me: sync targets, open document formats, and cross-note
377links. I don't ever plan to write full documents with a keyboard on the
378Supernote but it would still be nice. The other features would be
379absolutely killer for me as I would like to maintain a Zettelkasten (I
380wrote about [[/vim-as-a-markdown-editor/][using Vim to do so]] last year but didn't end up sticking with
381it) and manage document synchronisation with my own Nextcloud server.
382The community was quick to respond and confirm that Zettelkasten
383functionality would be implemented soon™. u/hex2asc responded /the day
384after/ and said that WebDAV would be supported but not earlier than May,
385ODF would likely not be supported, and cross-note links were definitely
386a possibility. Another community member has been avidly following the
387subreddit and even put together an [[https://app-rm.roadmunk.com/publish/03e6dca3d769e2b7015f7f48a649cb3f75f44d9e][unofficial roadmap]].
388
389*** Interfaces
390
391**** Home & Organisation
392
393***** TODO Record very short videos
394
395**** Settings
396
397***** TODO Record very short videos
398
399**** Writing & Annotating
400
401The following images are screenshots of the full page above with the
402possible UI variations while reading a book. This first one is default,
403with the editing bar at the top. It is exactly the same as what's
404displayed on the blank pages for hand writing full notes. From left to
405right is the Table of Contents toggle, the pen tools (fineliner,
406"fountain" pen[fn:5], and highlighter), the erasers, lasso select tool,
407undo/redo, context menu, palm rejection toggle, previous page, goto
408page, next page, and exit.
409
410[[/assets/pngs/supernote-reader-default.png]]
411
412You can hold your finger on that bar and drag it down to detach it from
413the top. The default width exposes all the tools without whitespace. You
414can move it around the screen by dragging the circle with a straight
415line through the middle on the far left.
416
417[[/assets/pngs/supernote-reader-medium.png]]
418
419If you tap that circle, the width shrinks and everything except the
420pens, erasers, and undo/redo buttons are hidden. It can be dragged the
421same was as in the previous image and tapping that circle will expand
422the bar again.
423
424[[/assets/pngs/supernote-reader-small.png]]
425
426The last mode is with the bar completely hidden. You achieve this just
427by dragging it to the right edge of the screen. Once hidden, you can
428swipe right to left from the edge and it will be revealed flush with the
429right edge.
430
431[[/assets/pngs/supernote-reader-minimal.png]]
432
433*** Experience
434**** Reading content
435I love e-ink. I think it looks beautiful and would love to have an e-ink
436monitor[fn:4]. That said, the Supernote has an especially nice display
437with 226 PPI (pixels per inch). The image below was taken with my
438phone's camera so it's not very good. However, if you zoom in a bit, you
439can see that the curved edges of some letters are /slightly/ pixellated.
440Viewing with my naked eye at a comfortable distance, it does look better
441/to me/ than some of my print books.
442
443[[/assets/pngs/supernote-resolution.png]]
444
445/At the moment,/ I am pretty disappointed with Table of Contents detection
446for ePUBs. A great many of my books seem to use a legacy ToC format that
447the Supernote sees and tries/fails to read before attempting to read the
448more up-to-date one. This is easily remedied by editing the ePUB in
449[[https://calibre-ebook.com/][Calibre]], going to Tools → Upgrade Book Internals → Remove the legacy
450Table of Contents in NCX format. You might need to make a small change
451to one of the HTML files and revert it before the save button is
452enabled. After that, just copy it back over to the Supernote and
453everything should work properly.
454
455**** Writing notes
456I write notes as often if not /more/ often than I read and annotate books.
457It's the main reason I purchased the device and I love the experience.
458The Supernote doesn't /really/ feel like paper despite what their
459marketing materials claim, though it doesn't feel /bad/ either. It's hard
460to describe but I would say it's something like writing with a
461rollerball pen on high-quality paper with a marble counter underneath:
462incredibly smooth with but a little bit of texture so it doesn't feel
463like writing on a glass display.
464
465While writing latency[fn:6] is noticeable, I really don't have a huge
466issue with it. I write very quickly but find that the slight latency
467actually makes writing /more/ enjoyable. It sounds weird and I'm not sure
468why, but I /really/ like writing on the Supernote; it's wonderfully
469smooth, pressure-sensitive, the latency makes things interesting, and
470[[https://supernote.com/#/part?id=SP-04][the Heart of Metal pen]] feels good in my hand.
471
472**** Surfacing Content
473While organisation is done using a regular filesystem hierarchy, the
474Supernote does have other ways to search for and surface your notes. As
475you're writing, you can use the lasso select tool and encircle a word. A
476little dialogue pops up and gives you a few buttons for things you can
477do with that selection: copy, move to another page, cut, add it to the
478Table of Contents, or mark it as a key word. If you select the key word
479icon, the Supernote does some incredible OCR[fn:7] on it and displays a
480dialogue where you can add it to the note file as a tag. This dialogue
481allows you to edit the word before adding it just in case the OCR was
482wonky. Even with my terrible handwriting, I've found that it works very
483well and I rarely have to make edits.
484
485*** TODO Pong Isi and Volpeon when finished
486** TODO Migrating repositories between git hosts
487** TODO A perfect email setup (for me)
488:PROPERTIES:
489:EXPORT_FILE_NAME: a-perfect-email-setup-for-me
490:EXPORT_HUGO_CUSTOM_FRONT_MATTER: :toc true
491:END:
492
493I've never been satisfied with any of the email clients most people use.
494I've tried [[https://www.thunderbird.net/en-GB/][Thunderbird]], [[https://wiki.gnome.org/Apps/Evolution][Evolution]], [[https://getmailspring.com/][Mailspring]], [[https://support.apple.com/mail][Mail.app]], [[https://roundcube.net/][Roundcube]],
495[[https://sogo.nu/][SOGo]], [[https://wiki.gnome.org/Apps/Geary][Geary]], and /many/ more. /None/ of them handle multiple accounts
496particularly well because all of the emails associated with that account
497are bound within it. Sure, you can make a new folder somewhere called
498~TODO~ and move all of your actionable emails to that folder but, when you
499go to move actionable emails from /another/ account into that folder,
500you'll likely find that the client simply doesn't let you. If it does,
501when you reply, it will likely be sent from the wrong account. This is a
502limitation of the IMAP protocol; everything is /managed/ locally but
503changes are pushed to the remote server and mixing things the way I want
504leads to broken setups.
505
506Before I go any further, these are a few characteristics of my ideal
507email tool.
508
509- Support for multiple accounts (obviously)
510- /Native desktop application/ (*not* [[https://github.com/electron/electron][Electron]])
511- Has stellar keyboard shortcuts
512- Doesn't require internet connectivity (other than downloading and
513 sending of course)
514- Organisation can be done with tags
515
516*** Why tags?
517Because they're better. Hierarchies are useful for prose and code but
518not for files, emails, notes, or anything where an item may fit within
519multiple categories. Imagine you get an email from your Computer Science
520professor that includes test dates, homework, and information about
521another assignment. In that same email, he asks every student to reply
522with something they learned from the previous class as a form of
523attendance. In a hierarchy, the best place for this might just be a ~TODO~
524folder /even though/ it would also fit under ~School~, ~CS~, ~Dates~, ~To read~,
525and ~Homework~. Maybe you have a few minutes and want to clear out some
526emails that don't require any interaction. In a tag-based workflow, this
527would be a good time to open ~To read~, get that email out of the way, and
528remove the ~To read~ tag. It would still show up under the other tags so
529you can find it later and take the time to fully answer the professor's
530question, add those dates to your calendar, and add the homework
531assignments to your ~TODO~ list. Hierarchies can be quite cumbersome to
532work with, especially when one folder ends up getting all the data. Tags
533ensure that you only see what you want when you want it. Tags are more
534efficient and they will remain my organisation system of choice.
535
536*** The tools
537In short, the tools we will be using are...
538+ [[https://www.offlineimap.org/][OfflineIMAP]] to download our emails
539+ [[https://notmuchmail.org/][~notmuch~]], the primary way emails will be organised
540+ [[https://afew.readthedocs.io/en/latest/][~afew~]] to apply initial ~notmuch~ tags based on subject, sender, recipient, etc.
541+ [[https://neomutt.org/][NeoMutt]] to interact with those emails, reply, compose, add/remove
542 tags, etc.
543+ [[https://marlam.de/msmtp/][~msmtp~]] for relaying our replies and compositions to our mail provider
544
545Yes, it's a lot. Yes, it's time-consuming to set up. Yes, it's worth it
546(in my opinion).
547
548*** OfflineIMAP
549As I said above, IMAP is limiting; we need to use some other method of
550downloading our emails. There's an awesome piece of software called
551[[https://www.offlineimap.org/][OfflineIMAP]] which is built for exactly this purpose. Its configuration
552can be rather daunting if you have as many accounts as I do (17) but
553it's not /terrible/.
554
555**** General
556#+BEGIN_SRC text
557[general]
558metadata = ~/.offlineimap
559accounts = use_exa
560maxsyncaccounts = 1
561ui = basic
562ignore-readonly = no
563pythonfile = ~/.offlineimap.py
564socktimeout = 60
565fsync = true
566#+END_SRC
567
568The first big option is ~accounts~; it tells OfflineIMAP what to actually
569sync. What to put there will be defined further down but ~use_exa~ is just
570filler text. The example account is ~user@example.com~ and I shortened
571that to ~use_exa~. ~maxsyncaccounts~ is also fairly important as it tells
572OfflineIMAP to only pull emails from one account at a time. This is
573certainly slower than multiple but it's also safer because we'll be
574running this in the background and don't want many OfflineIMAP processes
575executing concurrently and interfering with each other. ~pythonfile~ will
576be discussed later.
577
578**** Account
579#+BEGIN_SRC text
580[Account use_exa]
581localrepository = use_exa-local
582remoterepository = use_exa-remote
583quick = 10
584utf8foldernames = yes
585postsynchook = notmuch new
586#+END_SRC
587
588In the first block, ~localrepository~ and ~remoterepository~ tell OfflineIMAP where
589to look for your emails. ~use_exa-local~ is an arbitrary naming scheme I use to
590differentiate between the various local and remote accounts. It can easily be
591swapped with something else.
592
593**** Repository
594#+BEGIN_SRC text
595 [Repository use_exa-local]
596 type = Maildir
597 localfolders = ~/mail/use_exa
598 sync_deletes = yes
599
600 [Repository use_exa-remote]
601 type = IMAP
602 remotehost = imap.example.com
603 starttls = yes
604 ssl = no
605 remoteport = 143
606 remoteuser = user@example.com
607 remotepasseval = get_pass("use_exa")
608 auth_mechanisms = GSSAPI, XOAUTH2, CRAM-MD5, PLAIN, LOGIN
609 maxconnections = 1
610 createfolders = True
611 sync_deletes = yes
612#+END_SRC
613
614The repository sections describe how the emails are stored or retrieved.
615In the ~local~ block, you'll notice that the type is ~Maildir~. In this
616format, each email is given a unique filename and stored in a hierarchy
617of folders within your account. This is often how your emails are stored
618on your provider's mail server.
619
620~pythonfile~ is used here to authenticate with the remote server. This can
621be complicated and depends /entirely/ on how you manage your passwords. I
622use [[https://keepassxc.org/][KeePassXC]] and love it. When I set OfflineIMAP up, however, it didn't
623have ~libsecret~ compatibility. This would have made setup significantly
624easier but, as it already just works™, I don't really see a reason to
625change it.
626
627This new feature allows ~libresecret~-based applications to query
628KeePassXC for your passwords or store them there on your behalf. CLI/TUI
629applications that need a secure mechanism for background authentication
630can use ~secret-tool lookup Title "TITLE_OF_PASSWORD"~ as the password
631command. See [[https://github.com/keepassxreboot/keepassxc/pull/2726][the pull request]] for more details. Because this wasn't a
632feature when I first set it up, I put my passwords in plaintext files
633and encrypted them with the GPG key stored on my YubiKey. As long as my
634key is plugged in, OfflineIMAP can authenticate and download all my
635emails just fine. The process for using a GPG key /not/ stored on a
636hardware token is pretty much the same and I'll talk about that process
637instead.
638
639These are the contents of my ~~/.offlineimap.py~.
640
641#+BEGIN_SRC python
642 #! /usr/bin/env python2
643 from subprocess import check_output
644 def get_pass(account):
645 return check_output(["gpg", "-dq", f" ~/.mail_pass/{account}.gpg"]).strip("\n")
646#+END_SRC
647
648This runs ~gpg -dq ~/.mail_pass/use_exa.gpg~ then strips the newline
649character before returning it to OfflineIMAP. ~-d~ tells GPG that you're
650passing it a file you want decrypted and ~-q~ tells it not to give any
651output other than the file's contents. For a setup that works with this
652Python script, put your passwords in plaintext files with the account
653name as the file name (e.g. ~use_exa~). You'll then encrypt it with ~gpg
654-er <YOUR_KEY_ID> use_exa~. Running ~gpg -dq use_exa.gpg~ should display
655your password. Repeat for every account and store the resulting files in
656~~/.mail_pass/~.
657
658The other option, ~sync_deletes~, is whether or not to delete remote
659emails that have been deleted locally. I enabled that because I want to
660have easy control over how much remote storage is used.
661
662Here's the next block again so you don't have to scroll up:
663
664#+BEGIN_SRC text
665[Repository use_exa-remote]
666type = IMAP
667remotehost = imap.example.com
668starttls = yes
669ssl = no
670remoteport = 143
671remoteuser = user@example.com
672remotepasseval = get_pass("use_exa")
673auth_mechanisms = GSSAPI, XOAUTH2, CRAM-MD5, PLAIN, LOGIN
674maxconnections = 1
675createfolders = True
676sync_deletes = yes
677#+END_SRC
678
679This one's pretty self-explanatory. ~type~, ~remotehost~, ~starttls~, ~ssl~, and
680~remoteport~ should all be somewhere in your provider's documentation.
681~remoteuser~ is your email address and ~remotepasseval~ is the function that
682will return your password and allow OfflineIMAP to authenticate. You'll
683want enter the name of your password file without the ~.gpg~ extension;
684the script takes care of adding that. Leave ~auth_mechanisms~ alone and
685the same for ~maxconnections~ unless you know your provider won't rate
686limit you or something for opening multiple connections. ~sync_deletes~ is
687the same as in the previous block.
688
689Copy those three blocks for as many accounts as you want emails
690downloaded from. I have 510 lines just for ~Account~ and ~Repository~ blocks
691due to the number of address I'm keeping track of.
692
693*** ~notmuch~
694[[https://notmuchmail.org/][~notmuch~]] is /a fast, global-search, and tag-based email system/. This
695what does all of our organisation as well as what provides the "virtual"
696mailboxes NeoMutt will display later on. Configuration is incredibly
697simple. This file goes in ~~/.notmuch-config~.
698
699#+BEGIN_SRC text
700 [database]
701 path=/home/user/mail/
702
703 [user]
704 name=Amolith
705 primary_email=user@example.com
706
707 [new]
708 tags=unread;new;
709 ignore=Trash;
710
711 [search]
712 exclude_tags=deleted;spam;
713
714 [maildir]
715 synchronize_flags=true
716#+END_SRC
717
718First section is the path to where all of your archives are, the ~[user]~
719section is where you list all of your accounts, ~[new]~ adds ~tags~ to mail
720notmuch hasn't indexed yet and ignores indexing the ~Trash~ folder, and
721~[search]~ ignores mail tagged with ~deleted~ or ~spam~. The final section
722tells ~notmuch~ to add maildir flags which correspond with ~notmuch~ tags.
723These flags will be synced to the remote server the next time
724OfflineIMAP runs and things will be somewhat organised in your webmail
725interface.
726
727After creating the configuration file, run ~notmuch new~ and wait for all
728of your mail to be indexed. This could take a short amount of time or it
729could take minutes up to an hour, depending on how many emails you have.
730After it's finished, you'll be able to run queries and see matching
731emails:
732
733#+BEGIN_SRC text
734 $ notmuch search from:user@example.com
735 thread:0000000000002e9d December 28 [1/1] Example User; Random subject that means nothing
736#+END_SRC
737
738This is not terribly useful in and of itself because you can't read it
739or reply to it or anything. That's where the Mail User Agent (MUA) comes
740in.
741
742*** ~afew~
743[[https://afew.readthedocs.io/en/latest/][~afew~]] is /an initial tagging script for notmuch/. After calling ~notmuch
744new~, ~afew~ will add tags based on headers such as ~From:~, ~To:~, ~Subject:~,
745etc. as well as handle killed threads and spam. The official [[https://afew.readthedocs.io/en/latest/quickstart.html][quickstart
746guide]] is probably the best resource on getting started but I'll include
747a few tips here as well.
748
749*** NeoMutt
750*** ~msmtp~
751~msmtp~ is what's known as a /Mail Transfer Agent/ (MTA). You throw it an
752email and it will relay that to your mail provider's SMTP server so it
753can have the proper headers attached for authentication, it can be sent
754from the proper domain, etc. All the necessary security measures can be
755applied that prevent your email from going directly to spam or from
756being rejected outright.
757
758~msmtp~'s configuration is also fairly simple if a bit long, just like
759OfflineIMAP's.
760
761#+BEGIN_SRC text
762 # Set default values for all following accounts.
763 defaults
764
765 # Use the mail submission port 587 instead of the SMTP port 25.
766 port 587
767
768 # Always use TLS.
769 tls on
770#+END_SRC
771
772This section just sets the defaults. It uses port 587 (STARTTLS) for all
773SMTP servers unless otherwise specified and enables TLS.
774
775#+BEGIN_SRC
776 account user@example.com
777 host smtp.example.com
778 from user@example.com
779 auth on
780 user user@example.com
781 passwordeval secret-tool lookup Title "user@example.com"
782#+END_SRC
783
784This section is where things get tedious. When passing an email to
785~msmtp~, it looks at the ~From:~ header and searches for a block with a
786matching ~from~ line. If it finds one, it will use those configuration
787options to relay the email. ~host~ is simply the SMTP server of your mail
788provider, sometimes this is ~mail.example.com~, ~smtp.example.com~, etc.
789I've already explained ~from~, ~auth~ simply says that a username and
790password will have to be provided, ~user~ is that username, and
791~passwordeval~ is a method to obtain the password.
792
793When I got to configuring ~msmtp~, [[https://keepassxc.org/][KeePassXC]] had just released their
794~libsecret~ integration and I wanted to try it. ~secret-tool~ is a command
795line tool used to store and retrieve passwords from whatever keyring
796you're using. I think KDE has ~kwallet~ and GNOME has ~gnome-keyring~ if
797you already have those set up and want to use them; the process should
798be quite similar regardless.
799
800As mentioned above ~secret-tool~ stores and retrieves passwords. For
801retrieval, it expects the command to look like this.
802
803#+BEGIN_SRC text
804secret-tool lookup {attribute} {value} ...
805#+END_SRC
806
807I don't know what ~kwallet~ and ~gnome-keyring~'s attributes are but
808this can be used with KeePassXC by specifying the ~Title~ attribute. If
809the password to your email account is stored in KeePassXC with the
810address as the entry title, you can retrieve it by simply running...
811
812#+BEGIN_SRC text
813secret-tool lookup Title "user@example.com"
814#+END_SRC
815
816If you have a different naming system, you'll have to experiment and try
817different things; I don't know what KeePassXC's other attributes are so
818I can't give other examples.
819
820You could also just use the same method I described in [[*Repository][the Repository
821section]]! It will work perfectly fine here as well.
822
823#+BEGIN_SRC
824passwordeval gpg -dq ~/.mail_pass/use_exa.gpg
825#+END_SRC
826
827Now that the whole block is assembled, copy/paste/edit for as many
828accounts as you want to send email from.
829
830*** Summary
831
832*** TODO Pong fluffy when finished
833
834** TODO Making yourself overly available
835
836*** References
837[[https://www.nytimes.com/2021/03/05/opinion/ezra-klein-podcast-cal-newport.html?showTranscript=1][Stop. Breathe. We Can’t Keep Working Like This.]]
838
839*** Notes
840**** Get rid of information that isn't important
841**** Escalate the info that is
842**** Set /clear/ boundaries for when you are available
843**** Enforce those with automatic DnD rules or use timers
844**** With groups...
845***** Specialisation is /good/ and should be /encouraged/
846***** /All/ of the above points apply with coworkers as well
847
848*** TODO Pong Jake when finished
849
850** TODO Setting LXC up for local "cloud" development
851* Education :@Education:
852** TODO Homeschooling
853* Music :@Music:
854* Pipe Smoking :@Pipe__Smoking:
855* Dungeons & Dragons :@Dungeons__and__Dragons:
856
857* Footnotes
858
859[fn:19] [[https://github.com/audacity/audacity/pull/835#issuecomment-834451187][Link to the comment]] and [[/assets/pngs/audacity-pr/privatelibcurl.png][link to the screenshot]]
860
861[fn:18] [[https://github.com/audacity/audacity/pull/835#issuecomment-834010117][Link to the comment]] and [[/assets/pngs/audacity-pr/vendorproscons.png][link to the screenshot]]
862
863[fn:17] [[https://github.com/audacity/audacity/pull/835#issuecomment-836069326][Link to the comment]] and [[/assets/pngs/audacity-pr/tantacrulrole.png][link to the screenshot]]
864
865[fn:16] [[https://github.com/audacity/audacity/pull/835#discussion_r627762185][Link to the comment]] and [[/assets/pngs/audacity-pr/missingprivacypolicy.png][link to the screenshot]]
866
867[fn:15] [[https://github.com/audacity/audacity/pull/835#issuecomment-834286641][Link to the comment]] and [[/assets/pngs/audacity-pr/darkpattern.png][link to the screenshot]]
868
869[fn:14] [[https://github.com/audacity/audacity/pull/835#discussion_r627764300][Link to the comment]] and the screenshot is the same as previous
870
871[fn:13] [[https://github.com/audacity/audacity/pull/835#discussion_r627756976][Link to the comment]] and [[/assets/pngs/audacity-pr/vaguedialogue.png][link to the screenshot]]
872
873[fn:12] [[https://github.com/audacity/audacity/pull/835#discussion_r628124998][Link to the review]] and [[/assets/pngs/audacity-pr/defaultconsentbutton.png][link to the screenshot]]
874
875[fn:11] [[https://github.com/audacity/audacity/pull/835#discussion_r628008821][Link to the review]] and [[/assets/pngs/audacity-pr/externaldependency.png][link to the screenshot]]
876
877[fn:10] [[https://github.com/audacity/audacity/pull/835#discussion_r628005925][Link to the review]] and [[/assets/pngs/audacity-pr/vendorcurl.png][link to the screenshot]]
878
879[fn:9] Note that because I am not a C programmer, these reviews might
880not be entirely accurate and I wouldn't be able to catch the reviewer's
881error. I am relying on other community members to catch issues and
882comment on them; none of the reviews I link to have such comments so I'm
883assuming they are correct.
884
885[fn:8] [[https://github.com/audacity/audacity/pull/835#discussion_r627993755][Link to the review]] and [[/assets/pngs/audacity-pr/writeanalyticsid.png][link to the screenshot]]
886
887[fn:7] /Optical Character Recognition/: the program looks at your
888handwriting and tries to turn it into text.
889
890[fn:6] In this situation, latency refers to how long it takes for "ink"
891to show up on the "page" after writing something.
892
893[fn:5] It's not really a fountain pen even though that's what they call
894it; it's just pressure-sensitive.
895
896[fn:4] There does seem to be a group of people interested in just such a
897thing: /[[https://alexsoto.dev/challenges-building-an-open-source-eink-laptop.html][Challenges Building an Open-Source E Ink Laptop]]/
898
899[fn:3]Taken from their [[https://support.remarkable.com/hc/en-us/articles/360006699537-About-reMarkable-2-][support page about the reMarkable 2]]; search the
900page for /operating system/ and it should show up.
901
902[fn:2]I dislike Apple's operating system, their hardware, business
903model, privacy practises, and much of what they stand for as a company.
904Don't @ me.
905
906[fn:1]E-R/W is a play on media commonly being labelled as R/W when you
907can read from it and write to it.