1#+HUGO_BASE_DIR: ./
2#+HUGO_SECTION: posts
3#+HUGO_AUTO_SET_LASTMOD: t
4#+HUGO_FRONT_MATTER_FORMAT: yaml
5
6* Meta :@Meta:
7* Technology :@Technology:
8** TODO Audacity and the telemetry pull request :Open__source__culture:Audio__editing:Music:Drama:
9:PROPERTIES:
10:EXPORT_FILE_NAME: audacity-and-the-telemetry-pull-request
11:EXPORT_HUGO_CUSTOM_FRONT_MATTER: :toc true
12:END:
13
14Five days ago at the time of writing, [[https://github.com/crsib][Dmitry Vedenko]] opened a Pull
15Request (PR) in [[https://github.com/audacity/audacity/pull/835][Audacity's GitHub repository]] entitled [[https://github.com/audacity/audacity/pull/835][/Basic telemetry
16for the Audacity/.]] About two days later, all hell broke loose. That PR
17now has over 3.3 thousand downvotes and more than one thousand comments
18from nearly 400 individuals. I started reading the posts shortly after
19they began and kept up with them over the following days, reading every
20single new post. I recognise that few people are going to feel like
21wading through over 1k comments so this is my attempt to provide a
22summary of the PR itself using the community's code reviews along with a
23summary of the various opinions conveyed in the comments.
24
25When I reference comments, I'll provide a footnote that includes a link
26to the comment and a link to a screenshot just in case it's removed or
27edited in the future.
28
29*** Audacity's acquisition
30
31I haven't been able to find /much/ information in this area so forgive me
32if I'm scant on details.
33
34On 30 April, a company called [[https://mu.se/][Muse Group]] acquired [[https://www.audacityteam.org/][Audacity.]] According to
35[[https://mu.se][their website]], Muse is the parent company behind many musical
36applications and tools. It was founded by Eugeny Naidenov just days
37before it acquired Audacity. Before all of this, Eugeny Naidenov founded
38[[https://www.ultimate-guitar.com/][Ultimate Guitar]] (UG) in 1998. The service grew rather quickly and now
39has over 300 million users. UG acquired [[https://deanzelinsky.com/][Dean Zelinsky Guitars]] in 2012,
40[[http://agilepartners.com/][Agile Partners]] in 2013, [[https://musescore.org/][MuseScore]] in 2017, and [[http://trycrescendo.com/][Crescendo]] in 2018. Muse
41Group was established in 2021 and it seems as if all of the services UG
42acquired were (or will be) transferred to Muse Group, as well as UG
43itself. Immediately following its establishment, Muse not only acquired
44Audacity but also [[https://www.staffpad.net/][StaffPad.]]
45
46I say 30 April because that's when Muse published their [[https://mu.se/newsroom/tpost/6dhedma301-muse-group-acquires-audacity-expanding-c][press release]]
47and when Martin Keary (Tantacrul) published a video entitled [[https://www.youtube.com/watch?v=RMWNvwLiXIQ][/I’m now in
48charge of Audacity. Seriously./]] According to his comment,[fn:17] Martin
49will help with proposing Audacity's roadmap and many of its future
50features as well as working with the community. This has been his role
51with MuseScore since he joined that project and he will be continuing it
52here.
53
54~-----BEGIN PERSONAL OPINION-----~
55
56Looking at [[https://www.martinkeary.com/][his website,]] I also suspect he will play a large role in
57redesigning Audacity's interface. Considering that he was instrumental
58in designing [[https://www.martinkeary.com/#/ubuntu-touch-os/][the best mobile interface I've ever had the absolute
59pleasure of experiencing,]] I have high hopes that this is the case.
60
61~------END PERSONAL OPINION------~
62
63*** Telemetry implementation
64**** Implementation Basics
65
66A few days after the acquisition, a PR was opened that adds /Basic
67telemetry for the Audacity/. This implementation collects "application
68opened" events and sends those to Yandex to estimate the number of
69Audacity users. It also collects session start and end events, errors
70for debugging, file used for import and export, OS and Audacity
71versions, and the use of effects, generators, and analysis tools so they
72can prioritise future improvements. Sending this data would be optional
73and the user would be presented with a dialogue the first time they
74launch the application after installation or after they update to the
75including release. This description was mostly copied directly from [[https://github.com/audacity/audacity/pull/835#issue-629891447][the
76PR description itself.]]
77
78**** Frontend Implementation
79This is fairly straightforward and a pretty standard UI for prompting
80users to consent to analytics and crash logging. This section is
81included because the community has strong opinions regarding the
82language used and its design, but that will be discussed later. The
83screenshot below is copied directly from the PR.
84
85[[/assets/pngs/audacity-pr/consentdialogue.png]]
86
87**** Backend Implementation
88Many of the code reviews include the reviewer's personal opinion so I
89will summarise the comment, provide the code block in question, and link
90directly to the comment in a footnote.[fn:9]
91
92#+BEGIN_SRC c
93 if (!inputFile.Write (wxString::FromUTF8 (ClientID + "\n")))
94 return false;
95#+END_SRC
96[[https://github.com/crsib/audacity/blob/c9264d2478fe2af82aeb6e2a0295b00b3a27ce53/libraries/lib-telemetry/TelemetryManager.cpp#L199-L200][Lines 199-200 of TelemetryManager.cpp]] save the user's unique client ID
97to a file.[fn:8] This allows the analytics tool (in this case, Google
98Analytics) to aggregate data produced by a single user.
99
100#+BEGIN_SRC c
101 def_vars()
102
103 set( CURL_DIR "${_INTDIR}/libcurl" )
104 set( CURL_TAG "curl-7_76_0")
105#+END_SRC
106[[https://github.com/crsib/audacity/blob/c9264d2478fe2af82aeb6e2a0295b00b3a27ce53/cmake-proxies/libcurl/CMakeLists.txt#L3-L6][Lines 3-6 of CMakeLists.txt]] "vendor in" libcurl.[fn:10] This is when an
107application directly includes sources for a utility rather than making
108use utilities provided by the system itself.
109
110#+BEGIN_SRC c
111 ExternalProject_Add(curl
112 PREFIX "${CURL_DIR}"
113 INSTALL_DIR "${CURL_DIR}"
114 GIT_REPOSITORY https://github.com/curl/curl
115 GIT_TAG ${CURL_TAG}
116 GIT_SHALLOW Yes
117 CMAKE_CACHE_ARGS ${CURL_CMAKE_ARGS}
118 )
119#+END_SRC
120[[https://github.com/crsib/audacity/blob/c9264d2478fe2af82aeb6e2a0295b00b3a27ce53/cmake-proxies/libcurl/CMakeLists.txt#L29-L36][Lines 29-36 of CMakeLists.txt]] add curl as a remote dependency.[fn:11]
121This means that the machine building Audacity from its source code has
122to download curl during that build.
123
124#+BEGIN_SRC c
125 S.Id (wxID_NO).AddButton (rejectButtonTitle);
126 S.Id (wxID_YES).AddButton (acceptButtonTitle)->SetDefault ();
127#+END_SRC
128[[https://github.com/crsib/audacity/blob/c9264d2478fe2af82aeb6e2a0295b00b3a27ce53/src/telemetry/TelemetryDialog.cpp#L93-L94][Lines 93-94 of TelemetryDialog.cpp]] add buttons to the dialogue asking
129the user whether they consent to data collection.[fn:12] ~SetDefault~
130focuses the button indicating that the user does consent. This means
131that if the user doesn't really look at the dialogue and presses
132Spacebar or Enter, or if they do so accidentally by simply bumping the
133key, they unintentionally consent to data collection. If the user
134desires, this can later be changed in the settings menu. However, if
135they weren't aware what they were consenting to /or that they did
136consent/, they won't know to go back and opt out.
137
138There are other problems with the code that include [[https://github.com/audacity/audacity/pull/835#discussion_r628816050][simple mistakes,]]
139[[https://github.https://github.com/audacity/audacity/pull/835#discussion_r628774985][styling that's inconsistent with the rest of the project,]] [[https://github.com/audacity/audacity/pull/835#discussion_r628500849][unhandled
140return values resulting in skewed data,]] [[https://github.com/audacity/audacity/pull/835#discussion_r628792423][use of inappropriate functions,]]
141and [[https://github.com/audacity/audacity/pull/835#discussion_r628818054][spelling errors in the comments.]] I believe these are less important
142than those above so they won't be discussed.
143
144*** Community opinions
145There were many strong opinions regarding both the frontend and backend
146implementations of this PR, from the wording of the dialogue and
147highlighting the consent button to devices running something other than
148Windows and macOS not being able to send telemetry and thus skewing the
149data that /was/ collected.
150
151**** Opinions on the frontend
152
153Really, the only frontend here is the consent dialogue. However, there
154are /many/ comments about it, the most common of which is probably that
155the wording is not only too vague[fn:13] but also inaccurate.[fn:14] The
156assertion that Google Analytics are not anonymous and any data sent can
157be trivially de-anonymised (or de-pseudonymised) is repeated many times
158over. Below are a few links to comments stating such. I searched for the
159term /"anonymous"/, copied relevant links, and stopped when my scrollbar
160reached halfway down the page.
161
162- [[https://github.com/audacity/audacity/pull/835#discussion_r628156527][r628156527]]
163- [[https://github.com/audacity/audacity/pull/835#issuecomment-833969780][833969780]]
164- [[https://github.com/audacity/audacity/pull/835#issuecomment-833969933][833969933]]
165- [[https://github.com/audacity/audacity/pull/835#discussion_r627995927][r627995927]]
166- [[https://github.com/audacity/audacity/pull/835#issuecomment-834358022][834358022]]
167- [[https://github.com/audacity/audacity/pull/835#issuecomment-834377549][834377549]]
168- [[https://github.com/audacity/audacity/pull/835#issuecomment-834382007][834382007]]
169- [[https://github.com/audacity/audacity/pull/835#issuecomment-834385463][834385463]]
170- [[https://github.com/audacity/audacity/pull/835#issuecomment-834405825][834405825]]
171- [[https://github.com/audacity/audacity/pull/835#issuecomment-834531779][834531779]]
172- [[https://github.com/audacity/audacity/pull/835#issuecomment-834546874][834546874]]
173- [[https://github.com/audacity/audacity/pull/835#issuecomment-834638000][834638000]]
174
175The next most pervasive comment is regarding the consent buttons at the
176bottom of the dialogue where users opt in or out.[fn:15] Many individuals call
177this design a /dark pattern/. Harry Brignull, a UX specialist focusing on
178deceptive interface practises, describes dark patterns as [[https://www.darkpatterns.org/][/tricks used
179in websites and apps that make you do things that you didn't mean to/.]]
180The dark pattern in this situation is the opt-in button being
181highlighted. Many community members assert that users will see the big
182blue button and click it without actually reading the dialogue's
183contents. They just want to record their audio and this window is a
184distraction that prevents them from doing so; it needs to get out of the
185way and the quickest way to dismiss it is clicking that blue button.
186Below is a list of some comments criticising this design.
187
188 - [[https://github.com/audacity/audacity/pull/835#issuecomment-834286641][834286641]]
189 - [[https://github.com/audacity/audacity/pull/835#issuecomment-834358022][834358022]]
190 - [[https://github.com/audacity/audacity/pull/835#issuecomment-834399813][834399813]]
191 - [[https://github.com/audacity/audacity/pull/835#issuecomment-834479968][834479968]]
192 - [[https://github.com/audacity/audacity/pull/835#issuecomment-835250737][835250737]]
193 - [[https://github.com/audacity/audacity/pull/835#issuecomment-835253882][835253882]]
194 - [[https://github.com/audacity/audacity/pull/835#issuecomment-835291066][835291066]]
195 - [[https://github.com/audacity/audacity/pull/835#issuecomment-835445481][835445481]]
196
197Another issue that was brought up by a couple of individuals was the
198lack of a privacy policy.[fn:16] The consent dialogue links to one, but, at the
199time of writing, one does not exist at [[https://www.audacityteam.org/contact/privacy-policy/][the provided URL.]] I have [[https://web.archive.org/web/20210510012924/https://www.audacityteam.org/contact/privacy-policy/][archived
200the state of the page]] in case that changes in the future.
201
202**** Opinions on the backend
203
204#+BEGIN_SRC c
205 if (!inputFile.Write (wxString::FromUTF8 (ClientID + "\n")))
206 return false;
207#+END_SRC
208
209The issue many individuals take with this snippet is saving the
210~ClientID~. Say an individual has an odd file that causes Audacity to
211crash any time they try to open it. Say they attempt to open it a
212hundred times. Without giving the client a unique ID, it could look like
213there are 100 people having an issue opening a file instead of just the
214one. However, by virtue of each installation having an entirely unique
215ID, this telemetry /is not anonymous/. Anonymity would be sending
216statistics in such a way that connecting those failed attempts to a
217single user would be impossible. At best, this implementation is
218/pseudonymous/ because the client is given a random ID, you don't have to
219sign in with an account or something.
220
221#+BEGIN_SRC c
222 def_vars()
223
224 set( CURL_DIR "${_INTDIR}/libcurl" )
225 set( CURL_TAG "curl-7_76_0")
226#+END_SRC
227
228Timothe Litt's comment gives a good description of why "vendoring in"
229libcurl is a bad idea[fn:19] and Tyler True's comment gives a good
230overview of the pros and cons of doing so.[fn:18] Many people take issue
231with this /specifically/ because it's libcurl. Security flaws in it are
232/very/ common and Audacity's copy would need to be /manually/ kept up to
233date with every upstream release to ensure none of its vulnerabilities
234can be leveraged to compromise users. If the Audacity team was going to
235stay on top of all of the security fixes, they would need to release a
236new version every week or so.
237
238#+BEGIN_SRC c
239 ExternalProject_Add(curl
240 PREFIX "${CURL_DIR}"
241 INSTALL_DIR "${CURL_DIR}"
242 GIT_REPOSITORY https://github.com/curl/curl
243 GIT_TAG ${CURL_TAG}
244 GIT_SHALLOW Yes
245 CMAKE_CACHE_ARGS ${CURL_CMAKE_ARGS}
246 )
247#+END_SRC
248The problem with downloading curl at build-time is that it's simply
249disallowed for many Linux- and BSD-based operation systems. When a
250distribution builds an application from source, its build dependencies
251are often downloaded ahead of time and, as a security measure, the build
252machine is cut off from the internet to prevent any interference.
253Because this is disallowed, the build will fail and the application
254won't be available on those operation systems.
255
256Note, however, that these build machines would have the option to
257disable telemetry at build-time. This means the machine wouldn't attempt
258to download curl from GitHub and the build would succeed but, again,
259telemetry would be disabled for anyone not on Windows or macOS. This
260defeats the whole purpose of adding telemetry in the first place.
261
262#+BEGIN_SRC c
263 S.Id (wxID_NO).AddButton (rejectButtonTitle);
264 S.Id (wxID_YES).AddButton (acceptButtonTitle)->SetDefault ();
265#+END_SRC
266
267There was a lot of feedback about the decision to highlight the consent
268button but that was mentioned up in the frontend section; I won't rehash
269it here.
270
271**** Broader and particularly well-structured comments
272These are simply some comments I feel deserve particular attention.
273
274From SndChaser...
275- [[https://github.com/audacity/audacity/pull/835#issuecomment-834037351][834037351]]
276-
277
278*** The Audacity team's response
279
280
281-----------------
282
283*** The privacy policy modification
284
285https://github.com/audacity/audacity/issues/1213#issuecomment-875274890
286
287** TODO Catchy title about Supernote being "the new paper" :Supernote:Writing:Productivity:Organisation:
288:PROPERTIES:
289:EXPORT_FILE_NAME: something-about-supernote
290:EXPORT_HUGO_CUSTOM_FRONT_MATTER: :toc true
291:END:
292
293I like writing things down. I like the feel of the pen (preferably a
294fountain pen) gliding smoothly over the paper, that nice solid feeling
295of the tip against the table, seeing the ink dry as it flows from the
296nib, accidentally swiping my hand through it before it's finished and
297smearing a bit of ink across the page, then cursing under my breath as I
298dab it up with a handkerchief or a napkin or something else nearby. I
299also love that writing things by hand [[https://journals.sagepub.com/doi/abs/10.1177/0956797614524581][has an impact on memory and
300improves retention.]]
301
302*** The problem
303Unfortunately, I don't love keeping up with that paper. Across many
304different classes, even with dedicated folders for each one, something
305important inevitably gets lost. Notebooks are also bulky and can take up
306a lot of space. I tried [[https://bulletjournal.com/][bullet journalling]] for about a month earlier
307this year and, while the process was enjoyable, the maintenance was not.
308My brain moves faster than my pen (even though I have terrible
309handwriting) and I inevitably forget letters or even whole words. This
310is a problem while writing in pen because white-out looks ugly and I
311dislike wasting whole pages because of a couple mistakes.
312
313The obvious solution here is to get an iPad with an Apple Pen, right?
314Right??
315
316Wrong because Apple bad.[fn:2]
317
318*** The solution
319Enter the world of ... what are they even called? E-ink notebooks? Paper
320tablets? E-R/W?[fn:1] Do they even have a "device category" yet? I don't
321know, but they solve my problem in a wonderful way.
322
323As the names suggest, these are devices that can /usually/ open and read
324e-books (EPUBs, PDFs, etc.), annotate them, and create standalone pages
325of notes as if they were full notebooks. The most well-known of these
326devices is likely the [[https://remarkable.com/][reMarkable.]] They had a [[https://venturebeat.com/2019/10/08/remarkable-raises-15-million-to-bring-its-e-paper-tablets-to-more-scribblers/][hugely successful
327crowdfunding campaign]] and produced the reMarkable 1, followed by [[https://blog.remarkable.com/remarkable-2-the-next-generation-paper-tablet-91b47d0080cb][the
328reMarkable 2 in 2020.]] There are a few devices like these by now but
329we'll look at the reMarkable first.
330
331*** The reMarkable
332This device boasts all of the features I was looking for. It renders
333digital content, from books and manuals to comics and manga, allows you
334to mark those documents up as you would if they were physical media,
335create full notebooks of hand written text, organise them, search, and,
336if your handwriting is legible enough (mine certainly is not), perform
337OCR on your notes and email a transcription to yourself. It even runs
338Linux and the developers have opened SSH up so you can remote in and
339tinker with it as much as you like. Because of this, there's a pretty
340awesome [[https://github.com/reHackable/awesome-reMarkable][community of people creating third-party tools and integrations]]
341that add even further functionality. My favourite is probably [[https://github.com/bordaigorl/rmview][rMview,]] a
342really fast VNC client for the reMarkable that allows you to view your
343device's screen on any computer.
344
345After watching all of [[https://www.youtube.com/c/MyDeepGuide][MyDeepGuide's]] [[https://www.youtube.com/playlist?list=PLsSI9-gaSSmiXwb7Vjk5Vb-nB41UTnrXd][extensive playlist on the
346reMarkable,]] however, I decided to go with a different product.
347
348*** Enter the Supernote A5X
349The [[https://www.supernote.com/#/product?type=SN-A5-X][Supernote A5X]] has all of the basic features the reMarkable has:
350reading documents, writing notes, and organising your content. Its
351implementation, on the other hand, seems to be much more polished. It
352also lacks some features from the reMarkable while adding others.
353
354*** Operating System
355While the reMarkable runs Codex,[fn:3] a /"custom Linux-based OS
356optimised for low-latency e-paper"/, the Supernote just runs Android.
357There are both benefits and detriments to this; on one hand, they're
358running all of Android, bloated that it is, on a very lightweight
359tablet. On the other, they don't have to develop and maintain a custom
360operating system. This allows them to focus on other aspects that are
361arguably more important so I don't actually mind that it runs Android.
362
363The only place that Android stands out is in system operations; file
364transfer uses MTP and, when you swipe down from the top of the device, a
365small bar appears similar to what was in early Android. This lets you
366change WiFi networks, sync with the Supernote Cloud, take a screenshot,
367search, and access the system settings. Nothing else about the device
368really screams Android to me.
369
370*** Community
371I don't usually browse Reddit but [[https://old.reddit.com/r/Supernote/][the Supernote community]] there is
372fascinating. I haven't looked around enough to know exactly what his
373relationship is with the company, but one of the members, [[https://old.reddit.com/user/hex2asc][u/hex2asc,]]
374seems to represent Supernote in something of an official capacity. He's
375incredibly active and usually responds to posts and questions within a
376day or two.
377
378Before I purchased a Supernote, [[https://old.reddit.com/r/Supernote/comments/lhffyd/sync_targets_open_document_formats_and_crossnote/][I wrote a post]] asking about a couple of
379things that concerned me: sync targets, open document formats, and
380cross-note links. I don't really plan to write full documents on the
381device but having the option to do so would still be nice. The other
382features are absolutely killer for me as I would like to maintain a
383Zettelkasten (I wrote about [[/vim-as-a-markdown-editor/][using Vim to do so]] last year but didn't end
384up sticking with it) and manage document synchronisation with my own
385Nextcloud server. The community was quick to respond and confirm that
386Zettelkasten functionality would be implemented soon™. u/hex2asc
387responded /the day after/ and said that WebDAV would be supported but not
388earlier than May (September update: it's still not supported), ODF would
389likely not be supported, and cross-note links were definitely a
390possibility. Another community member has been avidly following the
391subreddit and even put together an [[https://app-rm.roadmunk.com/publish/03e6dca3d769e2b7015f7f48a649cb3f75f44d9e][unofficial roadmap.]]
392
393*** Interfaces
394
395**** Home & Organisation
396
397***** TODO Record very short video about home/organisation
398
399**** Settings
400
401***** TODO Record very short video about settings
402
403**** Writing & Annotating
404
405The following images are screenshots of the full page above with the
406possible UI variations while reading a book. This first one is default,
407with the editing bar at the top. It is exactly the same as what's
408displayed on the blank pages for hand writing full notes. From left to
409right is the Table of Contents toggle, the pen tools (fineliner,
410"fountain" pen,[fn:5] and highlighter), the erasers, lasso select tool,
411undo/redo, context menu, palm rejection toggle, previous page, goto
412page, next page, and exit.
413
414[[/assets/pngs/supernote-reader-default.png]]
415
416You can hold your finger on that bar and drag it down to detach it from
417the top. The default width exposes all the tools without whitespace. You
418can move it around the screen by dragging the circle with a straight
419line through the middle on the far left.
420
421[[/assets/pngs/supernote-reader-medium.png]]
422
423If you tap that circle, the width shrinks and everything except the
424pens, erasers, and undo/redo buttons are hidden. It can be dragged the
425same was as in the previous image and tapping that circle will expand
426the bar again.
427
428[[/assets/pngs/supernote-reader-small.png]]
429
430The last mode is with the bar completely hidden. You achieve this just
431by dragging it to the right edge of the screen. Once hidden, you can
432swipe right to left from the edge and it will be revealed flush with the
433right edge.
434
435[[/assets/pngs/supernote-reader-minimal.png]]
436
437*** Experience
438**** Reading content
439I love e-ink. I think it looks beautiful and would love to have an e-ink
440monitor.[fn:4] That said, the Supernote has an especially nice display
441with 226 PPI (pixels per inch). The image below was taken with my
442phone's camera so it's not very good. However, if you zoom in a bit, you
443can see that the curved edges of some letters are /slightly/ pixellated.
444Viewing with my naked eye at a comfortable distance, it does look better
445/to me/ than some of my print books, however.
446
447[[/assets/pngs/supernote-resolution.png]]
448
449/At the moment,/ I am pretty disappointed with Table of Contents detection
450for ePUBs. A great many of my books seem to use a legacy ToC format that
451the Supernote sees and tries/fails to read before attempting to read the
452more up-to-date one. This is easily remedied by editing the ePUB in
453[[https://calibre-ebook.com/][Calibre,]] going to Tools → Upgrade Book Internals → Remove the legacy
454Table of Contents in NCX format. You might need to make a small change
455to one of the HTML files and revert it before the save button is
456enabled. After that, just copy it back over to the Supernote and
457everything should work properly.
458
459**** Writing notes
460I write notes as often if not /more/ often than I read and annotate books.
461It's the main reason I purchased the device and I love the experience.
462The Supernote doesn't /really/ feel like paper despite what their
463marketing materials claim, though it doesn't feel /bad/ either. It's hard
464to describe but I would say it's something like writing with a
465rollerball pen on high-quality paper with a marble counter underneath:
466incredibly smooth with but a little bit of texture so it doesn't feel
467like writing on a glass display.
468
469While writing latency[fn:6] is noticeable, I really don't have a huge
470issue with it. I write very quickly but find that the slight latency
471actually makes writing /more/ enjoyable. It sounds weird and I'm not sure
472why, but I /really/ like writing on the Supernote; it's wonderfully
473smooth, pressure-sensitive, the latency makes things interesting, and
474[[https://supernote.com/#/part?id=SP-04][the Heart of Metal pen]] feels good in my hand.
475
476**** Surfacing Content
477While organisation is done using a regular filesystem hierarchy, the
478Supernote does have other ways to search for and surface your notes. As
479you're writing, you can use the lasso select tool and encircle a word. A
480little dialogue pops up and gives you a few buttons for things you can
481do with that selection: copy, move to another page, cut, add it to the
482Table of Contents, or mark it as a key word. If you select the key word
483icon, the Supernote does some incredible OCR[fn:7] on it and displays a
484dialogue where you can add it to the note file as a tag. This dialogue
485allows you to edit the word before adding it just in case the OCR was
486wonky. Even with my terrible handwriting, I've found that it works very
487well and I rarely have to make edits.
488
489*** TODO Ping Isi and Volpeon when finished
490** TODO Migrating repositories between git hosts
491** TODO A perfect email setup (for me) :Email:Workflow:
492:PROPERTIES:
493:EXPORT_FILE_NAME: a-perfect-email-setup-for-me
494:EXPORT_HUGO_CUSTOM_FRONT_MATTER: :toc true
495:END:
496
497I've never been satisfied with any of the email clients most people use.
498I've tried [[https://www.thunderbird.net/en-GB/][Thunderbird,]] [[https://wiki.gnome.org/Apps/Evolution][Evolution,]] [[https://getmailspring.com/][Mailspring,]] [[https://support.apple.com/mail][Mail.app,]] [[https://roundcube.net/][Roundcube,]]
499[[https://sogo.nu/][SOGo,]] [[https://wiki.gnome.org/Apps/Geary][Geary,]] and /many/ more. /None/ of them handle multiple accounts
500particularly well because all of the emails associated with that account
501are bound within it. Sure, you can make a new folder somewhere called
502~TODO~ and move all of your actionable emails to that folder but, when you
503go to move actionable emails from /another/ account into that folder,
504you'll likely find that the client simply doesn't let you. If it does,
505when you reply, it will likely be sent from the wrong account. This is a
506limitation of the IMAP protocol; everything is /managed/ locally but
507changes are pushed to the remote server and mixing things the way I want
508leads to broken setups.
509
510Before I go any further, these are a few characteristics of my ideal
511email tool.
512
513- Support for multiple accounts (obviously)
514- /Native desktop application/ (*not* [[https://github.com/electron/electron][Electron]])
515- Has stellar keyboard shortcuts
516- Doesn't require internet connectivity (other than downloading and
517 sending of course)
518- Organisation can be done with tags
519
520*** Why tags?
521Because they're better. Hierarchies are useful for prose and code but
522not for files, emails, notes, or anything where an item may fit within
523multiple categories. Imagine you get an email from your Computer Science
524professor that includes test dates, homework, and information about
525another assignment. In that same email, he asks every student to reply
526with something they learned from the previous class as a form of
527attendance. In a hierarchy, the best place for this might just be a ~TODO~
528folder /even though/ it would also fit under ~School~, ~CS~, ~Dates~, ~To read~,
529and ~Homework~. Maybe you have a few minutes and want to clear out some
530emails that don't require any interaction. In a tag-based workflow, this
531would be a good time to open ~To read~, get that email out of the way, and
532remove the ~To read~ tag. It would still show up under the other tags so
533you can find it later and take the time to fully answer the professor's
534question, add those dates to your calendar, and add the homework
535assignments to your ~TODO~ list. Hierarchies can be quite cumbersome to
536work with, especially when one folder ends up getting all the data. Tags
537ensure that you only see what you want when you want it. Tags are more
538efficient and they will remain my organisation system of choice.
539
540*** The tools
541In short, the tools we will be using are...
542+ [[https://isync.sourceforge.io/mbsync.html][~mbsync~]] to download our emails
543+ [[https://notmuchmail.org/][~notmuch~,]] the primary way emails will be organised
544+ [[https://afew.readthedocs.io/en/latest/][~afew~]] to apply initial ~notmuch~ tags based on subject, sender, recipient, etc.
545+ [[https://neomutt.org/][NeoMutt]] to interact with those emails, reply, compose, add/remove
546 tags, etc.
547+ [[https://marlam.de/msmtp/][~msmtp~]] for relaying our replies and compositions to our mail provider
548
549Yes, it's a lot. Yes, it's time-consuming to set up. Yes, it's worth it
550(in my opinion).
551
552*** ~mbsync~
553As I said above, IMAP is limiting; we need to use some other method of
554downloading our emails. There's an awesome piece of software called
555[[https://isync.sourceforge.io/mbsync.html][mbsync]] which is built for exactly this purpose. Its configuration
556can be rather daunting if you have as many accounts as I do (19) but
557it's not /terrible/.
558
559The following sections are named *Near*, *Far*, and *Sync*. Near and Far are
560terms mbsync uses to profile /how/ your emails are stored, /where/ they're
561stored, and how to interact with them. In this guide, Far will our mail
562provider's IMAP server and Near will be our local Maildir.
563
564**** Far
565#+BEGIN_SRC text
566IMAPAccount amo_ema
567Host imap.nixnet.email
568CertificateFile /etc/ssl/certs/ca-certificates.crt
569SSLType STARTTLS
570User amolith@nixnet.email
571PassCmd "secret-tool lookup Title amolith@nixnet.email"
572
573IMAPStore amo_ema-remote
574Account amo_ema
575#+END_SRC
576
577**** Near
578#+BEGIN_SRC text
579MaildirStore amo_ema-local
580SubFolders Verbatim
581Path ~/new-mail/amo_ema/
582Inbox ~/new-mail/amo_ema/INBOX/
583#+END_SRC
584
585In the first block, ~localrepository~ and ~remoterepository~ tell OfflineIMAP where
586to look for your emails. ~use_exa-local~ is an arbitrary naming scheme I use to
587differentiate between the various local and remote accounts. It can easily be
588swapped with something else.
589
590**** Sync
591#+BEGIN_SRC text
592Channel amo_ema
593Far :amo_ema-remote:
594Near :amo_ema-local:
595SyncState *
596Patterns *
597Create Both
598#+END_SRC
599
600The repository sections describe how the emails are stored or retrieved.
601In the ~local~ block, you'll notice that the type is ~Maildir~. In this
602format, each email is given a unique filename and stored in a hierarchy
603of folders within your account. This is often how your emails are stored
604on your provider's mail server.
605
606~pythonfile~ is used here to authenticate with the remote server. This can
607be complicated and depends /entirely/ on how you manage your passwords. I
608use [[https://keepassxc.org/][KeePassXC]] and love it. When I set OfflineIMAP up, however, it didn't
609have ~libsecret~ compatibility. This would have made setup significantly
610easier but, as it already just works™, I don't really see a reason to
611change it.
612
613This new feature allows ~libresecret~-based applications to query
614KeePassXC for your passwords or store them there on your behalf. CLI/TUI
615applications that need a secure mechanism for background authentication
616can use ~secret-tool lookup Title "TITLE_OF_PASSWORD"~ as the password
617command. See [[https://github.com/keepassxreboot/keepassxc/pull/2726][the pull request]] for more details. Because this wasn't a
618feature when I first set it up, I put my passwords in plaintext files
619and encrypted them with the GPG key stored on my YubiKey. As long as my
620key is plugged in, OfflineIMAP can authenticate and download all my
621emails just fine. The process for using a GPG key /not/ stored on a
622hardware token is pretty much the same and I'll talk about that process
623instead.
624
625These are the contents of my ~~/.offlineimap.py~.
626
627#+BEGIN_SRC python
628 #! /usr/bin/env python2
629 from subprocess import check_output
630 def get_pass(account):
631 return check_output(["gpg", "-dq", f" ~/.mail_pass/{account}.gpg"]).strip("\n")
632#+END_SRC
633
634This runs ~gpg -dq ~/.mail_pass/use_exa.gpg~ then strips the newline
635character before returning it to OfflineIMAP. ~-d~ tells GPG that you're
636passing it a file you want decrypted and ~-q~ tells it not to give any
637output other than the file's contents. For a setup that works with this
638Python script, put your passwords in plaintext files with the account
639name as the file name (e.g. ~use_exa~). You'll then encrypt it with ~gpg
640-er <YOUR_KEY_ID> use_exa~. Running ~gpg -dq use_exa.gpg~ should display
641your password. Repeat for every account and store the resulting files in
642~~/.mail_pass/~.
643
644The other option, ~sync_deletes~, is whether or not to delete remote
645emails that have been deleted locally. I enabled that because I want to
646have easy control over how much remote storage is used.
647
648Here's the next block again so you don't have to scroll up:
649
650#+BEGIN_SRC text
651[Repository use_exa-remote]
652type = IMAP
653remotehost = imap.example.com
654starttls = yes
655ssl = no
656remoteport = 143
657remoteuser = user@example.com
658remotepasseval = get_pass("use_exa")
659auth_mechanisms = GSSAPI, XOAUTH2, CRAM-MD5, PLAIN, LOGIN
660maxconnections = 1
661createfolders = True
662sync_deletes = yes
663#+END_SRC
664
665This one's pretty self-explanatory. ~type~, ~remotehost~, ~starttls~, ~ssl~, and
666~remoteport~ should all be somewhere in your provider's documentation.
667~remoteuser~ is your email address and ~remotepasseval~ is the function that
668will return your password and allow OfflineIMAP to authenticate. You'll
669want enter the name of your password file without the ~.gpg~ extension;
670the script takes care of adding that. Leave ~auth_mechanisms~ alone and
671the same for ~maxconnections~ unless you know your provider won't rate
672limit you or something for opening multiple connections. ~sync_deletes~ is
673the same as in the previous block.
674
675Copy those three blocks for as many accounts as you want emails
676downloaded from. I have 510 lines just for ~Account~ and ~Repository~ blocks
677due to the number of address I'm keeping track of.
678
679*** ~notmuch~
680[[https://notmuchmail.org/][~notmuch~]] is /a fast, global-search, and tag-based email system/. This
681what does all of our organisation as well as what provides the "virtual"
682mailboxes NeoMutt will display later on. Configuration is incredibly
683simple. This file goes in ~~/.notmuch-config~.
684
685#+BEGIN_SRC text
686 [database]
687 path=/home/user/mail/
688
689 [user]
690 name=Amolith
691 primary_email=user@example.com
692
693 [new]
694 tags=unread;new;
695 ignore=Trash;
696
697 [search]
698 exclude_tags=deleted;spam;
699
700 [maildir]
701 synchronize_flags=true
702#+END_SRC
703
704First section is the path to where all of your archives are, the ~[user]~
705section is where you list all of your accounts, ~[new]~ adds ~tags~ to mail
706notmuch hasn't indexed yet and ignores indexing the ~Trash~ folder, and
707~[search]~ ignores mail tagged with ~deleted~ or ~spam~. The final section
708tells ~notmuch~ to add maildir flags which correspond with ~notmuch~ tags.
709These flags will be synced to the remote server the next time
710OfflineIMAP runs and things will be somewhat organised in your webmail
711interface.
712
713After creating the configuration file, run ~notmuch new~ and wait for all
714of your mail to be indexed. This could take a short amount of time or it
715could take minutes up to an hour, depending on how many emails you have.
716After it's finished, you'll be able to run queries and see matching
717emails:
718
719#+BEGIN_SRC text
720 $ notmuch search from:user@example.com
721 thread:0000000000002e9d December 28 [1/1] Example User; Random subject that means nothing
722#+END_SRC
723
724This is not terribly useful in and of itself because you can't read it
725or reply to it or anything. That's where the Mail User Agent (MUA) comes
726in.
727
728*** ~afew~
729[[https://afew.readthedocs.io/en/latest/][~afew~]] is /an initial tagging script for notmuch/. After calling ~notmuch
730new~, ~afew~ will add tags based on headers such as ~From:~, ~To:~, ~Subject:~,
731etc. as well as handle killed threads and spam. The official [[https://afew.readthedocs.io/en/latest/quickstart.html][quickstart
732guide]] is probably the best resource on getting started but I'll include
733a few tips here as well.
734
735*** NeoMutt
736*** ~msmtp~
737~msmtp~ is what's known as a /Mail Transfer Agent/ (MTA). You throw it an
738email and it will relay that to your mail provider's SMTP server so it
739can have the proper headers attached for authentication, it can be sent
740from the proper domain, etc. All the necessary security measures can be
741applied that prevent your email from going directly to spam or from
742being rejected outright.
743
744~msmtp~'s configuration is also fairly simple if a bit long, just like
745OfflineIMAP's.
746
747#+BEGIN_SRC text
748 # Set default values for all following accounts.
749 defaults
750
751 # Use the mail submission port 587 instead of the SMTP port 25.
752 port 587
753
754 # Always use TLS.
755 tls on
756#+END_SRC
757
758This section just sets the defaults. It uses port 587 (STARTTLS) for all
759SMTP servers unless otherwise specified and enables TLS.
760
761#+BEGIN_SRC text
762 account user@example.com
763 host smtp.example.com
764 from user@example.com
765 auth on
766 user user@example.com
767 passwordeval secret-tool lookup Title "user@example.com"
768#+END_SRC
769
770This section is where things get tedious. When passing an email to
771~msmtp~, it looks at the ~From:~ header and searches for a block with a
772matching ~from~ line. If it finds one, it will use those configuration
773options to relay the email. ~host~ is simply the SMTP server of your mail
774provider, sometimes this is ~mail.example.com~, ~smtp.example.com~, etc.
775I've already explained ~from~, ~auth~ simply says that a username and
776password will have to be provided, ~user~ is that username, and
777~passwordeval~ is a method to obtain the password.
778
779When I got to configuring ~msmtp~, [[https://keepassxc.org/][KeePassXC]] had just released their
780~libsecret~ integration and I wanted to try it. ~secret-tool~ is a command
781line tool used to store and retrieve passwords from whatever keyring
782you're using. I think KDE has ~kwallet~ and GNOME has ~gnome-keyring~ if
783you already have those set up and want to use them; the process should
784be quite similar regardless.
785
786As mentioned above ~secret-tool~ stores and retrieves passwords. For
787retrieval, it expects the command to look like this.
788
789#+BEGIN_SRC text
790secret-tool lookup {attribute} {value} ...
791#+END_SRC
792
793I don't know what ~kwallet~ and ~gnome-keyring~'s attributes are but
794this can be used with KeePassXC by specifying the ~Title~ attribute. If
795the password to your email account is stored in KeePassXC with the
796address as the entry title, you can retrieve it by simply running...
797
798#+BEGIN_SRC text
799secret-tool lookup Title "user@example.com"
800#+END_SRC
801
802If you have a different naming system, you'll have to experiment and try
803different things; I don't know what KeePassXC's other attributes are so
804I can't give other examples.
805
806#+BEGIN_SRC text
807passwordeval gpg -dq ~/.mail_pass/use_exa.gpg
808#+END_SRC
809
810Now that the whole block is assembled, copy/paste/edit for as many
811accounts as you want to send email from.
812
813*** Summary
814
815*** TODO Pong fluffy when finished
816
817** TODO Making yourself overly available
818
819*** References
820[[https://www.nytimes.com/2021/03/05/opinion/ezra-klein-podcast-cal-newport.html?showTranscript=1][Stop. Breathe. We Can’t Keep Working Like This.]]
821
822*** Notes
823**** Get rid of information that isn't important
824**** Escalate the info that is
825**** Set /clear/ boundaries for when you are available
826**** Enforce those with automatic DnD rules or use timers
827**** With groups...
828***** Specialisation is /good/ and should be /encouraged/
829***** /All/ of the above points apply with coworkers as well
830
831*** TODO Pong Jake when finished
832
833** TODO Setting LXC up for local "cloud" development
834** DONE (Ab)using mesh networks for easy remote support :Mesh__networking:Open__source:Remote__support:
835CLOSED: [2021-11-01 Mon 02:51]
836:PROPERTIES:
837:EXPORT_FILE_NAME: abusing-mesh-networks-for-easy-remote-support
838:EXPORT_HUGO_CUSTOM_FRONT_MATTER: :toc true
839:END:
840One of the things many of us struggle with when setting friends and
841family up with Linux is remote support. Commercial solutions like
842[[https://www.realvnc.com/][RealVNC]] and [[https://rustdesk.com/][RustDesk]] do exist and function very well, but are often more
843expensive than we would like for answering the odd "I can't get Facebook
844open!" support call. I've been on the lookout for suitable alternatives
845for a couple years but nothing has been satisfying. Because of this, I
846have held off on setting others up with any Linux distribution, even the
847particularly user-friendly options such as [[https://linuxmint.com/][Linux Mint]] and [[https://elementary.io/][elementary OS;]]
848if I'm going drop someone in an unfamiliar environment, I want to be
849able to help with any issue within a couple hours, not days and
850/certainly/ not weeks.
851
852[[https://linuxunplugged.com/421][Episode 421 of LINUX Unplugged]] gave me an awesome idea to use [[https://github.com/slackhq/nebula][Nebula,]] a
853networking tool created by Slack, [[https://libvnc.github.io/][X11vnc,]] a very minimal VNC server, and
854[[https://remmina.org/][Remmina,]] a libre remote access tool available in pretty much every Linux
855distribution, to set up a scalable, secure, and simple setup reminiscent
856of products like RealVNC.
857
858*** Nebula
859The first part of our stack is Nebula, the tool that creates a network
860between all of our devices. With traditional VPNs, you have a client
861with a persistent connection to a central VPN server and other clients
862can communicate with the first by going through that central server.
863This works wonderfully in most situations, but there are a lot of
864latency and bandwidth restrictions that would make remote support an
865unpleasant experience. Instead of this model, what we want is a /mesh/
866network, where each client can connect directly to one another /without/
867going through a central system and slowing things down. This is where
868Nebula comes in.
869
870In Nebula's terminology, clients are referred to as /nodes/ and central
871servers are referred to as /lighthouses/, so those are the terms I'll use
872going forward.
873
874Mesh networks are usually only possible when dealing with devices that
875have static IP addresses. Each node has to know /how/ to connect with the
876other nodes; John can't meet up with Bob when Bob moves every other day
877without notifying anyone of his new address. This wouldn't be a problem
878if Bob phoned Jill and told her where he was moving; John would call
879Jill, Jill would tell him where Bob is, and the two would be able to
880find each other
881
882With Nebula, nodes are Bob and John and Jill is a lighthouse. Each node
883connects to a lighthouse and the lighthouse tells the nodes how to
884connect with one another when they ask. It /facilitates/ the P2P
885connection then /backs out of the way/ so the two nodes can communicate
886directly with each other.
887
888It allows any node to connect with any other node on any network from
889anywhere in the world, as long as one lighthouse is accessible that
890knows the connection details for both peers.
891
892**** Getting started
893The /best/ resource is [[https://github.com/slackhq/nebula][the official documentation,]] but I'll describe the
894process here as well.
895
896After [[https://github.com/slackhq/nebula#1-the-nebula-binaries-or-distribution-packages-for-your-specific-platform-specifically-youll-need-nebula-cert-and-the-specific-nebula-binary-for-each-platform-you-use][installing the required packages,]] make sure you have a VPS with a
897static IP address to use as a lighthouse. If you want something dirt
898cheap, I would recommend one of the small plans from [[https://buyvm.net][BuyVM.]] I do have a
899[[https://my.frantech.ca/aff.php?aff=3783][referral link]] if you want them to kick me a few dollars for your
900purchase. [[https://www.hetzner.com/cloud][Hetzner]] (referral: ~ckGrk4J45WdN~) or [[https://www.netcup.eu/][netcup]] (referral:
901~36nc15758387844~) would also be very good options; I've used them all and
902am very comfortable recommending them.
903
904**** Creating a Certificate Authority
905After picking a device with a static IP address, it needs to be set up
906as a lighthouse. This is done by first creating a Certificate Authority
907(CA) that will be used for signing keys and certificates that allow our
908other devices into the network. The ~.key~ file produced by the following
909command is incredibly sensitive; with it, anyone can authorise a new
910device and give it access to your network. Store it in a safe,
911preferably encrypted location.
912
913#+BEGIN_SRC bash
914 nebula-cert ca -name "nebula.example.com"
915#+END_SRC
916
917I'll explain why we used a Fully-Qualified Domain Name (FQDN) as the
918CA's name in a later section. If you have your own domain, feel free to
919use that instead; it doesn't really matter what domain is used as long
920as the format is valid.
921
922**** Generating lighthouse credentials
923Now that we have the CA's ~.crt~ and ~.key~ files, we can create and sign
924keys and certificates for the lighthouse.
925
926#+BEGIN_SRC bash
927 nebula-cert sign -name "buyvm.lh.nebula.example.com" -ip "192.168.100.1/24"
928#+END_SRC
929
930Here, we're using a FQDN for the same reason as we did in the CA. You
931can use whatever naming scheme you like, I just prefer
932~<vps-host>.lh.nebula...~ for my lighthouses. The IP address can be on any
933of the following private IP ranges, I just happened to use ~192.168.100.X~
934for my network.
935
936| IP Range | Number of addresses |
937|-------------------------------+---------------------|
938| 10.0.0.0 – 10.255.255.255 | 16 777 216 |
939| 172.16.0.0 – 172.31.255.255 | 10 48 576 |
940| 192.168.0.0 – 192.168.255.255 | 65 536 |
941
942**** Creating a config file
943The next step is creating our lighthouse's config file. The reference
944config can be found in [[https://github.com/slackhq/nebula/blob/master/examples/config.yml][Nebula's repo.]] We only need to change a few of
945the lines for the lighthouse to work properly. If I don't mention a
946specific section here, I've left the default values.
947
948The section below is where we'll define certificates and keys. ~ca.crt~
949will remain ~ca.crt~ when we copy it over but I like to leave the node's
950cert and key files named as they were when generated; this makes it easy
951to identify nodes by their configs. Once we copy everything over to the
952server, we'll add the proper paths to the ~cert~ and ~key~ fields.
953
954#+BEGIN_SRC yaml
955 pki:
956 ca: /etc/nebula/ca.crt
957 cert: /etc/nebula/
958 key: /etc/nebula/
959#+END_SRC
960
961The next section is for identifying and mapping your lighthouses. This
962needs to be present in /all/ of the configs on /all/ nodes, otherwise they
963won't know how to reach the lighthouses and will never actually join the
964network. Make sure you replace ~XX.XX.XX.XX~ with whatever your VPS's
965public IP address is. If you've used a different private network range,
966those changes need to be reflected here as well.
967
968#+BEGIN_SRC yaml
969 static_host_map:
970 "192.168.100.1": ["XX.XX.XX.XX:4242"]
971#+END_SRC
972
973Below, we're specifying how the node should behave. It is a lighthouse,
974it should answer DNS requests, the DNS server should listen on all
975interfaces on port 53, it sends its IP address to lighthouses every 60
976seconds (this option doesn't actually have any effect when ~am_lighthouse~
977is set to ~true~ though), and this lighthouse should not send reports to
978other lighthouses. The bit about DNS will be discussed later.
979
980#+BEGIN_SRC yaml
981 lighthouse:
982 am_lighthouse: true
983 serve_dns: true
984 dns:
985 host: 0.0.0.0
986 port: 53
987 interval: 60
988 hosts:
989#+END_SRC
990
991The next bit is about [[https://en.wikipedia.org/wiki/Hole_punching_%28networking%29][hole punching]], also called /NAT punching/, /NAT
992busting/, and a few other variations. Make sure you read the comments for
993better explanations than I'll give here. ~punch: true~ enables hole
994punching. I also like to enable ~respond~ just in case nodes are on
995particularly troublesome networks; because we're using this as a support
996system, we have no idea what networks our nodes will actually be
997connected to. We want to make sure devices are available no matter where
998they are.
999
1000#+BEGIN_SRC yaml
1001 punchy:
1002 punch: true
1003 respond: true
1004 delay: 1s
1005#+END_SRC
1006
1007~cipher~ is a big one. The value /must/ be identical on /all/ nodes /and/
1008lighthouses. ~chachapoly~ is more compatible so it's used by default. The
1009devices /I/ want to connect to are all x86 Linux, so I can switch to ~aes~
1010and benefit from [[https://www.reddit.com/r/networking/comments/iksyuu/comment/g3ra5cv/?utm_source=share&utm_medium=web2x&context=3][a small performance boost.]] Unless you know /for sure/
1011that you won't need to work with /anything/ else, I recommend leaving it
1012set to ~chachapoly~.
1013
1014#+BEGIN_SRC yaml
1015 cipher: chachapoly
1016#+END_SRC
1017
1018The last bit I modify is the firewall section. I leave most everything
1019default but /remove/ the bits after ~port: 443~. I don't /need/ the ~laptop~ and
1020~home~ groups (groups will be explained later) to access port ~443~ on this
1021node, so I shouldn't include the statement. If you have different needs,
1022take a look at the comment explaining how the firewall portion works and
1023make those changes.
1024
1025Again, I /remove/ the following bit from the config.
1026
1027#+BEGIN_SRC yaml
1028 - port: 443
1029 proto: tcp
1030 groups:
1031 - laptop
1032 - home
1033#+END_SRC
1034
1035**** Setting the lighthouse up
1036We've got the config, the certificates, and the keys. Now we're ready to
1037actually set it up. After SSHing into the server, grab the [[https://github.com/slackhq/nebula/releases/latest][latest
1038release of Nebula for your platform,]] unpack it, make the ~nebula~ binary
1039executable, then move it to ~/usr/local/bin~ (or some other location
1040fitting for your platform).
1041
1042#+BEGIN_SRC bash
1043 wget https://github.com/slackhq/nebula/releases/download/vX.X.X/nebula-PLATFORM-ARCH.tar.gz
1044 tar -xvf nebula-*
1045 chmod +x nebula
1046 mv nebula /usr/local/bin/
1047 rm nebula-*
1048#+END_SRC
1049
1050Now we need a place to store our config file, keys, and certificates.
1051
1052#+BEGIN_SRC bash
1053 mkdir /etc/nebula/
1054#+END_SRC
1055
1056The next step is copying the config, keys, and certificates to the
1057server. I use ~rsync~ but you can use whatever you're comfortable with.
1058The following four files need to be uploaded to the server.
1059
1060- ~config.yml~
1061- ~ca.crt~
1062- ~buyvm.lh.nebula.example.com.crt~
1063- ~buyvm.lh.nebula.example.com.key~
1064
1065With ~rsync~, that would look something like this. Make sure ~rsync~ is also
1066installed on the VPS before attempting to run the commands though;
1067you'll get an error otherwise.
1068
1069#+BEGIN_SRC bash
1070 rsync -avmzz ca.crt user@example.com:
1071 rsync -avmzz config.yml user@example.com:
1072 rsync -avmzz buyvm.lh.* user@example.com:
1073#+END_SRC
1074
1075SSH back into the server and move everything to ~/etc/nebula/~.
1076
1077#+BEGIN_SRC bash
1078 mv ca.crt /etc/nebula/
1079 mv config.yml /etc/nebula/
1080 mv buyvm.lh* /etc/nebula/
1081#+END_SRC
1082
1083Edit the config file and ensure the ~pki:~ section looks something like
1084this, modified to match your hostnames of course.
1085
1086#+BEGIN_SRC yaml
1087 pki:
1088 ca: /etc/nebula/ca.crt
1089 cert: /etc/nebula/buyvm.lh.nebula.example.com.crt
1090 key: /etc/nebula/buyvm.lh.nebula.example.com.key
1091#+END_SRC
1092
1093Run the following command to make sure everything works properly.
1094
1095#+BEGIN_SRC bash
1096 nebula -config /etc/nebula/config.yml
1097#+END_SRC
1098
1099The last step is daemonizing Nebula so it runs every time the server
1100boots. If you're on a machine using systemd, dropping the following
1101snippet into ~/etc/systemd/system/nebula.service~ should be sufficient. If
1102you're using something else, check the [[https://github.com/slackhq/nebula/tree/master/examples/][the examples directory]] for more
1103options.
1104
1105#+BEGIN_SRC text
1106 [Unit]
1107 Description=nebula
1108 Wants=basic.target
1109 After=basic.target network.target
1110 Before=sshd.service
1111
1112 [Service]
1113 SyslogIdentifier=nebula
1114 ExecReload=/bin/kill -HUP $MAINPID
1115 ExecStart=/usr/local/bin/nebula -config /etc/nebula/config.yml
1116 Restart=always
1117
1118 [Install]
1119 WantedBy=multi-user.target
1120#+END_SRC
1121
1122We're almost done!
1123
1124**** Setting individual nodes up
1125This process is almost exactly the same as setting lighthouses up. All
1126you'll need to do is generate a couple of certs and keys then tweak the
1127configs a bit.
1128
1129The following command creates a new cert/key for USER's node with the IP
1130address ~192.168.100.2~. The resulting files would go on the /remote/ node
1131not yours. Replace ~HOST~ and ~USER~ with fitting values.
1132
1133#+BEGIN_SRC bash
1134 nebula-cert sign -name "HOST.USER.nebula.example.com" -ip "192.168.100.2/24"
1135#+END_SRC
1136
1137The following command will create a /similar/ cert/key but it will be part
1138of the ~support~ group. The files resulting from this should go on /your/
1139nodes. With the config we'll create next, nodes in the ~support~ group
1140will be able to VNC and SSH into other nodes. Your nodes need to be in
1141the ~support~ group so you'll have access to the others.
1142
1143#+BEGIN_SRC bash
1144 nebula-cert sign -name "HOST.USER.nebula.example.com" -ip "192.168.100.2/24" -groups "support"
1145#+END_SRC
1146
1147On to the config now. This tells the node that it is /not/ a lighthouse,
1148it should /not/ resolve DNS requests, it /should/ ping the lighthouses and
1149tell them its IP address every 60 seconds, and the node at ~192.168.100.1~
1150is one of the lighthouses it should report to and query from. If you
1151have more than one lighthouse, add them to the list as well.
1152
1153#+BEGIN_SRC yaml
1154 lighthouse:
1155 am_lighthouse: false
1156 #serve_dns: false
1157 #dns:
1158 #host: 0.0.0.0
1159 #port: 53
1160 interval: 60
1161 hosts:
1162 - "192.168.100.1"
1163#+END_SRC
1164
1165The other bit that should be modified is the ~firewall:~ section and this
1166is where the groups we created earlier are important. Review its
1167comments and make sure you understand how it works before proceeding.
1168
1169We want to allow inbound connections on ports 5900, the standard port
1170for VNC, and 22, the standard for SSH. Additionally, we /only/ want to
1171allow connections from nodes in the ~support~ group. Any /other/ nodes
1172should be denied access.
1173
1174Note that including this section is not necessary on /your/ nodes, those
1175in the ~support~ group. It's only necessary on the remote nodes that
1176you'll be connecting to. As long as the ~outbound:~ section in the config
1177on /your/ node allows any outbound connection, you'll be able to access
1178other nodes.
1179
1180#+BEGIN_SRC yaml
1181 - port: 5900
1182 proto: tcp
1183 groups:
1184 - support
1185
1186 - port: 22
1187 proto: tcp
1188 groups:
1189 - support
1190#+END_SRC
1191
1192The certs, key, config, binary, and systemd service should all be copied
1193to the same places on all of these nodes as on the lighthouse.
1194
1195*** X11vnc
1196/Alright./ The hardest part is finished. Now on to setting ~x11vnc~ up on
1197the nodes you'll be supporting.
1198
1199All you should need to do is install ~x11vnc~ using the package manager
1200your distro ships with, generate a 20 character password with ~pwgen -s
120120 1~, run the following command, paste the password, wait for ~x11vnc~ to
1202start up, make sure it's running correctly, press ~Ctrl~ + ~C~, then add the
1203command to the DE's startup applications!
1204
1205#+BEGIN_SRC bash
1206 x11vnc --loop -usepw -listen <nebula-ip> -display :0
1207#+END_SRC
1208
1209~--loop~ tells ~x11vnc~ to restart once you disconnect from the session.
1210~-usepw~ is pretty self-explanatory. ~-listen <nebula-ip>~ is important; it
1211tells ~x11vnc~ to only listen on the node's Nebula IP address. This
1212prevents randos in a coffee shop from seeing an open VNC port and trying
1213to brute-force the credentials. ~-display :0~ just defines which X11
1214server display to connect to.
1215
1216Some distributions like elementaryOS and those that use KDE and GNOME
1217will surface a dialogue for managing startup applications if you just
1218press the Windows (Super) key and type ~startup~. If that doesn't work,
1219you'll have to root around in the settings menus, consult the
1220distribution's documentation, or ask someone else that might know.
1221
1222After adding it to the startup application, log out and back in to make
1223sure it's running in the background.
1224
1225*** Remmina
1226Now that our network is functioning properly and the VNC server is set
1227up, we need something that connects to the VNC server over the fancy
1228mesh network. Enter [[https://remmina.org/][Remmina.]] This one goes on /your/ nodes.
1229
1230Remmina is a multi-protocol remote access tool available in pretty much
1231ever distribution's package archive as ~remmina~. Install it, launch it,
1232add a new connection profile in the top left, give the profile a
1233friendly name (I like to use the name of the person I'll be supporting),
1234assign it to a group, such as ~Family~ or ~Friends~, set the Protocol to
1235~Remmina VNC Plugin~, enter the node's Nebula IP address in the Server
1236field, then enter their username and the 20 character password you
1237generated earlier. I recommend setting the quality to Poor, but Nebula
1238is generally performant enough that any of the options are suitable. I
1239just don't want to have to disconnect and reconnect with a lower quality
1240if the other person happens to be on a slow network.
1241
1242Save and test the connection!
1243
1244If all goes well and you see the other device's desktop, you're done
1245with the VNC section! Now on to SSH.
1246
1247*** SSH
1248First off, make sure ~openssh-server~ is installed on the remote node;
1249~openssh-client~ would also be good to have, but from what I can tell,
1250it's not strictly necessary. You /will/ need ~openssh-client~ on /your/ node,
1251however. If you already have an SSH key, copy it over to
1252~~/.ssh/authorized_keys~ on the remote node. If you don't, generate one
1253with ~ssh-keygen -t ed25519~. This will create an Ed25519 SSH key pair.
1254Ed25519 keys are shorter and faster than RSA and more secure than ECDSA
1255or DSA. If that means nothing to you, don't worry about it. Just note
1256than this key might not interact well with older SSH servers; you'll
1257know if you need to stick with the default RSA. Otherwise, Ed25519 is
1258the better option. After key generation has finished, copy
1259~~/.ssh/id_ed25519.pub~ (note the ~.pub~ extension) from your node to
1260~~/.ssh/authorized_keys~ on the remote node. The file /without/ ~.pub~ is your
1261/private/ key. Like the Nebula CA certificate we generated earlier, this
1262is extremely sensitive and should never be shared with anyone else.
1263
1264Next is configuring SSH to only listen on Nebula's interface; as with
1265~x11vnc~, this prevents randos in a coffee shop from seeing an open SSH
1266port and trying to brute-force their way in. Set the ~ListenAddress~
1267option in ~/etc/ssh/sshd_config~ to the remote node's Nebula IP address.
1268If you want to take security a step further, search for
1269~PasswordAuthentication~ and set it to ~no~. This means your SSH key is
1270/required/ for gaining access via SSH. If you mess up Nebula's firewall
1271rules and accidentally give other Nebula devices access to this machine,
1272they still won't be able to get in unless they have your SSH key. I
1273/personally/ recommend disabling password authentication, but it's not
1274absolutely necessary. After making these changes, run ~systemctl restart
1275sshd~ to apply them.
1276
1277Now that the SSH server is listening on Nebula's interface, it will
1278actually fail to start when the machine (re)boots. The SSH server starts
1279faster than Nebula does, so it will look for the interface before Nebula
1280has even had a chance to connect. We need to make sure systemd waits for
1281Nebula to start up and connect before it tells SSH to start; run
1282~systemctl edit --full sshd~ and add the following line in the ~[Unit]~
1283section, above ~[Service]~.
1284
1285#+BEGIN_SRC text
1286 After=nebula.service
1287#+END_SRC
1288
1289Even now, there's still a bit of a hiccup. Systemd won't start SSH until
1290Nebula is up and running, which is good. Unfortunately, even after
1291Nebula has started, it still takes a minute to bring the interface up,
1292causing SSH to crash. To fix /this/, add the following line directly below
1293~[Service]~.
1294
1295#+BEGIN_SRC text
1296 ExecStartPre=/usr/bin/sleep 30
1297#+END_SRC
1298
1299If the ~sleep~ executable is stored in a different location, make sure you
1300use that path instead. You can check by running ~which sleep~.
1301
1302When the SSH /service/ starts up, it will now wait an additional 30
1303seconds before actually starting the SSH /daemon/. It's a bit of a hacky
1304solution but it works™. If you come up with something better, please
1305send it to me and I'll include it in the post! My contact information is
1306at the bottom of [[/][this site's home page.]]
1307
1308After you've made these changes, run ~systemctl daemon-reload~ to make
1309sure systemd picks up on the modified service file, then run ~systemctl
1310restart sshd~. You should be able to connect to the remote node from your
1311node using the following command.
1312
1313#+BEGIN_SRC bash
1314 ssh USER@<nebula-ip>
1315#+END_SRC
1316
1317If you want to make the command a little simpler so you don't have to
1318remember the IP every time, create ~~/.ssh/config~ on your node and add
1319these lines to it.
1320
1321#+BEGIN_SRC text
1322 Host USER
1323 Hostname <nebula-ip>
1324 User USER
1325#+END_SRC
1326
1327Now you can just run ~ssh USER~ to get in. If you duplicate the above
1328block for all of the remote nodes you need to support, you'll only have
1329to remember the person's username to SSH into their machine.
1330
1331*** Going further with Nebula
1332This section explains why we used FQDNs in the certs and why the DNS
1333resolver is enabled on the lighthouse.
1334
1335Nebula ships with a built-in resolver meant specifically for mapping
1336Nebula node hostnames to their Nebula IP addresses. Running a public DNS
1337resolver is very much discouraged because it can be abused in terrible
1338ways. However, the Nebula resolver mitigates this risk because it /only/
1339answers queries for Nebula nodes. It doesn't forward requests to any
1340other servers nor does it attempt to resolve any domain other than what
1341was defined in its certificate. If you use the example I gave above,
1342that would be ~nebula.example.com~; the lighthouse will attempt to resolve
1343any subdomain of ~nebula.example.com~ but it will just ignore ~example.com~,
1344~nebula.duckduckgo.com~, ~live.secluded.site~, etc.
1345
1346Taking advantage of this resolver requires setting it as your secondary
1347resolver on any device you want to be able to resolve hostnames from.
1348If you were to add the lighthouse's IP address as your secondary
1349resolver on your PC, you could enter ~host.user.nebula.example.com~ in
1350Remmina's server settings /instead of/ ~192.168.1.2~.
1351
1352But how you do so is beyond the scope of this post!
1353
1354If you're up for some /more/ shenanigans later on down the line, you could
1355set up a Pi-Hole instance backed by Unbound and configure Nebula as
1356Unbound's secondary resolver. With this setup, you'd get DNS-level ad
1357blocking /and/ the ability to resolve Nebula hostname. Pi-Hole would query
1358Unbound for ~host.user.nebula.example.com~, Unbound would receive no
1359answer from the root servers because the domain doesn't exist outside of
1360your VPN, Unbound would fall back to Nebula, Nebula would give it an
1361answer, Unbound would cache the answer, tell Pi-Hole, Pi-Hole would
1362cache the answer, tell your device, then your device would cache the
1363answer, and you can now resolve any Nebula host!
1364
1365Exactly how you do /that/ is */definitely/* beyond the scope of this post :P
1366
1367If you set any of this up, I would be interested to hear how it goes! As
1368stated earlier, my contact information is at the bottom of the site's
1369home page :)
1370
1371** TODO Stop using Gmail!
1372* Education :@Education:
1373** TODO Homeschooling
1374* Music :@Music:
1375* Pipe Smoking :@Pipe__Smoking:
1376* Dungeons & Dragons :@Dungeons__and__Dragons:
1377
1378* Footnotes
1379
1380[fn:19] [[https://github.com/audacity/audacity/pull/835#issuecomment-834451187][Link to the comment]] and [[/assets/pngs/audacity-pr/privatelibcurl.png][link to the screenshot]]
1381
1382[fn:18] [[https://github.com/audacity/audacity/pull/835#issuecomment-834010117][Link to the comment]] and [[/assets/pngs/audacity-pr/vendorproscons.png][link to the screenshot]]
1383
1384[fn:17] [[https://github.com/audacity/audacity/pull/835#issuecomment-836069326][Link to the comment]] and [[/assets/pngs/audacity-pr/tantacrulrole.png][link to the screenshot]]
1385
1386[fn:16] [[https://github.com/audacity/audacity/pull/835#discussion_r627762185][Link to the comment]] and [[/assets/pngs/audacity-pr/missingprivacypolicy.png][link to the screenshot]]
1387
1388[fn:15] [[https://github.com/audacity/audacity/pull/835#issuecomment-834286641][Link to the comment]] and [[/assets/pngs/audacity-pr/darkpattern.png][link to the screenshot]]
1389
1390[fn:14] [[https://github.com/audacity/audacity/pull/835#discussion_r627764300][Link to the comment]] and the screenshot is the same as previous
1391
1392[fn:13] [[https://github.com/audacity/audacity/pull/835#discussion_r627756976][Link to the comment]] and [[/assets/pngs/audacity-pr/vaguedialogue.png][link to the screenshot]]
1393
1394[fn:12] [[https://github.com/audacity/audacity/pull/835#discussion_r628124998][Link to the review]] and [[/assets/pngs/audacity-pr/defaultconsentbutton.png][link to the screenshot]]
1395
1396[fn:11] [[https://github.com/audacity/audacity/pull/835#discussion_r628008821][Link to the review]] and [[/assets/pngs/audacity-pr/externaldependency.png][link to the screenshot]]
1397
1398[fn:10] [[https://github.com/audacity/audacity/pull/835#discussion_r628005925][Link to the review]] and [[/assets/pngs/audacity-pr/vendorcurl.png][link to the screenshot]]
1399
1400[fn:9] Note that because I am not a C programmer, these reviews might
1401not be entirely accurate and I wouldn't be able to catch the reviewer's
1402error. I am relying on other community members to catch issues and
1403comment on them; none of the reviews I link to have such comments so I'm
1404assuming they are correct.
1405
1406[fn:8] [[https://github.com/audacity/audacity/pull/835#discussion_r627993755][Link to the review]] and [[/assets/pngs/audacity-pr/writeanalyticsid.png][link to the screenshot]]
1407
1408[fn:7] /Optical Character Recognition/: the program looks at your
1409handwriting and tries to turn it into text.
1410
1411[fn:6] In this situation, latency refers to how long it takes for "ink"
1412to show up on the "page" after writing something.
1413
1414[fn:5] It's not really a fountain pen even though that's what they call
1415it; it's just pressure-sensitive.
1416
1417[fn:4] There does seem to be a group of people interested in just such a
1418thing: /[[https://alexsoto.dev/challenges-building-an-open-source-eink-laptop.html][Challenges Building an Open-Source E Ink Laptop]]/
1419
1420[fn:3]Taken from their [[https://support.remarkable.com/hc/en-us/articles/360006699537-About-reMarkable-2-][support page about the reMarkable 2;]] search the
1421page for /operating system/ and it should show up.
1422
1423[fn:2]I dislike Apple's operating system, their hardware, business
1424model, privacy practises, and much of what they stand for as a company.
1425Don't @ me.
1426
1427[fn:1]E-R/W is a play on media commonly being labelled as R/W when you
1428can read from it and write to it.