1#+HUGO_BASE_DIR: ./
2#+HUGO_SECTION: posts
3#+HUGO_AUTO_SET_LASTMOD: t
4
5* Meta :@Meta:
6* Technology :@Technology:
7** TODO Audacity and the telemetry pull request :Open__source__culture:Audio__editing:Music:Drama:
8:PROPERTIES:
9:EXPORT_FILE_NAME: audacity-and-the-telemetry-pull-request
10:EXPORT_HUGO_CUSTOM_FRONT_MATTER: :toc true
11:END:
12
13Five days ago at the time of writing, [[https://github.com/crsib][Dmitry Vedenko]] opened a Pull
14Request (PR) in [[https://github.com/audacity/audacity/pull/835][Audacity's GitHub repository]] entitled [[https://github.com/audacity/audacity/pull/835][/Basic telemetry
15for the Audacity/.]] About two days later, all hell broke loose. That PR
16now has over 3.3 thousand downvotes and more than one thousand comments
17from nearly 400 individuals. I started reading the posts shortly after
18they began and kept up with them over the following days, reading every
19single new post. I recognise that few people are going to feel like
20wading through over 1k comments so this is my attempt to provide a
21summary of the PR itself using the community's code reviews along with a
22summary of the various opinions conveyed in the comments.
23
24When I reference comments, I'll provide a footnote that includes a link
25to the comment and a link to a screenshot just in case it's removed or
26edited in the future.
27
28*** Audacity's acquisition
29
30I haven't been able to find /much/ information in this area so forgive me
31if I'm scant on details.
32
33On 30 April, a company called [[https://mu.se/][Muse Group]] acquired [[https://www.audacityteam.org/][Audacity.]] According to
34[[https://mu.se][their website]], Muse is the parent company behind many musical
35applications and tools. It was founded by Eugeny Naidenov just days
36before it acquired Audacity. Before all of this, Eugeny Naidenov founded
37[[https://www.ultimate-guitar.com/][Ultimate Guitar]] (UG) in 1998. The service grew rather quickly and now
38has over 300 million users. UG acquired [[https://deanzelinsky.com/][Dean Zelinsky Guitars]] in 2012,
39[[http://agilepartners.com/][Agile Partners]] in 2013, [[https://musescore.org/][MuseScore]] in 2017, and [[http://trycrescendo.com/][Crescendo]] in 2018. Muse
40Group was established in 2021 and it seems as if all of the services UG
41acquired were (or will be) transferred to Muse Group, as well as UG
42itself. Immediately following its establishment, Muse not only acquired
43Audacity but also [[https://www.staffpad.net/][StaffPad.]]
44
45I say 30 April because that's when Muse published their [[https://mu.se/newsroom/tpost/6dhedma301-muse-group-acquires-audacity-expanding-c][press release]]
46and when Martin Keary (Tantacrul) published a video entitled [[https://www.youtube.com/watch?v=RMWNvwLiXIQ][/I’m now in
47charge of Audacity. Seriously./]] According to his comment,[fn:17] Martin
48will help with proposing Audacity's roadmap and many of its future
49features as well as working with the community. This has been his role
50with MuseScore since he joined that project and he will be continuing it
51here.
52
53~-----BEGIN PERSONAL OPINION-----~
54
55Looking at [[https://www.martinkeary.com/][his website,]] I also suspect he will play a large role in
56redesigning Audacity's interface. Considering that he was instrumental
57in designing [[https://www.martinkeary.com/#/ubuntu-touch-os/][the best mobile interface I've ever had the absolute
58pleasure of experiencing,]] I have high hopes that this is the case.
59
60~------END PERSONAL OPINION------~
61
62*** Telemetry implementation
63**** Implementation Basics
64
65A few days after the acquisition, a PR was opened that adds /Basic
66telemetry for the Audacity/. This implementation collects "application
67opened" events and sends those to Yandex to estimate the number of
68Audacity users. It also collects session start and end events, errors
69for debugging, file used for import and export, OS and Audacity
70versions, and the use of effects, generators, and analysis tools so they
71can prioritise future improvements. Sending this data would be optional
72and the user would be presented with a dialogue the first time they
73launch the application after installation or after they update to the
74including release. This description was mostly copied directly from [[https://github.com/audacity/audacity/pull/835#issue-629891447][the
75PR description itself.]]
76
77**** Frontend Implementation
78This is fairly straightforward and a pretty standard UI for prompting
79users to consent to analytics and crash logging. This section is
80included because the community has strong opinions regarding the
81language used and its design, but that will be discussed later. The
82screenshot below is copied directly from the PR.
83
84[[/assets/pngs/audacity-pr/consentdialogue.png]]
85
86**** Backend Implementation
87Many of the code reviews include the reviewer's personal opinion so I
88will summarise the comment, provide the code block in question, and link
89directly to the comment in a footnote.[fn:9]
90
91#+BEGIN_SRC c
92 if (!inputFile.Write (wxString::FromUTF8 (ClientID + "\n")))
93 return false;
94#+END_SRC
95[[https://github.com/crsib/audacity/blob/c9264d2478fe2af82aeb6e2a0295b00b3a27ce53/libraries/lib-telemetry/TelemetryManager.cpp#L199-L200][Lines 199-200 of TelemetryManager.cpp]] save the user's unique client ID
96to a file.[fn:8] This allows the analytics tool (in this case, Google
97Analytics) to aggregate data produced by a single user.
98
99#+BEGIN_SRC c
100 def_vars()
101
102 set( CURL_DIR "${_INTDIR}/libcurl" )
103 set( CURL_TAG "curl-7_76_0")
104#+END_SRC
105[[https://github.com/crsib/audacity/blob/c9264d2478fe2af82aeb6e2a0295b00b3a27ce53/cmake-proxies/libcurl/CMakeLists.txt#L3-L6][Lines 3-6 of CMakeLists.txt]] "vendor in" libcurl.[fn:10] This is when an
106application directly includes sources for a utility rather than making
107use utilities provided by the system itself.
108
109#+BEGIN_SRC c
110 ExternalProject_Add(curl
111 PREFIX "${CURL_DIR}"
112 INSTALL_DIR "${CURL_DIR}"
113 GIT_REPOSITORY https://github.com/curl/curl
114 GIT_TAG ${CURL_TAG}
115 GIT_SHALLOW Yes
116 CMAKE_CACHE_ARGS ${CURL_CMAKE_ARGS}
117 )
118#+END_SRC
119[[https://github.com/crsib/audacity/blob/c9264d2478fe2af82aeb6e2a0295b00b3a27ce53/cmake-proxies/libcurl/CMakeLists.txt#L29-L36][Lines 29-36 of CMakeLists.txt]] add curl as a remote dependency.[fn:11]
120This means that the machine building Audacity from its source code has
121to download curl during that build.
122
123#+BEGIN_SRC c
124 S.Id (wxID_NO).AddButton (rejectButtonTitle);
125 S.Id (wxID_YES).AddButton (acceptButtonTitle)->SetDefault ();
126#+END_SRC
127[[https://github.com/crsib/audacity/blob/c9264d2478fe2af82aeb6e2a0295b00b3a27ce53/src/telemetry/TelemetryDialog.cpp#L93-L94][Lines 93-94 of TelemetryDialog.cpp]] add buttons to the dialogue asking
128the user whether they consent to data collection.[fn:12] ~SetDefault~
129focuses the button indicating that the user does consent. This means
130that if the user doesn't really look at the dialogue and presses
131Spacebar or Enter, or if they do so accidentally by simply bumping the
132key, they unintentionally consent to data collection. If the user
133desires, this can later be changed in the settings menu. However, if
134they weren't aware what they were consenting to /or that they did
135consent/, they won't know to go back and opt out.
136
137There are other problems with the code that include [[https://github.com/audacity/audacity/pull/835#discussion_r628816050][simple mistakes,]]
138[[https://github.https://github.com/audacity/audacity/pull/835#discussion_r628774985][styling that's inconsistent with the rest of the project,]] [[https://github.com/audacity/audacity/pull/835#discussion_r628500849][unhandled
139return values resulting in skewed data,]] [[https://github.com/audacity/audacity/pull/835#discussion_r628792423][use of inappropriate functions,]]
140and [[https://github.com/audacity/audacity/pull/835#discussion_r628818054][spelling errors in the comments.]] I believe these are less important
141than those above so they won't be discussed.
142
143*** Community opinions
144There were many strong opinions regarding both the frontend and backend
145implementations of this PR, from the wording of the dialogue and
146highlighting the consent button to devices running something other than
147Windows and macOS not being able to send telemetry and thus skewing the
148data that /was/ collected.
149
150**** Opinions on the frontend
151
152Really, the only frontend here is the consent dialogue. However, there
153are /many/ comments about it, the most common of which is probably that
154the wording is not only too vague[fn:13] but also inaccurate.[fn:14] The
155assertion that Google Analytics are not anonymous and any data sent can
156be trivially de-anonymised (or de-pseudonymised) is repeated many times
157over. Below are a few links to comments stating such. I searched for the
158term /"anonymous"/, copied relevant links, and stopped when my scrollbar
159reached halfway down the page.
160
161- [[https://github.com/audacity/audacity/pull/835#discussion_r628156527][r628156527]]
162- [[https://github.com/audacity/audacity/pull/835#issuecomment-833969780][833969780]]
163- [[https://github.com/audacity/audacity/pull/835#issuecomment-833969933][833969933]]
164- [[https://github.com/audacity/audacity/pull/835#discussion_r627995927][r627995927]]
165- [[https://github.com/audacity/audacity/pull/835#issuecomment-834358022][834358022]]
166- [[https://github.com/audacity/audacity/pull/835#issuecomment-834377549][834377549]]
167- [[https://github.com/audacity/audacity/pull/835#issuecomment-834382007][834382007]]
168- [[https://github.com/audacity/audacity/pull/835#issuecomment-834385463][834385463]]
169- [[https://github.com/audacity/audacity/pull/835#issuecomment-834405825][834405825]]
170- [[https://github.com/audacity/audacity/pull/835#issuecomment-834531779][834531779]]
171- [[https://github.com/audacity/audacity/pull/835#issuecomment-834546874][834546874]]
172- [[https://github.com/audacity/audacity/pull/835#issuecomment-834638000][834638000]]
173
174The next most pervasive comment is regarding the consent buttons at the
175bottom of the dialogue where users opt in or out.[fn:15] Many individuals call
176this design a /dark pattern/. Harry Brignull, a UX specialist focusing on
177deceptive interface practises, describes dark patterns as [[https://www.darkpatterns.org/][/tricks used
178in websites and apps that make you do things that you didn't mean to/.]]
179The dark pattern in this situation is the opt-in button being
180highlighted. Many community members assert that users will see the big
181blue button and click it without actually reading the dialogue's
182contents. They just want to record their audio and this window is a
183distraction that prevents them from doing so; it needs to get out of the
184way and the quickest way to dismiss it is clicking that blue button.
185Below is a list of some comments criticising this design.
186
187 - [[https://github.com/audacity/audacity/pull/835#issuecomment-834286641][834286641]]
188 - [[https://github.com/audacity/audacity/pull/835#issuecomment-834358022][834358022]]
189 - [[https://github.com/audacity/audacity/pull/835#issuecomment-834399813][834399813]]
190 - [[https://github.com/audacity/audacity/pull/835#issuecomment-834479968][834479968]]
191 - [[https://github.com/audacity/audacity/pull/835#issuecomment-835250737][835250737]]
192 - [[https://github.com/audacity/audacity/pull/835#issuecomment-835253882][835253882]]
193 - [[https://github.com/audacity/audacity/pull/835#issuecomment-835291066][835291066]]
194 - [[https://github.com/audacity/audacity/pull/835#issuecomment-835445481][835445481]]
195
196Another issue that was brought up by a couple of individuals was the
197lack of a privacy policy.[fn:16] The consent dialogue links to one, but, at the
198time of writing, one does not exist at [[https://www.audacityteam.org/contact/privacy-policy/][the provided URL.]] I have [[https://web.archive.org/web/20210510012924/https://www.audacityteam.org/contact/privacy-policy/][archived
199the state of the page]] in case that changes in the future.
200
201**** Opinions on the backend
202
203#+BEGIN_SRC c
204 if (!inputFile.Write (wxString::FromUTF8 (ClientID + "\n")))
205 return false;
206#+END_SRC
207
208The issue many individuals take with this snippet is saving the
209~ClientID~. Say an individual has an odd file that causes Audacity to
210crash any time they try to open it. Say they attempt to open it a
211hundred times. Without giving the client a unique ID, it could look like
212there are 100 people having an issue opening a file instead of just the
213one. However, by virtue of each installation having an entirely unique
214ID, this telemetry /is not anonymous/. Anonymity would be sending
215statistics in such a way that connecting those failed attempts to a
216single user would be impossible. At best, this implementation is
217/pseudonymous/ because the client is given a random ID, you don't have to
218sign in with an account or something.
219
220#+BEGIN_SRC c
221 def_vars()
222
223 set( CURL_DIR "${_INTDIR}/libcurl" )
224 set( CURL_TAG "curl-7_76_0")
225#+END_SRC
226
227Timothe Litt's comment gives a good description of why "vendoring in"
228libcurl is a bad idea[fn:19] and Tyler True's comment gives a good
229overview of the pros and cons of doing so.[fn:18] Many people take issue
230with this /specifically/ because it's libcurl. Security flaws in it are
231/very/ common and Audacity's copy would need to be /manually/ kept up to
232date with every upstream release to ensure none of its vulnerabilities
233can be leveraged to compromise users. If the Audacity team was going to
234stay on top of all of the security fixes, they would need to release a
235new version every week or so.
236
237#+BEGIN_SRC c
238 ExternalProject_Add(curl
239 PREFIX "${CURL_DIR}"
240 INSTALL_DIR "${CURL_DIR}"
241 GIT_REPOSITORY https://github.com/curl/curl
242 GIT_TAG ${CURL_TAG}
243 GIT_SHALLOW Yes
244 CMAKE_CACHE_ARGS ${CURL_CMAKE_ARGS}
245 )
246#+END_SRC
247The problem with downloading curl at build-time is that it's simply
248disallowed for many Linux- and BSD-based operation systems. When a
249distribution builds an application from source, its build dependencies
250are often downloaded ahead of time and, as a security measure, the build
251machine is cut off from the internet to prevent any interference.
252Because this is disallowed, the build will fail and the application
253won't be available on those operation systems.
254
255Note, however, that these build machines would have the option to
256disable telemetry at build-time. This means the machine wouldn't attempt
257to download curl from GitHub and the build would succeed but, again,
258telemetry would be disabled for anyone not on Windows or macOS. This
259defeats the whole purpose of adding telemetry in the first place.
260
261#+BEGIN_SRC c
262 S.Id (wxID_NO).AddButton (rejectButtonTitle);
263 S.Id (wxID_YES).AddButton (acceptButtonTitle)->SetDefault ();
264#+END_SRC
265
266There was a lot of feedback about the decision to highlight the consent
267button but that was mentioned up in the frontend section; I won't rehash
268it here.
269
270**** Broader and particularly well-structured comments
271These are simply some comments I feel deserve particular attention.
272
273From SndChaser...
274- [[https://github.com/audacity/audacity/pull/835#issuecomment-834037351][834037351]]
275-
276
277*** The Audacity team's response
278
279
280-----------------
281
282*** The privacy policy modification
283
284https://github.com/audacity/audacity/issues/1213#issuecomment-875274890
285
286** TODO Catchy title about Supernote being "the new paper" :Supernote:Writing:Productivity:Organisation:
287:PROPERTIES:
288:EXPORT_FILE_NAME: something-about-supernote
289:EXPORT_HUGO_CUSTOM_FRONT_MATTER: :toc true
290:END:
291
292I like writing things down. I like the feel of the pen (preferably a
293fountain pen) gliding smoothly over the paper, that nice solid feeling
294of the tip against the table, seeing the ink dry as it flows from the
295nib, accidentally swiping my hand through it before it's finished and
296smearing a bit of ink across the page, then cursing under my breath as I
297dab it up with a handkerchief or a napkin or something else nearby. I
298also love that writing things by hand [[https://journals.sagepub.com/doi/abs/10.1177/0956797614524581][has an impact on memory and
299improves retention.]]
300
301*** The problem
302Unfortunately, I don't love keeping up with that paper. Across many
303different classes, even with dedicated folders for each one, something
304important inevitably gets lost. Notebooks are also bulky and can take up
305a lot of space. I tried [[https://bulletjournal.com/][bullet journalling]] for about a month earlier
306this year and, while the process was enjoyable, the maintenance was not.
307My brain moves faster than my pen (even though I have terrible
308handwriting) and I inevitably forget letters or even whole words. This
309is a problem while writing in pen because white-out looks ugly and I
310dislike wasting whole pages because of a couple mistakes.
311
312The obvious solution here is to get an iPad with an Apple Pen, right?
313Right??
314
315Wrong because Apple bad.[fn:2]
316
317*** The solution
318Enter the world of ... what are they even called? E-ink notebooks? Paper
319tablets? E-R/W?[fn:1] Do they even have a "device category" yet? I don't
320know, but they solve my problem in a wonderful way.
321
322As the names suggest, these are devices that can /usually/ open and read
323e-books (EPUBs, PDFs, etc.), annotate them, and create standalone pages
324of notes as if they were full notebooks. The most well-known of these
325devices is likely the [[https://remarkable.com/][reMarkable.]] They had a [[https://venturebeat.com/2019/10/08/remarkable-raises-15-million-to-bring-its-e-paper-tablets-to-more-scribblers/][hugely successful
326crowdfunding campaign]] and produced the reMarkable 1, followed by [[https://blog.remarkable.com/remarkable-2-the-next-generation-paper-tablet-91b47d0080cb][the
327reMarkable 2 in 2020.]] There are a few devices like these by now but
328we'll look at the reMarkable first.
329
330*** The reMarkable
331This device boasts all of the features I was looking for. It renders
332digital content, from books and manuals to comics and manga, allows you
333to mark those documents up as you would if they were physical media,
334create full notebooks of hand written text, organise them, search, and,
335if your handwriting is legible enough (mine certainly is not), perform
336OCR on your notes and email a transcription to yourself. It even runs
337Linux and the developers have opened SSH up so you can remote in and
338tinker with it as much as you like. Because of this, there's a pretty
339awesome [[https://github.com/reHackable/awesome-reMarkable][community of people creating third-party tools and integrations]]
340that add even further functionality. My favourite is probably [[https://github.com/bordaigorl/rmview][rMview,]] a
341really fast VNC client for the reMarkable that allows you to view your
342device's screen on any computer.
343
344After watching all of [[https://www.youtube.com/c/MyDeepGuide][MyDeepGuide's]] [[https://www.youtube.com/playlist?list=PLsSI9-gaSSmiXwb7Vjk5Vb-nB41UTnrXd][extensive playlist on the
345reMarkable,]] however, I decided to go with a different product.
346
347*** Enter the Supernote A5X
348The [[https://www.supernote.com/#/product?type=SN-A5-X][Supernote A5X]] has all of the basic features the reMarkable has:
349reading documents, writing notes, and organising your content. Its
350implementation, on the other hand, seems to be much more polished. It
351also lacks some features from the reMarkable while adding others.
352
353*** Operating System
354While the reMarkable runs Codex,[fn:3] a /"custom Linux-based OS
355optimised for low-latency e-paper"/, the Supernote just runs Android.
356There are both benefits and detriments to this; on one hand, they're
357running all of Android, bloated that it is, on a very lightweight
358tablet. On the other, they don't have to develop and maintain a custom
359operating system. This allows them to focus on other aspects that are
360arguably more important so I don't actually mind that it runs Android.
361
362The only place that Android stands out is in system operations; file
363transfer uses MTP and, when you swipe down from the top of the device, a
364small bar appears similar to what was in early Android. This lets you
365change WiFi networks, sync with the Supernote Cloud, take a screenshot,
366search, and access the system settings. Nothing else about the device
367really screams Android to me.
368
369*** Community
370I don't usually browse Reddit but [[https://old.reddit.com/r/Supernote/][the Supernote community]] there is
371fascinating. I haven't looked around enough to know exactly what his
372relationship is with the company, but one of the members, [[https://old.reddit.com/user/hex2asc][u/hex2asc,]]
373seems to represent Supernote in something of an official capacity. He's
374incredibly active and usually responds to posts and questions within a
375day or two.
376
377Before I purchased a Supernote, [[https://old.reddit.com/r/Supernote/comments/lhffyd/sync_targets_open_document_formats_and_crossnote/][I wrote a post]] asking about a couple of
378things that concerned me: sync targets, open document formats, and
379cross-note links. I don't really plan to write full documents on the
380device but having the option to do so would still be nice. The other
381features are absolutely killer for me as I would like to maintain a
382Zettelkasten (I wrote about [[/vim-as-a-markdown-editor/][using Vim to do so]] last year but didn't end
383up sticking with it) and manage document synchronisation with my own
384Nextcloud server. The community was quick to respond and confirm that
385Zettelkasten functionality would be implemented soon™. u/hex2asc
386responded /the day after/ and said that WebDAV would be supported but not
387earlier than May (September update: it's still not supported), ODF would
388likely not be supported, and cross-note links were definitely a
389possibility. Another community member has been avidly following the
390subreddit and even put together an [[https://app-rm.roadmunk.com/publish/03e6dca3d769e2b7015f7f48a649cb3f75f44d9e][unofficial roadmap.]]
391
392*** Interfaces
393
394**** Home & Organisation
395
396***** TODO Record very short videos
397
398**** Settings
399
400***** TODO Record very short videos
401
402**** Writing & Annotating
403
404The following images are screenshots of the full page above with the
405possible UI variations while reading a book. This first one is default,
406with the editing bar at the top. It is exactly the same as what's
407displayed on the blank pages for hand writing full notes. From left to
408right is the Table of Contents toggle, the pen tools (fineliner,
409"fountain" pen,[fn:5] and highlighter), the erasers, lasso select tool,
410undo/redo, context menu, palm rejection toggle, previous page, goto
411page, next page, and exit.
412
413[[/assets/pngs/supernote-reader-default.png]]
414
415You can hold your finger on that bar and drag it down to detach it from
416the top. The default width exposes all the tools without whitespace. You
417can move it around the screen by dragging the circle with a straight
418line through the middle on the far left.
419
420[[/assets/pngs/supernote-reader-medium.png]]
421
422If you tap that circle, the width shrinks and everything except the
423pens, erasers, and undo/redo buttons are hidden. It can be dragged the
424same was as in the previous image and tapping that circle will expand
425the bar again.
426
427[[/assets/pngs/supernote-reader-small.png]]
428
429The last mode is with the bar completely hidden. You achieve this just
430by dragging it to the right edge of the screen. Once hidden, you can
431swipe right to left from the edge and it will be revealed flush with the
432right edge.
433
434[[/assets/pngs/supernote-reader-minimal.png]]
435
436*** Experience
437**** Reading content
438I love e-ink. I think it looks beautiful and would love to have an e-ink
439monitor.[fn:4] That said, the Supernote has an especially nice display
440with 226 PPI (pixels per inch). The image below was taken with my
441phone's camera so it's not very good. However, if you zoom in a bit, you
442can see that the curved edges of some letters are /slightly/ pixellated.
443Viewing with my naked eye at a comfortable distance, it does look better
444/to me/ than some of my print books, however.
445
446[[/assets/pngs/supernote-resolution.png]]
447
448/At the moment,/ I am pretty disappointed with Table of Contents detection
449for ePUBs. A great many of my books seem to use a legacy ToC format that
450the Supernote sees and tries/fails to read before attempting to read the
451more up-to-date one. This is easily remedied by editing the ePUB in
452[[https://calibre-ebook.com/][Calibre,]] going to Tools → Upgrade Book Internals → Remove the legacy
453Table of Contents in NCX format. You might need to make a small change
454to one of the HTML files and revert it before the save button is
455enabled. After that, just copy it back over to the Supernote and
456everything should work properly.
457
458**** Writing notes
459I write notes as often if not /more/ often than I read and annotate books.
460It's the main reason I purchased the device and I love the experience.
461The Supernote doesn't /really/ feel like paper despite what their
462marketing materials claim, though it doesn't feel /bad/ either. It's hard
463to describe but I would say it's something like writing with a
464rollerball pen on high-quality paper with a marble counter underneath:
465incredibly smooth with but a little bit of texture so it doesn't feel
466like writing on a glass display.
467
468While writing latency[fn:6] is noticeable, I really don't have a huge
469issue with it. I write very quickly but find that the slight latency
470actually makes writing /more/ enjoyable. It sounds weird and I'm not sure
471why, but I /really/ like writing on the Supernote; it's wonderfully
472smooth, pressure-sensitive, the latency makes things interesting, and
473[[https://supernote.com/#/part?id=SP-04][the Heart of Metal pen]] feels good in my hand.
474
475**** Surfacing Content
476While organisation is done using a regular filesystem hierarchy, the
477Supernote does have other ways to search for and surface your notes. As
478you're writing, you can use the lasso select tool and encircle a word. A
479little dialogue pops up and gives you a few buttons for things you can
480do with that selection: copy, move to another page, cut, add it to the
481Table of Contents, or mark it as a key word. If you select the key word
482icon, the Supernote does some incredible OCR[fn:7] on it and displays a
483dialogue where you can add it to the note file as a tag. This dialogue
484allows you to edit the word before adding it just in case the OCR was
485wonky. Even with my terrible handwriting, I've found that it works very
486well and I rarely have to make edits.
487
488*** TODO Ping Isi and Volpeon when finished
489** TODO Migrating repositories between git hosts
490** TODO A perfect email setup (for me) :Email:Workflow:
491:PROPERTIES:
492:EXPORT_FILE_NAME: a-perfect-email-setup-for-me
493:EXPORT_HUGO_CUSTOM_FRONT_MATTER: :toc true
494:END:
495
496I've never been satisfied with any of the email clients most people use.
497I've tried [[https://www.thunderbird.net/en-GB/][Thunderbird,]] [[https://wiki.gnome.org/Apps/Evolution][Evolution,]] [[https://getmailspring.com/][Mailspring,]] [[https://support.apple.com/mail][Mail.app,]] [[https://roundcube.net/][Roundcube,]]
498[[https://sogo.nu/][SOGo,]] [[https://wiki.gnome.org/Apps/Geary][Geary,]] and /many/ more. /None/ of them handle multiple accounts
499particularly well because all of the emails associated with that account
500are bound within it. Sure, you can make a new folder somewhere called
501~TODO~ and move all of your actionable emails to that folder but, when you
502go to move actionable emails from /another/ account into that folder,
503you'll likely find that the client simply doesn't let you. If it does,
504when you reply, it will likely be sent from the wrong account. This is a
505limitation of the IMAP protocol; everything is /managed/ locally but
506changes are pushed to the remote server and mixing things the way I want
507leads to broken setups.
508
509Before I go any further, these are a few characteristics of my ideal
510email tool.
511
512- Support for multiple accounts (obviously)
513- /Native desktop application/ (*not* [[https://github.com/electron/electron][Electron]])
514- Has stellar keyboard shortcuts
515- Doesn't require internet connectivity (other than downloading and
516 sending of course)
517- Organisation can be done with tags
518
519*** Why tags?
520Because they're better. Hierarchies are useful for prose and code but
521not for files, emails, notes, or anything where an item may fit within
522multiple categories. Imagine you get an email from your Computer Science
523professor that includes test dates, homework, and information about
524another assignment. In that same email, he asks every student to reply
525with something they learned from the previous class as a form of
526attendance. In a hierarchy, the best place for this might just be a ~TODO~
527folder /even though/ it would also fit under ~School~, ~CS~, ~Dates~, ~To read~,
528and ~Homework~. Maybe you have a few minutes and want to clear out some
529emails that don't require any interaction. In a tag-based workflow, this
530would be a good time to open ~To read~, get that email out of the way, and
531remove the ~To read~ tag. It would still show up under the other tags so
532you can find it later and take the time to fully answer the professor's
533question, add those dates to your calendar, and add the homework
534assignments to your ~TODO~ list. Hierarchies can be quite cumbersome to
535work with, especially when one folder ends up getting all the data. Tags
536ensure that you only see what you want when you want it. Tags are more
537efficient and they will remain my organisation system of choice.
538
539*** The tools
540In short, the tools we will be using are...
541+ [[https://isync.sourceforge.io/mbsync.html][~mbsync~]] to download our emails
542+ [[https://notmuchmail.org/][~notmuch~,]] the primary way emails will be organised
543+ [[https://afew.readthedocs.io/en/latest/][~afew~]] to apply initial ~notmuch~ tags based on subject, sender, recipient, etc.
544+ [[https://neomutt.org/][NeoMutt]] to interact with those emails, reply, compose, add/remove
545 tags, etc.
546+ [[https://marlam.de/msmtp/][~msmtp~]] for relaying our replies and compositions to our mail provider
547
548Yes, it's a lot. Yes, it's time-consuming to set up. Yes, it's worth it
549(in my opinion).
550
551*** ~mbsync~
552As I said above, IMAP is limiting; we need to use some other method of
553downloading our emails. There's an awesome piece of software called
554[[https://isync.sourceforge.io/mbsync.html][mbsync]] which is built for exactly this purpose. Its configuration
555can be rather daunting if you have as many accounts as I do (19) but
556it's not /terrible/.
557
558The following sections are named *Near*, *Far*, and *Sync*. Near and Far are
559terms mbsync uses to profile /how/ your emails are stored, /where/ they're
560stored, and how to interact with them. In this guide, Far will our mail
561provider's IMAP server and Near will be our local Maildir.
562
563**** Far
564#+BEGIN_SRC text
565IMAPAccount amo_ema
566Host imap.nixnet.email
567CertificateFile /etc/ssl/certs/ca-certificates.crt
568SSLType STARTTLS
569User amolith@nixnet.email
570PassCmd "secret-tool lookup Title amolith@nixnet.email"
571
572IMAPStore amo_ema-remote
573Account amo_ema
574#+END_SRC
575
576**** Near
577#+BEGIN_SRC text
578MaildirStore amo_ema-local
579SubFolders Verbatim
580Path ~/new-mail/amo_ema/
581Inbox ~/new-mail/amo_ema/INBOX/
582#+END_SRC
583
584In the first block, ~localrepository~ and ~remoterepository~ tell OfflineIMAP where
585to look for your emails. ~use_exa-local~ is an arbitrary naming scheme I use to
586differentiate between the various local and remote accounts. It can easily be
587swapped with something else.
588
589**** Sync
590#+BEGIN_SRC text
591Channel amo_ema
592Far :amo_ema-remote:
593Near :amo_ema-local:
594SyncState *
595Patterns *
596Create Both
597#+END_SRC
598
599The repository sections describe how the emails are stored or retrieved.
600In the ~local~ block, you'll notice that the type is ~Maildir~. In this
601format, each email is given a unique filename and stored in a hierarchy
602of folders within your account. This is often how your emails are stored
603on your provider's mail server.
604
605~pythonfile~ is used here to authenticate with the remote server. This can
606be complicated and depends /entirely/ on how you manage your passwords. I
607use [[https://keepassxc.org/][KeePassXC]] and love it. When I set OfflineIMAP up, however, it didn't
608have ~libsecret~ compatibility. This would have made setup significantly
609easier but, as it already just works™, I don't really see a reason to
610change it.
611
612This new feature allows ~libresecret~-based applications to query
613KeePassXC for your passwords or store them there on your behalf. CLI/TUI
614applications that need a secure mechanism for background authentication
615can use ~secret-tool lookup Title "TITLE_OF_PASSWORD"~ as the password
616command. See [[https://github.com/keepassxreboot/keepassxc/pull/2726][the pull request]] for more details. Because this wasn't a
617feature when I first set it up, I put my passwords in plaintext files
618and encrypted them with the GPG key stored on my YubiKey. As long as my
619key is plugged in, OfflineIMAP can authenticate and download all my
620emails just fine. The process for using a GPG key /not/ stored on a
621hardware token is pretty much the same and I'll talk about that process
622instead.
623
624These are the contents of my ~~/.offlineimap.py~.
625
626#+BEGIN_SRC python
627 #! /usr/bin/env python2
628 from subprocess import check_output
629 def get_pass(account):
630 return check_output(["gpg", "-dq", f" ~/.mail_pass/{account}.gpg"]).strip("\n")
631#+END_SRC
632
633This runs ~gpg -dq ~/.mail_pass/use_exa.gpg~ then strips the newline
634character before returning it to OfflineIMAP. ~-d~ tells GPG that you're
635passing it a file you want decrypted and ~-q~ tells it not to give any
636output other than the file's contents. For a setup that works with this
637Python script, put your passwords in plaintext files with the account
638name as the file name (e.g. ~use_exa~). You'll then encrypt it with ~gpg
639-er <YOUR_KEY_ID> use_exa~. Running ~gpg -dq use_exa.gpg~ should display
640your password. Repeat for every account and store the resulting files in
641~~/.mail_pass/~.
642
643The other option, ~sync_deletes~, is whether or not to delete remote
644emails that have been deleted locally. I enabled that because I want to
645have easy control over how much remote storage is used.
646
647Here's the next block again so you don't have to scroll up:
648
649#+BEGIN_SRC text
650[Repository use_exa-remote]
651type = IMAP
652remotehost = imap.example.com
653starttls = yes
654ssl = no
655remoteport = 143
656remoteuser = user@example.com
657remotepasseval = get_pass("use_exa")
658auth_mechanisms = GSSAPI, XOAUTH2, CRAM-MD5, PLAIN, LOGIN
659maxconnections = 1
660createfolders = True
661sync_deletes = yes
662#+END_SRC
663
664This one's pretty self-explanatory. ~type~, ~remotehost~, ~starttls~, ~ssl~, and
665~remoteport~ should all be somewhere in your provider's documentation.
666~remoteuser~ is your email address and ~remotepasseval~ is the function that
667will return your password and allow OfflineIMAP to authenticate. You'll
668want enter the name of your password file without the ~.gpg~ extension;
669the script takes care of adding that. Leave ~auth_mechanisms~ alone and
670the same for ~maxconnections~ unless you know your provider won't rate
671limit you or something for opening multiple connections. ~sync_deletes~ is
672the same as in the previous block.
673
674Copy those three blocks for as many accounts as you want emails
675downloaded from. I have 510 lines just for ~Account~ and ~Repository~ blocks
676due to the number of address I'm keeping track of.
677
678*** ~notmuch~
679[[https://notmuchmail.org/][~notmuch~]] is /a fast, global-search, and tag-based email system/. This
680what does all of our organisation as well as what provides the "virtual"
681mailboxes NeoMutt will display later on. Configuration is incredibly
682simple. This file goes in ~~/.notmuch-config~.
683
684#+BEGIN_SRC text
685 [database]
686 path=/home/user/mail/
687
688 [user]
689 name=Amolith
690 primary_email=user@example.com
691
692 [new]
693 tags=unread;new;
694 ignore=Trash;
695
696 [search]
697 exclude_tags=deleted;spam;
698
699 [maildir]
700 synchronize_flags=true
701#+END_SRC
702
703First section is the path to where all of your archives are, the ~[user]~
704section is where you list all of your accounts, ~[new]~ adds ~tags~ to mail
705notmuch hasn't indexed yet and ignores indexing the ~Trash~ folder, and
706~[search]~ ignores mail tagged with ~deleted~ or ~spam~. The final section
707tells ~notmuch~ to add maildir flags which correspond with ~notmuch~ tags.
708These flags will be synced to the remote server the next time
709OfflineIMAP runs and things will be somewhat organised in your webmail
710interface.
711
712After creating the configuration file, run ~notmuch new~ and wait for all
713of your mail to be indexed. This could take a short amount of time or it
714could take minutes up to an hour, depending on how many emails you have.
715After it's finished, you'll be able to run queries and see matching
716emails:
717
718#+BEGIN_SRC text
719 $ notmuch search from:user@example.com
720 thread:0000000000002e9d December 28 [1/1] Example User; Random subject that means nothing
721#+END_SRC
722
723This is not terribly useful in and of itself because you can't read it
724or reply to it or anything. That's where the Mail User Agent (MUA) comes
725in.
726
727*** ~afew~
728[[https://afew.readthedocs.io/en/latest/][~afew~]] is /an initial tagging script for notmuch/. After calling ~notmuch
729new~, ~afew~ will add tags based on headers such as ~From:~, ~To:~, ~Subject:~,
730etc. as well as handle killed threads and spam. The official [[https://afew.readthedocs.io/en/latest/quickstart.html][quickstart
731guide]] is probably the best resource on getting started but I'll include
732a few tips here as well.
733
734*** NeoMutt
735*** ~msmtp~
736~msmtp~ is what's known as a /Mail Transfer Agent/ (MTA). You throw it an
737email and it will relay that to your mail provider's SMTP server so it
738can have the proper headers attached for authentication, it can be sent
739from the proper domain, etc. All the necessary security measures can be
740applied that prevent your email from going directly to spam or from
741being rejected outright.
742
743~msmtp~'s configuration is also fairly simple if a bit long, just like
744OfflineIMAP's.
745
746#+BEGIN_SRC text
747 # Set default values for all following accounts.
748 defaults
749
750 # Use the mail submission port 587 instead of the SMTP port 25.
751 port 587
752
753 # Always use TLS.
754 tls on
755#+END_SRC
756
757This section just sets the defaults. It uses port 587 (STARTTLS) for all
758SMTP servers unless otherwise specified and enables TLS.
759
760#+BEGIN_SRC text
761 account user@example.com
762 host smtp.example.com
763 from user@example.com
764 auth on
765 user user@example.com
766 passwordeval secret-tool lookup Title "user@example.com"
767#+END_SRC
768
769This section is where things get tedious. When passing an email to
770~msmtp~, it looks at the ~From:~ header and searches for a block with a
771matching ~from~ line. If it finds one, it will use those configuration
772options to relay the email. ~host~ is simply the SMTP server of your mail
773provider, sometimes this is ~mail.example.com~, ~smtp.example.com~, etc.
774I've already explained ~from~, ~auth~ simply says that a username and
775password will have to be provided, ~user~ is that username, and
776~passwordeval~ is a method to obtain the password.
777
778When I got to configuring ~msmtp~, [[https://keepassxc.org/][KeePassXC]] had just released their
779~libsecret~ integration and I wanted to try it. ~secret-tool~ is a command
780line tool used to store and retrieve passwords from whatever keyring
781you're using. I think KDE has ~kwallet~ and GNOME has ~gnome-keyring~ if
782you already have those set up and want to use them; the process should
783be quite similar regardless.
784
785As mentioned above ~secret-tool~ stores and retrieves passwords. For
786retrieval, it expects the command to look like this.
787
788#+BEGIN_SRC text
789secret-tool lookup {attribute} {value} ...
790#+END_SRC
791
792I don't know what ~kwallet~ and ~gnome-keyring~'s attributes are but
793this can be used with KeePassXC by specifying the ~Title~ attribute. If
794the password to your email account is stored in KeePassXC with the
795address as the entry title, you can retrieve it by simply running...
796
797#+BEGIN_SRC text
798secret-tool lookup Title "user@example.com"
799#+END_SRC
800
801If you have a different naming system, you'll have to experiment and try
802different things; I don't know what KeePassXC's other attributes are so
803I can't give other examples.
804
805#+BEGIN_SRC text
806passwordeval gpg -dq ~/.mail_pass/use_exa.gpg
807#+END_SRC
808
809Now that the whole block is assembled, copy/paste/edit for as many
810accounts as you want to send email from.
811
812*** Summary
813
814*** TODO Pong fluffy when finished
815
816** TODO Making yourself overly available
817
818*** References
819[[https://www.nytimes.com/2021/03/05/opinion/ezra-klein-podcast-cal-newport.html?showTranscript=1][Stop. Breathe. We Can’t Keep Working Like This.]]
820
821*** Notes
822**** Get rid of information that isn't important
823**** Escalate the info that is
824**** Set /clear/ boundaries for when you are available
825**** Enforce those with automatic DnD rules or use timers
826**** With groups...
827***** Specialisation is /good/ and should be /encouraged/
828***** /All/ of the above points apply with coworkers as well
829
830*** TODO Pong Jake when finished
831
832** TODO Setting LXC up for local "cloud" development
833** DONE (Ab)using mesh networks for easy remote support :Mesh__networking:Open__source:Remote__support:
834CLOSED: [2021-11-01 Mon 02:51]
835:PROPERTIES:
836:EXPORT_FILE_NAME: abusing-mesh-networks-remote-support
837:EXPORT_HUGO_CUSTOM_FRONT_MATTER: :toc true
838:END:
839One of the things many of us struggle with when setting friends and
840family up with Linux is remote support. Commercial solutions like
841[[https://www.realvnc.com/][RealVNC]] and [[https://rustdesk.com/][RustDesk]] do exist and function very well, but are often more
842expensive than we would like for answering the odd "I can't get Facebook
843open!" support call. I've been on the lookout for suitable alternatives
844for a couple years but nothing has been satisfying. Because of this, I
845have held off on setting others up with any Linux distribution, even the
846particularly user-friendly options such as [[https://linuxmint.com/][Linux Mint]] and [[https://elementary.io/][elementary OS;]]
847if I'm going drop someone in an unfamiliar environment, I want to be
848able to help with any issue within a couple hours, not days and
849/certainly/ not weeks.
850
851[[https://linuxunplugged.com/421][Episode 421 of LINUX Unplugged]] gave me an awesome idea to use [[https://github.com/slackhq/nebula][Nebula,]] a
852networking tool created by Slack, [[https://libvnc.github.io/][X11vnc,]] a very minimal VNC server, and
853[[https://remmina.org/][Remmina,]] a libre remote access tool available in pretty much every Linux
854distribution, to set up a scalable, secure, and simple setup reminiscent
855of products like RealVNC.
856
857*** Nebula
858The first part of our stack is Nebula, the tool that creates a network
859between all of our devices. With traditional VPNs, you have a client
860with a persistent connection to a central VPN server and other clients
861can communicate with the first by going through that central server.
862This works wonderfully in most situations, but there are a lot of
863latency and bandwidth restrictions that would make remote support an
864unpleasant experience. Instead of this model, what we want is a /mesh/
865network, where each client can connect directly to one another /without/
866going through a central system and slowing things down. This is where
867Nebula comes in.
868
869In Nebula's terminology, clients are referred to as /nodes/ and central
870servers are referred to as /lighthouses/, so those are the terms I'll use
871going forward.
872
873Mesh networks are usually only possible when dealing with devices that
874have static IP addresses. Each node has to know /how/ to connect with the
875other nodes; John can't meet up with Bob when Bob moves every other day
876without notifying anyone of his new address. This wouldn't be a problem
877if Bob phoned Jill and told her where he was moving; John would call
878Jill, Jill would tell him where Bob is, and the two would be able to
879find each other
880
881With Nebula, nodes are Bob and John and Jill is a lighthouse. Each node
882connects to a lighthouse and the lighthouse tells the nodes how to
883connect with one another when they ask. It /facilitates/ the P2P
884connection then /backs out of the way/ so the two nodes can communicate
885directly with each other.
886
887It allows any node to connect with any other node on any network from
888anywhere in the world, as long as one lighthouse is accessible that
889knows the connection details for both peers.
890
891**** Getting started
892The /best/ resource is [[https://github.com/slackhq/nebula][the official documentation,]] but I'll describe the
893process here as well.
894
895After [[https://github.com/slackhq/nebula#1-the-nebula-binaries-or-distribution-packages-for-your-specific-platform-specifically-youll-need-nebula-cert-and-the-specific-nebula-binary-for-each-platform-you-use][installing the required packages,]] make sure you have a VPS with a
896static IP address to use as a lighthouse. If you want something dirt
897cheap, I would recommend one of the small plans from [[https://buyvm.net][BuyVM.]] I do have a
898[[https://my.frantech.ca/aff.php?aff=3783][referral link]] if you want them to kick me a few dollars for your
899purchase. [[https://www.hetzner.com/cloud][Hetzner]] (referral: ~ckGrk4J45WdN~) or [[https://www.netcup.eu/][netcup]] (referral:
900~36nc15758387844~) would also be very good options; I've used them all and
901am very comfortable recommending them.
902
903**** Creating a Certificate Authority
904After picking a device with a static IP address, it needs to be set up
905as a lighthouse. This is done by first creating a Certificate Authority
906(CA) that will be used for signing keys and certificates that allow our
907other devices into the network. The ~.key~ file produced by the following
908command is incredibly sensitive; with it, anyone can authorise a new
909device and give it access to your network. Store it in a safe,
910preferably encrypted location.
911
912#+BEGIN_SRC bash
913 nebula-cert ca -name "nebula.example.com"
914#+END_SRC
915
916I'll explain why we used a Fully-Qualified Domain Name (FQDN) as the
917CA's name in a later section. If you have your own domain, feel free to
918use that instead; it doesn't really matter what domain is used as long
919as the format is valid.
920
921**** Generating lighthouse credentials
922Now that we have the CA's ~.crt~ and ~.key~ files, we can create and sign
923keys and certificates for the lighthouse.
924
925#+BEGIN_SRC bash
926 nebula-cert sign -name "buyvm.lh.nebula.example.com" -ip "192.168.100.1/24"
927#+END_SRC
928
929Here, we're using a FQDN for the same reason as we did in the CA. You
930can use whatever naming scheme you like, I just prefer
931~<vps-host>.lh.nebula...~ for my lighthouses. The IP address can be on any
932of the following private IP ranges, I just happened to use ~192.168.100.X~
933for my network.
934
935| IP Range | Number of addresses |
936|-------------------------------+---------------------|
937| 10.0.0.0 – 10.255.255.255 | 16 777 216 |
938| 172.16.0.0 – 172.31.255.255 | 10 48 576 |
939| 192.168.0.0 – 192.168.255.255 | 65 536 |
940
941**** Creating a config file
942The next step is creating our lighthouse's config file. The reference
943config can be found in [[https://github.com/slackhq/nebula/blob/master/examples/config.yml][Nebula's repo.]] We only need to change a few of
944the lines for the lighthouse to work properly. If I don't mention a
945specific section here, I've left the default values.
946
947The section below is where we'll define certificates and keys. ~ca.crt~
948will remain ~ca.crt~ when we copy it over but I like to leave the node's
949cert and key files named as they were when generated; this makes it easy
950to identify nodes by their configs. Once we copy everything over to the
951server, we'll add the proper paths to the ~cert~ and ~key~ fields.
952
953#+BEGIN_SRC yaml
954 pki:
955 ca: /etc/nebula/ca.crt
956 cert: /etc/nebula/
957 key: /etc/nebula/
958#+END_SRC
959
960The next section is for identifying and mapping your lighthouses. This
961needs to be present in /all/ of the configs on /all/ nodes, otherwise they
962won't know how to reach the lighthouses and will never actually join the
963network. Make sure you replace ~XX.XX.XX.XX~ with whatever your VPS's
964public IP address is. If you've used a different private network range,
965those changes need to be reflected here as well.
966
967#+BEGIN_SRC yaml
968 static_host_map:
969 "192.168.100.1": ["XX.XX.XX.XX:4242"]
970#+END_SRC
971
972Below, we're specifying how the node should behave. It is a lighthouse,
973it should answer DNS requests, the DNS server should listen on all
974interfaces on port 53, it sends its IP address to lighthouses every 60
975seconds (this option doesn't actually have any effect when ~am_lighthouse~
976is set to ~true~ though), and this lighthouse should not send reports to
977other lighthouses. The bit about DNS will be discussed later.
978
979#+BEGIN_SRC yaml
980 lighthouse:
981 am_lighthouse: true
982 serve_dns: true
983 dns:
984 host: 0.0.0.0
985 port: 53
986 interval: 60
987 hosts:
988#+END_SRC
989
990The next bit is about [[https://en.wikipedia.org/wiki/Hole_punching_%28networking%29][hole punching]], also called /NAT punching/, /NAT
991busting/, and a few other variations. Make sure you read the comments for
992better explanations than I'll give here. ~punch: true~ enables hole
993punching. I also like to enable ~respond~ just in case nodes are on
994particularly troublesome networks; because we're using this as a support
995system, we have no idea what networks our nodes will actually be
996connected to. We want to make sure devices are available no matter where
997they are.
998
999#+BEGIN_SRC yaml
1000 punchy:
1001 punch: true
1002 respond: true
1003 delay: 1s
1004#+END_SRC
1005
1006~cipher~ is a big one. The value /must/ be identical on /all/ nodes /and/
1007lighthouses. ~chachapoly~ is more compatible so it's used by default. The
1008devices /I/ want to connect to are all x86 Linux, so I can switch to ~aes~
1009and benefit from [[https://www.reddit.com/r/networking/comments/iksyuu/comment/g3ra5cv/?utm_source=share&utm_medium=web2x&context=3][a small performance boost.]] Unless you know /for sure/
1010that you won't need to work with /anything/ else, I recommend leaving it
1011set to ~chachapoly~.
1012
1013#+BEGIN_SRC yaml
1014 cipher: chachapoly
1015#+END_SRC
1016
1017The last bit I modify is the firewall section. I leave most everything
1018default but /remove/ the bits after ~port: 443~. I don't /need/ the ~laptop~ and
1019~home~ groups (groups will be explained later) to access port ~443~ on this
1020node, so I shouldn't include the statement. If you have different needs,
1021take a look at the comment explaining how the firewall portion works and
1022make those changes.
1023
1024Again, I /remove/ the following bit from the config.
1025
1026#+BEGIN_SRC yaml
1027 - port: 443
1028 proto: tcp
1029 groups:
1030 - laptop
1031 - home
1032#+END_SRC
1033
1034**** Setting the lighthouse up
1035We've got the config, the certificates, and the keys. Now we're ready to
1036actually set it up. After SSHing into the server, grab the [[https://github.com/slackhq/nebula/releases/latest][latest
1037release of Nebula for your platform,]] unpack it, make the ~nebula~ binary
1038executable, then move it to ~/usr/local/bin~ (or some other location
1039fitting for your platform).
1040
1041#+BEGIN_SRC bash
1042 wget https://github.com/slackhq/nebula/releases/download/vX.X.X/nebula-PLATFORM-ARCH.tar.gz
1043 tar -xvf nebula-*
1044 chmod +x nebula
1045 mv nebula /usr/local/bin/
1046 rm nebula-*
1047#+END_SRC
1048
1049Now we need a place to store our config file, keys, and certificates.
1050
1051#+BEGIN_SRC bash
1052 mkdir /etc/nebula/
1053#+END_SRC
1054
1055The next step is copying the config, keys, and certificates to the
1056server. I use ~rsync~ but you can use whatever you're comfortable with.
1057The following four files need to be uploaded to the server.
1058
1059- ~config.yml~
1060- ~ca.crt~
1061- ~buyvm.lh.nebula.example.com.crt~
1062- ~buyvm.lh.nebula.example.com.key~
1063
1064With ~rsync~, that would look something like this. Make sure ~rsync~ is also
1065installed on the VPS before attempting to run the commands though;
1066you'll get an error otherwise.
1067
1068#+BEGIN_SRC bash
1069 rsync -avmzz ca.crt user@example.com:
1070 rsync -avmzz config.yml user@example.com:
1071 rsync -avmzz buyvm.lh.* user@example.com:
1072#+END_SRC
1073
1074SSH back into the server and move everything to ~/etc/nebula/~.
1075
1076#+BEGIN_SRC bash
1077 mv ca.crt /etc/nebula/
1078 mv config.yml /etc/nebula/
1079 mv buyvm.lh* /etc/nebula/
1080#+END_SRC
1081
1082Edit the config file and ensure the ~pki:~ section looks something like
1083this, modified to match your hostnames of course.
1084
1085#+BEGIN_SRC yaml
1086 pki:
1087 ca: /etc/nebula/ca.crt
1088 cert: /etc/nebula/buyvm.lh.nebula.example.com.crt
1089 key: /etc/nebula/buyvm.lh.nebula.example.com.key
1090#+END_SRC
1091
1092Run the following command to make sure everything works properly.
1093
1094#+BEGIN_SRC bash
1095 nebula -config /etc/nebula/config.yml
1096#+END_SRC
1097
1098The last step is daemonizing Nebula so it runs every time the server
1099boots. If you're on a machine using systemd, dropping the following
1100snippet into ~/etc/systemd/system/nebula.service~ should be sufficient. If
1101you're using something else, check the [[https://github.com/slackhq/nebula/tree/master/examples/][the examples directory]] for more
1102options.
1103
1104#+BEGIN_SRC text
1105 [Unit]
1106 Description=nebula
1107 Wants=basic.target
1108 After=basic.target network.target
1109 Before=sshd.service
1110
1111 [Service]
1112 SyslogIdentifier=nebula
1113 ExecReload=/bin/kill -HUP $MAINPID
1114 ExecStart=/usr/local/bin/nebula -config /etc/nebula/config.yml
1115 Restart=always
1116
1117 [Install]
1118 WantedBy=multi-user.target
1119#+END_SRC
1120
1121We're almost done!
1122
1123**** Setting individual nodes up
1124This process is almost exactly the same as setting lighthouses up. All
1125you'll need to do is generate a couple of certs and keys then tweak the
1126configs a bit.
1127
1128The following command creates a new cert/key for USER's node with the IP
1129address ~192.168.100.2~. The resulting files would go on the /remote/ node
1130not yours. Replace ~HOST~ and ~USER~ with fitting values.
1131
1132#+BEGIN_SRC bash
1133 nebula-cert sign -name "HOST.USER.nebula.example.com" -ip "192.168.100.2/24"
1134#+END_SRC
1135
1136The following command will create a /similar/ cert/key but it will be part
1137of the ~support~ group. The files resulting from this should go on /your/
1138nodes. With the config we'll create next, nodes in the ~support~ group
1139will be able to VNC and SSH into other nodes. Your nodes need to be in
1140the ~support~ group so you'll have access to the others.
1141
1142#+BEGIN_SRC bash
1143 nebula-cert sign -name "HOST.USER.nebula.example.com" -ip "192.168.100.2/24" -groups "support"
1144#+END_SRC
1145
1146On to the config now. This tells the node that it is /not/ a lighthouse,
1147it should /not/ resolve DNS requests, it /should/ ping the lighthouses and
1148tell them its IP address every 60 seconds, and the node at ~192.168.100.1~
1149is one of the lighthouses it should report to and query from. If you
1150have more than one lighthouse, add them to the list as well.
1151
1152#+BEGIN_SRC yaml
1153 lighthouse:
1154 am_lighthouse: false
1155 #serve_dns: false
1156 #dns:
1157 #host: 0.0.0.0
1158 #port: 53
1159 interval: 60
1160 hosts:
1161 - "192.168.100.1"
1162#+END_SRC
1163
1164The other bit that should be modified is the ~firewall:~ section and this
1165is where the groups we created earlier are important. Review its
1166comments and make sure you understand how it works before proceeding.
1167
1168We want to allow inbound connections on ports 5900, the standard port
1169for VNC, and 22, the standard for SSH. Additionally, we /only/ want to
1170allow connections from nodes in the ~support~ group. Any /other/ nodes
1171should be denied access.
1172
1173Note that including this section is not necessary on /your/ nodes, those
1174in the ~support~ group. It's only necessary on the remote nodes that
1175you'll be connecting to. As long as the ~outbound:~ section in the config
1176on /your/ node allows any outbound connection, you'll be able to access
1177other nodes.
1178
1179#+BEGIN_SRC yaml
1180 - port: 5900
1181 proto: tcp
1182 groups:
1183 - support
1184
1185 - port: 22
1186 proto: tcp
1187 groups:
1188 - support
1189#+END_SRC
1190
1191The certs, key, config, binary, and systemd service should all be copied
1192to the same places on all of these nodes as on the lighthouse.
1193
1194*** X11vnc
1195/Alright./ The hardest part is finished. Now on to setting ~x11vnc~ up on
1196the nodes you'll be supporting.
1197
1198All you should need to do is install ~x11vnc~ using the package manager
1199your distro ships with, generate a 20 character password with ~pwgen -s
120020 1~, run the following command, paste the password, wait for ~x11vnc~ to
1201start up, make sure it's running correctly, press ~Ctrl~ + ~C~, then add the
1202command to the DE's startup applications!
1203
1204#+BEGIN_SRC bash
1205 x11vnc --loop -usepw -listen <nebula-ip> -display :0
1206#+END_SRC
1207
1208~--loop~ tells ~x11vnc~ to restart once you disconnect from the session.
1209~-usepw~ is pretty self-explanatory. ~-listen <nebula-ip>~ is important; it
1210tells ~x11vnc~ to only listen on the node's Nebula IP address. This
1211prevents randos in a coffee shop from seeing an open VNC port and trying
1212to brute-force the credentials. ~-display :0~ just defines which X11
1213server display to connect to.
1214
1215Some distributions like elementaryOS and those that use KDE and GNOME
1216will surface a dialogue for managing startup applications if you just
1217press the Windows (Super) key and type ~startup~. If that doesn't work,
1218you'll have to root around in the settings menus, consult the
1219distribution's documentation, or ask someone else that might know.
1220
1221After adding it to the startup application, log out and back in to make
1222sure it's running in the background.
1223
1224*** Remmina
1225Now that our network is functioning properly and the VNC server is set
1226up, we need something that connects to the VNC server over the fancy
1227mesh network. Enter [[https://remmina.org/][Remmina.]] This one goes on /your/ nodes.
1228
1229Remmina is a multi-protocol remote access tool available in pretty much
1230ever distribution's package archive as ~remmina~. Install it, launch it,
1231add a new connection profile in the top left, give the profile a
1232friendly name (I like to use the name of the person I'll be supporting),
1233assign it to a group, such as ~Family~ or ~Friends~, set the Protocol to
1234~Remmina VNC Plugin~, enter the node's Nebula IP address in the Server
1235field, then enter their username and the 20 character password you
1236generated earlier. I recommend setting the quality to Poor, but Nebula
1237is generally performant enough that any of the options are suitable. I
1238just don't want to have to disconnect and reconnect with a lower quality
1239if the other person happens to be on a slow network.
1240
1241Save and test the connection!
1242
1243If all goes well and you see the other device's desktop, you're done
1244with the VNC section! Now on to SSH.
1245
1246*** SSH
1247First off, make sure ~openssh-server~ is installed on the remote node;
1248~openssh-client~ would also be good to have, but from what I can tell,
1249it's not strictly necessary. You /will/ need ~openssh-client~ on /your/ node,
1250however. If you already have an SSH key, copy it over to
1251~~/.ssh/authorized_keys~ on the remote node. If you don't, generate one
1252with ~ssh-keygen -t ed25519~. This will create an Ed25519 SSH key pair.
1253Ed25519 keys are shorter and faster than RSA and more secure than ECDSA
1254or DSA. If that means nothing to you, don't worry about it. Just note
1255than this key might not interact well with older SSH servers; you'll
1256know if you need to stick with the default RSA. Otherwise, Ed25519 is
1257the better option. After key generation has finished, copy
1258~~/.ssh/id_ed25519.pub~ (note the ~.pub~ extension) from your node to
1259~~/.ssh/authorized_keys~ on the remote node. The file /without/ ~.pub~ is your
1260/private/ key. Like the Nebula CA certificate we generated earlier, this
1261is extremely sensitive and should never be shared with anyone else.
1262
1263Next is configuring SSH to only listen on Nebula's interface; as with
1264~x11vnc~, this prevents randos in a coffee shop from seeing an open SSH
1265port and trying to brute-force their way in. Set the ~ListenAddress~
1266option in ~/etc/ssh/sshd_config~ to the remote node's Nebula IP address.
1267If you want to take security a step further, search for
1268~PasswordAuthentication~ and set it to ~no~. This means your SSH key is
1269/required/ for gaining access via SSH. If you mess up Nebula's firewall
1270rules and accidentally give other Nebula devices access to this machine,
1271they still won't be able to get in unless they have your SSH key. I
1272/personally/ recommend disabling password authentication, but it's not
1273absolutely necessary. After making these changes, run ~systemctl restart
1274sshd~ to apply them.
1275
1276Now that the SSH server is listening on Nebula's interface, it will
1277actually fail to start when the machine (re)boots. The SSH server starts
1278faster than Nebula does, so it will look for the interface before Nebula
1279has even had a chance to connect. We need to make sure systemd waits for
1280Nebula to start up and connect before it tells SSH to start; run
1281~systemctl edit --full sshd~ and add the following line in the ~[Unit]~
1282section, above ~[Service]~.
1283
1284#+BEGIN_SRC text
1285 After=nebula.service
1286#+END_SRC
1287
1288Even now, there's still a bit of a hiccup. Systemd won't start SSH until
1289Nebula is up and running, which is good. Unfortunately, even after
1290Nebula has started, it still takes a minute to bring the interface up,
1291causing SSH to crash. To fix /this/, add the following line directly below
1292~[Service]~.
1293
1294#+BEGIN_SRC text
1295 ExecStartPre=/usr/bin/sleep 30
1296#+END_SRC
1297
1298If the ~sleep~ executable is stored in a different location, make sure you
1299use that path instead. You can check by running ~which sleep~.
1300
1301When the SSH /service/ starts up, it will now wait an additional 30
1302seconds before actually starting the SSH /daemon/. It's a bit of a hacky
1303solution but it works™. If you come up with something better, please
1304send it to me and I'll include it in the post! My contact information is
1305at the bottom of [[/][this site's home page.]]
1306
1307After you've made these changes, run ~systemctl daemon-reload~ to make
1308sure systemd picks up on the modified service file, then run ~systemctl
1309restart sshd~. You should be able to connect to the remote node from your
1310node using the following command.
1311
1312#+BEGIN_SRC bash
1313 ssh USER@<nebula-ip>
1314#+END_SRC
1315
1316If you want to make the command a little simpler so you don't have to
1317remember the IP every time, create ~~/.ssh/config~ on your node and add
1318these lines to it.
1319
1320#+BEGIN_SRC text
1321 Host USER
1322 Hostname <nebula-ip>
1323 User USER
1324#+END_SRC
1325
1326Now you can just run ~ssh USER~ to get in. If you duplicate the above
1327block for all of the remote nodes you need to support, you'll only have
1328to remember the person's username to SSH into their machine.
1329
1330*** Going further with Nebula
1331This section explains why we used FQDNs in the certs and why the DNS
1332resolver is enabled on the lighthouse.
1333
1334Nebula ships with a built-in resolver meant specifically for mapping
1335Nebula node hostnames to their Nebula IP addresses. Running a public DNS
1336resolver is very much discouraged because it can be abused in terrible
1337ways. However, the Nebula resolver mitigates this risk because it /only/
1338answers queries for Nebula nodes. It doesn't forward requests to any
1339other servers nor does it attempt to resolve any domain other than what
1340was defined in its certificate. If you use the example I gave above,
1341that would be ~nebula.example.com~; the lighthouse will attempt to resolve
1342any subdomain of ~nebula.example.com~ but it will just ignore ~example.com~,
1343~nebula.duckduckgo.com~, ~live.secluded.site~, etc.
1344
1345Taking advantage of this resolver requires setting it as your secondary
1346resolver on any device you want to be able to resolve hostnames from.
1347If you were to add the lighthouse's IP address as your secondary
1348resolver on your PC, you could enter ~host.user.nebula.example.com~ in
1349Remmina's server settings /instead of/ ~192.168.1.2~.
1350
1351But how you do so is beyond the scope of this post!
1352
1353If you're up for some /more/ shenanigans later on down the line, you could
1354set up a Pi-Hole instance backed by Unbound and configure Nebula as
1355Unbound's secondary resolver. With this setup, you'd get DNS-level ad
1356blocking /and/ the ability to resolve Nebula hostname. Pi-Hole would query
1357Unbound for ~host.user.nebula.example.com~, Unbound would receive no
1358answer from the root servers because the domain doesn't exist outside of
1359your VPN, Unbound would fall back to Nebula, Nebula would give it an
1360answer, Unbound would cache the answer, tell Pi-Hole, Pi-Hole would
1361cache the answer, tell your device, then your device would cache the
1362answer, and you can now resolve any Nebula host!
1363
1364Exactly how you do /that/ is */definitely/* beyond the scope of this post :P
1365
1366If you set any of this up, I would be interested to hear how it goes! As
1367stated earlier, my contact information is at the bottom of the site's
1368home page :)
1369
1370* Education :@Education:
1371** TODO Homeschooling
1372* Music :@Music:
1373* Pipe Smoking :@Pipe__Smoking:
1374* Dungeons & Dragons :@Dungeons__and__Dragons:
1375
1376* Footnotes
1377
1378[fn:19] [[https://github.com/audacity/audacity/pull/835#issuecomment-834451187][Link to the comment]] and [[/assets/pngs/audacity-pr/privatelibcurl.png][link to the screenshot]]
1379
1380[fn:18] [[https://github.com/audacity/audacity/pull/835#issuecomment-834010117][Link to the comment]] and [[/assets/pngs/audacity-pr/vendorproscons.png][link to the screenshot]]
1381
1382[fn:17] [[https://github.com/audacity/audacity/pull/835#issuecomment-836069326][Link to the comment]] and [[/assets/pngs/audacity-pr/tantacrulrole.png][link to the screenshot]]
1383
1384[fn:16] [[https://github.com/audacity/audacity/pull/835#discussion_r627762185][Link to the comment]] and [[/assets/pngs/audacity-pr/missingprivacypolicy.png][link to the screenshot]]
1385
1386[fn:15] [[https://github.com/audacity/audacity/pull/835#issuecomment-834286641][Link to the comment]] and [[/assets/pngs/audacity-pr/darkpattern.png][link to the screenshot]]
1387
1388[fn:14] [[https://github.com/audacity/audacity/pull/835#discussion_r627764300][Link to the comment]] and the screenshot is the same as previous
1389
1390[fn:13] [[https://github.com/audacity/audacity/pull/835#discussion_r627756976][Link to the comment]] and [[/assets/pngs/audacity-pr/vaguedialogue.png][link to the screenshot]]
1391
1392[fn:12] [[https://github.com/audacity/audacity/pull/835#discussion_r628124998][Link to the review]] and [[/assets/pngs/audacity-pr/defaultconsentbutton.png][link to the screenshot]]
1393
1394[fn:11] [[https://github.com/audacity/audacity/pull/835#discussion_r628008821][Link to the review]] and [[/assets/pngs/audacity-pr/externaldependency.png][link to the screenshot]]
1395
1396[fn:10] [[https://github.com/audacity/audacity/pull/835#discussion_r628005925][Link to the review]] and [[/assets/pngs/audacity-pr/vendorcurl.png][link to the screenshot]]
1397
1398[fn:9] Note that because I am not a C programmer, these reviews might
1399not be entirely accurate and I wouldn't be able to catch the reviewer's
1400error. I am relying on other community members to catch issues and
1401comment on them; none of the reviews I link to have such comments so I'm
1402assuming they are correct.
1403
1404[fn:8] [[https://github.com/audacity/audacity/pull/835#discussion_r627993755][Link to the review]] and [[/assets/pngs/audacity-pr/writeanalyticsid.png][link to the screenshot]]
1405
1406[fn:7] /Optical Character Recognition/: the program looks at your
1407handwriting and tries to turn it into text.
1408
1409[fn:6] In this situation, latency refers to how long it takes for "ink"
1410to show up on the "page" after writing something.
1411
1412[fn:5] It's not really a fountain pen even though that's what they call
1413it; it's just pressure-sensitive.
1414
1415[fn:4] There does seem to be a group of people interested in just such a
1416thing: /[[https://alexsoto.dev/challenges-building-an-open-source-eink-laptop.html][Challenges Building an Open-Source E Ink Laptop]]/
1417
1418[fn:3]Taken from their [[https://support.remarkable.com/hc/en-us/articles/360006699537-About-reMarkable-2-][support page about the reMarkable 2;]] search the
1419page for /operating system/ and it should show up.
1420
1421[fn:2]I dislike Apple's operating system, their hardware, business
1422model, privacy practises, and much of what they stand for as a company.
1423Don't @ me.
1424
1425[fn:1]E-R/W is a play on media commonly being labelled as R/W when you
1426can read from it and write to it.