Various Electrum Checks and Fixes

Christopher Vollick created 4 years ago

There were some issues with Electrum, and we lost a bit of confidence,
so I built these to help with that.

bin/check_electrum_wallet_completeness
- This one is meant to be run in cron. It checks for addresses we've
  given a user that Electrum doesn't know we have. It just prints out,
  so we get an email and can go look.
  The purpose of this is to know before our users that we're missing
  something.

bin/detect_duplicate_addrs
- This one is meant to be run in cron. It looks through the addresses
  that users has have been given to make sure the same address hasn't
  been given out to more than one person.
  It just prints out the issues, so we'll be notified and can take a
  look

bin/correct_duplicate_addrs
- This is one potential solution that can be run in response to
  duplicate addresses.
  Since I'm expecting an email from bin/detect_duplicate_addrs, this
  takes as input the text that was sent to us.
  It goes through each address and re-assigns it away from all users,
  parking the addresses on the support account so we still get notified
  when people send money, etc
  Because it takes output as input, they could be piped together in
  theory, but I never tested that because I assume some investigation
  would be warranted

bin/reassert_electrum_notification
- This script goes through every bitcoin address that's been given to a
  customer and makes sure that electrum knows to tell us about changes
  to that address

Change summary

bin/check_electrum_wallet_completeness | 22 ++++++++++++++++
bin/correct_duplicate_addrs            | 38 ++++++++++++++++++++++++++++
bin/detect_duplicate_addrs             | 12 ++++++++
bin/reassert_electrum_notification     | 29 +++++++++++++++++++++
lib/electrum.rb                        |  8 +++++
lib/redis_addresses.rb                 | 24 +++++++++++++++++
6 files changed, 133 insertions(+)

Detailed changes

bin/check_electrum_wallet_completeness

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+require 'redis'
+require 'dhall'
+require_relative '../lib/redis_addresses'
+require_relative '../lib/electrum'
+
+config =
+	Dhall::Coder
+	.new(safe: Dhall::Coder::JSON_LIKE + [Symbol, Proc])
+	.load(ARGV[0], transform_keys: :to_sym)
+
+redis = Redis.new
+electrum = Electrum.new(**config)
+
+electrum_addrs = electrum.listaddresses
+
+get_addresses_with_users(redis).each do |addr, keys|
+	unless electrum_addrs.include?(addr)
+		puts "The address #{addr} (included in #{keys.join(", ")}) isn't included in electrum's list"
+	end
+end

bin/correct_duplicate_addrs

@@ -0,0 +1,38 @@
+# frozen_string_literal: true
+
+# This is meant to be run with the output of detect_duplicate_addrs on stdin
+# The assumption is that some logging will dump that, and then someone will
+# run this after looking into why
+# Theoretically they could be piped together directly for automated fixing
+
+require 'redis'
+
+redis = Redis.new
+
+customer_id = ENV['DEFAULT_CUSTOMER_ID']
+unless customer_id
+	puts "The env-var DEFAULT_CUSTOMER_ID must be set to the ID of the customer who will receive the duplicated addrs, preferably a support customer or something linked to notifications when stray money is sent to these addresses"
+	exit 1
+end
+
+
+STDIN.each_line do |line|
+	match = line.match(/^(\w+) is used by the following \d+ keys: (.*)/)
+	unless match
+		puts "The following line can't be understood and is being ignored"
+		puts "  #{line}"
+		next
+	end
+
+	addr = match[1]
+	keys = match[2].split(" ")
+
+	# This is the customer ID of the support chat
+	# All duplicates are moved to the support addr so we still hear when people
+	# send money there
+	redis.sadd("jmp_customer_btc_addresses-#{customer_id}", addr)
+
+	keys.each do |key|
+		redis.srem(key, addr)
+	end
+end

bin/detect_duplicate_addrs

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+require 'redis'
+require_relative '../lib/redis_addresses'
+
+redis = Redis.new
+
+get_addresses_with_users(redis).each do |addr, keys|
+	if keys.length > 1
+		puts "#{addr} is used by the following #{keys.length} keys: #{keys.join(" ")}"
+	end
+end

bin/reassert_electrum_notification

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+require 'redis'
+require 'dhall'
+require_relative '../lib/redis_addresses'
+require_relative '../lib/electrum'
+
+config =
+	Dhall::Coder
+	.new(safe: Dhall::Coder::JSON_LIKE + [Symbol, Proc])
+	.load(ARGV[0], transform_keys: :to_sym)
+
+redis = Redis.new
+electrum = Electrum.new(**config)
+
+get_addresses_with_users(redis).each do |addr, keys|
+	match = keys.first.match(/.*-(\d+)$/)
+	unless match
+		puts "Can't understand key #{keys.first}, skipping"
+		next
+	end
+
+	customer_id = match[1]
+	url = "https://pay.jmp.chat/electrum_notify?address=#{addr}&customer_id=#{customer_id}"
+
+	unless electrum.notify(addr, url)
+		puts "Failed to setup #{addr} to notify #{url}. Skipping"
+	end
+end

lib/electrum.rb

@@ -27,6 +27,14 @@ class Electrum
 		rpc_call(:get_tx_status, txid: tx_hash)["result"]
 	end
 
+	def listaddresses
+		rpc_call(:listaddresses, {})["result"]
+	end
+
+	def notify(address, url)
+		rpc_call(:notify, address: address, URL: url)["result"]
+	end
+
 	class Transaction
 		def initialize(electrum, tx_hash, tx)
 			@electrum = electrum

lib/redis_addresses.rb

@@ -0,0 +1,24 @@
+# frozen_string_literal: true
+
+require "redis"
+
+# This returns a hash
+# The keys are the bitcoin addresses, the values are all of the keys which
+#   contain that address
+# If there are no duplicates, then each value will be a singleton list
+def get_addresses_with_users(redis)
+	addrs = Hash.new { |h, k| h[k] = [] }
+
+	# I picked 1000 because it made a relatively trivial case take 15 seconds
+	#   instead of forever.
+	# Basically it's "how long does each command take"
+	# The lower it is (default is 10), it will go back and forth to the client a
+	#   ton
+	redis.scan_each(match: "jmp_customer_btc_addresses-*", count: 1000) do |key|
+		redis.smembers(key).each do |addr|
+			addrs[addr] << key
+		end
+	end
+
+	addrs
+end