Regtest support

Since electrum doesn’t support regtest out of the box, current testing is done on testnet, but this is not optimal for a few reasons. Testnet is slow, some miners do not include segwit transactions in their blocks and at this point even the coins are scarce enough that if some are lost, it becomes quite annoying to get more. Also testing the more exotic things like double spending and reorgs requires mining (and competing against other miners). Even at difficulty 1, cpu mining is cumbersome and can take minutes to mine even one block.

For the EPS setup to support regtest, first we need to patch Electrum for regtest support. Between themselves, testnet and regtest are not so different, except for two things (might be more that I didn’t consider):

1. Bech32 addresses are different. Testnet uses tb as the human readable part while regtest uses bcrt
2. Checkpoints make no sense on regtest. The only block that would be the same for a new regtest run is the genesis block.

For EPS itself, point (1) does not seem like issue. It relies on the underlying bitcoind for getting the addresses and as long as it supports bech32 on regtest (which Core does), it should be alright. As for the checkpoints, it seems that both get_block_header() and the blockchain.block.get_chunk method in handle_query() both expect a previousblockhash in the json return by getblockheader. This value doesn’t exist when the genesis block’s header is requested.

This causes EPS to fail with :

Traceback (most recent call last):                                  
  File "p2pkh/server.py", line 473, in <module>                              
    main()                                                                                                                                     
  File "p2pkh/server.py", line 470, in main                                                                      
    poll_interval_connected, certfile, keyfile)                                                                                                
  File "p2pkh/server.py", line 290, in run_electrum_server                                                       
    txmonitor)                                                                                                                                 
  File "p2pkh/server.py", line 153, in handle_query                                                              
    header = get_block_header(rpc, blockhash)                                                                                                  
  File "p2pkh/server.py", line 224, in get_block_header                                                          
    "prev_block_hash": rpc_head["previousblockhash"],                                                                                          
KeyError: 'previousblockhash'

and :

Traceback (most recent call last):                                                                               
  File "p2pkh/server.py", line 482, in <module>                                                                                                
    main()                                                                                                       
  File "p2pkh/server.py", line 479, in main                                                                                                    
    poll_interval_connected, certfile, keyfile)                                                                  
  File "p2pkh/server.py", line 299, in run_electrum_server                                                                                     
    txmonitor)                                                                                                                                 
  File "p2pkh/server.py", line 170, in handle_query                                     
    binascii.unhexlify(header["previousblockhash"])[::-1],      
KeyError: 'previousblockhash'

Luckily, the previous block hash of genesis is defined as 0000000000000000000000000000000000000000000000000000000000000000 (at least in the header of the block), so it’s possible to patch that in as well, in case the previousblockhash doesn’t exist.

I’ve done the needed patching for both Electrum and EPS and it seems to work well (I only tested p2pkh and p2wpkh for now). Electrum is at: https://github.com/fivepiece/electrum/tree/add-network-regtest

And the diff is (done against version 3.1.2 of electrum):

diff --git a/electrum b/electrum
index 783d5c13..caa0b2a4 100755
--- a/electrum
+++ b/electrum
@@ -417,6 +417,9 @@ if __name__ == '__main__':
     if config.get('testnet'):
         constants.set_testnet()
 
+    if config.get('regtest'):
+        constants.set_regtest()
+
     # run non-RPC commands separately
     if cmdname in ['create', 'restore']:
         run_non_RPC(config)
diff --git a/lib/commands.py b/lib/commands.py
index 8435cf9e..35b41e62 100644
--- a/lib/commands.py
+++ b/lib/commands.py
@@ -806,6 +806,7 @@ def add_global_options(parser):
     group.add_argument("-P", "--portable", action="store_true", dest="portable", default=False, help="Use local 'electrum_data' directory")
     group.add_argument("-w", "--wallet", dest="wallet_path", help="wallet path")
     group.add_argument("--testnet", action="store_true", dest="testnet", default=False, help="Use Testnet")
+    group.add_argument("--regtest", action="store_true", dest="regtest", default=False, help="Use Regtest")
 
 def get_parser():
     # create main parser
diff --git a/lib/constants.py b/lib/constants.py
index ec35cbe3..620db80e 100644
--- a/lib/constants.py
+++ b/lib/constants.py
@@ -93,6 +93,34 @@ class BitcoinTestnet:
     }
 
 
+class BitcoinRegtest:
+
+    TESTNET = True
+    WIF_PREFIX = 0xef
+    ADDRTYPE_P2PKH = 111
+    ADDRTYPE_P2SH = 196
+    SEGWIT_HRP = "bcrt"
+    GENESIS = "0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206"
+    DEFAULT_PORTS = {'t': '51001', 's': '51002'}
+    DEFAULT_SERVERS = read_json('servers_regtest.json', {})
+    CHECKPOINTS = []
+
+    XPRV_HEADERS = {
+        'standard':    0x04358394,  # tprv
+        'p2wpkh-p2sh': 0x044a4e28,  # uprv
+        'p2wsh-p2sh':  0x024285b5,  # Uprv
+        'p2wpkh':      0x045f18bc,  # vprv
+        'p2wsh':       0x02575048,  # Vprv
+    }
+    XPUB_HEADERS = {
+        'standard':    0x043587cf,  # tpub
+        'p2wpkh-p2sh': 0x044a5262,  # upub
+        'p2wsh-p2sh':  0x024285ef,  # Upub
+        'p2wpkh':      0x045f1cf6,  # vpub
+        'p2wsh':       0x02575483,  # Vpub
+    }
+
+
 # don't import net directly, import the module instead (so that net is singleton)
 net = BitcoinMainnet
 
@@ -105,3 +133,8 @@ def set_mainnet():
 def set_testnet():
     global net
     net = BitcoinTestnet
+
+
+def set_regtest():
+    global net
+    net = BitcoinRegtest
diff --git a/lib/servers_regtest.json b/lib/servers_regtest.json
new file mode 100644
index 00000000..c54339ed
--- /dev/null
+++ b/lib/servers_regtest.json
@@ -0,0 +1,8 @@
+{
+    "127.0.0.1": {
+        "pruning": "-",
+        "s": "50002",
+        "t": "50001",
+        "version": "1.2"
+    }
+}
diff --git a/lib/simple_config.py b/lib/simple_config.py
index 34c62d3f..7e6ff2d6 100644
--- a/lib/simple_config.py
+++ b/lib/simple_config.py
@@ -116,6 +116,10 @@ class SimpleConfig(PrintError):
             path = os.path.join(path, 'testnet')
             make_dir(path)
 
+        if self.get('regtest'):
+            path = os.path.join(path, 'regtest')
+            make_dir(path)
+
         self.print_error("electrum directory", path)
         return path
 
diff --git a/setup.py b/setup.py
index 58c938e5..a513d3b5 100755
--- a/setup.py
+++ b/setup.py
@@ -74,6 +74,7 @@ setup(
         'electrum': [
             'servers.json',
             'servers_testnet.json',
+            'servers_regtest.json',
             'currencies.json',
             'checkpoints.json',
             'checkpoints_testnet.json',

EPS PR is coming up at #7 .

(edit to electrum diff : added creation of the regtest directory)

[fivepiece]

Regtest support merged to electrum https://github.com/spesmilo/electrum/pull/4242 \o/

[fivepiece]

I figured it’s best to ask for any rejects first : https://github.com/spesmilo/electrum/issues/4241