1# Go SQLite VFS API
2
3This package implements the SQLite [OS Interface](https://sqlite.org/vfs.html) (aka VFS).
4
5It replaces the default SQLite VFS with a **pure Go** implementation,
6and exposes [interfaces](https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs#VFS)
7that should allow you to implement your own [custom VFSes](#custom-vfses).
8
9See the [support matrix](https://github.com/ncruces/go-sqlite3/wiki/Support-matrix)
10for the list of supported OS and CPU architectures.
11
12Since this is a from scratch reimplementation,
13there are naturally some ways it deviates from the original.
14It's also not as battle tested as the original.
15
16The main differences to be aware of are
17[file locking](#file-locking) and
18[WAL mode](#write-ahead-logging) support.
19
20### File Locking
21
22POSIX advisory locks,
23which SQLite uses on [Unix](https://github.com/sqlite/sqlite/blob/5d60f4/src/os_unix.c#L13-L14),
24are [broken by design](https://github.com/sqlite/sqlite/blob/5d60f4/src/os_unix.c#L1074-L1162).
25Instead, on Linux and macOS, this package uses
26[OFD locks](https://www.gnu.org/software/libc/manual/html_node/Open-File-Description-Locks.html)
27to synchronize access to database files.
28
29This package can also use
30[BSD locks](https://man.freebsd.org/cgi/man.cgi?query=flock&sektion=2),
31albeit with reduced concurrency (`BEGIN IMMEDIATE` behaves like `BEGIN EXCLUSIVE`,
32[docs](https://sqlite.org/lang_transaction.html#immediate)).
33BSD locks are the default on BSD and illumos,
34but you can opt into them with the `sqlite3_flock` build tag.
35
36On Windows, this package uses `LockFileEx` and `UnlockFileEx`,
37like SQLite.
38
39You can also opt into a cross-platform locking implementation
40with the `sqlite3_dotlk` build tag.
41
42Otherwise, file locking is not supported, and you must use
43[`nolock=1`](https://sqlite.org/uri.html#urinolock)
44(or [`immutable=1`](https://sqlite.org/uri.html#uriimmutable))
45to open database files.
46To use the [`database/sql`](https://pkg.go.dev/database/sql) driver
47with `nolock=1` you must disable connection pooling by calling
48[`db.SetMaxOpenConns(1)`](https://pkg.go.dev/database/sql#DB.SetMaxOpenConns).
49
50You can use [`vfs.SupportsFileLocking`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs#SupportsFileLocking)
51to check if your build supports file locking.
52
53### Write-Ahead Logging
54
55On Unix, this package uses `mmap` to implement
56[shared-memory for the WAL-index](https://sqlite.org/wal.html#implementation_of_shared_memory_for_the_wal_index),
57like SQLite.
58
59On Windows, this package uses `MapViewOfFile`, like SQLite.
60
61You can also opt into a cross-platform, in-process, memory sharing implementation
62with the `sqlite3_dotlk` build tag.
63
64Otherwise, [WAL support is limited](https://sqlite.org/wal.html#noshm),
65and `EXCLUSIVE` locking mode must be set to create, read, and write WAL databases.
66To use `EXCLUSIVE` locking mode with the
67[`database/sql`](https://pkg.go.dev/database/sql) driver
68you must disable connection pooling by calling
69[`db.SetMaxOpenConns(1)`](https://pkg.go.dev/database/sql#DB.SetMaxOpenConns).
70
71You can use [`vfs.SupportsSharedMemory`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs#SupportsSharedMemory)
72to check if your build supports shared memory.
73
74### Blocking Locks
75
76On Windows and macOS, this package implements
77[Wal-mode blocking locks](https://sqlite.org/src/doc/tip/doc/wal-lock.md).
78
79### Batch-Atomic Write
80
81On Linux, this package may support
82[batch-atomic writes](https://sqlite.org/cgi/src/technote/714)
83on the F2FS filesystem.
84
85### Checksums
86
87This package can be [configured](https://pkg.go.dev/github.com/ncruces/go-sqlite3#Conn.EnableChecksums)
88to add an 8-byte checksum to the end of every page in an SQLite database.
89The checksum is added as each page is written
90and verified as each page is read.\
91The checksum is intended to help detect database corruption
92caused by random bit-flips in the mass storage device.
93
94The implementation is compatible with SQLite's
95[Checksum VFS Shim](https://sqlite.org/cksumvfs.html).
96
97### Build Tags
98
99The VFS can be customized with a few build tags:
100- `sqlite3_flock` forces the use of BSD locks.
101- `sqlite3_dotlk` forces the use of dot-file locks.
102
103> [!IMPORTANT]
104> The default configuration of this package is compatible with the standard
105> [Unix and Windows SQLite VFSes](https://sqlite.org/vfs.html#multiple_vfses);
106> `sqlite3_flock` builds are compatible with the
107> [`unix-flock` VFS](https://sqlite.org/compile.html#enable_locking_style);
108> `sqlite3_dotlk` builds are compatible with the
109> [`unix-dotfile` VFS](https://sqlite.org/compile.html#enable_locking_style).
110
111> [!CAUTION]
112> Concurrently accessing databases using incompatible VFSes
113> will eventually corrupt data.
114
115### Custom VFSes
116
117- [`github.com/ncruces/go-sqlite3/vfs/memdb`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs/memdb)
118 implements an in-memory VFS.
119- [`github.com/ncruces/go-sqlite3/vfs/readervfs`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs/readervfs)
120 implements a VFS for immutable databases.
121- [`github.com/ncruces/go-sqlite3/vfs/adiantum`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs/adiantum)
122 wraps a VFS to offer encryption at rest.
123- [`github.com/ncruces/go-sqlite3/vfs/xts`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs/xts)
124 wraps a VFS to offer encryption at rest.