[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
08/19: guix: register-path: do deduplication.
From: |
guix-commits |
Subject: |
08/19: guix: register-path: do deduplication. |
Date: |
Tue, 29 Jan 2019 14:19:50 -0500 (EST) |
reepca pushed a commit to branch guile-daemon
in repository guix.
commit 48ce4ddfa3c252717b23de565daab922596c8339
Author: Caleb Ristvedt <address@hidden>
Date: Tue Jun 6 02:44:41 2017 -0500
guix: register-path: do deduplication.
* guix/store.scm (get-temp-link, replace-with-link, deduplicate): new
procedures.
(register-path): uses deduplicate now.
---
guix/store.scm | 47 ++++++++++++++++++++++++++++++++++++++++-------
1 file changed, 40 insertions(+), 7 deletions(-)
diff --git a/guix/store.scm b/guix/store.scm
index e60fa16..bc0a057 100644
--- a/guix/store.scm
+++ b/guix/store.scm
@@ -50,7 +50,6 @@
#:use-module (ice-9 threads)
#:use-module (ice-9 format)
#:use-module (web uri)
- #:use-module (sqlite3)
#:use-module (guix store database)
#:use-module (gnu build install)
#:export (%daemon-socket-uri
@@ -1558,11 +1557,44 @@ makes a wrapper around a port which implements
GET-POSITION."
(values hash
size)))))
-;; TODO: Run a "deduplication pass", whatever that involves. Also, handle
-;; databases not existing yet (what should the default behavior be? Figuring
-;; out how the C++ stuff currently does it sounds like a lot of grepping for
-;; global variables...). Also, return #t on success like the documentation
-;; says we should.
+(define (get-temp-link target)
+ "Like mkstemp!, but instead of creating a new file and giving you the name,
+it creates a new hardlink to TARGET and gives you the name."
+ (let try-again ((tempname (tmpnam)))
+ (catch
+ #t
+ (lambda ()
+ (link target tempname)
+ tempname)
+ (lambda ()
+ (try-again (tmpnam))))))
+
+(define (replace-with-link target to-replace)
+ "Replaces the file TO-REPLACE with a hardlink to TARGET"
+ ;; According to the C++ code, this is how you replace it with a link
+ ;; "atomically".
+ (let ((temp-link (get-temp-link target)))
+ (delete-file to-replace)
+ (rename-file temp-link to-replace)))
+
+;; TODO: handling in case the .links directory doesn't exist? For now I'll
+;; just assume it's the responsibility of whoever makes the store to create
+;; it.
+(define (deduplicate path store hash)
+ "Checks if a store item with hash HASH already exists. If so, replaces PATH
+with a hardlink to the already-existing one. If not, it registers PATH so that
+future duplicates can hardlink to it."
+ (let ((links-path (string-append store
+ "/.links/"
+ (bytevector->base16-string hash))))
+ (if (file-exists? links-path)
+ (replace-with-link links-path path)
+ (link path links-path))))
+
+;; TODO: Handle databases not existing yet (what should the default behavior
+;; be? Figuring out how the C++ stuff currently does it sounds like a lot of
+;; grepping for global variables...). Also, return #t on success like the
+;; documentation says we should.
(define* (register-path path
#:key (references '()) deriver prefix state-directory)
@@ -1621,7 +1653,8 @@ be used internally by the daemon's build hook."
(with-output-to-port
(%make-void-port "w")
(lambda ()
- (reset-timestamps real-path))))))
+ (reset-timestamps real-path)))
+ (deduplicate real-path store-dir hash))))
;;;
- branch guile-daemon created (now 10b0562), guix-commits, 2019/01/29
- 07/19: guix: register-path: reset timestamps after registering., guix-commits, 2019/01/29
- 01/19: patches: honor NIX_STORE in site.py., guix-commits, 2019/01/29
- 04/19: .dir-locals.el: properly indent sql macros., guix-commits, 2019/01/29
- 08/19: guix: register-path: do deduplication.,
guix-commits <=
- 12/19: linux-container: new use-output argument., guix-commits, 2019/01/29
- 02/19: guix: register-path: Implement prototype in scheme., guix-commits, 2019/01/29
- 05/19: guix: sql.scm: split into generic and store-specific parts., guix-commits, 2019/01/29
- 09/19: guix: register-path: return #t on success., guix-commits, 2019/01/29
- 03/19: guix: register-path: Honor environment variables., guix-commits, 2019/01/29
- 06/19: guix: register-path: use new %store-database-directory, guix-commits, 2019/01/29
- 10/19: guix: register-path: use new %store-database-directory, guix-commits, 2019/01/29
- 16/19: build-derivations: Leaked environment variables more robust., guix-commits, 2019/01/29
- 19/19: gnu: linux-container: Make it more suitable for derivation-building., guix-commits, 2019/01/29
- 17/19: guix: store: Make register-items transactional, register drv outputs, guix-commits, 2019/01/29