I have a theory about what is causing this - I think it's me not thinking about deferred evaluation carefully enough.
To try to work around this, I separated my scripts into a library module to be unit tested, and a short runner which will call into the library module to generate the manifest, however this approach produces unexpected behaviour - for example if I have a simple gurobi-manifest.scm like so:
(add-to-load-path (dirname (current-filename))) ;; put the script's location on the module path
(use-modules (gurobi-transform-lib)) ;; load the script library
(newline)
(packages->manifest (list my-package-with-my-gurobi
(specification->package "python"))) ;; add python for convenience
The procedure we call to generate the package - my-package-with-my-gurobi, is in a file called gurobi-transform-lib.scm in the same directory as the manifest above.
This is crux of the issue - I think it's an issue with where the code for my package is evaluated.
If I move just 2 functions back from my library into the manifest itself, then everything works exactly as I'd expect.
;; Prompt for Gurobi package if not set in env var
(define my-gurobi-package ((transform) (specification->package "gurobipy")))
(define my-package-with-my-gurobi
(let* ((test-package-string (get-env-var-or-prompt "GUIX_TEST_PACKAGE"))
(test-package (specification->package test-package-string)))
(format #t "~%Setting Test Package: ~a~%" test-package-string)
(package/inherit test-package
(propagated-inputs
`(("gurobipy" ,my-gurobi-package) ;; add my new gurobi
,@(alist-delete "gurobipy" (package-propagated-inputs test-package))))))) ;; remove the original gurobi and splice
What I think is happening is that the propagated-inputs in my newly created test-package will have their evaluation delayed until the build occurs in the daemon. I presume at this point my local library I've used add-to-load-path to include in the manifest will no longer be accessible and hence the eval failure being returned.
IIRC there is a key you can add to package definitons to request module loading on the daemon side as part of the build process - if something like was to exist then it might be possible to move these 2 functions back to the library rather than having them in the manifest, but as it stands it seems the daemon has visibilty of the manifest script itself, but not load-path and modules as set in the manifest script.