Re: Some ravanan feedback

Re: Some ravanan feedback

[Nicolas Graves]

I still experience quite some errors, for instance with a random
repository online :

https://github.com/nfdi4plants/cwl-examples

and the following changes : 

modified   run-python-script/test.cwl
@@ -3,8 +3,9 @@
 cwlVersion: v1.2
 class: CommandLineTool
 hints:
-  DockerRequirement:
-    dockerPull: python:latest
+  SoftwareRequirement:
+    packages:
+      python
 baseCommand: python
 stdout: output.txt
 inputs:
modified   run-python-script/testRun.yml
@@ -1,5 +1,5 @@
 src:
   class: File
-  path: test.py
+  location: test.py
 firstArg: memem
 secondArg: soos


I happen to have such a backtrace :


Backtrace:
In ice-9/eval.scm:
    619:8 19 (_ #(#(#(#(#(#(#(#(#(#(#(…) …) …) …) …) …) …) …) …) …) …))
   293:34 18 (_ #(#(#(#(#(#(#(#(#(#(#(…) …) …) …) …) …) …) …) …) …) …))
In ice-9/boot-9.scm:
    724:2 17 (call-with-prompt ("prompt") #<procedure 7f2b83800180 …> …)
  1752:10 16 (with-exception-handler _ _ #:unwind? _ # _)
    152:2 15 (with-fluid* _ _ _)
In ice-9/exceptions.scm:
   406:15 14 (run-workflow _ _ _ _ _ _ _ _ #:guix-daemon-socket _)
In ice-9/boot-9.scm:
  1752:10 13 (with-exception-handler _ _ #:unwind? _ # _)
In ravanan/workflow.scm:
   592:26 12 (_)
   293:29 11 (schedule _ _ _)
In ravanan/command-line-tool.scm:
   361:10 10 (run-command-line-tool "test" #f #f (("baseCom…" . #) …) …)
   828:13  9 (build-command-line-tool-script "test" _ #f _ ((# …) …) …)
   831:21  8 (_ _)
In ravanan/work/vectors.scm:
     62:3  7 (vector-map->list _ _ . _)
In srfi/srfi-43.scm:
    376:5  6 (vector-fold #<procedure 7f2b838258a0 at ravanan/work/…> …)
In ice-9/boot-9.scm:
  1685:16  5 (raise-exception _ #:continuable? _)
  1683:16  4 (raise-exception _ #:continuable? _)
In ice-9/eval.scm:
   293:34  3 (_ #(#(#<directory (guile-user) 7f2ba26d8c80> ("p…")) #))
   182:19  2 (proc #(#(#<directory (guile-user) 7f2ba26d8c80> (#)) #))
   142:16  1 (compile-top-call #<directory (guile-user) 7f2ba26d8c80> …)
In unknown file:
           0 (%resolve-variable (7 . manifest-file-error?) #<directo…>)




https://www.commonwl.org/implementations/

I don't know if there's a database to test against, here they have the
notion of "self-reported compliance" but it seems unclear / case-by-case
to see where those compliance tests are implemented and if you can use
them as a base to test against.

I think those are pretty good examples of poor backtraces too.

Cheers,
Nicolas



On 2025-03-11 18:14, Arun Isaac wrote:

> Hi Nicolas,
>
> Thanks for trying out ravanan!
>
>> For some reason (script-module evaluates to #f), using ravanan, the
>> manifest needs to have a proper define-module (as opposed to the more
>> classic use-modules). As it's non-standard, it probably should be
>> documented.
>
> I do use manifest files with use-modules, that is, no define-module.
>
>> I manage to run a step but it blocks at that point :
>>
>> Running /gnu/store/wvi50qxx03ibvx39zam8704agw2z3y8a-download_and_extract_data
>> /gnu/store/wvi50qxx03ibvx39zam8704agw2z3y8a-download_and_extract_data failed; logs at /home/graves/proj
>> ects/src/competences_vertes_rncp/store/logs/wvi50qxx03ibvx39zam8704agw2z3y8a-download_and_extract_data.
>> stdout and /home/graves/projects/src/competences_vertes_rncp/store/logs/wvi50qxx03ibvx39zam8704agw2z3y8
>> a-download_and_extract_data.stderr
>>
>> Backtrace:
>> In ice-9/boot-9.scm:
>>   1752:10 13 (with-exception-handler _ _ #:unwind? _ # _)
>> In unknown file:
>>           12 (apply-smob/0 #<thunk 7fc0c8f132a0>)
>> In ice-9/boot-9.scm:
>>     724:2 11 (call-with-prompt _ _ #<procedure default-prompt-handle…>)
>> In ice-9/eval.scm:
>>     619:8 10 (_ #(#(#<directory (guile-user) 7fc0c8f19c80>)))
>>     619:8  9 (_ #(#(#(#(#(#(#(#(#(#(#(…) …) …) …) …) …) …) …) …) …) …))
>>    293:34  8 (_ #(#(#(#(#(#(#(#(#(#(#(…) …) …) …) …) …) …) …) …) …) …))
>> In ice-9/boot-9.scm:
>>     724:2  7 (call-with-prompt _ _ #<procedure 7fc0a9fcbc80 at ice-9…>)
>>   1752:10  6 (with-exception-handler _ _ #:unwind? _ # _)
>>     152:2  5 (with-fluid* _ _ _)
>>   1685:16  4 (raise-exception _ #:continuable? _)
>> In ice-9/eval.scm:
>>    293:34  3 (_ #(#(#<directory (guile-user) 7fc0c8f19c80> ("p…")) #))
>>    182:19  2 (proc #(#(#<directory (guile-user) 7fc0c8f19c80> (#)) #))
>>    142:16  1 (compile-top-call #<directory (guile-user) 7fc0c8f19c80> …)
>> In unknown file:
>>            0 (%resolve-variable (7 . manifest-file-error?) #<directo…>)
>>
>> ERROR: In procedure %resolve-variable:
>> error: manifest-file-error?: unbound variable
>> zsh: exit 1     ravanan workflow.cwl inputs.yaml --store=store
>> --guix-manifest=manifest.scm
>
> Could you share the ravanan command that resulted in this error? I'd
> also be interested in seeing the workflow.cwl, inputs.yaml and
> manifest.scm files, or some minimal version of them.
>
> And, btw, there is also a ravanan public-inbox you can write to at
> ravanan@systemreboot.net. https://lists.systemreboot.net/ravanan/
>
> Cheers!
> Arun

-- 
Best regards,
Nicolas Graves

Re: Some ravanan feedback

[Arun Isaac]

Hi Nicolas,

> https://github.com/nfdi4plants/cwl-examples
>
> and the following changes : 
>
> modified   run-python-script/test.cwl

I have fixed the issue with running this workflow. It was a bug in
canonicalizing File type inputs.
https://git.systemreboot.net/ravanan/commit/?id=9be0612b4b486f7d96ddd71c156be9284eff30b2
I have also added tests to catch this.
https://git.systemreboot.net/ravanan/commit/?id=af43d1fa1bafd8f9e641da16eca45e0795af7935

> @@ -3,8 +3,9 @@
>  cwlVersion: v1.2
>  class: CommandLineTool
>  hints:
> -  DockerRequirement:
> -    dockerPull: python:latest
> +  SoftwareRequirement:
> +    packages:
> +      python
>  baseCommand: python

The baseCommand needs to be changed to python3. If not, you can also use
the python-wrapper package.

With these changes, I run ravanan like so:

ravanan --store=store /tmp/cwl-examples/run-python-script/test.cwl
/tmp/cwl-examples/run-python-script/testRun.yml

and I get the following. It works.

--8<---------------cut here---------------start------------->8---
WARNING: (guix ui): imported module (guix diagnostics) overrides core binding `info'
WARNING: (ravanan command-line-tool): imported module (rnrs conditions) overrides core binding `&error'
/tmp/cwl-examples/run-python-script/test.py previously interned into store as /home/arun/Projects/ravanan/store/files/fc29e96825193827a8ab943f9201b8a935be4780-test.py
building path(s) `/gnu/store/9mqspgkknz3q55x7b12pg830anml975h-ca-certificate-bundle'
building path(s) `/gnu/store/8agn2i7ykqhbx4jkkiwnfxaknlkx4bsg-emacs-subdirs'
building path(s) `/gnu/store/h4k77l6xhrbwqwgin0bc5ljj5bxy19nz-fonts-dir'
building path(s) `/gnu/store/cvz9046gm0i2nqpr5nphsr7jg4hbw3ai-info-dir'
building path(s) `/gnu/store/xckk1ns8xy49m6m09cn4fx4k4hncrwwf-profile'
building path(s) `/gnu/store/4vrfghzrjd4kkkzr32yrjmq0jkvf2pfj-test'
Running /gnu/store/4vrfghzrjd4kkkzr32yrjmq0jkvf2pfj-test
/gnu/store/4vrfghzrjd4kkkzr32yrjmq0jkvf2pfj-test completed; logs at /home/arun/Projects/ravanan/store/logs/4vrfghzrjd4kkkzr32yrjmq0jkvf2pfj-test.stdout and /home/arun/Projects/ravanan/store/logs/4vrfghzrjd4kkkzr32yrjmq0jkvf2pfj-test.stderr
{
  "example_out": {
    "class": "File",
    "basename": "output.txt",
    "nameroot": "output",
    "nameext": ".txt",
    "size": 110,
    "checksum": "sha1$7c41bfd87ed9d66d64c44f433f7a7c40bec5d52f",
    "location": "file:///home/arun/Projects/ravanan/store/files/4vrfghzrjd4kkkzr32yrjmq0jkvf2pfj-test/output.txt",
    "path": "/home/arun/Projects/ravanan/store/files/4vrfghzrjd4kkkzr32yrjmq0jkvf2pfj-test/output.txt"
  }
}
--8<---------------cut here---------------end--------------->8---

> I happen to have such a backtrace :
>
> Backtrace:
> In ice-9/eval.scm:
>     619:8 19 (_ #(#(#(#(#(#(#(#(#(#(#(…) …) …) …) …) …) …) …) …) …) …))
>    293:34 18 (_ #(#(#(#(#(#(#(#(#(#(#(…) …) …) …) …) …) …) …) …) …) …))
> In ice-9/boot-9.scm:
>     724:2 17 (call-with-prompt ("prompt") #<procedure 7f2b83800180 …> …)
>   1752:10 16 (with-exception-handler _ _ #:unwind? _ # _)
>     152:2 15 (with-fluid* _ _ _)
> In ice-9/exceptions.scm:
>    406:15 14 (run-workflow _ _ _ _ _ _ _ _ #:guix-daemon-socket _)
> In ice-9/boot-9.scm:
>   1752:10 13 (with-exception-handler _ _ #:unwind? _ # _)
> In ravanan/workflow.scm:
>    592:26 12 (_)
>    293:29 11 (schedule _ _ _)
> In ravanan/command-line-tool.scm:
>    361:10 10 (run-command-line-tool "test" #f #f (("baseCom…" . #) …) …)
>    828:13  9 (build-command-line-tool-script "test" _ #f _ ((# …) …) …)
>    831:21  8 (_ _)
> In ravanan/work/vectors.scm:
>      62:3  7 (vector-map->list _ _ . _)
> In srfi/srfi-43.scm:
>     376:5  6 (vector-fold #<procedure 7f2b838258a0 at ravanan/work/…> …)
> In ice-9/boot-9.scm:
>   1685:16  5 (raise-exception _ #:continuable? _)
>   1683:16  4 (raise-exception _ #:continuable? _)
> In ice-9/eval.scm:
>    293:34  3 (_ #(#(#<directory (guile-user) 7f2ba26d8c80> ("p…")) #))
>    182:19  2 (proc #(#(#<directory (guile-user) 7f2ba26d8c80> (#)) #))
>    142:16  1 (compile-top-call #<directory (guile-user) 7f2ba26d8c80> …)
> In unknown file:
>            0 (%resolve-variable (7 . manifest-file-error?)
> #<directo…>)

Sorry the backtraces are terrible. I haven't yet invested much effort in
fixing them. These are early days for ravanan, I'm afraid.

> I don't know if there's a database to test against, here they have the
> notion of "self-reported compliance" but it seems unclear / case-by-case
> to see where those compliance tests are implemented and if you can use
> them as a base to test against.

The CWL conformance test suite is at
https://github.com/common-workflow-language/cwl-v1.2/tree/main/tests

Information on how to run them is at
https://github.com/common-workflow-language/cwl-v1.2/blob/main/CONFORMANCE_TESTS.md

I have run the test suite on my local machine. I need to wire this up
into the CI. I plan to do that in the coming weeks.

Regards,
Arun