[Larceny-users] [long] identifier bindings and visibility across phases

[Marco Maggi]

Ciao,

I am trying to implement an expansion time feature for my
R6RS libraries; basically it works like this, using
identifier macros:

(define-record-extension <string>
(fields (length string-length #f)
(upcase string-upcase #f)
(dncase string-downcase #f)))

(let ((S "Ciao"))

(with-virtual-fields ((length <string> S))
length)
; => 4

(with-virtual-fields (((upcase dncase) <string> S))
(list upcase dncase))
; => ("CIAO" "ciao"))

I have prepared an archive of test files[1] with everything
in it, which can be downloaded from the "Downloads" page of
Nausicaa[2].

As the tests show, the basic feature works when the record
extension is in the same file that makes use of it; but
there is a problem when the extension is defined in a
library ("lib-test-2.sls").

With the test in "test.sps":

(with-virtual-fields ((def <beta> o))
def)

I get the following error:

------------------------------------------------------------

Syntax violation: invalid reference

No binding available for beta-def-ref in library (...program...)

Form: beta-def-ref

Trace:

(beta-def-ref the-record)

def

(lambda (the-record)
(let-syntax
((def (identifier-syntax
(_ (beta-def-ref the-record))
((set! _ e) (beta-def-set! the-record e)))))
(with-virtual-fields () (set! def '(90 91)) def)))

((lambda (the-record)
(let-syntax
((def (identifier-syntax
(_ (beta-def-ref the-record))
((set! _ e) (beta-def-set! the-record e)))))
(with-virtual-fields () (set! def '(90 91)) def)))
o)

(let-syntax
((dummy (lambda (stx)
(syntax-case
stx
()
((_ ?kontext)
(let ((RTD (record-type-descriptor <beta>)))
(with-syntax
(((EXPR) (datum->syntax #'?kontext '(o)))
((VAR ...) (datum->syntax #'?kontext '(def)))
((ACCESSOR ...)
`(,(%virtual-field-accessor '<beta> RTD 'def)))
((MUTATOR ...)
`(,(%virtual-field-mutator '<beta> RTD 'def)))
((FORMS ...)
(datum->syntax
#'?kontext
'((with-virtual-fields () (set! def '(90 91)) def)))))
#'(let ((the-record EXPR))
(let-syntax
((VAR (identifier-syntax
(_ (ACCESSOR the-record))
((set! _ e) (MUTATOR the-record e))))
...)
FORMS
...)))))))))
(dummy <beta>))

(with-virtual-fields
((((def def)) <beta> o))
(with-virtual-fields () (set! def '(90 91)) def))

(with-virtual-fields
(((def) <beta> o))
(set! def '(90 91))
def)

(write (with-virtual-fields
(((def) <beta> o))
(set! def '(90 91))
def))

(lambda (o)
(write (with-virtual-fields ((def <beta> o)) def))
(newline)
(write (with-virtual-fields
(((def) <beta> o))
(set! def '(90 91))
def))
(newline))

((lambda (o)
(write (with-virtual-fields ((def <beta> o)) def))
(newline)
(write (with-virtual-fields
(((def) <beta> o))
(set! def '(90 91))
def))
(newline))
(make <beta> 1 2 3 4 5 6))


Error: no handler for exception #<record &compound-condition>
Compound condition has these components:
#<record &who>
who : "invalid reference"
#<record &message>
message : "No binding available for beta-def-ref in library (...program...)"
#<record &syntax>
form : beta-def-ref
subform : #f

Terminating program execution.

------------------------------------------------------------

the test fully works with Ikarus and Ypsilon (and probably
it will work with Mosh, too, when Issue 102 is solved); only
Larceny fails, so I understand it is a phasing problem. I
am not able to understand how to change the import
specifications of the libraries to make the code work; I am
still learning how phasing works...

Anyway, in the definition of DEFINE-RECORD-EXTENSION I do:

(datum->syntax #'?kontext '?accessor)

where, IMHO, "?kontext" holds the correct context
informations to make everything work, whatever the context
in which the output form of WITH-VIRTUAL-FIELDS ends up to
be.

What is wrong?

TIA

[1] <http://cloud.github.com/downloads/marcomaggi/nausicaa/virtual-fields.tar.gz>
[2] <http://github.com/marcomaggi/nausicaa/downloads>

[Marco Maggi]

I have prepared an archive of test files[1] with
everything in it, which can be downloaded from the
"Downloads" page of Nausicaa[2].

I admit that there is too much code in the archive, so I
reduced it to the following:

(library (sublib)
(export %bury %disinter)
(import (rnrs))
(define *objects*
(make-eq-hashtable))
(define (%bury key obj)
(hashtable-set! *objects* key obj))
(define (%disinter key)
(hashtable-ref *objects* key #f)))

(library (lib)
(export define-thing thing)
(import (rnrs) (for (sublib) expand))
(define-syntax define-thing
(lambda (stx)
(syntax-case stx ()
((_ ?key ?id)
(begin
(%bury (syntax->datum #'?key) #'?id)
#'(define dummy #f))))))
(define-syntax thing
(lambda (stx)
(syntax-case stx ()
((_ ?key)
(with-syntax ((obj (%disinter (syntax->datum #'?key))))
#'(obj)))))))

(library (def)
(export)
(import (rnrs) (lib))
(define (woppa) 123)
(define-thing woppa woppa))

;;;; the program follows
(import (rnrs) (def) (lib))
(write (thing woppa))
(newline)

which works for Ikarus and (somewhat) for Mosh, while it
fails with Ypsilon (my original code worked with Ypsilon,
too); with Larceny, running the program prints:

------------------------------------------------------------
Syntax violation: invalid reference

No binding available for woppa in library (...program...)

Form: woppa

Trace:

(woppa)

(thing woppa)

(write (thing woppa))


Error: no handler for exception #<record &compound-condition>
Compound condition has these components:
#<record &who>
who : "invalid reference"
#<record &message>
message : "No binding available for woppa in library (...program...)"
#<record &syntax>
form : woppa
subform : #f

Terminating program execution.
------------------------------------------------------------

now, DEFINE-THING stores a syntax object into a hashtable,
and THING retrieves it; by changing the code in the (sublib)
library to:

(define (%bury key obj)
(write 'bury)(newline)
(hashtable-set! *objects* key obj))
(define (%disinter key)
(write 'disinter)(newline)
(hashtable-ref *objects* key #f)))

I see that "bury" and "disinter" are printed, so the error
seems to come from bad insertion of the disinterred syntax
object in the output form of the THING macro use. Why is it
so?

TIA
--
Marco Maggi

[Andre van Tonder]

WOPPA is not exported from library DEF, so any attempt to refer to it from
outside DEF should fail. The behavior of Larceny and Ypsilon are correct,
and that of Ikarus and Mosh are incorrect.

Post by Marco Maggi

I have prepared an archive of test files[1] with
everything in it, which can be downloaded from the
"Downloads" page of Nausicaa[2].

I admit that there is too much code in the archive, so I
(library (sublib)
(export %bury %disinter)
(import (rnrs))
(define *objects*
(make-eq-hashtable))
(define (%bury key obj)
(hashtable-set! *objects* key obj))
(define (%disinter key)
(hashtable-ref *objects* key #f)))
(library (lib)
(export define-thing thing)
(import (rnrs) (for (sublib) expand))
(define-syntax define-thing
(lambda (stx)
(syntax-case stx ()
((_ ?key ?id)
(begin
(%bury (syntax->datum #'?key) #'?id)
#'(define dummy #f))))))
(define-syntax thing
(lambda (stx)
(syntax-case stx ()
((_ ?key)
(with-syntax ((obj (%disinter (syntax->datum #'?key))))
#'(obj)))))))
(library (def)
(export)
(import (rnrs) (lib))
(define (woppa) 123)
(define-thing woppa woppa))
;;;; the program follows
(import (rnrs) (def) (lib))
(write (thing woppa))
(newline)
which works for Ikarus and (somewhat) for Mosh, while it
fails with Ypsilon (my original code worked with Ypsilon,
------------------------------------------------------------
Syntax violation: invalid reference
No binding available for woppa in library (...program...)
Form: woppa
(woppa)
(thing woppa)
(write (thing woppa))
Error: no handler for exception #<record &compound-condition>
#<record &who>
who : "invalid reference"
#<record &message>
message : "No binding available for woppa in library (...program...)"
#<record &syntax>
form : woppa
subform : #f
Terminating program execution.
------------------------------------------------------------
now, DEFINE-THING stores a syntax object into a hashtable,
and THING retrieves it; by changing the code in the (sublib)
(define (%bury key obj)
(write 'bury)(newline)
(hashtable-set! *objects* key obj))
(define (%disinter key)
(write 'disinter)(newline)
(hashtable-ref *objects* key #f)))
I see that "bury" and "disinter" are printed, so the error
seems to come from bad insertion of the disinterred syntax
object in the output form of the THING macro use. Why is it
so?
TIA
--
Marco Maggi
_______________________________________________
Larceny-users mailing list
https://lists.ccs.neu.edu/bin/listinfo/larceny-users

[Andre van Tonder]

Post by Andre van Tonder
WOPPA is not exported from library DEF, so any attempt to refer to it from
outside DEF should fail. The behavior of Larceny and Ypsilon are correct,
and that of Ikarus and Mosh are incorrect.

Reconsidering, I think this might be underspecified in R6RS, so both behaviours
might be fine.

Andre

[Marco Maggi]

Post by Andre van Tonder

Post by Andre van Tonder
WOPPA is not exported from library DEF, so any attempt to refer to it from
outside DEF should fail. The behavior of Larceny and Ypsilon are correct,
and that of Ikarus and Mosh are incorrect.

Reconsidering, I think this might be underspecified in R6RS, so both
behaviours might be fine.

Thanks, I see that changing (def) to:

(library (def)
(export woppa)
(import (rnrs) (lib))
(define (woppa) 123)
(define-thing woppa woppa))

makes everything work; are there relevant paragraphs in the
R6RS documents? It seems to me that the chapter on
SYNTAX-CASE says nothing about non-exported bindings in
syntax objects.

But, if it is an export problem, why the following works?
BLUE is not exported by (sublib):

(library (sublib)
(export h)
(import (rnrs))
(define (blue)
456)
(define h
(make-eq-hashtable))
(hashtable-set! h 'b blue))

(library (lib)
(export thing)
(import (rnrs) (for (sublib) expand run))
(define-syntax thing
(lambda (stx)
(syntax-case stx ()
((_)
(with-syntax ((f (hashtable-ref h 'b #f)))
#'('f)))))))

;;; the program
(import (rnrs) (lib))
(write (thing))

is it a specific interaction between exporting and syntax
objects? The following works with Ikarus, Mosh and Ypsilon;
it fails only on Larceny:

(library (sublib)
(export h)
(import (rnrs))
(define (blue)
456)
(define h
(make-eq-hashtable))
(hashtable-set! h 'b (syntax blue)))

(library (lib)
(export thing)
(import (rnrs) (for (sublib) expand run))
(define-syntax thing
(lambda (stx)
(syntax-case stx ()
((_)
(with-syntax ((f (hashtable-ref h 'b #f)))
#'(f)))))))

;;; the program
(import (rnrs) (lib))
(write (thing))

[Marco Maggi]

Post by Marco Maggi
But, if it is an export problem, why the following works?
(library (sublib)
(export h)
(import (rnrs))
(define (blue)
456)
(define h
(make-eq-hashtable))
(hashtable-set! h 'b blue))
(library (lib)
(export thing)
(import (rnrs) (for (sublib) expand run))
(define-syntax thing
(lambda (stx)
(syntax-case stx ()
((_)
(with-syntax ((f (hashtable-ref h 'b #f)))
#'('f)))))))
;;; the program
(import (rnrs) (lib))
(write (thing))

Okay, that is a stupid example... Anyway, why is it

Post by Marco Maggi
(library (sublib)
(export h)
(import (rnrs))
(define (blue)
456)
(define h
(make-eq-hashtable))
(hashtable-set! h 'b (syntax blue)))
(library (lib)
(export thing)
(import (rnrs) (for (sublib) expand run))
(define-syntax thing
(lambda (stx)
(syntax-case stx ()
((_)
(with-syntax ((f (hashtable-ref h 'b #f)))
#'(f)))))))
;;; the program
(import (rnrs) (lib))
(write (thing))

? Creating a syntax object is done exactly to maintain the
lexical context informations, no?

[Andre van Tonder]

Post by Marco Maggi
But, if it is an export problem, why the following works?
(library (sublib)
(export h)
(import (rnrs))
(define (blue)
456)
(define h
(make-eq-hashtable))
(hashtable-set! h 'b blue))
(library (lib)
(export thing)
(import (rnrs) (for (sublib) expand run))
(define-syntax thing
(lambda (stx)
(syntax-case stx ()
((_)
(with-syntax ((f (hashtable-ref h 'b #f)))
#'('f)))))))

This should not work (portably) at all.

You are constructing a syntax object with an embedded procedure object.
That has no portable meaning in R6RS. Although certain implementations
might allow it for their own inscrutable purposes, it would really be
better for an implementation to raise a syntax error in cases like
these.

[Andre van Tonder]

Post by Marco Maggi
The following works with Ikarus, Mosh and Ypsilon;
(library (sublib)
(export h)
(import (rnrs))
(define (blue)
456)
(define h
(make-eq-hashtable))
(hashtable-set! h 'b (syntax blue)))
(library (lib)
(export thing)
(import (rnrs) (for (sublib) expand run))
(define-syntax thing
(lambda (stx)
(syntax-case stx ()
((_)
(with-syntax ((f (hashtable-ref h 'b #f)))
#'(f)))))))
;;; the program
(import (rnrs) (lib))
(write (thing))

This looks like the same issue as previously.
The identifier BLUE is never exported from SUBLIB,
and so should not be available outside SUBLIB.

The fact that it works with other implementations
are likely an accident of their specific
implementation technique.

[Marco Maggi]

WOPPA is not exported from library DEF, so any attempt
to refer to it from outside DEF should fail. The
behavior of Larceny and Ypsilon are correct, and that of
Ikarus and Mosh are incorrect.

Reconsidering, I think this might be underspecified in
R6RS, so both behaviours might be fine.

Last note before giving up; the following works for Ikarus,
Larceny, Mosh and Ypsilon:

(library (subsublib)
(export blue)
(import (rnrs))
(define (blue) 123))

(library (sublib)
(export h)
(import (rnrs) (for (subsublib) (meta -1)))
(define h
(make-eq-hashtable))
(hashtable-set! h 'b (syntax blue)))

(library (lib)
(export thing)
(import (rnrs) (for (sublib) expand))
(define-syntax thing
(lambda (stx)
(syntax-case stx ()
((_)
(with-syntax ((f (hashtable-ref h 'b #f)))
#'(f)))))))

(import (rnrs) (lib))
(write (thing))

so the fact that the identifier has to be imported in the
library defining the macro transformer is not the whole
story...

[Andre van Tonder]

Post by Marco Maggi
Last note before giving up; the following works for Ikarus,
(library (subsublib)
(export blue)
(import (rnrs))
(define (blue) 123))
(library (sublib)
(export h)
(import (rnrs) (for (subsublib) (meta -1)))
(define h
(make-eq-hashtable))
(hashtable-set! h 'b (syntax blue)))
(library (lib)
(export thing)
(import (rnrs) (for (sublib) expand))
(define-syntax thing
(lambda (stx)
(syntax-case stx ()
((_)
(with-syntax ((f (hashtable-ref h 'b #f)))
#'(f)))))))
(import (rnrs) (lib))
(write (thing))
so the fact that the identifier has to be imported in the
library defining the macro transformer is not the whole
story...

Very interesting.

Maybe r7rs can be persuaded to address this kind of thing. It should certainly
be raised for discussion when the time comes.

Until then, I don't think this kind of thing can be expected to
work portably across implementations, or even consistently within a
given implementation, since use cases similar to this can subvert
the normal static determination of import dependencies.

I would consider the permissiveness of all the above implementations
to be a bug, but it would be interesting to hear different points of
view.

Andre