Issue EXTERNAL-FORMAT-FOR-EVERY-FILE-CONNECTION Writeup

Status:       MINUMUM proposal accepted 4/1/92

Issue:        EXTERNAL-FORMAT-FOR-EVERY-FILE-CONNECTION

Forum:        Cleanup

References:   OPEN, WITH-OPEN-FILE, LOAD, COMPILE-FILE, STREAM-EXTERNAL-FORMAT

Category:     EXTEND

Edit history: 10-Jan-92, Version 1 by Masa Ida

              14-Jan-92, Version 2 ibid. 

                (correct a typo. add Editorial Impact field. Add Discussion.)

              1-Apr-92, Version 3 by Barrett

                (compile-file-pathname, similarity of constants constraint)

              2-Apr-92, Version 4 by Barrett (amendments from meeting)


Problem Description:


        OPEN and WITH-OPEN-FILE have :EXTERNAL-FORMAT keyword

        to specify the external format of files, but other file

        connecting interfaces like LOAD have no such keyword.

        As a result uniform handling of external formats are

        impossible with the current draft.


Proposal (EXTERNAL-FORMAT-FOR-EVERY-FILE-CONNECTION:MINIMUM);


        Add the :EXTERNAL-FORMAT keyword to LOAD and COMPILE-FILE.

        The default value is :DEFAULT as in OPEN.  This argument specifies the

        character representation scheme to be used by LOAD and COMPILE-FILE

        when reading a source file, in the same manner as for OPEN.


        This changes the syntax descriptions to

                Compile-file input-pathname &key :output-file :verbose

                        :print :external-format

                Load filename &key :verbose :print :if-does-not-exist

                        :external-format


        Add to the descriptions of COMPILE-FILE and LOAD a cross reference to

        (or copy of) the description of the :EXTERNAL-FORMAT argument for OPEN.


        The :EXTERNAL-FORMAT argument is ignored by LOAD when opening a

        compiled file.  Instead, COMPILE-FILE and LOAD must cooperate (using

        some implementation-dependent mechanism) in such a way that similarity

        of constants is maintained across the interface between these two

        functions.


Editorial Impact:

        Trivial. If STREAM-EXTERNAL-FORMAT description will refer

        the usage of :EXTERNAL-FORMAT in the future, editorial consideration

        will be needed.


Test Case:


        (LOAD "MY-PROG.LISP" :EXTERNAL-FORMAT :NIHONGO-EUC)

        will load the file MY-PROG.LISP which contains nihongo-euc

        coded characters defined by UI-OSF joint japanization definition.


        (COMPILE-FILE "YY.CL")

        will compile the file YY.CL with the default understanding.


Rationale:

        Uniform handling of external formats are enabled by

        adopting this proposal.


Current Practice:

        There is no way to specify the coding scheme for

        loading/compiling files.

        Several Japanese commercial CL implementations use

        this scheme.


Cost to Implementors:

        If implementation supports :default only, the cost is negligible.


Cost to Users:

        The existing japanized software with their own tricks for

        converting the codes must rewrite the interface to the

        system supplied one to have a portability.


Cost of Non-Adoption:

        Incompatibility with the Jeida Guideline draft.


Benefits:

        Performance up.

        Compatible definition.


Aesthetics:

        unified syntax.



Proposal (EXTERNAL-FORMAT-FOR-EVERY-FILE-CONNECTION:JEIDA-GUIDELINE);


        Only :DEFAULT is defined for the value of :EXTERNAL-FORMAT now.

        Other possible selections are not defined.

        Add the following selections for :external-format value.

                :nihongo-jis  file contains JIS X0202 encoding JIS characters.

                :nihongo-euc  file contains nihongo-euc defined by

                              the joint convention defined by UI-OSF.

                :nihongo-ms   file contains Microsoft Kanji Code.


Editorial Impact:

        Trivial. If STREAM-EXTERNAL-FORMAT description will refer

        the usage of :EXTERNAL-FORMAT in the future, editorial consideration

        will be needed.


Test Case:

        See the Test Case for MINIMUM


Rationale:

        Every practical conversion scheme is provided with this definition.


Current Practice:

        Users write their own codes for conversion. Or

        Several CL cannot handle japanese characters.


Cost to Implementors:

        Requiers the supporting mechanisms/libraries for japanese

        character handling.


Cost to Users:


Cost of Non-Adoption:

        Incompatibility with the Jeida Guideline draft.


Benefits:

        By adopting this proposal, implementations can have a portable

        way for japanization.


Aesthetics:

        If there is consideration for other nations,

        say, we can add :arabic-XXX, :chinese-XXX, ...


Discussion:

        Masa Ida recomended to have the MINIMUM proposal.

        The proposal JEIDA-GUIDELINE seems to be too sudden for X3J13

        and may require more detailed definition.

        The proposal JEIDA-GUIDELINE seems to be not needed for USA domestic

        Applications.


        Date: Mon, 13 Jan 92 11:16:40 EST

        From: kab@cambridge.apple.com (Kim Barrett)

        I don't yet have an informed opinion on the proposal (after the above

        bug is fixed), but my initial reaction is that this certainly seems

        plausible.  I couldn't think of any operators besides the two mentioned

        that implicitly call OPEN.


        Date: Mon, 13 Jan 1992 16:37-0500

        From: Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>

        I do believe this is an important issue.  I think it's nice for us to

        try to cater to the legitimate technical needs of the international

        community in order to better position CL for acceptance in the

        international arena, both in the standards arena and in the commercial

        arena.  I hope that at the appropriate time we will give it due

        consideration.

        My personal opinion is that the proposal looks solid (modulo the

        obvious typo--missing "not"--mentioned by Kim Barrett) and worth doing,

        both for general consistency and because I can easily see where the

        absence of this feature would lead to some serious problems.


        From: Larry Masinter <masinter@parc.xerox.com>

        Date:    Mon, 13 Jan 1992 15:49:14 PST

        I like the :MINIMUM proposal, but I would be wary of including the

        JEIDA guideline without also including other values for

        :EXTERNAL-FORMAT.

        There is apprently good progress in the ISO 10646 standard. (The DIS-2

        ISO 10646 was supposed to be issued January 1992 for a 4-month

        vote...)

        Even if we listed interpretations for :external-format that included

        JIS, EUC and Microsoft Kanji codes, it would seem to be important to

        include a better reference for those coding schemes, since they tend

        to get revised from time to time. 

        It might be that the standard should not include these keywords any

        more than a listing of features, but that some other informal or

        formal consortium of implementors could keep a registry of keyword

        assignments.