Because not enough of the people here have their Lips on my weenie.



  • Originally posted here (following up on the threat I made earlier in that thread : "If someone doesn't either get this discussion back on topic, or lock the thread, I gonna start discussing the advantages and disadvantages of hygienic macro systems as applied to assembly-level kernel development. At length. And no one wants that.". They didn't, so I did.)


    OK, I've got the first part of my monologue on macros ready. Let me begin by defining some terms:

    A [i]macro definition[/i] (or simply a [i]macro[/i]) is a piece of code within a program which describes a set of transformations to be performed on another set of code prior to that code being translated (either compiled or
    interpreted).

    A [i]macro application[/i] is an instance of a macro being used to alter a section of code.

    The [i]macro expansion[/i] of a given macro application is the resulting code that is actual translated and executed.

    To give a simple example using the familiar C pre-processor, the following macro definition,
    [code]#define square(x) x * x[/code]
    is illustrative of a typical textual macro as seen in C, C++ and many assembly languages. It defines the name of the macro, a parameter, and a simple set of string changes to make to the argument. One could apply it to a piece of code, in this case a numerical literal,
    [code] a = square(42);[/code]
    and get the following macro expansion:
    [code] a = 42 * 42;[/code]
    Now, it has to be understood that this simple type of macro, called a textual macro, is simply performing a string interpolation on the argument; that is to say, it is replacing the the parameter with the text of the argument given to the macro application, and inserting it verbatim into the body of the macro, which then replaces the macro application. one could have written [code] a = square(48 - 6);[/code]
    and gotten the expansion
    [code] a = 48 - 6 * 48 - 6;[/code]
    The result of which is clearly not the same as the previous application, even though they would appear on the surface to be identical semantically. This highlights one of the flaws in naive macros of this sort, and introduces the most basic issue of [i]macro hygiene[/i]: avoiding duplicate evaluation.

    The macros possible with the C preprocessor are extremely limited; they do not compose effectively without heroic effort, they are generally restricted to a single line of code (though they can span multiple lines with a line extension) and they are rather simplistic in how they are expanded. They also represent something of an exceptional case for the C language, as they do not match the syntactic structures of the language overall. Macros, and the C pre-processor directives in general, form a separate mini-language independent from C itself, and indeed the pre-processor can be run separately from the compiler in most implementations.

    Similarly, the m4 macro processor, frequently used in conjunction with the Unix as assembler, is actually a stand-alone program in it's own right, and invoked automatically by the assembler. It is a significantly more sophisticated macro processor than CPP, but it is still basically a textual macro expansion.

    A key reason why these textual macros are limited is because they are separate from the languages they pre-process. However, it is possible, in languages which possess the property of [i]homoiconicity[/i] - that is to say, the code in which the language is written is itself a data structure of the language and can be manipulated programmatically within the code without needing to call out to a separate translator - can support a more powerful form of macro, known as [i]lexical macros[/i]. The classic examples of lexical macros are to be found in the various dialects of the Lisp family of languages.

    In Common Lisp. a macro is a piece of Lisp code the same as any other, except that the compiler recognizes that it is a macro when it scans it and calls the macro expander to perform the transformation at compile time, prior to compiling the code. To use the simple example from before,
    [code](defmacro square (x)
    `(* ,x ,x))[/code]
    As it happens, Common Lisp provides a simple means with which to check what the expansion of a given macro application would be, the [b]macroexpand-1[/b] special form. So, if we write
    [code](macroexpand-1 '(square 3))[/code]
    at the Lisp listener, we get
    [code](* 3 3)[/code]
    as expected. Unfortunately, we still haven't solved the duplicate evaluation problem, as shown here:
    [code] > (macroexpand-1 '(square (- 3 1)))
    (* (- 3 1) (- 3 1)) ;[/code]
    (the right angle bracket here is the listener prompt.) As it happens, this does [i]not[/i] have the consequences it did in C, thanks to a quirk of Lisp syntax: because Lisp functions are always fully parenthesized, the specific
    fault of mismatching the order of operations doesn't happen, though this doesn't mean we're free of the duplicate evaluation problem.

    If you look at the macro again, you should note the backquote and the commas; these are relevant, as they indicate what parts of the code should be evaluated at compile time and which at run time. To put it succinctly, a Lisp macro is a Lisp function that is run before compile time, with it's expansion being the function's output. The backquote says that the following part of the code should be passed through to the expansion unaltered, [i]except[/i] that it may contain elements which should be evaluated, which are indicated by the commas.
    Had we omitted these indicators,
    [code](defmacro square2 (x)
    (* x x))[/code]
    It would result in the whole code being evaluated a compile time,
    [code]

    (macroexpand-1 '(square2 3))
    9 ;
    (macroexpand-1 '(square2 (- 3 1)))

    *** - *: (- 3 1) is not a number[/code]
    You can see how, in the first case, it is can be a very useful thing to have the expression evaluated at compile time, as the expression essentially becomes a simple constant; but you can see in the second example the limitations of this, as the arguments passed to the macro are not evaluated at all.

    Because they have the full Common Lisp language available to them for generating their expansion, Common Lisp macros do not have many of the limitations that textual macros had. For example, one could write a macro that [i]does[/i] evaluate an arbitrary numeric argument to its square at compile time:
    [code]
    (defmacro square3 (x)
    (let ((y (eval x)))
    (if (numberp y)
    (* y y)
    nil)))[/code]
    which does indeed expand as desired:
    [code]> (macroexpand-1 '(square3 (- 3 1)))
    4 ;
    [/code]
    Furthermore, because it evaluates the value of [b]x[/b] prior to binding it to [b]y[/b] [i]before[/i] it is squared, it avoids re-evaluating the expression; no matter how complex the expression is, so long as the value is a number, it will be evaluated exactly once, at compile time. Of course, this still has its limitations: the expression must be evaluable at compile time, so it can't depend on any variables entered by the user at run time, for example. Still, this can be a useful property for certain applications.



  • I just really want to change the thread title to something about people not knowing about your fantastic weenie lips.

    I just keep looking at the title, thinking about changing it to that, and then laughing like a child.



  • Scheme. Hygeine. (sytax-rules) uh, rules...


  • Winner of the 2016 Presidential Election Banned

    @ScholRLEA said:

    languages which possess the property of homoiconicity - that is to say, the code in which the language is written is itself a data structure of the language and can be manipulated programmatically within the code without needing to call out to a separate translator

    So, what you're saying is, Skynet is a Lisp program.

    But seriously, that was a rather interesting read. I've never used (or even seen code from) Lisp, but it sounds like it might have some interesting features


  • Notification Spam Recipient

    Fun fun? Words and things, brain set to process during Deep Dream tonight...



  • What use do macros have over functions? Is this anything like what C++'s templates are used for?

    template<class T> T square3(T x) { return x*x; }


  • Thanks. Very educational, and I've been wondering what use macros are in lisp.



  • I find it interesting that self-modifying code is virtually never seen today.



  • @anonymous234 said:

    self-modifying code is virtually never seen today

    except every single time anybody launches something that isn't statically linked.



  • Short-circuit evaluation is one possible use
    Consider: macroSS(simpleTest(arg), complexTest(arg));
    If macroSS is a macro, it can evaluate simpleTest(arg) and conditionally skip complexTest(arg) altogether, whereas if it is a function, both simpleTest and complexTest will be evaluated.



  • @MZH said:

    What use do macros have over functions? Is this anything like what C++'s templates are used for?

    Depends on the language and what you need them for. In C, and most assembly languages that have macros, the main things they would be used for are either to name an operation that is to run inline, or to insert code or declarations rather than a value. For example, you could have a macro like this:

    #define gen_list_type(typename, value_type) \
        struct typename { \
            value_type datal; \
            typename *prev, *next; \
        }
    
    gen_list_type(int_node, int data);
    

    which would expand to

    struct int_node { 
        int data;
        int_node *prev, *next; 
    }
    

    Because C macros are really terrible about macro hygiene, they are rarely used in such a manner, though you'll see quite a few of this kind in the Linux kernel. C++ goes out of it's way to reduce the need for macros, with the cconst type modifier, inline function hints, templates, and so forth; a lot of the things you would use macros for in C would be better done with templates in C++. Templates can be seen (and are usually implemented) as a specialized type of macro, with type-safety and other advantages; the restrictions on them actually make them safer and hence more useful. It is

    In Lisp, macros become something else entirely; in fact, much of the 'standard syntax' of most Lisp languages is actually composed of macros. For example, the (unless) form might be implemented as

    (defmacro unless (state &rest body)
       `(if (not ,state)
            (progn
              ,@body)
            nil)))
    

    Which for the application

     (unless (eq foo baz)
       (bar 23 17)
       (quux 'frob))
    

    expands to

    (if  (not (eq foo baz))
         (progn
            (bar 23 17)
            (quux 'frob))
         nil))
    

    This is still a really simple example; a more elaborate example (from the book On Lisp, ch. 15) shows a macro that returns a new function consisting of the composition of two or more functions (with three utility functions to simplify it):

    (defmacro fn (expr) ‘#’,(rbuild expr))
    
    (defun rbuild (expr)
        (if (or (atom expr) (eq (car expr) ’lambda))
            expr
            (if (eq (car expr) ’compose)
                (build-compose (cdr expr))
                (build-call (car expr) (cdr expr)))))
    
    (defun build-call (op fns)
      (let ((g (gensym)))
        ‘(lambda (,g)
          (,op ,@(mapcar #’(lambda (f)
                                        ‘(,(rbuild f) ,g))
                                   fns)))))
    
    (defun build-compose (fns)
      (let ((g (gensym)))
        ‘(lambda (,g)
          ,(labels ((rec (fns)
                         (if fns
                           ‘(,(rbuild (car fns))
                             ,(rec (cdr fns)))
                           g)
              (rec fns)))))
    

    I can explain the details of this one later if you want, but the practical upshot is that for the application

    (fn and integerp oddp)
    

    it generates the function form

    (#`lambda (x) (and (integerp x) (oddp( x)))
    

    (Actually, the variable x would in this case be a gensym, an automatically generated variable whose name is a uniquely generated symbol that cannot be directly referenced. It is Common Lisp's solution for certain hygiene issues with regards to variable capture.)

    I've got to go somewhere right now, but I can get back to this later if you want.



  • @anonymous234 said:

    I find it interesting that self-modifying code is virtually never seen today.

    Arguably runtime linkers are exactly this. The app I work on behaves differently based on which client-specific apps are linked into it, which is decided in an XML configuration file.

    Anyway, it's not seen much because it's frankly not that useful to anybody outside of AI research.



  • @ScholRLEA, I look forward to the actual discussion of hygienic, but crippled, R⁵RS Scheme (define-syntax), powerful, but unfortunately unhygienic D's mixins, the failure of Rust's $crate kludge and, of course, actually discussing their use for generating assembly code.


    Filed Under: And of course any other macro system I haven't heard of yet.



  • Quick macro example from some Tinyscheme code I happen to have lying around. I decided that it'd be nice to have prefix ++ and -- operators in Scheme, like C/C++ have. (Bonus question for reader: why can't I have postfix operators?) Anyway, they're pretty easy to implement:

    (define-macro (++ i) `(set! ,i (+ ,i 1)))
    (define-macro (-- i) `(set! ,i (- ,i 1)))
    

    And a quick test run:

    ts> (define x 10)
    x
    ts> (++ x)
    11
    ts> (-- x)
    10
    

    Filed under: macros are great!



  • @tar said:

    Bonus question for reader: why can't I have postfix operators?

    You can have their semantics though.



  • @Bulb said:

    any other macro system I haven't heard of yet.

    Nemerle has pretty decent macro capabilities.



  • And how hygienic is it?



  • Hygienic, in that I believe there's a notation system for defining the scopes of any terms involved in the macros...

    (Look for the discussion of "color" in the context of quasi-quotation and splicing here: https://github.com/rsdn/nemerle/wiki/Nemerle-macros-intro)



  • Hundreds of words and not a single example of how to solve an actual problem.

    Yup, that's LISP.


  • Java Dev

    At least show us 99 bottles or something to show how awesome your language is.


  • BINNED

    What about a self-modifying code in Python[1] to add goto to it?

    [1] I loved the implementation, and can think of other evil things one can do with that approach


  • BINNED

    I cannot believe I read this (and liked it too)


  • BINNED

    @Bulb said:

    And of course any other macro system I haven't heard of yet.



  • @tar said:

     https://github.com/rsdn/nemerle/wiki/Nemerle-macros-intro

    That's compiler extensions. Compiler extensions can do almost anything, but they are unwieldy.

    @dse said:

    self-modifying code in Python

    Python is suspiciously malleable for a language intended to use one true way of doing things. But that is good, because it makes it usable. This is firmly in the hack land though (and the other implementation that also includes comefrom even more so).

    It does not really count as macro though, because it manipulates bytecode, not AST.

    @antiquarian said:

    http://docs.factorcode.org/content/article-macros.html

    Um, that documentation is really incomprehensible, at least without having some grasp of the syntax already. Additionally obfuscated by the nonobvious click-throughs.


  • BINNED

    @Bulb said:

    Um, that documentation is really incomprehensible, at least without having some grasp of the syntax already.

    So you expect to understand documentation of an advanced language feature without even knowing the most basic syntax? You may want to see a doctor about that. :trollface:



  • Maybe that's just in elisp, but I've been using the 1+ and 1- operator for that...

    (setq x 10)
    (1+ x)
    11
    


  • Lisp macroes are fine I guess, but they are nothing compared to my homemade templating system for node.js.

    // Main tools library
    
    function Tools() {}
    module.exports = new Tools();
    
    // Basic tools
    //@>
    $pn.moduleRequire("./tools/basic");
    $pn.moduleExports("./tools/basic", "module.exports.%s = ");
    //<@
    
    

    "expands" into

    // Main tools library
    
    function Tools() {}
    module.exports = new Tools();
    
    // Basic tools
    var libToolsBasic = require("./tools/basic");
    module.exports.format = libToolsBasic.format;
    module.exports.inspect = libToolsBasic.inspect;
    module.exports.path = libToolsBasic.path;
    module.exports.dateUTC = libToolsBasic.dateUTC;
    module.exports.inherits = libToolsBasic.inherits;
    module.exports.inheritEventEmitter = libToolsBasic.inheritEventEmitter;
    

    Watch the beauty of it and feel the bitter envy bubbling in your veins!


  • BINNED

    @cartman82 said:

    Watch the beauty of it and feel the bitter envy bubbling in your veins!

    That would have worked had you posted something that isn't possible using Lisp macros. :trollface:


  • BINNED

    @ScholRLEA said:

    Fair enough; I even said as much at the start of the rant, I just didn't think to use 'reply as linked topic' at the time. Call it a brain-fart I guess - it is a feature few other fora have, so I'm not used to using it. If the mods want to relocate it to a new thread, I have no problem with that.

    Well, you do have two Lisp threads, one currently active.



  • So far this is probably the best lisp thread around. I still remember mine:

    @me said:

    Hey guys, I wanna learn Lisp! I don't feel like also learning Emacs, so anyone know any proper IDEs that work with Lisp?

    @Everyone else said:

    Use Emacs.

    @me said:

    Hey cool, Eclipse seems to work somewhat. Except only on Windows, because it's looking for something rooted in C: on Linux.

    @Everyone else said:

    Use Emacs.



  • @antiquarian said:

    That would have worked had you posted something that isn't possible using Lisp macros. :trollface:

    EVERYTHING IS POSSIBLE USING LISP MACROS!!1
    FILED UNDER: MY PHONE HAS A CAPS LOCK KEY!!1


  • Notification Spam Recipient

    @tar said:

    FILED UNDER: MY PHONE HAS A CAPS LOCK KEY!!1
    I always wondered why double-tapping Shift did that. Spent quite some time discovering that Sticky Keys is apparently forever disconabled.



  • I actually have a physical keyboard on my phone. I'd send a photo of it, but both of my phone cameras face away from the keyboard...

    Filed under: I only use the software keyboard when I want to misspell words and make myself cross.


  • BINNED

    @Everyone else said:

    Use Emacs.

    I thought that was just me.

    Granted, I didn't do a very good job of explaining why at the time, but since you're here I'll try again:

    There is a small community of Lisp experts on comp.lang.lisp. (There are also some trolls, but you post here so you should be accustomed to that.) Most of them have been using Lisp for years if not decades, and some of them have even managed to make a living doing it. Here's the important part. If you're using Eclipse and run into problems, none of them will be able to help you because they are all either using Emacs or the IDE that comes with their commercial implementation (which also works like Emacs).



  • So it's been around for like 30 years, and literally nobody has EVER made an IDE for it?


  • BINNED

    Did you miss this part?

    @antiquarian said:

    they are all either using Emacs or the IDE that comes with their commercial implementation (which also works like Emacs).

    Or is your real question "why are they using Emacs"?



  • You said the IDE is the same thing as Emacs, and since Emacs isn't an IDE, I assumed what you meant is that it's an IDE in name only.

    But whatever. LISP is shit. Nobody involved in LISP has ever done anything practical with it, ever. It's all just abstract pointless rants like the OP in this thread which don't even bother to even slightly attempt to solve real-life problems.


  • BINNED

    @blakeyrat said:

    Emacs isn't an IDE

    For the sake of this argument, try to imagine a parallel universe in which things that don't work in exactly the way you expect can be used effectively and aren't buggy, broken pieces of shit. In this parallel universe, Emacs effectively is an IDE for Lisp because it can do many of the same things with Lisp code that an IDE would do.

    @blakeyrat said:

    Nobody involved in LISP has ever done anything practical with it, ever.

    ITA Software would like to have a word with you.

    http://franz.com/success/customer_apps/data_mining/itastory.php3

    Filed under: yet another blakeyfact



  • @antiquarian said:

    In this parallel universe, Emacs effectively is an IDE for Lisp because it can do many of the same things with Lisp code that an IDE would do.

    I can do many of the same things a dog does. I am not a dog.

    ITA Software selected Allegro CL as their original development environment. "We use Lisp for the high level structure, in conjunction with a variety of other languages such as C and Java throughout the application."

    LISP is so great you can't write a practical application with it without adding in some Java!!!!!



  • @antiquarian said:

    Emacs effectively is an IDE

    At what point is something an IDE and not just a text editor? When it has a build button? Can't you shell out commands in most text editors?

    Is this like the difference between a "programming language" and a "scripting language"?


  • BINNED

    @Bort said:

    At what point is something an IDE and not just a text editor? When it has a build button?

    The common workflow for Lisp coding using Emacs is to start a REPL (read-eval-print loop) in one of the buffers and load the code you're working on and any dependencies into the REPL. There's a built-in command that will load a file, which means that it basically evaluates everything into the file into the REPL. So all of your code is in the REPL at this point. Then after changing a function in a source code file, there's a command to execute the function definition in the REPL, effectively loading the new version into it. So it's a bit like a build button, but not exactly.

    https://www.youtube.com/watch?v=BKFcznZ28BE

    @Bort said:

    Is this like the difference between a "programming language" and a "scripting language"?

    Yes.

    @blakeyrat said:

    LISP is so great you can't write a practical application with it without adding in some Java!!!!!

    :moving_goal_post:


  • Discourse touched me in a no-no place

    @anonymous234 said:

    I find it interesting that self-modifying code is virtually never seen today.

    Except that arguably all scripting languages are entirely made of self-modifying code.



  • @blakeyrat said:

    Emacs isn't an IDE

    It isn't? Pray tell, what do you find missing for Emacs to be an IDE?

    I'll not go into your Lisp-troll, since you are obviously wrong there. You are right in that the lisp community doesn't get out much, and many of the applications developed in lisp have not seen the widespread adoption of other non-lisp applications, but that's neither here nor there.



  • I would say it lacks an integrated debugger, but I'm sure you're going to bring up some extremely shitty plug-in that kind of makes kind of a sort of text-based debugger with almost no features and say I'm wrong.

    @Mikael_Svahnberg said:

    You are right in that the lisp community doesn't get out much,

    I just want someone to promote LISP but telling me how it solves an actual problem in reality. Instead of all the abstract abstractness.



  • Even i can step in here somewhat. Lisp variants have debugging built into them, and you'd normally do that in the repl. Yes, it's text based, so you'd probably hate it, but it's better than most languages get even so.


  • Discourse touched me in a no-no place

    Highly interactive languages (of which Lisp is a member) need debuggers much less, because they can always just use the language itself. This in turn creates all sorts of challenges, but debugging is a very different experience. It's pretty common to just start an interactive session, hack away until you've done the operation you want, then cut-n-paste that session into a file before editing it into a program. Which is actually much faster to do and a lot more intuitive than it sounds like from this description. 😄

    Some languages (e.g., Oberon did this) go as far as making the saved program be the dumped interactive session. I think that's going too far though; you get the saved mistakes as well…



  • Yes, but that sounds like 10% of what debugging is like in lisp, so why even mention it?


  • Discourse touched me in a no-no place

    @Magus said:

    Yes, but that sounds like 10% of what debugging is like in lisp, so why even mention it?

    Well, someone's got to put the bugs into the Lisp code first!



  • @blakeyrat said:

    I would say it lacks an integrated debugger, but I'm sure you're going to bring up some extremely shitty plug-in that kind of makes kind of a sort of text-based debugger with almost no features and say I'm wrong.

    @Mikael_Svahnberg said:

    You are right in that the lisp community doesn't get out much,

    I just want someone to promote LISP but telling me how it solves an actual problem in reality. Instead of all the abstract abstractness.

    It does have a great built-in debugger for lisp, and plenty of debuggers for other languages. Yes, they are text based. No, that is not a bad thing.

    Lisp solves actual problems just the same way as C/C++, c#, java, or any other programming language does. By enabling the programmer.

    I find it interesting that many "modern" trends in progamming and progamming languages are builtin into lisp. For example, callback programming that is all the rage in node.js? It's done right with hooks in lisp. Aspect Oriented Programming? defadvice in lisp. Event-driven languages, such as erlang and (I think, I have not actually looked) Go, that do have their place at least in the telecom industry? That's the default programming mindset in lisp. Rule-engines (is it Bizhub that Ms touts?)? Ditto. Namespaces? No such luck, but there are surprisingly few conflicts in all the packages that you can download (and that comes shipped by default) in Emacs, and, because of the hooks (see earlier) they can integrate nicely with each other.


  • Notification Spam Recipient

    @dkf said:

    just start an interactive session, hack away until you've done the operation you want, then cut-n-paste that session
    I do this regularly, across most of my languages.
    Just put down some boilerplate code, slap a breakpoint in, and muck around in Immediate Mode (or whatever is the equivalent). Once something has been accomplished, stop the debugger (Why can't I make changes to the code while running? It's broken, right?), rinse and repeat. @CodingHorrorBot?


    Filed under: So that's why they call it spaghetti code. Mmm! Delicious!


Log in to reply