tokens are removed after the lexical phase.
may not contain a LineTerminator
. Since any token is as long as possible, this means SingleLineComment
are parsed to the end of the line. The LineTerminator
at the end will not be part of the Comment though.
that contains at least one LineTerminator
is not discarded from the token stream but replaced by a LineTerminator Token
Comments are allowed to contain any character except for a LineTerminator
. MultiLineComment tokens can not contain an embedded */
sequence. This is not affected by the greedy rule (take the longest sequence possible) because the productions are crafted thusly that it cannot occur.
In my personal oppionion, the production MultiLineComment
could be done much easier. Read here
for more information about that.