Fontification Problems with alltt

[Jan T. Kim]

Dear AucTeXers,

while writing a LaTeX file with frequently interspersed code
fragments typeset in the alltt environment, I experience that
AucTeX has a tendency to somehow get confused about whether it's
"inside" or "outside" an alltt environment. The visual effect of
this is that everything is fontified as though it was in alltt,
except the portions inside an alltt environment.

Switching font-lock-mode off and on again fixes the problem, but
it eventually returns.

Unfortunately, I am not able to systematically reproduce this
effect. I have the impression that the chances of this happening
get higher as I work more on the end of the text, but it occasionally
happens even with very short documents.

This happens on emacs-21.3-12, running on a Fedora core 2 installation,
but I've also seen it on my notebook which is a recently installed
Debian system.

Is this a known problem, is there anything that can be done about this?

Kind regards,
Jan T. Kim

[Ralf Angeli]

Post by Jan T. Kim
while writing a LaTeX file with frequently interspersed code
fragments typeset in the alltt environment, I experience that
AucTeX has a tendency to somehow get confused about whether it's
"inside" or "outside" an alltt environment. The visual effect of
this is that everything is fontified as though it was in alltt,
except the portions inside an alltt environment.
Switching font-lock-mode off and on again fixes the problem, but
it eventually returns.

[...]

Post by Jan T. Kim
This happens on emacs-21.3-12, running on a Fedora core 2 installation,
but I've also seen it on my notebook which is a recently installed
Debian system.
Is this a known problem, is there anything that can be done about this?

There was a similar report a few days ago. It is likely Emacs not
updating the font locking correctly. Do you still see the problem
with a development version of Emacs?

[Jan T. Kim]

Post by Ralf Angeli

Post by Jan T. Kim
while writing a LaTeX file with frequently interspersed code
fragments typeset in the alltt environment, I experience that
AucTeX has a tendency to somehow get confused about whether it's
"inside" or "outside" an alltt environment. The visual effect of
this is that everything is fontified as though it was in alltt,
except the portions inside an alltt environment.
Switching font-lock-mode off and on again fixes the problem, but
it eventually returns.

[...]

Post by Jan T. Kim
This happens on emacs-21.3-12, running on a Fedora core 2 installation,
but I've also seen it on my notebook which is a recently installed
Debian system.
Is this a known problem, is there anything that can be done about this?

There was a similar report a few days ago. It is likely Emacs not
updating the font locking correctly. Do you still see the problem
with a development version of Emacs?

I've just installed emacs-21.4a, dated 17-Feb-2005 on
http://ftp.gnu.org/pub/gnu/emacs/ , hope that's sufficiently "bleeding
edge"...

Anyway, after editing the file in question with that a little, I haven't
seen the effect again, but as I don't have any way to reliably trigger
it, I don't know how much this means.

It's a bit strange, though, that with auctex-11.55, installed from the
source, my alltt stuff is fontified in black (default colour), whereas
with emacs-21.3, and auctex-11.55-1 installed from Debian packages, it's
fontified in brown (SaddleBrown, as far as I can tell from M-x
customize-face font-latex-verbatim-face). Therefore, I'm not certain
whether I would even notice the problem, as it has manifested itself by
painting everything *outside* the alltt environments in brown.

I don't know if this variation of fontification colour schemes is due
to the version of emacs, or perhaps caused by alterations made by the
maintainers of the Debian package (and the same change made by the
RPM maintainer too).

Best regards, Jan

[David Kastrup]

Post by Jan T. Kim

Post by Ralf Angeli

Post by Jan T. Kim
while writing a LaTeX file with frequently interspersed code
fragments typeset in the alltt environment, I experience that
AucTeX has a tendency to somehow get confused about whether it's
"inside" or "outside" an alltt environment. The visual effect of
this is that everything is fontified as though it was in alltt,
except the portions inside an alltt environment.
Switching font-lock-mode off and on again fixes the problem, but
it eventually returns.

[...]

Post by Jan T. Kim
This happens on emacs-21.3-12, running on a Fedora core 2 installation,
but I've also seen it on my notebook which is a recently installed
Debian system.
Is this a known problem, is there anything that can be done about this?

There was a similar report a few days ago. It is likely Emacs not
updating the font locking correctly. Do you still see the problem
with a development version of Emacs?

I've just installed emacs-21.4a, dated 17-Feb-2005 on
http://ftp.gnu.org/pub/gnu/emacs/ , hope that's sufficiently "bleeding
edge"...

It isn't. Apart from two fixes(?), it is the same as 21.3, and that
is just bugfixes over 21.1. No bleeding edge at all.

[Ralf Angeli]

[fontification bleeding out of alltt environment]

Post by Jan T. Kim

Post by Ralf Angeli
There was a similar report a few days ago. It is likely Emacs not
updating the font locking correctly. Do you still see the problem
with a development version of Emacs?

I've just installed emacs-21.4a, dated 17-Feb-2005 on
http://ftp.gnu.org/pub/gnu/emacs/ , hope that's sufficiently "bleeding
edge"...

There haven't been any changes to the font locking code in this
release compared to Emacs 21.3. So I'd say no. It would have to be a
CVS snapshot or checkout. But of course installing and using such a
snapshot is not everybody's cup of tea.

Post by Jan T. Kim
Anyway, after editing the file in question with that a little, I haven't
seen the effect again, but as I don't have any way to reliably trigger
it, I don't know how much this means.
It's a bit strange, though, that with auctex-11.55, installed from the
source, my alltt stuff is fontified in black (default colour), whereas
with emacs-21.3, and auctex-11.55-1 installed from Debian packages, it's
fontified in brown (SaddleBrown, as far as I can tell from M-x
customize-face font-latex-verbatim-face). Therefore, I'm not certain
whether I would even notice the problem, as it has manifested itself by
painting everything *outside* the alltt environments in brown.

If an alltt environment is not fontified in brown you either forgot to
activate parsing or something else went wrong. In the latter case you
will probably find some hint in the *Messages* buffer.

It is probably more reliable to check for correct fontification with
the verbatim environment instead of the alltt environment because the
former does not require an additional AUCTeX style file ot be loaded.
In case the verbatim environment is fontified but not the alltt
environment it is likely you forgot to enable parsing as mentioned
above.

[Jan T. Kim]

Post by Ralf Angeli
[fontification bleeding out of alltt environment]

Post by Jan T. Kim

Post by Ralf Angeli
There was a similar report a few days ago. It is likely Emacs not
updating the font locking correctly. Do you still see the problem
with a development version of Emacs?

I've just installed emacs-21.4a, dated 17-Feb-2005 on
http://ftp.gnu.org/pub/gnu/emacs/ , hope that's sufficiently "bleeding
edge"...

There haven't been any changes to the font locking code in this
release compared to Emacs 21.3. So I'd say no. It would have to be a
CVS snapshot or checkout. But of course installing and using such a
snapshot is not everybody's cup of tea.

Ok -- I have compiled Emacs 22.0.50.1 from CVS. Thus far, I have not
been able to trigger the bug again, but this may not mean much, as I
don't know a reliable way to trigger it. Here are some potentially
relevant observations, though. The fontification does get inversed
(i.e. it's not just "bleeding out of the alltt environment",
everything is brown outside and content inside alltt is multicoloured,
depending on commands) if

* the alltt environment is empty

* \begin{alltt} is followed by a backslash and a newline.

Both these effects also occur with the 21.3 version, although sometimes
very slowly (which is why I haven't noticed this pattern before -- the
CVS version seems to be substantially faster) and are usually reversed
upon adding stuff.

Now, assuming for the moment that upgrading to 22.0.50.1 fixes the
problem: Does that mean that this upgrade is the only way to get
AUCTeX working correctly? Can you give me any recommendations on how
to work this? Any recommendations?

Best regards & thanks in advance, Jan

[Ralf Angeli]

Post by Jan T. Kim
Ok -- I have compiled Emacs 22.0.50.1 from CVS. Thus far, I have not
been able to trigger the bug again, but this may not mean much, as I
don't know a reliable way to trigger it. Here are some potentially
relevant observations, though. The fontification does get inversed
(i.e. it's not just "bleeding out of the alltt environment",
everything is brown outside and content inside alltt is multicoloured,
depending on commands) if
* the alltt environment is empty
* \begin{alltt} is followed by a backslash and a newline.
Both these effects also occur with the 21.3 version, although sometimes
very slowly (which is why I haven't noticed this pattern before -- the
CVS version seems to be substantially faster) and are usually reversed
upon adding stuff.

In the case the environment is empty, the fontification code places
syntax properties for marking the start and end of the verbatim
content onto the same point in the buffer and this confuses the rest
of the code. In the case of a backslash at the end of the first line,
the escape syntax of the backslash overrides the syntax property
marking the start of the verbatim content.

These constellations usually are non-permanent, so it should not be a
problem if Emacs updates the display reliably. Obviously Emacs 21
does not get this right fast/reliable enough. I don't think we can do
much about it by changing something in AUCTeX.

Post by Jan T. Kim
Now, assuming for the moment that upgrading to 22.0.50.1 fixes the
problem: Does that mean that this upgrade is the only way to get
AUCTeX working correctly? Can you give me any recommendations on how
to work this? Any recommendations?

You can force an update of fontification with `M-x
font-lock-fontify-buffer RET'. In case the wrong fontification
happens often to you, you can bind `font-lock-fontify-buffer' to a
keyboard shortcut.