Greetings! I have a very small request...

Right now, there are a limited amount of colors that we can use to markup posts. In addition to these, I would like to see arbitrary RGB values allowed - something like:

[ff0000]this text would be red[/ff0000]

or

[ff0000]this text would be red[/color]

This would be very easy to implement, taking 15 minutes to your time.

I have written a script that automatically syntax highlights over 300 languages and file types for this message board. However, the full spectrum of colors must be mapped down to the limited colors available on this board, so the results are sometimes very ugly. The script would be far more useful and powerful if this mapping was not necessary, if arbitrary RGB values were accepted.

A secondary, but less important request: it would be nice to be able to set the background color of CODE blocks.

Thanks!
Eric

If you just don't have the time to do this, I will gladly add it myself. I would really like this feature. You could just send me the module responsible for parsing post to convert style-code to HTML, and I would make the necessary changes. Language is unimportant.

Cheers,
Eric

Thanks for your offer, I will implement it by my self as soon as I got time

If you want to contribute with your syntax-coloring, then send me an email because that would be really cool to have here.

Regarding the syntax I think something like

this text would be red would be more
suitable.

What do you think?


/WEBMASTER

: Thanks for your offer, I will implement it by my self as soon as I got time
:
: If you want to contribute with your syntax-coloring, then send me an email because that would be really cool to have here.

Well, my syntax highlighting script is for the editor VIM. The script queries VIM for highlighting information, which it then uses to markup the text with Programmer's Heaven style-codes. With the correct command line arguments, you could use VIM to markup text from a CGI script. Of course, this would require an extension the CODE tag so that you could specify which language to use for highlighting. Something like:

or

[code language=pascal]

: Regarding the syntax I think something like 
: 
: this text would be red would be more
: suitable.
:
: What do you think? 

That's fine, though a little verbose. Some other alternatives...

[#ff0000][/color]

or simply:

[#ff0000][/#]


These are just as easy to parse (if not easier), requires less typing, and are still unlikely to conflict with valid language constructs (unlike some style codes, like 'strikethrough' and 'bold').

In the end, it doesn't make much difference to me, as my editor will be doing the markup automatically.

Cheers,
Eric


P.S.  I'm sure you aware that some people's code is being incorrectly marked up when an expression in their code is mistaken for style-code.  For instance, this was a common problem with [i] until you changed the style-code for italic to [italic].  However, this did not solve the problem for other style-codes like [x] and [b].  I see this all the time in code like:

for( int x=0; x<10; ++x )
   cout << myarray[x] << '\n'

One way to prevent this in the future, without needed to modify the syntax of style-code, and without requiring style-code characters to be 'escaped' in normal text, would be to only markup text if the closing tag is found.  In the case of the code snippet above, [x] would not be mistaken for a style-code because there is no closing [/x] tag.

I today rewrote the message rendering code and added support for any color and and code background.

So, now you can do this


Programmers Heaven


by using the [ color=xxxxxx].....[/color] tag, where the xxxxxx is the RGB HEX values.

Also, i also now support the use of
[ code=xxxxxx]
Where you can set the default background color of a code block, like

case "blue"
    if closed=false then
        strBuf = strBuf & "<font color=""Blue"">"
    else
        strBuf = strBuf & "</font>"
    end if

/WEBMASTER

Awesome! You rock dude!

Here's an example of my script highlighting some Java code, with two different colorization schemes...

import javax.swing.text.*;
import javax.swing.undo.*;
import javax.swing.event.*;
import javax.swing.*;

/**
 * Sample application using the simple text editor component that
 * supports only one font.
 */
class Notepad extends JPanel {

    private static ResourceBundle resources;

    static {
        try {
            resources = ResourceBundle.getBundle("resources.Notepad",
                                                 Locale.getDefault());
        } catch (MissingResourceException mre) {
            System.err.println("resources/Notepad.properties not found");
            System.exit(1);
        }
    }

    Notepad() {
  super(true);

  // Force SwingSet to come up in the Cross Platform L&F
  try {
      UIManager.setLookAndFeel(UIManager.getCrossPlatformLookAndFeelClassName());
      // If you want the System L&F instead, comment out the above line and
      // uncomment the following:
      // UIManager.setLookAndFeel(UIManager.getSystemLookAndFeelClassName());
  } catch (Exception exc) {
      System.err.println("Error loading L&F: " + exc);
  }

import javax.swing.text.*;
import javax.swing.undo.*;
import javax.swing.event.*;
import javax.swing.*;

/**
 * Sample application using the simple text editor component that
 * supports only one font.
 */
class Notepad extends JPanel {

    private static ResourceBundle resources;

    static {
        try {
            resources = ResourceBundle.getBundle("resources.Notepad", 
                                                 Locale.getDefault());
        } catch (MissingResourceException mre) {
            System.err.println("resources/Notepad.properties not found");
            System.exit(1);
        }
    }

    Notepad() {
  super(true);

  // Force SwingSet to come up in the Cross Platform L&F
  try {
      UIManager.setLookAndFeel(UIManager.getCrossPlatformLookAndFeelClassName());
      // If you want the System L&F instead, comment out the above line and
      // uncomment the following:
      // UIManager.setLookAndFeel(UIManager.getSystemLookAndFeelClassName());
  } catch (Exception exc) {
      System.err.println("Error loading L&F: " + exc);
  }

That's nice!

What I need now is a ASP script or COM component that could do real realtime syntaxcoloring!


/WEBMASTER

Well, you could easily use VIM as an external text filter. Let's say you encounter:

[code=ff0000 lang=fortran]
... text ...
[/code]

You gather up the text and pass it off to VIM along with the language name. VIM would markup the text with the appropriate style-code and return it to you. You could feed the text to VIM via stdin, and receive the altered text via stdout (like a CGI app), or you could just save it the text to a temp file and pass the filename to VIM.

I've not had the occasion to use ASP before, but if it requires external code to reside in a COM component, we could easily write one that exported a single conversion function and took care of invoking VIM...

  string markupText( in string text, in string language )

Of course, spawning external processes to do work is not as fast as memory resident server extensions, but that wouldn't really be an issue here. This would only need to be done once with each new post.

Once this was in place, Programmer's Heaven would be able to automatically syntax highlighting for over 300 filetypes!

abaqus, abc, abel, acedb, ada, aflex, ahdl, amiga, aml, antlr, apache, apachestyle, asm, asm68k, asmh8300, asn, aspperl, aspvbs, atlas, automake, ave, awk, ayacc, b, basic, bc, bdf, bib, bindzone, blank, btm, c, catalog, cf, cfg, ch, change, changelog, cl, clean, clipper, cobol, colortest, conf, config, cpp, crontab, csc, csh, csp, css, cterm, ctrlh, cupl, cuplsim, cvs, cweb, cynlib, cynpp, dcl, debchangelog, debcontrol, def, diff, diva, dns, docbk, dosbatch, dosini, dot, dracula, dsl, dtd, dtml, dylan, dylanintr, dylanlid, ecd, eiffel, elf, elmfilt, erlang, esqlc, eterm, expect, exports, fgl, focexec, form, forth, fortran, foxpro, fvwm, gdb, gdmo, gedcom, gnuplot, gp, groff, gsp, gtkrc, haskell, hb, help, hercules, hitest, hog, html, htmlm4, htmlos, ia64, icon, idl, idlang, indent, inform, inittab, ishd, iss, ist, jam, jargon, java, javacc, javascript, jess, jgraph, jproperties, jsp, kix, kscript, kwt, lace, latte, lex, lftp, lhaskell, lilo, lisp, lite, lotos, lout, lprolog, lss, lua, lynx, m4, mail, make, man, manual, maple, masm, mason, master, matlab, mel, mf, mgp, mib, mma, model, modsim3, modula2, modula3, mp, msql, mush, muttrc, named, nasm, nastran, natural, ncf, nosyntax, nqc, nroff, objc, ocaml, omnimark, openroad, opl, ora, papp, pascal, pcap, pccts, perl, pfmain, php, phtml, pic, pike, pilrc, pine, plm, plsql, po, pod, postscr, pov, povini, ppwiz, procmail, progress, prolog, psf, ptcap, purifylog, python, qf, r, radiance, ratpoison, rc, rcslog, readline, rebol, registry, remind, rexx, robots, rpcgen, rtf, ruby, samba, sas, sather, scheme, screen, sdl, sed, sgml, sgmldecl, sgmllnx, sh, sicad, simula, sinda, sindacmp, sindaout, skill, sl, slang, slrnrc, slrnsc, sm, smil, smith, sml, snnsnet, snnspat, snnsres, snobol4, spec, specman, spice, spup, sql, sqlforms, sqr, squid, st, stp, strace, syncolor, synload, syntax, tads, tags, tak, takcmp, takout, tasm, tcl, terminfo, tex, texinfo, texmf, tf, tidy, tli, trasys, tsalt, tsscl, tssgm, tssop, types, uc, uil, vb, verilog, vgrindefs, vhdl, vim, viminfo, virata, vrml, vsejcl, wdiff, web, webmacro, wget, whitespace, winbatch, wml, wsh, xdefaults, xkb, xmath, xml, xpm, xpm2, xs, xxd, yacc, z8a, zsh

Personally, it doesn't make a difference to me. My editor already does this for me. But if you want to add this for everyone elses benefit, I could help you out with the VIM part of it, supplying the conversion script and the appropriate command-line. If you think it would be feasible (you know more about your website setup than I do), I could even write the COM component for you.

Cheers,
Eric

Hi

calling processes/execute programs externally from the server for rendering the code sounds like a to much resource-intensive task for the server. The rendering of the posts is also done in realtime to get the best flexibility.

Of course it could be done once when posting, but that is not a very flexible solution because you will get problems if you got bugs in the rendering and you want to re-render the messages. Its quite difficult to "remove" the rendering from the original post. And storing two versions of the same message (one un-rendered and one renderend) is not a solution either. So it got to be realtime with low overhead.

I agree that it could be a very simple way of doing it, but perhaps not the best way for a high-volume WEB-site. It needs to be a COM-component or a ASP script to do this efficiently.

It could also be a simple thing to do if I moved to .NET (that I hope to be able to do in the future).

I think making a parser would be the simplest solution at the moment.



/WEBMASTER

: calling processes/execute programs externally from the server for rendering the code sounds like a to much resource-intensive task for the server. The rendering of the posts is also done in realtime to get the best flexibility.

I assume 'rendering in realtime' means that style-code is converted to HTML each time the page is served, rather than once when the message is posted. In that case, yeah, it would be a bit resource intensive, mostly because VIM does not perform the conversion very quickly (can take over a second if the code is long enough).

On the other hand, while spawning external processes is not ideal, it's not as bad as you think. After all, this is how all dynamic content was generated for most of the Web's history (i.e. CGI).

: Of course it could be done once when posting, but that is not a very flexible solution because you will get problems if you got bugs in the rendering and you want to re-render the messages. Its quite difficult to "remove" the rendering from the original post. And storing two versions of the same message (one un-rendered and one renderend) is not a solution either. So it got to be realtime with low overhead.

Well, clearly it doesn't have to be (no one is holding a gun to your head), but I agree that it is the most elegant way to do it.

: I agree that it could be a very simple way of doing it, but perhaps not the best way for a high-volume WEB-site. It needs to be a COM-component or a ASP script to do this efficiently.

Even with the overhead of spawning an external process, it would still be faster to do the conversion with a native app than with an ASP script. I mean, ASP uses ActiveScript, right? I've not used ASP before, but I'm familar with the Microsoft's JScript implementation. It's extremly fast - orders of magnitude faster than the Mozilla implementation of ECMA-262. Yes it is still far, far slower than native code.

Another alternative would be to turn VIM into a COM component (it is open source), but that's a lot of work. Too much for me anyway. ;) Like I said, I already have this feature via my editor.

: It could also be a simple thing to do if I moved to .NET (that I hope to be able to do in the future).

Why, does .NET come with built in syntax highlighting libraries. ;)

: I think making a parser would be the simplest solution at the moment.

Writing a C++ parser is the 'simplest solution'? OK. Good luck!

If you're going to do the parsing yourself, I would not recommend writing full-blown, grammar based language parsers - this is overkill, and would make the cost of adding support for new languages prohibitively high. I would do what VIM does - use regular expressions to get "close enough". That makes it easy to add new languages, which is precisely why VIM supports so many.

Cheers,
Eric

: On the other hand, while spawning external processes is
: not ideal, it's not as bad as you think. After all, this
: is how all dynamic content was generated for most of
: the Web's history (i.e. CGI).
:
I know that , but still, its much more resource intensive than a ASP/COM solution.


:
: Even with the overhead of spawning an external process,
: it would still be faster to do the conversion with a native
: app than with an ASP script. I mean, ASP uses ActiveScript,
: right? I've not used ASP before, but I'm familar with the
: Microsoft's JScript implementation. It's extremly
: fast - orders of magnitude faster than the Mozilla
: implementation of ECMA-262. Yes it is still far, far slower
: than native code.

I have found ASP to be quite fast, its not as fast (of course) as a COM object or native code, but the IIS is caching the interpreted ASP scripts , so its quite fast. I think a good start could be to do some simpler color syntax thing to get started ,like making the comments in one color and the rest in a another color.

I guess I need to do some regular expression hacking to get this done. I am using that to render the messages today.


:
: Another alternative would be to turn VIM into a COM
: component (it is open source), but that's a
: lot of work. Too much for me anyway. ;) Like I said, I
: already have this feature via my editor.

That sounds like to much work, its not worth that effort. but I am sure there must be some open-source or public domain sourcecode for a tool like that.

:
: Why, does .NET come with built in syntax highlighting libraries. ;)

No, but in .NET you can create more advanced WEB-solution/parsers more easily. doing COM is just a lot of trouble, COM is quite a nightmare to develop, test, register, unregister, restarting the WEB-server each time....


/WEBMASTER

: I think a good start could be to do some simpler color syntax thing to get started ,like making the comments in one color and the rest in a another color.

If you do move in that direction - please make it optional. ;)

: I guess I need to do some regular expression hacking to get this done. I am using that to render the messages today.

If you made your regular expression based highlighting system compatible with VIM, then you could take advantage of VIM's plethora of syntax definitions.

Come to think of it, VIM is extremely modular - the syntax highlighting portion of the code is totally optional, and can be easily #defined out of a build. So the highlighting code might be easy to hack out and turn into a standalone highlighting component. I'll look into it tonight.

: That sounds like to much work, its not worth that effort. but I am sure there must be some open-source or public domain sourcecode for a tool like that.

Not really. I've seen one that supports C++ and Java, but it uses full-blown language parsers (via LEX and YACC), so extending it to support other languages is non-trivial (writing a whole new scanner-parser!).

: doing COM is just a lot of trouble, COM is quite a nightmare to develop, test, register, unregister, restarting the WEB-server each time....

Why do you have to test the component in the web server? You write the object first in a regular, stand-alone app, where testing is easy. Perfect it there. You then "COM-ify" it, testing it with another stand-alone host app. MSVC++ registers the component for you automatically each time you re-build, so it's pretty painless. Only after the component is verified rock-solid do you move it over the the web server.

Yes, writing the COM-glue from scratch can be painful - but ATL makes it much easier, and MFC makes it trivial. Besides, you really only need a COM object that exposes one method "markup( code, language )". Not too hard. ;)

Hmmm... as long as you were going to pass part of the text to a native app, you might as well just pass all of it. Something like "StyleCodeToHTML(buffer)". If I wrote such a thing, would you be willing to try it out?

Cheers,
Eric

: Hmmm... as long as you were going to pass part of the text to a native app, you might as well just pass all of it. Something like "StyleCodeToHTML(buffer)". If I wrote such a thing, would you be willing to try it out?
:

Sure, I always appreciate all kinds of contributions and you will of course get full credits for that on the site.

I would require to get access to the sourcecode to make sure its not a trojan horse or something I don't want to run strange COM objects that I don't trust, I hope you understand that.

But I think before you do anything that we have to lookaround for existing solutions. PErhaps there are some Perl/PHP stuff that can be converted? There's no point in inventing the wheel again.




/WEBMASTER

: : Hmmm... as long as you were going to pass part of the text to a native app, you might as well just pass all of it. Something like "StyleCodeToHTML(buffer)". If I wrote such a thing, would you be willing to try it out?
: :
:
: Sure, I always appreciate all kinds of contributions and you will of course get full credits for that on the site.
:
: I would require to get access to the sourcecode to make sure its not a trojan horse or something I don't want to run strange COM objects that I don't trust, I hope you understand that.

I understand completely.

: But I think before you do anything that we have to lookaround for existing solutions. PErhaps there are some Perl/PHP stuff that can be converted? There's no point in inventing the wheel again.

Sure. Of course, a Perl implementation would be no better than invoking VIM, right? Anyway, I'll look around.

Cheers,
Eric

: I today rewrote the message rendering code and added support for any color and and code background.
:
: So, now you can do this
:
:
: Programmers Heaven
:
:
: by using the [ color=xxxxxx].....[/color] tag, where the xxxxxx is the RGB HEX values.
:
: Also, i also now support the use of
: [ code=xxxxxx]
: Where you can set the default background color of a code block, like
:
:

: case "blue"
:     if closed=false then
:         strBuf = strBuf & "<font color=""Blue"">"
:     else
:         strBuf = strBuf & "</font>"
:     end if
: 

:
:
:
:
: /WEBMASTER
:
:

Very cool! Thanks a million!

: : I today rewrote the message rendering code and added support for any color and and code background.
: :
: : So, now you can do this
: :
: :
: : Programmers Heaven
: :
: :
: : by using the [ color=xxxxxx].....[/color] tag, where the xxxxxx is the RGB HEX values.
: :
: : Also, i also now support the use of
: : [ code=xxxxxx]
: : Where you can set the default background color of a code block, like
: :
: :

: : case "blue"
: :     if closed=false then
: :         strBuf = strBuf & "<font color=""Blue"">"
: :     else
: :         strBuf = strBuf & "</font>"
: :     end if
: : 

: :
: :
: :
: :
: : /WEBMASTER
: :
: :
:
: Very cool! Thanks a million!
:

 testing 

testing

Ignore this post

 if this post works this is damn cool man

: : : I today rewrote the message rendering code and added support for any color and and code background.
: : :
: : : So, now you can do this
: : :
: : :
: : : Programmers Heaven
: : :
: : :
: : : by using the [ color=xxxxxx].....[/color] tag, where the xxxxxx is the RGB HEX values.
: : :
: : : Also, i also now support the use of
: : : [ code=xxxxxx]
: : : Where you can set the default background color of a code block, like
: : :
: : :

: : : case "blue"
: : :     if closed=false then
: : :         strBuf = strBuf & "<font color=""Blue"">"
: : :     else
: : :         strBuf = strBuf & "</font>"
: : :     end if
: : : 

: : :
: : :
: : :
: : :
: : : /WEBMASTER
: : :
: : :
: :
: : Very cool! Thanks a million!
: :
: testing
: testing
: Ignore this post
:
:

:  if this post works this is damn cool man
: 

: