NAML Feedback

[DavidJCobb]

I just released an update for the NAML on Reaching Perfection. As I developed it, I noted things to ask about or suggest; here are the notes I came up with:

PROBLEM: Strings that contain null

If a string contains line breaks in addition to <n.null/>, that string becomes a ticking time bomb. The standard null-handling macros -- even binary ones like "is_null" and "default" -- will fail to detect the null, but it will still cause errors if the value is written to the page or passed to any string-handling macro.

The situation that triggered this bug for me was a macro that output <n.null/> deliberately if a set of exception checks failed. Even though the macro used the "unindent" attribute, the line breaks contained in its code were still returned along with the null. The caller stored the return value in a var, essentially making that var "toxic" in the manner I have described.

I worked around the bug by simply wrapping the processing parts of the null-returning macro in a call to "no_output", such that no line breaks were produced. However, it does seem as though "is_null" and friends ought to be able to detect strings that contain null, in addition to values that are null and only null.

Simplified test case: <n.default to="No bug" text="[n.crlf/][n.null/]"/>

PROBLEM: Some HTML tags are not recognized

THEAD and TFOOT are not recognized as valid tags, and will trigger errors if used.

SUGGESTION: Arbitrary node lists

I couldn't find any way to persistently store an arbitrary node list, or to easily convert something similar (i.e. a string of comma-separated IDs) into one.

For the Portfolio Index, I used a string list stored as a property on the root node. I rigged up a macro that allowed me to paginate the list, but since I'm not using a real node list, I can't sort the nodes by their subjects, last post dates, or anything that a real node list could offer.

This is more of a nice-to-have thing than an urgent feature.

QUESTION about translations

I noticed that some strings are wrapped in "T" tags, and after doing some research, I learned that this is part of your translation system. My question is, if I use "T" tags to wrap a string that doesn't have a translation, would that cause errors?

(My thinking is that if others want to use any features I've created with NAML, wrapping the strings in "T" tags would allow them to translate them without having to manually edit the macros.)

SUGGESTION: Reading cookies from NAML

I don't know how feasible this would be, but I can see a few uses for reading and writing cookies from NAML.

For example, one might allow mobile users to view a "gateway" page, which sets a cookie to signal that the user is on a mobile device. Then, NAML code could be used to check the cookie and tailor certain pieces of output for mobile devices (i.e. no embedding redirection since many mobiles don't support IFRAMEs, and writing a SCRIPT tag for jQuery Mobile instead of standard jQuery). Granted, it'd probably be better to build an entirely separate part of the app for mobiles, rather than just tweaking a desktop-oriented app, but the latter could serve as a decent step toward the former.

Much of what this idea would accomplish could be done through JavaScript alone. Doing it through NAML may make the code cleaner, though. Nice-to-have, but not essential.

PROBLEM: Error pages don't escape tags

On the Error 500 page, angle brackets in the stack trace are escaped when the trace is written to the page's TITLE, but they are not escaped in the page's BODY. This means that one must view the page source to see the trace.

SUGGESTION: Control over unhandled exception output

If an exception occurs and is caught with handle_exception, but there is no n.exception call for that specific exception name, then a message is printed at the bottom of the handle_exception block, noting the unhandled exception and its name.

This is a very convenient feature, but it would be nice to have more control over the output (just in case some unanticipated exception occurs in a final product rather than on a test app). Perhaps it would be possible to add macros like "has_unhandled_exception" and "print_unhandled_exception" for this?

[DavidJCobb]

Also, today I discovered that the "filter" parameter on some of the node list macros (i.e. "children_list") actually takes a perfectly normal string with a special syntax. Two of the binary filters (pinned_filter and no_pinned_or_subapps_filter) just output these values:

(pin is null or is_app = 'f' or is_app is null)
pin is not null

In theory, this allows one to create an "pinned_filter" like "pin is null", which has its uses.

Is this filter syntax stable/frozen, or is it an internal thing subject to frequent change? (IOW is it safe to use?)

[Hugo <Nabble>]

Thanks a lot for your feedback, David!
I will reply to all your questions and suggestions very soon.

[Hugo <Nabble>]

DavidJCobb wrote

PROBLEM: Strings that contain null

Great feedback. As you realized, our code throws a NullPointerException when a "null" value is printed together with other information in the output stream (in your case, it was printed together with line breaks). This exception has an important role in the code because it brings bugs to the attention of the developer (you don't want nulls to be printed when NAML is building a page, for example). But your case is different and the exception becomes indeed a problem. So we have created a new command called "null_exception_to_null" that catches the NullPointerException and prints a single "null" to the stream. Here is some code that works:

<macro name="test">
	<n.safe_default to="No bug" text="[n.crlf/][n.null/]"/>
</macro>

<macro name="safe_default" parameters="to,text" >
	<n.default to="[n.to/]" text="[n.null_exception_to_null.text/]"/>
</macro>

The name of this new command is a bit strange, but we couldn't find anything better (suggestions?). It should solve your problem though.

DavidJCobb wrote

PROBLEM: Some HTML tags are not recognized

You can introduce new tags with the "static" tag, just like this:

<static>
  thead
  tfoot
</static>

Each line represents a new tag that you want to use.

DavidJCobb wrote

SUGGESTION: Arbitrary node lists

This is possible, but we need a way to select arbitrary nodes in the database and retrieve them as a list. We believe the solution is to use our tag table, which is pretty much like a label that is applied to a node or user (think of labels in Gmail). So somehow you would stick a label in some nodes and then use a filter to retrieve them. Would this solve your problem?

DavidJCobb wrote

QUESTION about translations

That's correct, "T" tags are used for translations. Basically, if you wrap a string with <t>...</t> tags, you tell the code that a replacement for that text may be available. Here is an example: imagine a macro that has this text:

... <t>some text</t>...

If no translation is available, this is not a problem at all. The code will simply use the text between the T tags. But you may override that code with:

<translation><from>some text</from><to>SoMe TeXt</to></translation>

you can even override the translation with:

<override_translation><from>some text</from><to>s-o-m-e t-e-x-t</to></override_translation>

This is nice because Nabble has some ready-to-use translations, but you can use the last case above to customize it even more.

DavidJCobb wrote

SUGGESTION: Reading cookies from NAML

We just added a new command called "get_cookie_value", which can be used like this:

<n.get_cookie_value name="mycookie"/>

DavidJCobb wrote

PROBLEM: Error pages don't escape tags

I was not able to reproduce this. Please let me know if you have a URL or some steps to reproduce it.

DavidJCobb wrote

SUGGESTION: Control over unhandled exception output

I am investigating this and will come with a solution soon.

[Hugo <Nabble>]

DavidJCobb wrote

Is this filter syntax stable/frozen, or is it an internal thing subject to frequent change? (IOW is it safe to use?)

This is stable. Actually, this is a SQL condition that will be applied to the main SQL when retrieving nodes from the database. So the syntax is certainly stable. Ideally we should create more and more filters that anyone can use, but custom filters like that should work fine.

[DavidJCobb]

Hugo <Nabble> wrote

Great feedback. As you realized, our code throws a NullPointerException when a "null" value is printed together with other information in the output stream (in your case, it was printed together with line breaks). This exception has an important role in the code because it brings bugs to the attention of the developer (you don't want nulls to be printed when NAML is building a page, for example). But your case is different and the exception becomes indeed a problem. So we have created a new command called "null_exception_to_null" that catches the NullPointerException and prints a single "null" to the stream. Here is some code that works:

<macro name="test">
	<n.safe_default to="No bug" text="[n.crlf/][n.null/]"/>
</macro>

<macro name="safe_default" parameters="to,text" >
	<n.default to="[n.to/]" text="[n.null_exception_to_null.text/]"/>
</macro>

The name of this new command is a bit strange, but we couldn't find anything better (suggestions?). It should solve your problem though.

Name seems fine to me. Thanks for the new command; it should prove useful. :)

Hugo <Nabble> wrote

You can introduce new tags with the "static" tag, just like this:

<static>
  thead
  tfoot
</static>

Each line represents a new tag that you want to use.

Good to know.

Hugo <Nabble> wrote

This is possible, but we need a way to select arbitrary nodes in the database and retrieve them as a list. We believe the solution is to use our tag table, which is pretty much like a label that is applied to a node or user (think of labels in Gmail). So somehow you would stick a label in some nodes and then use a filter to retrieve them. Would this solve your problem?

That could work for persistent storage, and could help to solve the issue, yes.

I don't know much about databases, but that does sound like it could be hard to keep efficient -- retrieving the nodes would require looping through every node on the site, no? I think a simpler solution might be to add a function that'll take a comma-delimited list of IDs as a string, and return a nodelist that could then be sorted. Given a safe storage area for the comma-separated list (i.e. as a property on the root_node), I think this would accomplish the same thing as a tag table, but with the benefit of being able to grab nodes directly by ID rather than having to check all of them for a property.

Either way'll get the job done. I propose the alternative simply because I think it could have a performance advantage, though I've no clue how large the difference would be.

Hugo <Nabble> wrote

DavidJCobb wrote

QUESTION about translations

That's correct, "T" tags are used for translations. Basically, if you wrap a string with <t>...</t> tags, you tell the code that a replacement for that text may be available. Here is an example: imagine a macro that has this text:

... <t>some text</t>...

If no translation is available, this is not a problem at all. The code will simply use the text between the T tags. But you may override that code with:

<translation><from>some text</from><to>SoMe TeXt</to></translation>

you can even override the translation with:

<override_translation><from>some text</from><to>s-o-m-e t-e-x-t</to></override_translation>

This is nice because Nabble has some ready-to-use translations, but you can use the last case above to customize it even more.

So if I were to find some people who spoke other languages, I could actually start packing some basic translations with the NAML code for the features... Very interesting. Thanks for the info.

Hugo <Nabble> wrote

DavidJCobb wrote

SUGGESTION: Reading cookies from NAML

We just added a new command called "get_cookie_value", which can be used like this:

<n.get_cookie_value name="mycookie"/>

Much appreciated. :)

Hugo <Nabble> wrote

DavidJCobb wrote

PROBLEM: Error pages don't escape tags

I was not able to reproduce this. Please let me know if you have a URL or some steps to reproduce it.

I'll try to remember to post here next time I cause an Error 500.

Hugo <Nabble> wrote

DavidJCobb wrote

SUGGESTION: Control over unhandled exception output

I am investigating this and will come with a solution soon.

Thanks for looking into it.

Hugo <Nabble> wrote

DavidJCobb wrote

Is this filter syntax stable/frozen, or is it an internal thing subject to frequent change? (IOW is it safe to use?)

This is stable. Actually, this is a SQL condition that will be applied to the main SQL when retrieving nodes from the database. So the syntax is certainly stable. Ideally we should create more and more filters that anyone can use, but custom filters like that should work fine.

Perfect! One of our UI tweaks takes advantage of that, so it's good to know that we can rely on it when we need it.

[Hugo <Nabble>]

DavidJCobb wrote

Hugo <Nabble> wrote

DavidJCobb wrote

SUGGESTION: Control over unhandled exception output

I am investigating this and will come with a solution soon.

Thanks for looking into it.

We have added the unknown_exception command. You can use it like this (see lines 9-12):

<n.format_error.handle_exception. for="save-block">
	<n.exception. name="exception1">
		Message 1
	</n.exception.>
	<n.exception. name="exception2">
		Message 2
	</n.exception.>
	...
	<n.unknown_exception.>
		An error has ocurred: <n.message/>
		<n.log.>Exception in NAML code: <n.message/></n.log.>
	</n.unknown_exception.>
</n.format_error.handle_exception.>

Please note that I added a log line to the unknown exception block (line 11), which would be printed in the NAML log of your app. Those exceptions should be properly handled, so the log would help you with this task.

[DavidJCobb]

Excellent! I'll add it to my NAML to-do list, and edit the existing forms to use it once some other site stuff is cleared up. Thanks for the addition. :)

[Hugo <Nabble>]

DavidJCobb wrote

Hugo <Nabble> wrote

This is possible, but we need a way to select arbitrary nodes in the database and retrieve them as a list. We believe the solution is to use our tag table, which is pretty much like a label that is applied to a node or user (think of labels in Gmail). So somehow you would stick a label in some nodes and then use a filter to retrieve them. Would this solve your problem?

That could work for persistent storage, and could help to solve the issue, yes.

I don't know much about databases, but that does sound like it could be hard to keep efficient -- retrieving the nodes would require looping through every node on the site, no? I think a simpler solution might be to add a function that'll take a comma-delimited list of IDs as a string, and return a nodelist that could then be sorted. Given a safe storage area for the comma-separated list (i.e. as a property on the root_node), I think this would accomplish the same thing as a tag table, but with the benefit of being able to grab nodes directly by ID rather than having to check all of them for a property.

Either way'll get the job done. I propose the alternative simply because I think it could have a performance advantage, though I've no clue how large the difference would be.

Selecting nodes by ID is certainly a good and flexible solution. Do you expect the number of nodes to be very high? Our idea is to retrieve all nodes with just one SQL. So basically we would build the SQL with all node IDs in the "where" clause and this could become a problem if the number of nodes is very high (e.g., > 200). We want to do this with just one SQL because sorting the nodes would be much easier. The command would look more or less like this:

<n.custom_node_list
     node_ids="11, 12, 13, 14, 15, 16, 17"
     sort_by="subject asc, last_node_date desc">
          <n.loop.>
               <n.current_node....>
          </n.loop.>
</n.custom_node_list>

As you can see, there would be a "sort_by" parameter that follows the SQL syntax. You would be able to sort nodes by any column in that table. This is why we need just one SQL. Does that work for you? Suggestions?

[DavidJCobb]

Right now, we have about 70 nodes we'd need to track using this system. If we were to get enough new active members, hundreds of portfolio nodes wouldn't be out of the question.

If this is something that'd be too difficult to implement, we can continue to use the current system (parse the IDs as a string list -- not sortable, but we can paginate it, so it gets the job done). The ability to build nodelists is nice-to-have but not essential.

[Hugo <Nabble>]

Thanks for the explanation. The more I think about the requirements, the more I realize that the tag/label solution is the right way to go. Basically, we need three things: (1) high number of nodes (e.g., 1000+), (2) sorting and (3) pagination (i.e., read only a section of the list and ignore the rest). We can do these three things with just one SQL if we have labels in place. So if you agree, I can create a ticket for this feature, but we will be able to work on this only in some months (we have to finish other top priority features first). For now, you can keep using your comma-separated list of IDs and contact us if you have issues. Does that make sense?

[DavidJCobb]

Ah, I see. Now that I think about it, tag/label fields in the database probably would be more efficient than a processed-at-run-time list of IDs.

It makes perfect sense, and it sounds like the way to go. I've no issues with the feature having to wait a few months; you're providing an excellent service, and I understand that as part of that, there are things that need more immediate attention.

Should I keep an eye on this topic, or would it be possible for you to e-mail me when the tag/label system is ready?

[GregChapman]

DavidJCobb wrote

Should I keep an eye on this topic

Don't forget the OPTIONS > SUBSCRIBE feature. If you subscribe at topic level you'll only get emails posted to that topic. :-)

[DavidJCobb]

GregChapman wrote

DavidJCobb wrote

Should I keep an eye on this topic

Don't forget the OPTIONS > SUBSCRIBE feature. If you subscribe at topic level you'll only get emails posted to that topic. :-)

Ah, thanks for the tip. *sub'd*

[Hugo <Nabble>]

DavidJCobb wrote

Should I keep an eye on this topic, or would it be possible for you to e-mail me when the tag/label system is ready?

You will see label controls on the UI when this is ready. I am still not sure how they are going to look, but they should be easy to spot. Also, you should announce this in our Latest Features page. In any case, the ticket I created has a link to this topic, so I can post a reply to let you know.