I was recently reading John Lienhard's Engines of Our Ingenuity and he said something that really struck me. "Engineering is really just communication."

That struck me, because that's what I believe about programming. Obviously, coding is the act of communicating with a machine, and our software must be able to communicate with the user, and we must be able to communicate with the user, his bosses, other departments in IT, our own chain of command, etc. etc. etc. Writing the code is the easy part.

In our shop, the Notes developers and administrators were in different groups, and that's as it should be. My job as a developer is to get functionality out as quickly and cheaply as possible. The administrator's job is to keep the infrastructure up and running 24/7. In other words, they're there to keep me from breaking it. We are good natured rivals, and should never be too cozy. But we are different sides of the same coin.

A few years after I started in Notes, our company was acquired by a much larger company. We went through the usual adjustments, and as you might expect, our larger, more bureaucratic new headmasters wanted everything done their way. I can still recall the first time I called one of those new administrators (in another city no less) to get an agent stamped in an emergency.

But, instead of making adversaries of these folks, instead of bad mouthing them when the speaker phone was on mute or during the restroom break, I just tried my best to over communicate. When I had something in the works that might affect admin, I'd write it up carefully and at length, and then I'd call and answer any questions. Pretty soon, my boss asked me to sit in on their weekly meetings. What could be more fun than 6 admins sitting around talking about server upgrades for an hour?

But I found that, from time to time, they would be planning something that affected us. Sometimes it was an adverse impact that they could not foresee--moving a server was going to break some icons, or location documents would need to be updated or what have you. And often I would see that they needed my help. Pretty soon, I had one or two little projects in the works at all times, in support of my administrators. I'd write them a tool to help do analysis in the NAB, or to assist with tracking the R5 upgrade, or to assign new internet addresses. With each project, we'd all get to know each other better. And eventually, they started calling to ask for my input at the outset of new projects.

Most of these folks have gone on to other things through divestiture, downsizing or just plain old attrition, and today I have a whole new crop of administrators and DBAs and infrastructure folks to put up with. But I will never forget that those strangers who came in and wanted us to follow their silly rules all turned out to be people with their own problems, abilities and developers to put up with. And those strangers, with a lot of communication, became colleagues and friends. And I could not have done anything without them.

I mention this because it seems to me the IT people tend to be rather insular. We live off in our little technical universe that no one else can peer into. The administrators and the infrastructure folks can't understand half of what we are doing, and you can forget about the managers and the accountants and the pin-heads in auditing. But the thing is, all those people are just like us. They are all trying to get work done, and they can't understand why those pin-heads in IT can't just stop talking techie and build their application.

I have had this conversation with IT folks a number of times. They always want to know why this group or that group does not understand ANYTHING about that it takes to get their applications built. Well, not to put to fine a point on it, but have we told them? Have we actually told our users how we expect them to use their files shares? Have we followed up to make sure they understand corporate retention policies? Have we bothered to ask them if their names are spelled correctly in our ERP system? Do we ever go to THEIR planning meetings and learn about what they do (run the company--make the bucks)? If your users are forever coming to you with ill-conceived projects that need to be completed yesterday (and that have been in the pipeline for 6 months), it is because not enough communication is happening.

And if we see it, then it is OUR RESPONSIBILITY to take action.
Posted by Cregg Hardwick Generic view/Form actions
21 SEP 2006 00:18 EDT (04:18, GMT)
A significant way to reduce support costs is by eliminating parallel code -- code that is almost-but-not-quite identical, but must be slightly different to accommodate different situations. One big source of parallel code is view and form actions in Lotus Notes. Here is a way to write a single action that will work correctly from either a view or a form.

If you have been working in Notes for any time, or have ever downloaded a sample database from the Internet, you have probably encountered "parallel" form and view actions. These are actions that do essentially the same thing except that one acts on the document that is currently open in the UI and the other runs against documents selected in a view.

For simple actions, both can call a formula agent that runs against selected documents. (i.e. @Command([ToolsRunMacro];"(YourHiddenAgent)"). But formula language is not sufficient for more complex tasks; say if you need to ask the user for input that will be used for processing each document. And sometimes you need the action to behave differently depending on how it was called, like by writing to a log for selected documents but displaying a pop-up results box for the current document.

It is common to code a Lotusscript form action to run against a ws.currentdocument.document, where ws is the NotesUiWorkspace. View actions intended to run against selected documents usually run against db.UnprocessedDocuments, where db is the current database (obtains from the NotesSession). When you try to combine the form and view actions, however, you run into a snag.

If a user selects documents in a view while a document is open in the UI, Db.UnprocessedDocuments.Count will be greater than 0, ws.CurrentDocument will be defined. So how do you tell which action to perform? None of the properties of the UnprocessedDocument collection or the UiWorkspace.CurrentDocument will help. There is, however, a way.

What you do is fetch the first document in the UnprocessedDocuments collection and check to see if it has an instantiated "ParentView" property. If it does, then it was selected in a view. If not then it is open in the UI. Simple, once you know how.

Sample code is below:

       Dim session As New NotesSession
       Dim ws As New NotesUiWorkspace
       Dim dc As NotesDocumentCollection
 Set dc=session.CurrentDatabase.UnprocessedDocuments
 . . .
 Set doc=dc.GetFirstDocument
 If  doc.parentview Is Nothing Then
  Set doc=ws.CurrentDocument.Document
  Process(doc)
 Else
  For i=1 To dc.count
   Set doc=dc.GetNthDocument(i)
   Process(doc)
  Next
 End If

Outlines are a wonderful addition to Notes, but they ignore this setting. To use outlines, you have to use the launch setting "Open Designated Frameset." The frameset then displays a page which contains the outline. Normally you have the outline (page) in the navigation frame on the left and display a view on the right. This provides everything you need to simulate "Restore as last viewed by user."

First, in the queryclose event of the page you are using to display your outline, add the following code to save the name of the current view (You cannot get this information from the database queryclose event but it is available when the Page closes):

    Sub Queryclose(Source As Notesuidocument, Continue As Variant)
        ' when page is closed, save name of current 
        ' view so we can open to it next time. 
        Dim ws As New notesuiworkspace
        If Not ws.currentview Is Nothing Then
            Dim session As New notessession
            Call session.SetEnvironmentVar("##" +_
              session.currentdatabase.title +_
              "_LastView", ws.currentview.view.name)
        End If
    End Sub

Sub Queryopen(Source As Notesuidocument, Mode As Integer, Isnewdoc As Variant, Continue As Variant)
        Dim session As New NotesSession
        Dim workspace As New NotesUIWorkspace
        Dim DB As NotesDatabase
        Dim view As NotesView
        Dim lastview As String

        db = session.CurrentDatabase
        lastview = session.GetEnvironmentString("##" +_
           DB.title + "_Lastview", False)

        If lastview <> "" Then view = db.GetView(lastview)

        ' Last view was not recorded in environment or no longer exists
        If view Is Nothing Then
            ' See if a view with default alias exists... 
            view = db.GetView("Main")
            If Not view Is Nothing Then
                Call session.SetEnvironmentVar("##" +_
                  session.currentdatabase.title + "_LastView", "Main")
            Else ' If not, find default view the hard way
   Forall Victor In session.currentdatabase.views 
                Print "Searching for default view..." 
                If Victor.IsDefaultView Then
                    Call session.SetEnvironmentVar("##" +_
                    session.currentdatabase.title +_
                    "_LastView", Victor.name)
                    Print ' clear status line

     Exit Forall
                End If
   End Forall
            End If
        End If
    End Sub

Finally, you set the frame you are using to display views to calculate its initial content based on the value you saved. This is done by setting the frame content to "Named element" -- "View" -- "computed" with the following formula:

View:=@Environment("##"+@DbTitle+"_Lastview");
@If(view="";"Main";view)

Where "Main" is the alias of the default view.

That's all there is to it. As long as the user navigates using your outline (They don't your View-Database_goto to open a view not on your outline) they will be in the outline and in the page when they exit the database. The page will save the name of the current view in the user's Notes.ini file, and when they next open the database (and your frameset) the initial view will be calculated based on this value.
Posted by Cregg Hardwick Only you can prevent orphans
19 SEP 2006 21:10 EDT (01:10, GMT)
Lotus Notes, as we all have heard a million times, does not support relational data. It does, however, support response hierarchies. Every Notes document has a property called its "Responses" collection, which maintains a list of its zero or more responses (and every response document has a $REF field and a ParentID property, both of which hold the document UNID of its parent). This hierarchy was created for the sole purpose of supporting discussion threads, but it can be used to handle some pretty useful (if rather simple) data relations. It can also create orphan documents.

Pick any large database that has a lot of records, at least one response hierarchy, and that has been around for a while. Create two new views that both select @All. Set ONLY one view to "display response documents in a hierarchy." Open both the views and press control-A to select all documents and observe the count. Chances are, if the database has been around for a while, you will see that the view that does NOT show the response hierarchy will have a larger count. What are these extra documents?

When a view is set to use the response hierarchy, it displays response documents (that is, any document that has a $REF field), each indented under its parent. But alas, if the parent does not exist, the document will not appear in the view at all. When a parent is deleted there is no mechanism to delete its responses. They might disappear from view, but they are still in the database, slowly accumulating until the server has a nervous breakdown!

The ease with which Notes allows responses to become orphans is so glaring, it's hard to figure why Lotus has never fixed it, or why it hasn't become a crippling Achilles heel, or why they ever shipped the product in this condition in the first place. If you think I'm delusional, test me. Create a little test database and try it. In fact, because I figured this problem out in 1997, I thought I might actually BE delusional (or at least that Lotus might have fixed it without telling me). But as of R6.5, it's still there.

There is a programmatic way to identify these documents and clean them, but I'll leave that for your own early morning hours of desperation. Suffice it to say, you need a LotusScript agent that runs on all response documents and looks up their parents, although that's over-simplifying a bit. No, what I want to share with you is the code that Lotus SHOULD have compiled into Notes R3.0, but didn't.

As you know, Notes has events. You may or may not be aware of the database events, but they work just like any other event in Notes, they just apply to the whole database. If you don't want your databases to swell up with orphans, assign this code to the database script module of any database using a response hierarchy:

  Sub Postdocumentdelete(ByVal Source As Notesuidatabase)
  ' Remove any responses to just deleted documents so orphans are not created. 
  Dim DeletedDocuments As NotesDocumentCollection
  Dim doc As notesdocument, child As notesdocument

  DeletedDocuments = source.documents

  Dim i As Long, j As Long
  For i = 1 To DeletedDocuments.count
    doc = DeletedDocuments.getnthdocument(i)
    If Not doc.responses Is Nothing Then
      For j = doc.responses.count To 1 Step -1
        child = doc.responses.getnthdocument(j)
        If Isarray(child.items) Then Call child.remove(True)
      Next
    End If
  Next

End Sub

It's clear enough what this does. It runs AFTER a delete operation is committed (through the Notes UI by a user selecting them in a view and clicking the delete key), but BEFORE the documents are actually deleted. It simply scans all the documents that are being deleted and, if any have responses, deletes the responses. You might wonder why I didn't just call .RemoveAll on each document's responses collection. The reason is that Notes sometimes allows this list to become corrupted. When that happens, the operation would fail and users would be unable to delete the document in question. To avoid this, the script checks each entry in the responses collection (starting at the bottom and backing through the list). If it finds a corrupted entry, the referenced response document will not have an instantiated Items collection. The Items collection is the array of NotesItems or fields, and every valid Notes document has one.

So, that will prevent orphans from being created through the UI. You're welcome. If you delete documents using LotusScript, use similar code to make sure you don't leave any orphans behind. Now the next bit of code is the all important human factors part. Why is the user deleting a document that has responses? Does he know about the responses? In a discussion thread it might not matter, but in an inventory in which the responses are equipment and the parent is the owner, it might matter very, very much. The following code takes care of this:

Sub Querydocumentdelete(Source As Notesuidatabase, Continue As Variant)
    Dim Session As New notessession
    Dim SelectedDocuments As NotesDocumentCollection
    Dim doc As notesdocument
    Dim child As notesdocument
    Dim TotalResponses As Long
    Dim ConflictResponses As Long
    Dim Conflicts As Long

    Dim i As Integer, j As Integer
    SelectedDocuments = source.documents
    For i = 1 To SelectedDocuments.count
      doc = SelectedDocuments.getnthdocument(i)
      If doc.hasitem("$Conflict") Then Conflicts = Conflicts + 1
      For j = 1 To doc.responses.count
        child = doc.responses.getnthdocument(j)
        If Isarray(child.items) Then
          If child.hasitem("$conflict") Then
            ConflictResponses = ConflictResponses + 1
          Else
            TotalResponses = TotalResponses + 1
          End If
        End If
      Next
    Next

    If ConflictResponses > 0 Then
  continue = Messagebox("Selected document(s) have"+Str(ConflictResponses+TotalResponses)+_
      " response(s) including"+Str(ConflictResponses)+ " conflict(s)." +Chr(10)+_
      "Are you sure you mean to delete all selected documents including all responses?" ,4, "Are you sure?" )=6
    Else
      If TotalResponses > 0 Then
   continue=Messagebox("Do you want to delete"+_
        Str(SelectedDocuments.count)+{ document(s)along with}+Str(TotalResponses)+_
        " response documents?" ,4, "Are you sure?" )=6
      End If
    End If
  End Sub

Yes. I know. I've created another of those damnable "Are you sure" prompts. In this it works because people don't generally delete a whole lot of Notes documents, and because the prompts all default in the safe direction.

Now one last note about responses. In the form designer, there is a property that lets you specify that a form is a "Document," a "Response," or a "Response to a response." This property confuses most Notes developers that I have ever met, so let me explain it to you.

This property has nothing to do with the parent-child relationship and the response hierarchy I explained above. This property ONLY governs how the Notes UI treats the document when it is composed or edited in the UI. If you create responses through LotusScript by calling their .MakeResponse method, and if those responses are never edited in the UI, then that form setting does not come into play (but then, why create the form at all).

A "ResponseToResponse" type form, when used to compose a new document through the built-in Notes UI, will automatically become a response to whatever document was open (or selected in the view) at the moment it was composed. A "Response" type form, when used to compose a new document through the build-in Notes UI, will become a response to the ultimate parent of whatever document was selected (or open) whether the selected document was the parent itself, or a great-great-great granddaughter thereof.

As a practical matter, this is not very important. If you are writing applications in notes, you should generally NOT be letting users compose documents in an un-controlled manner via the "Compose" menu. You should instead hide all documents from the menu and give them actions through a carefully designed application UI. But, you still need to know about the form property because, if a response document is ever edited using a "Document" type form, it stops being a response!
Posted by Cregg Hardwick The golden rule: Keep it simple
18 SEP 2006 06:12 EDT (10:12, GMT)
Remember Pascal? When I was in school, we studied Pascal, but everyone "knew" that C was where the action was. Pascal was far easier to learn and to read. Its strong typing protected us from all manner of evil, and it was powerful enough to express any concept that came up in a college CS curriculum. But of course, the operating systems of the day were written in C, so C was what the young turks like myself all aspired to. Then I went to work in the real world.

At SoftDisk Publishing, we programmed in everything. We took software submissions and licensed shareware and everything had to be cleaned up, normalized and/or stabilized. All of the in-house development was done in Pascal, C++, or an early database language called Clipper (which was really just an early compiled dialect of BASIC with data access constructs like Microsoft's "new" LINQS). I worked in all of these languages, but I haughtily thought of myself as a "real" programmer. So when I was given my first major project to run, it had to be C++.

Three horrible months of high blood pressure, lost sleep, and nightmares later, I had learned a valuable lesson. Every mistake and omission that Pascal prevented, C++ invited. The syntax, which aficionados referred to as "mathematical," was obscure, case was important, types could not be trusted, buffers could overflow, memory management had to be attended to, pointers could point to anything and crashes were simply a way of life.

For the "gamer guys" down the hall, C++ offered an easy way to write directly to the hardware, and was far superior to writing directly in assembly language. They were able to get the most out of the language by applying standards, working insanely long hours, and dedicating a team to each project. But in my group, each developer was responsible for one new development project every other month, plus a host of monthly editorial duties, including work in other programming languages. For our group, Pascal was a vastly superior tool.

I am reminded of this fateful lesson each and every day at work. Yes, friends, I am engaged in the treasonous practice of creating a simple little ASP.Net application. We have at our disposal the full suite of Microsoft tools, from the impressive IDE to their shiny new "Guidance Packages." The latter are really kind of neat. Imagine a cross between a Lotus Notes template and a hands-on tutorial, and like everything Microsoft makes, it's extensible. And it's a darn good thing too, because as nice as all these tools are, and as impressive the intellectual firepower that clearly stands behind them, they are mind-bogglingly complex.

Microsoft has spent billions solving problems that we Notes programmers had licked ten years ago. Sure, their solutions are better (in theory) in many ways, and granted, some of the Lotus solutions still have nagging weaknesses that haven't been improved in years (can anyone explain why it took until R7 for record locking, and there are still no good tools for automating replication conflict resolution?). But sometimes "better" means different things to different people.

Imagine if you went off to school to become an aerospace engineer and they told you that, in order to build rockets, you first had to understand quantum mechanics. After all, the old Newtonian mechanics don't work for objects traveling close to the speed of light, and most of it hasn't been improved in over a century. If you are like me, you might be sorely tempted to pull back your tuition check and inquire, "so what?"

History is filled with examples of this kind of reasoning error, and we have all seen their effect in business. Mainframes were bigger and more reliable than microcomputers for a long time, but that didn't stop microcomputer technology from dominating the industry. Microsoft's vast suite of development tools is impressive in its breadth and intellectual foundations, but is that what really matters? I mentioned the Guidance packages earlier. These "assist" the developer in building a complex multi-tier application, which has all kinds of theoretical benefits. My guess is that, after it matures and I get enough experience, I'll be able to do in a week what I can now do in a month. But I could get the same results in Lotus Notes in two days.

After the Space Shuttle Challenger exploded, one of the people who made up the Rogers commission investigating the accident was Nobel laureate physicist Richard Feynman. He's the fellow that many people remember for bringing a piece of rubber o-ring material and a glass of ice water out before the cameras to demonstrate easily and clearly how the material failed at low temperatures. What many people do not know is that Feynman conducted his own personal investigation of the entire space program, by the expedient of leaving all the reports behind and walking around talking to people. One of the areas he investigated was the computer system on the shuttle. He found that, over all, it was probably the best designed part of the shuttle system. What made it so good? It was simple. The shuttle designers deliberately under-designed the computer and built it using ancient transistor technology with tape cartridge to load its simple programs. It doesn't do much. But it gets the shuttle up and back down... every single time.

When it comes to choosing technologies, we should all keep this example in mind. The goal is not to have the best looking, or the most interesting, or the most "Enterprise class" or the most "scalable." The goal, is to solve whatever the problem is as simply and cheaply as possible. Anything else is a waste of resources, or an investment in artistic expression.
Posted by Cregg Hardwick Resistance is futile
15 SEP 2006 09:13 EDT (13:13, GMT)
I got a memo from my corporate LAN group today that put me in mind of a favorite topic of mine -- you guessed it -- human factors engineering. The memo said that someone had identified a new security threat and blah blah blah, they are going to lock out zip file attachments for some period of time. Now don't get me wrong. I'm not ragging on the LAN team, they do a great job and don't get nearly enough credit and I'm sure they are right.

But here's the thing. People are not going to stop exchanging files. When we got a new IM system that didn't allow URLs in a message, we just substituted "htp" for "http" and sent them anyway. When they put filters on our intranet sites and email so that we couldn't send any kind of executables, we just started packing our executables in Zip files. Now they are going to block Zip files and we will just change the file extension. And if they find a way to block that we'll find a way around it! WE ARE GOING TO EXCHANGE FILES.

Now, my point is not that these measures are silly, paranoid, and better handled by the firewall and virus protection system. My point is not how foolish it is to tell employees not to use a tool they need to do their jobs. Heck, we in corporate America are used to routinely having valuable data destroyed so it can't be used in litigation for things we won't be doing if we are just allowed to do our jobs, or having the best source of free technical information on the planet today (Blogs) blocked by the corporate firewall (because someone in Lan once saw a Blog about how to strangle a bureaucrat with one hand I guess -- but that's nothing compared to a local school system that blocked Yahoo when the school system's IT head (a librarian) found out it could be used to access pornography.

No, my point is a point that was made in a materials resource planning class I took in college. "If you create a process that the workers perceive as getting in the way of their work, THEY WILL GO AROUND IT," it said. That's fairly significant I think. No, "it's a training issue." No, "they just don't get it." No "support starts at the top." No, your workers will go around you if you get in their way. Why that nerve! The insubordination! To go around corporate policy and actually do the work they are being paid for??? The FIENDS!

Okay, I'm being a bit facetious. And my corporate infrastructure guys know we are going to continue exchanging files, and that's not what they are worried about. They are worried about files coming in from outside the company to unsuspecting secretaries. But here's the thing. When you force people to change file names in order to get around your technology so they can do their work, you are training the entire workforce to habitually circumvent the technology intended to protect them. Am I the only one who thinks that's a bad thing?

Did you know that when the Apollo XIII spacecraft explosion occurred, NASA had turned off a critical alarm because the previous evening it had kept going off due to a trivial glitch in another system and prevented the Astronauts from getting any sleep (not that that would have prevented the accident, but...). Then there is the common corporate practice of forcing people to change their passwords so often that they just write them down on a piece of paper stuck to the side of the monitor.

This is an important issue in these days of increased security concerns. If you create any security mechanism so onerous that people must circumvent it in order to go about their lives, then it is of zero utility. Better by far to have a less effective, but practical measure that is actually used. If it took three minutes to disarm your house alarm each evening, you would just stop setting it!

But there is a broader principle at work here, too. For example, when is the last time your computer asked you "Are you sure you want to delete this file" and you stopped and said "Heck no, am I deleting? Why, I didn't mean to do that!" Bunk! The part of your brain that has become acclimated to dispensing with that silly box can respond to it in 1/5 the time it takes for you to become consciously aware of it. Unless you are one of those hunt and peck people who carefully read each and every word on each and every page (geeeeez! I can't even stand to write it!), you're lucky if you even notice that little box one time out of twenty, which is why they invented undo and undelete which are much better solutions to the same problem.

And the principle is still more general than that. There are fundamental limits to how reliable, precise, accurate or detailed anything involving a human being can be. If you go into a restaurant and ask 20 people whether it's daytime or nighttime, at most hours you will get a pretty consistent set of answers. But ask the same 20 people "what time is it," and it would be most surprising if any two of the answers matched to within less than a minute. This principle crops up a lot in business, which means it crops up a lot in application design.

When I used to have to allocate my time to SBU that was hard. I was supporting 10 or 12 SBUs at the time, but I usually only did work for five or six in a given month. So I could do that. Then I had a boss who wanted to track all IT work in one of what grew to 23 categories. I finally convinced him that it was better to have a reliable report broken down into four categories (like "Enhancement," "Break-Fix" and "Researching") than to have a hundred employees each guessing which of 23 categories to fit their efforts into. Then a few years later, the new CIO decided that she wanted all work broken down into one of four work orders for each of 24 billing points and broken out into regular and overtime. Yeah. I'll be doing that. Then the next year she added up to a dozen activity codes that had to be allocated to each of the above. Needless to say, there is NO CHANCE AT ALL that a single person in IT who was not fully dedicated to a single long term project could even hope to comply with such foolishness, even if they tried (and tried and tried and tried). Well, managers ask for all sorts of things. The trouble is this CIO was getting reports based on the assumption that everyone in IT was obediently following the official time tracking procedures. What was on the report? I have no clue, but I can tell you what was NOT on that report. It was NOT an accurate reflection of where time and money was being spent. In fact, with that many buckets, I doubt if work was even being allocated to the right SBU!

Of course, since they passed Sarbanes Oxley, we don't have any problems like that any more. I know everything runs as smooth as a baby's bottom. And no one exchanges any of those nasty files. And the guy in the next cube figured out a way to right a worm that will hack all the porno sites in the world and put them on the path to righteousness. Thanks Mr. Lay!
Posted by Cregg Hardwick DialogBoxes in LotusScript
14 SEP 2006 07:38 EDT (11:38, GMT)
If you don't routinely use dialog boxes in your Notes apps, you should. A good user interface calls to the user's attention those things that should be attended to, minimizes the amount of information that the user should attend to at one time, and allows the user to break a task into subtasks that can be independently aborted. Dialog boxes let our applications do all these things and more. And, although they should be an indispensable, part of the Lotus Notes developer's bag of tricks, it seems that lots of folks still don't know how to use them effectively.

First, let's be clear that we are talking about custom dialog boxes presented using the NotesUIWorkspace.Dialogbox method. There are lots of other commands and methods that present modal dialogs, such as MessageBox and NotesUIworkspace.PickListCollection and .PickListStrings. These are excellent at what they do, but are not what I want to tell you about.

The first thing to understand is the separation between documents and forms. Now bear with me if this seems obvious -- some people don't see it at first. In Notes, a form is a design for a data entry screen that will be used to compose or edit documents. Documents are actual records stored in the database, and displayed using views. By default, all forms appear in the "Compose" menu, and when selected from it, create new documents that take the name of the form used to create them.

So, by default, a document seems to be part and parcel of the form used to compose it. But this is not so at all. In fact, you can create a document totally from lotusscript and associate it with any form you choose:

Dim session as new notesSession
Dim doc as notesdocument
Set doc=session.currentDatabase.CreateDocument
Doc.form="fruit"
. . .
Doc.myField="Bannana"
Call Doc.save(true,false,false)

There. We have a new document. The database does not have to have a form called "fruit" (but to be of any use, there ought to be a view that selects documents where Form="fruit"). In fact, we can edit our "fruit" document using a different form. We can do this in several ways which I won't go into unless someone wants to ask about that.

So, forms are data entry screens. Documents are records. The default behavior in Notes is to use a form to compose a document and then always use the same form to edit that document. The DialogBox method is one way to open a document in the UI using any form you choose.

Why would you want to do this? Many reasons. You might want to ask the user some questions and only store the result. You might want to pull in data from another data source. You might want to allow the user to use a wizard or calculator and have the option of aborting the process in a controlled way.

So, enough of this, here is the syntax for the DialogBox method of NotesUIWorkspace:

DialogBox(formName,true,true,false,false,false,false,Title, NotesDocument,true)

Now, if you want to know what all those options are, look in designer help. As a practical matter, the three shown in bold are the only ones that count. By default, DialogBox will try to edit the document that is currently selected in a view or open in the UI. But, as is often the case in Lotus Notes, we are never going to use the default behavior. We are always going to pass a document to the method so that we can specify the last parameter, which causes the dialog box to display only the part of the form that is inside the first table on the form. That's usefull because it is far, far easier to lay screens out using tables than it is with the older layout regions.

We can call DialogBox from code that will run in an agent or from a form event say. Let's say we want to run an agent on selected documents. The agent will present a dialog box to ask what we want to do, then do it to all selected documents. Here's how to do it.

1. Create a form called "Wizard."
1. Set the background color to the workspace color (gray).
2. Create a table, and turn of it's border. For our purpose, a 1x1 tabel if fine.
3. Add a few fields. Let's say, a control to allow users to select on of several target fields, and a control to allow them to enter a new value for that field.
4. Set everything above and below to have a hidewhen formula of "1" -- this is to avoid one of those annoying Notes glitches.
5. Create a hidden, computed for display field (at the top of the form) called "SaveOptions" set to "0". This will keep any document from being accidently composed with this form. We don't want to create "Wizard" document -- it is used to edit other documents.
2. Create a new agent called "Set any field."
1. Set the agent to run on selected docuements.
2. Make it a LotusScript agent.
3. Add the following code to the initialize event:

Sub Initialize()

Dim ws As New notesUiWorkspace

Dim session As New notesSession

Dim db As notesDatabase

db = session.currentDatabase

'This document will never be saved anywhere

Dim dialog As notesdocument

dialog = db.CreateDocument

' You could add code here to initialize the dialog, say

' to pull a list of fields from the first selected document for

' use in the field selection

Dim continue As Integer

Do While ws.DialogBox("Wizard",True, True, False, False, False, False,_ "Change any field",Dialog, True)

continue=True

If Dialog.FieldName(0)= "" Then continue=False

If Dialog.FieldValue(0)= "" Then continue=False

' add any other validation you might need here...

If continue Then

Exit Do

Else

Messagebox {Hey! Enter a valid field name and value or press "Cancel"!},_

MB_INFORMATION, db.title + " - " + session.CurrentAgent

End If

Loop

If continue Then

Call db.UnprocessedDocuments._
StampAll(dialog.fieldName(0), dialog.fieldValue)

End If

End Sub

Notice a few things about this code:

• Strings can be wrapped in curley braces and can then contain quotes.
• Long lines can be split with an underscore (personally, I never do this except for internet consumption).
• We are creating a real bona-fide document called Dialog, which will be used to display our dialog box, but which will never, itself, be saved to disk.
• We are calling DialogBox within a loop so that we can validate it before acting on it. If the user presses escape or clicks the cancel button, the loop will abort without performing this validation.

In this example, we validate the dialog after it closes, so the dialog gets redisplayed again and again until the user either enters valid entries or aborts. Here, that is appropriate. But when you are using a pop-up dialog to present a calculator or sub editor for a document that the user has open for edit, it is sometimes more appropriate to validate the form BEFORE ALLOWING IT TO CLOSE. To do this, but still allow the user to cancel the dialog, put this code in the QueryClose event of the form used by the dialog box:

Sub Queryclose(Source As Notesuidocument, Continue As Variant)

If Not source.DialogBoxCanceled Then

' add code to set Continue=false if dialog

' (source.Document) fields don't pass...

End If

End Sub

This way, you can prevent the user from accidentally ending the dialog when they have made some preventable mistake. In this case, it may not be acceptable to validate what they have done after the dialog closes, because the current document has already been updated with changes made in the dialog box, and the user might be forces to exit without saving other edits (unrelated to the dialog) if they decide they have bothed the dialog operation.

Note, however, that I have not acctually shown you how to edit the current document using DialogBox, because even though it is the default behavior, I consider it bad design. It is almost always better to create an in-memory document and explicitely load it, then display it in a dialog, then explicitly load its values back to the current document. This takes a small amount of extra work, and can save a huge amount of debugging later on.

So there you have it. If you've never used custom dialog boxes before, you now know everthing you need to give your Notes databases a giant boost in usability.
Posted by Cregg Hardwick Psychology in software design
13 SEP 2006 10:43 EDT (14:43, GMT)
During my lifetime, tremendous strides have been made toward improving every aspect of software development, reliability and yes, usability. GUIs, Mice, UI metaphors, "Information Hiding" and better operating systems have vastly improved the user experience even as the sophistication and complexity of our systems have grown. And users have become more technically savvy, less easily intimidated, and more able and willing to help themselves.

Despite all this goodness, users are still making many of the same errors. Those darn users! When will they learn? Well, not to put too fine a point on it, but never. Not because they aren't smart people, but because they shouldn't have to learn how to avoid tripping over software, software should stay out of the way.

Let me be more concrete. I once read of a study in which psychologists tested responses to various traffic light configurations. They found that when two traffic lights are placed one right after the other, drivers were several times more likely to run the second light. Go figure. I can almost hear the people at city hall crying "well of all the nerve! After all, the law IS the law!" But, the fact is this lawless behavior is the predictable result of exposing human beings to a poorly designed environment. People were not running the light to "stick it to those traffic engineering pin heads" (well, most of them weren't). They were making predictable perceptual and judgmental errors. Errors that the traffic engineers should have foreseen and protected them from.

The world as you know it is an illusion. I'm not being philosophical. Much of what you think is real is fabricated by your brain to keep the real world and your own cantankerous nervous system from giving you the willies. Don't believe me? Check out this site. This page illustrates two illusions that arise out of the operation of the brain. One is a motion induced blindness, in which dots disappear before the eyes; another is an aftereffect of apparent "spooky" motion, when the actual motion is stopped. The site lists dozens of similar illusions, and the book "Mind Hacks" (Stafford & Webb, O'Reilly) dozens more. These "illusions" are not merely quaint side show tricks, they are, in fact, ways of exposing flaws in the way we perceive the world. Flaws like the fact that while you think you can see your entire computer screen, you actually have a measurable blind hole in the middle of your field of view. You will never see the hole, because your eyes flit around several times a second so that you know what's in the hole. Of course, you ignore everything coming from your eyes while they are flitting around... and so on and so on.

I don't mean to digress into a treatise on nervous system function. The point is, people have predicable perceptual "issues." They make predictable mistakes. The software we design should consider this, and help prevent these mistakes. Keeping up on emerging trends in UI design can help, and following Microsoft's (and other) usability guidelines will help more. But many things that trip users up are common sense once you start to think about them. Personally, I highly recommend that any developer spend at least some time studying product design, ergonomics or human factors design. There are many good books on these subjects, but since you are sitting at your computer, you might check out BadDesigns.com.

This is a fascinating site, but quickly, here are a few of their categories of bad design that apply to software design:

• Things that don't work the way you expect
• Things that are hard to see
• Things that get in your way
• Different displays that are too similar
• Incompatible mapping of controls to devices
• Controls that are too far away from devices
• Too many controls
• Unexpected mapping between functions and controls
• Controls that are too easy to activate
• Controls that have unexpected functions
• Controls with conflicting cues
• Controls with unintuitive labels
• Controls that are too similar to each other

I won't take up thirty pages listing goofy examples I have seen of these and many other design flaws in software. But I will tell you one thing; I've written a lot of software for a lot of smart people. When they get into trouble, it's nearly always because I threw them a bad curve. I forgot to call their attention to the right thing, I forgot to anticipate a foreseeable error, I called attention to the wrong thing or I just failed to see the business process through their eyes.

No one can avoid putting things into a design that turn out to be goofy. But individually, many of these things are avoidable. The only way to minimize them is to study the ways that the user can go wrong. Perhaps not every developer is going to take an interest in brain science or product design or psychology. And that's okay, that's why we have a growing body of design knowledge. But if you really want to help your users, that's the way.
Posted by Cregg Hardwick The keys to IT
12 SEP 2006 17:18 EDT (21:18, GMT)
Lots of people get into IT because they enjoy solving intellectual puzzles. And that's fine; development is full of such puzzles. Problems arise, however, when we lose touch of the fact that those puzzles are the means, and not the end. We in IT are tasked with solving business problems, not with writing code. Writing code is just one of many means of solving problems. Other important tools include: asking good questions, listening and considering alternatives.

When I first became a technical lead for our Lotus Notes team, I had a sign on my wall that read:

1. It is impossible to over-communicate.
2. What's the business problem we are trying to solve?
3. Function first, efficiency later.

Many of the problems that I have encountered in development stem from technical people (or myself) missing these basics. I've seen people lose jobs that they did well, get into pointless fights over issues they didn't fundamentally disagree over, and waste unfathomable amounts of time and money solving problems that just didn't matter -- all for want of keeping these three rules in mind. As one of those people who got into IT for the love of solving problems I have come to think of them as old friends.

I could probably write a book on how things go wrong when these rules are forgotten. But instead, let's look at a few examples of how they help in LotusScript programming, starting with rule #1: It is impossible to overcommunicate.

In my career as a Lotus Notes developer, I started out with one app, then another that became three and, between new development and annexation of apps from other parts of the company, I ended up as technical lead for a small team that supported a portfolio of over 1,500 databases, about 200 of which we active, non-trivial applications.

If you were to ask me what the most important aspect of supporting such a portfolio was, I could only answer: "communication." Communication with clients to find out what is really important to them, communication with management to understand where resources are being focused, communication with teammates and non-IT developers of varying technical abilities across multiple states and, most importantly, communication with one's elf, in the form of system documentation and code comments.

No one likes documenting how a system works or documenting the code, but, as my support portfolio has grown over the years, I've come to see such documentation as a gift to myself. The only think worse than wasting time and money trying to figure out someone else's code is trying to figure out my own. The body of knowledge in this regard is now so broad that it would be a waste of effort to give much advice, except to recommend Steve Mc Connell's Code Complete. There is no better guide to code documentation.

There is another, widely overlooked, area where communication intersects coding; the area of training and technical writing. I won't digress here into why I think the developer cannot, and should not, ever be too far removed from this area; instead I will give one piece of advice. When writing on a technical subject, whether for a training manual, to document a technique, or to explain an issue to management, over-explain. One must always assume a certain level of knowledge on the part of one's audience, but developers nearly always assume too high a level.

More to the point, people nearly always over-estimate what they are communicating to others. Let's say that we can describe your consciousness, what you experience in life, as a series of little pictures that exist in your head. The goal of technical communication is to re-create those pictures in the reader's head. Merely putting down some words that are more or less relevant is simply not good enough. You have to establish a framework and then build upon that until the reader cannot avoid developing an understanding similar to yours. This, of course, takes effort. But most of the time, we simply run the pictures in our minds while writing (or talking) and assume that the reader sees the same pictures we do. This works fine when chatting about the Astros or choosing a restaurant, but when describing a technical problem a little more specificity is called for.

To get a better idea of my meaning, watch the movie Apollo 13, and pay special attention to the scene where the carbon dioxide level starts rising. The Lunar Lander and Command Module were built by different companies and used different canisters in the CO2 scrubbers needed to keep the air breathable. When the canisters in the LEM were exhausted, NASA had to come up with a way to use the other canisters. In the movie, you see a box of parts being dumped on a table, and later a fellow running down the hallway with his prototype -- parts falling off as he goes. When the instructions are read over the radio, one astronaut curses after ripping a plastic bag -- has he botched the job? What you don't see is the work that went into creating those instructions.

NASA understands how difficult it is to transform a design into a working implementation, and that when lives are at stake; having a good solution on the ground is only half the job. And so, NASA leases the Jules Undersea Lodge in Key Largo Florida, so that engineers can spend a week living under 30 feet of water and experience what it's like not to be able to walk down the hall and have a hands-on conversation. And if your children ever have a chance to participate in one of several NASA-affiliated Space Camps held around the country, they will have experienced one of the solutions, an exercise in which children must construct a complex device out of Tinker-Toys™ and then enable students in another room to reconstruct it using only written instructions. Try it some time.
Posted by Cregg Hardwick Defending LotusScript
11 SEP 2006 09:44 EDT (13:44, GMT)
I'm not a luddite. I'm just busy. I don't have my own Blog and I don't have any immediate plans to create one. Sure -- I'm as brimming with fascinating knowledge and pithy humor as the next guy. I just have a life. Plus, I've been slow to embrace the whole IM / Blog approach to writing. I've always seen written communication as a fundamentally different and more formal animal than more ad-hoc forms, like talking on the phone or chatting in the hallway. But technology moves on, and I suppose the Blog is developing into a sort of literary form in its own right. In addition, I do like to write. So when TechTarget asked me to participate in this project, it immediately made sense.

The only question I had was "what am I going to Blog about LotusScript? After all, let's face it, LotusScript is BASIC. It is essentially the same language I learned on a TRS-80 Model-I, only without the line numbers. In fact, LotusScript is somewhat primitive. I mean, it doesn't have the cute curly braces of one of those flashy new guys, it's only sort of kind of object oriented a little, and its debugger is, well... how 'bout them Mets?

All kidding aside, LotusScript has never developed into the universal power-tool that it once seemed was in the offing. In some ways, it seems as if Lotus has stood still while Microsoft has sailed on by. In fact, LotusScript has not really had any substantive additions from its initial release up until R7. But, on the other hand, has it really needed any?

In my 8+ years as a Lotus Notes developer, I've encountered all kinds of, frankly, irrational objections to Lotus Notes. From managers who quoted magazine articles claiming that Notes is not cost competitive with "other email systems", to those who complain that it "isn't scalable," to IT professionals who eschew its non-relational data-store as practically sacrilegious, I've often felt like the only atheist at a Baptist ball. The ones who have never complained, and who in fact have generally heaped praise in my direction, have been the clients -- the people who ultimately pay my salary and enjoy getting agile and effective service in return.

I won't waste space answering these dubious charges except to say that they totally miss the point. Yes, Lotus has many failings and it isn't the right fit for every development project. But I can crank out a fairly complex application in Notes in a day or two, get 90% of the code ready for prototype acceptance in a week, and role it out to users in ten seconds. I can morph a mailbox into a workflow app without breaking anything or having to retrain my users, and I can deploy a complex workflow application across 5 states by sending an email. Try that with Microsoft Exchange, Outlook, SharePoint Services, SharePoint Portal Server, IIS, ASP.Net and Active Directory.

Over the last two years, I have been training in the Microsoft fold. Microsoft's products are breathtaking. Microsoft does not trot out a little feature to solve a specific client issue only to leave out obvious functions that would make the feature valuable to everyone. They create a framework in which every feature fits logically into an over-arching strategic approach to development, and when that framework suggests a capability, it is usually real and practical. And that's all good, and IBM can and should take a page or two from Microsoft's book (and spend 10 times as much on product development).

On the other hand, I sometimes think I could go back to school and get a PhD in fluid mechanics in less time than it will take me to really understand how all the parts fit into the wonderful strategic approach underlying the many, many Microsoft products that have to work together to do what one Notes server does. As a Notes developer, I used to go home infuriated after wrestling with what seemed like a simple task for two or three times as long as expected. As a Microsoft developer, I have learned to be patient when things take 6 or 10 or 15 times as long as expected! Unlike in Notes, I can nearly always get the problem solved to my satisfaction, but figuring out how to do so brings up detail after complication after quirk after related side-technology until I'm ready to bonk the next person to glibly praise the era of "Internet standards" in the nose!

Software development, especially in the corporate world, is not an abstract science. What matters is solving problems quickly and cheaply. Sure, Notes ties the developer's hands in countless ways. But on the other hand, Notes gives us a preset object model that is logically structured around the parts of the client UI. Once you understand the NotesUIWorkspace, NotesSession, NotesDatabase, and NotesDocument classes and basic product events, you're ready to solve problems. In .Net, there are currently over 22 major namespaces that the developer must be reasonably comfortable with before he can write any meaningful code!

So the anti-Notes zealots can have their quips. LotusScript is still a powerful, productive problem solving power-tool. Over the next two weeks, I will share with you how I have used this tool to leverage the strengths of Notes, and work-around some of its weaknesses. Along the way, I'll show some of my thinking about software development, and I hope we'll all have a fun time.
Posted by Cregg Hardwick