Excel import workaround that’s better than the original!

Well, you learn something new every day, and sometimes from the most surprising sources. Although we use Outlook for email, we had created an Asset Management application in Notes (using rich text fields to store scans of original invoices etc). We needed a way to export information from this application to an Excel spreadsheet to further manipulate it. This was where I discovered that Excel 2007 no longer imports Lotus 123 - see blog post from April 23.

A few days later the finance person using the Asset Tracker came back to me and said she had solved the export / import problem - just use "Copy as table" and paste into the spreadsheet! Wow - this finance person had never used Notes before, and come up with a work around that was much better / easier to use than the original process. (Note: Although cell background colors are preserved in Open Office, they are miss aligned. Background colors don't paste into MS Excel.) This just goes to show that you never stop learning, and that potentially anybody can teach you something.

Excel 2007 can’t open Lotus 123 spreadsheets

We hit this problem today trying to export a Notes view as a spreadsheet, and import it into Excel. Older versions of Excel could import a wk4 spreadsheet, but not so in Excel 2007. Maybe IBM / Lotus will finally provide a means to export native Excel files directly from Notes (or maybe this is just wishful thinking...!)

Lotusphere: Worst Practices "Pre" video

... and here is the promised video, posted on You Tube. (Editing it was MUCH more work that I expected, mainly because it was done on a slow laptop.)

Some feedback on Lotusphere 2007

I am sitting on a plane returning from Lotusphere 2007. Thought about live blogging, but decided that it's better to pay attention during the presentations. Besides a laptop gets very heavy by the end of the day. For the first time in years, I can say that IBM finally "gets" Notes. What I mean is that Notes now finally has a decent roadmap for the future.

Because of my work, I was looking to see the plans for the future of Notes. I well remember the collective groan that went up at the opening session of Lotusphere a few years ago when Lotus introduced the "dual lane" approach. So I was very interested in where Notes is going. My real interest is "How effectively can I use this tool to solve business problems that other tools haven't been able to solve.

With that in mind, I attended a designer session. It was mildly interesting until last 15 minutes when Maureen Leyland arrived and demoded where the Designer client is going; based on Eclipse, the Designer now has an open architecture. Same for the Notes client, also based on Eclipse. Finally, I saw IBM with a decent, logical, plausible future for Notes.

With these new directions, I think Microsoft finally has a reason to look over their shoulder. When IBM and Microsoft battle it out, I score as a customer. Remember how long IE6 was Microsoft's browser? Years. Only when Firefox appeared and started taking market share did Microsoft start improving IE. I see the same thing happening with the Apple / Windows race. These races are ones that we, as customers, do not want any vendor to win! As long as these companies are competing we score.

Much to my surprise, I bumped in Gary Devendorf at the Dolphin on Wednesday. Gary used to be "Mr LotusScript" until IBM laid him off. A few months later he was working at Microsoft, helping them integrate their products with Notes. It was very interesting to hear his comments on the culture, and on how people relate at Microsoft, compared to IBM. Personally, I think IBM rather shot themselves in the foot letting someone like Gary go.

This year my favorite presentation was "Worst Practices" by Bill Bucan & Paul Mooney. When you put an Irishman and a Scot of doubtful sobriety on the stage together you will have an interesting banter. I hope to post a video on this soon...

Reusability with Standards

In February this year I did a presentation for Teamstudio on Notes Best Practices. Part of that presentation covered standards, and I want to extend an idea on standards by making the following claim: "In the corporate IT world a standard amounts to an agreed upon solution to a recurring problem."

In the software world there is a lot of talk about reusing code. Well, this same thought can be extended (reused?) to go beyond just code. When you create a standard (rule, procedure etc.), you effectively create a solution to a recurring problem. Each time you implement or use that standard, you are taking all the thought & effort that went into creating the standard the first time and reusing it. Let me supply some examples...

* You develop a standard configuration for a Notes client install (or any other software package). Now you company's help desk has just one client install to work with. Every time new help desk people come on board they only have to learn how to support that one client install. Compare this to the situation where there are multiple client installs, each one slightly different.
* You create a standard for your Notes development: all projects must go into under the "Dev\Projects\ProjectName folder, where "ProjectName" is the name of the project. Thus all project files will be grouped in one place. Now, every time a new project is started there is no need to decide where to put the project: just follow the standard. If an old project suddenly needs work, you know exactly where to find it. Compare this to the situation where developers can put projects anywhere they want on a development server, and chaotic mess that results from this approach.


Now, I have often had people say to me "But with all these rules, standards, procedures etc. you are taking away our freedom to be creative". But reality is quite different: standards actually provide greater freedom to be creative. Why? Because standards solve the mundane, recurring problems and leave you free to be creative on the more interesting problems. Do you really want to solve the same problems again and again, or would you rather focus on new problems?

To sum up, creating standards to solve recurring problems and you have reused the effort of solving those problems the first time. While there is a cost to manage standards, the reusability component can lead to significant cost savings. Another benefit is that a standards driven environment is much simpler to manage (i.e. complexity is reduced), and tends to be significantly more reliable. Finally, standards free people from mundane problems, allowing them to focus on more interesting problems. And you though reusability applied only to code...?

Drill Down Views in Notes

In The View Nov/Dec 2001 (page15) Damien Katz wrote a short article on a technique for "faking" response documents. (Also see his very interesting articles on the R6 formula engine rewrite and the Formula Language's dirty secret.) This "fake response docs" technique can be used to create drill down lists in Notes. A very useful example of this is a hierarchical corporate directory, showing who reports to whom, and how many direct and indirect reports people have. Use this to drill down from the CEO to the janitor, and see the complete chain of command. Typically this information is updated automatically from the HR system every day, and is a LOT easier to manage and use that the endless out of date Visio diagrams that float around organizations. Recently I posted a screen shot of this type of directory, and Scott Giesbrecht asked me to describe how to do it - so here are the details.

Response documents have $Ref field which contains the UNID of the parent document. If a view's properties are set to "Show response documents in a hierarchy", and $Ref field exists on document "A" containing the UNID of parent document "B", then the view index makes document "A" a response of document "B".

Damien's technique is to use a FakeRef field with the UNID of the intended parent, and then use the view selection formula to set (i.e. fake) $Ref on all documents to the value in the FakeRef field - just for that view. This causes the view to be built with those documents as responses to their parents.

In the sample code posted at the  end of this article, each person document has a FakeRef field called "ManagerUNID" that contains the UNID of their manager. (See Notes 1 & 2 below) The only way I have got this to work is to use the LotusScript "MakeResponse" method of the document - see the code fragment below, which is part of a loop processing all documents. The parent doc is "docManager", and the person document is "doc'. This code runs in an agent that updates the values of the ManagerUNID fields after updated data has been manually entered or imported from the HR system. (See Note 3 below)

Call doc.MakeResponse ( docManager )      'Makes the person doc a response to the manager doc.
Set item = doc.GetFirstItem ( "$Ref" )
Call doc.ReplaceItemValue ( "ManagerUNID", item )      'Creates the FakeRef field
Call doc.RemoveItem ( "$Ref" )      'Removes $Ref so the person doc is no longer a response.
Call doc.Save ( True, True )

Now that evey document has it's parent defined, we are ready to create the drill down view (the Org Chart view in the sample code). To do this we fool the view into thinking that a document has a $Ref field by adding a line to the view's selection formula:

DEFAULT $Ref := ManagerUNID ;         'ManagerUNID is the FakeRef field in the sample code

And, in Damien's words, "This is all it takes". Remember that Notes limits you to 32 levels of response documents, but this is more than adequate for corporate hierarchies.

When you open the sample code, go to People by... Org Chart. Then expand all. You can experiment by adding people to the hierarchy. After adding people run the "Manual Org Chart refresh" agent (Under the View button).

Click link to download file
CorporateDirectory.zip


Note 1
For a document to appear as a response in a view, the FakeRef field (ManagerUNID in this example) MUST contain the hex / binary value of the UNID, and NOT the ASCII version (i.e. @text(@documentUnid) will not work.). The original article in The View was a little unclear here. If a field contains the hex / binary value of a UNID and that field shows on a form, then the contents of the field will appears as a doclink to the parent document. To see this in action, put the $Ref field on a response document form. Then open any response document created with that form in the Notes client. Instead of the UNID of the parent doc showing, you see a doclink to that parent document.

Note 2
You are not limited to just one FakeRef field - you can have several. Each view that uses fake response documents would select the appropriate FakeRef field. This is how one document can have different parents in different views.

Note 3
Heaven is when the HR system exports it's data with canonical names. This makes it trivial to update people's managers, and add the ManagerUNID field. Unfortunately this seldom seems to be the case, so you must find some way of linking manager names from the HR system to the canonical names used by Notes. The next best is when you have unique employee ID numbers, but some HR departments treat these like social security numbers and do their best to hide them. If that happens, you need a routine that somehow uniquely identifies people so that the update agent can put the correct name in the manager field. For more thoughts on the problems of names, see When is a name not a name?.