How a Notes view is internally navigated

A Lotus Notes view is a collection of documents. Internally talking (from the Notes API documenation) a view is also a "collection."

So what happens when the user opens a view in the Lotus Notes workstation? Does Lotus Domino server return the entire view to the Notes workstation? In fact when the user navigates a view, Domino only returns a "chunk" of documents to the Notes workstation. Else (if the entire view was read) it would take lot of time to read the view content.
So when the user opens a view, Notes requests only some documents. For example let's suppose a plain view (without categories) and in one window Notes can show 20 documents. When the user is viewing a view Notes may request for example 100 documents: the 20 documents to fit the window plus 40 documents before the first shown document plus other 40 documents after the last document being seen in the window. By this way if the user navigates the view scrolling down then the information is already in the client side and Notes does not need to ask the server for these documents... when Notes is approaching the end (or the beginning) of a chunk, it queries another chunk.

When you expand a category Notes needs to get another chunk, the same if you go to the end or beginning of a view, or if you go to a given position of the view by dragging the scrollbar control.

As you can see this is a very efficient way to navigate a view.

Notes uses the Notes API functions to navigate a view. The main used function is NIFReadEntries. This function admits 12 parameters:

STATUS LNPUBLIC NIFReadEntries(
HCOLLECTION hCollection,
COLLECTIONPOSITION far *IndexPos,
WORD SkipNavigator,
DWORD SkipCount,
WORD ReturnNavigator,
DWORD ReturnCount,
DWORD ReturnMask,
HANDLE far *rethBuffer,
WORD far *retBufferLength,
DWORD far *retNumEntriesSkipped,
DWORD far *retNumEntriesReturned,
WORD far *retSignalFlags);

IndexPos is the exact position in the view from where the chunk needs to be read.
ReturnCount indicates how many documents needs to be read (in the previous example it should be set to 100).
The remaining parameters can be found in the documentation, but what I try to show you is how Notes uses this function to navigate the view.

What does this function return? Does it return the NoteID or UNID and then Notes opens each document to retrieve the fields to be shown in the view? Not really. If Notes opened each document to retrieve, for example in an email database, the subject or the delivered date then it would loose performance. What it does is to retrieve only the information to be shown in the view (it is called Summary buffer, I will explain it later in another post). So Notes does not need to open any document to show the view content. For each row (or document) in the view, NIFReadEntries returns packed data with the subject, delivered date, size, ...

What about readers? When a document is not shown in the view (because it contains reader fields) the job is done by the server... and it is an intensive job! It moreless needs to sequentially skip each document that does not need to be shown. So suppose that you are browsing a view containing 1.000.000 of documents but you are only allowed to see the first and the last document from this view. The Notes client will simply call NIFReadEntries and the server is the one that will need to skip 999.998 documents. The CPU (and I/O) consumption will be done by the server and not by the workstation. As you can see, reading a view that is affected by reader fields causes serious performance issues.

Is there an accurate way to know how many documents are in a view? It will sound strange but there is no accurate way to know how many documents a view contains! Only if your view is plain (that is, it does not contain response documents neither categories) then you can call this Notes API function: NIFGetCollectionData. The other alternative to get the total number of documents is to fully navigate the view.

This is the reason why the scrollbar control does not have a good behaviour for views having categories (specially when they are expanded).

I will propose a new feature that could be useful and very easy to implement by the Lotus Notes client developers at IBM. Suppose you have a view that shows all the bids and this view is sorted by the bid date. Now suppose that you want to view only bids from a given date (e.g. 01/01/2008) to another given date (e.g. 12/01/2008). In this case the view object should expose a new method like "FilterView". And you can simply pass these two dates in this method. Then internally talking is pretty straighforward to implement it: It's just a matter of locating the first date by calling the NIFFindByKey API and then locating the last date by calling the same NIFFindByKey API. And here you will have the "initial" and "last" COLLECTIONPOSITIONs that will indicate the bounds of the view. The only requirement is that the view should be sorted by bid date.
This is similar to the show-single category feature but more flexible.
Suppose you have a view showing names... and the first sorted column is the surname. And you want to see all documents where the surname starts with S or T or U. In this case you can apply the same exact logic.
So with this proposal the LotusScript and Formula developers will easily filter the view dinamically without causing extra costs (like creating views with different formulas).

Well, in this article I shown how views internally works and I proposed an enhancement.