Lotus Notus (English)

Two critical HTTP problems in Domino 9...

Serdar Basegmez — Fri, 29 Mar 2013 14:19:23 +0300

After I upgraded my servers to Domino 9, I have found two problems affecting HTTP task.

1. Redirect TCP to SSL problem...

My HTTP task stopped responding just after the upgrade. When I look into thread logs I saw that it was redirecting every requests to the same URL! After a couple of tests, I found that if you have "Redirect TCP to SSL" checked in your Internet Site document, it fails with infinite redirection problem.

I posted the issue into the N/D 9.0 Social Edition forum and saw that I'm not alone. It seems like the same issue happens when you check "Require SSL Connection" in specific databases.

IBMers couldn't repeat the problem. After dozens of tests I found out a specific setting in my configuration was the root cause.

Since one of my servers was behind WAS plugin before, I had a notes.ini setting in my configuration document. After deleting this, it got back to normal. So,

If you are forcing SSL redirection either by "Redirect TCP to SSL" or "Require SSL Connection", delete "HTTPEnableConnectorHeaders=1" setting from notes.ini file.

Forum entry: http://www-10.lotus.com/ldd/ndseforum.nsf/xpTopicThread.xsp?documentId”692BEB1A4244C685257B39003F4264

2. Traveler behind IHS (IBM HTTP Server)

The second problem is with Traveler Server and a new feature of Domino 9: IHS. By this new version, you have the chance to position IHS in front of your Domino servers running in Windows platform. It has a couple of advantages but the most obvious one for me is to be able to use 'modern' SSL certificates :)

However, if you are running Traveler server at the same time, you experience a strange problem. It works for a while and then stops working for iOS (I didn't test Android).

The problem is, when you make a profile change, Traveler asks the client to send a 'PROVISION' command with a non-standard HTTP response code (449). But the IHS module for Domino does not transfer this code between the server and mobile client. So iOS device stops working and you cannot register any more devices.

What you should do is to open a PMR and receive updated "mod_domino.so" file.

Forum entry: http://www-10.lotus.com/ldd/ndseforum.nsf/xpTopicThread.xsp?documentId=4F341BE6082FF4A985257B3600374B38

Happy Pi Day present: Pi Calculator for XPages...

Serdar Basegmez — Thu, 14 Mar 2013 09:45:00 +0300

Today is the "Pi Day"...

Last week I was playing with Threads and Jobs project from OpenNTF. I needed a very long job to be run n the server-side. I have thought calculating pi digits would be a cool selection :)

I found a Javascript-based Spigot algorithm by Stephen R. Schmitt and easily converted to Java (sorry about the code!).

Notice that, on an average machine it calculates 10,000 digits in less than a minute. Now consider, ENIAC has calculated 2037 digits in 70 hours in 1949!

However, to calculate trillion digits would be tough with this algorithm. Because we need a large String buffer and a large integer array for calculation. Integer array is around 3.5 times the digit count and string buffer has one chars per digit. So we are limited with the heap size. In a non-XPages environment, if you don't keep string buffer, you would be able to calculate 2.5 million digits with standard memory :)

Anyway, it was fun and I wanted to share it. Here is the screencast:

You can download the database here.

DOTS Deep Dive 4: I can schedule myself...

Serdar Basegmez — Thu, 21 Feb 2013 14:05:00 +0300

Finally, we will be able to enable FeedMonster for CollaborationToday project.

While doing final touches, I have been challenged by a question: "Can we schedule DOTS tasklets programmatically?"

Actually, this is in the wish list for the next version of DOTS. But we can do some trick here. I didn't test this on Domino 9 but it should work.

Here is the code:

package org.openntf.news.playground.tasklets; 
 
 import org.eclipse.core.runtime.CoreException; 
 import org.eclipse.core.runtime.IProgressMonitor; 
 
 import com.ibm.dots.annotation.RunOnStart; 
 import com.ibm.dots.task.AbstractServerTaskExt; 
 import com.ibm.dots.task.RunWhen; 
 import com.ibm.dots.task.RunWhen.RunUnit; 
 import com.ibm.dots.task.ServerTaskManager; 
 
 public class CustomSchedule extends AbstractServerTaskExt { 
 
         @Override 
         public void dispose() { 
 
         } 
         
         @RunOnStart 
         public void scheduleMe(IProgressMonitor monitor) { 
                 ServerTaskManager stm=ServerTaskManager.getInstance(); 
                 
                 // Every 10 seconds 
                 RunWhen sched=new RunWhen(RunUnit.second, 10, 0, 0); 
                 
                 try { 
                         // custom01 is this tasklet's id as defined in plugin.xml 
                         stm.scheduleTasklet("custom01", sched); 
                 } catch (CoreException e) { 
                         e.printStackTrace(); 
                 } 
                 
                 logMessage("I have scheduled myself for "+sched); 
                 
         } 
         
         @Override 
         protected void doRun(RunWhen runWhen, IProgressMonitor monitor) { 
                 logMessage("CustomSchedule: "+runWhen.toString()); 
         } 
         
 }

So it's simple. There is a ServerTaskManager singleton object which manages all tasklets. We are using ".schedule(...)" method (luckily it's public).

When we load DOTS:

> load dots 
 Listening for transport dt_socket at address: 8001 
 21.02.2013 13:55:30   Domino OSGi Tasklet Container started ( profile DOTS ) 
 21.02.2013 13:55:31   [DOTS] (custom01) I have scheduled myself for Runs every 10 - second 
 21.02.2013 13:55:33   [DOTS] (custom01) CustomSchedule: Runs every 10 - second 
 21.02.2013 13:55:43   [DOTS] (custom01) CustomSchedule: Runs every 10 - second

You can use some kind of parameterization to customize your tasklet scheduling. For instance, I am planning to use this functionality for FeedMonster such that If there are only 5 feeds to be read, it's useless to receive new stories every 5 minutes. However, there is no way to 'unschedule' a tasklet. We'll find another trick for this.

ServerTaskManager is a great object BTW. For instance;

stm.runTasklet("someTasklet", "argument1", "argument2");

I didn't find it safe though. Since it doesn't submit a CommandInterpreter object, there might be a problem in error trapping. Noted to the list of feedbacks :)

DOTS Deep Dive 3: Warning for Deadlocks

Serdar Basegmez — Thu, 14 Feb 2013 16:00:00 +0300

Last time, I have blogged about the importance of the importantance of canceling tasklets...

In most of the time, canceling a task is a 'choice' you have. You might want to stop the task for a reason. However, a very important problem is falling into deadlocks. If somehow your code falls into a deadlock or stuck situation, that would lock your DOTS container entirely.

DOTS uses a basic mechanism for identifying scheduled tasklets that are stuck. Every tasklet starts its life with a predefined timeout value. This is 5 minutes by default. You can use "@HungPossibleAfter( timeInMinutes=n )" annotation to change this value.

Native part of DOTS implementation (ndots task) continuously signals the service thread (thread responsible from running scheduled tasklets) for a new tasklet. As we said before, it is designed to run a single tasklet at a time. But service thread still stores the next tasklet in the queue (that's why scheduling becomes complicated as in the first post of this series).

If added tasklet is already in progress, service thread will check if this tasklet has exceeded the timeout value. If so, it warns us...

Here, we have an issue. Because DOTS do nothing to cancel the tasklet that has been stuck with a loop. It just throws stack traces and warnings into console to let you know the situation. You have to cancel your tasklet manually. I hope this would be changed in the future.

Let's look around the issue with a sample code:

@RunEvery( every=15, unit=RunUnit.second ) 
 public void run15(IProgressMonitor monitor) throws Exception { 
         logMessage("RUN: 15 seconds"); 
 } 
 
 @RunEvery( every=1, unit=RunUnit.minute ) 
 @HungPossibleAfter( timeInMinutes=1 ) 
 public void run5(IProgressMonitor monitor) throws Exception { 
         logMessage("START: 1 minutes "); 
 
         while(true) { 
                 Thread.sleep(1000); 
 
                 if(monitor.isCanceled()) { 
                         logMessage("Cancel request taken..."); 
                         break; 
                 } 
 
                 if(getSession().getEnvironmentString("waiting_for_star_wars_disney_edition", true).equals("never")) { 
                         break; 
                 } 
         } 
         
         logMessage("END: 1 minutes"); 
         
 }

There are two tasklets here. The first one is simple. The second one is an infinite loop. When we load DOTS, after a point it will be hung.

14.02.2013 15:43:09   [DOTS] (stest01) RUN: 15 seconds 
 14.02.2013 15:43:09   [DOTS] (stest01) START: 1 minutes 
 14.02.2013 15:45:09   [DOTS] Service Thread Scheduled Runs appears to be stuck on task stest01@run5. It has been more than 2 minute(s) 0 seconds since the last time a task ran. Throwing away new Task instances. Thread Stack trace to follow 
 14.02.2013 15:45:09   [DOTS] Thread Scheduled Runs appears to be deadlocked: 
 14.02.2013 15:45:09   [DOTS]    java.lang.Thread.sleep(Native Method) 
 14.02.2013 15:45:09   [DOTS]    java.lang.Thread.sleep(Thread.java:853) 
 14.02.2013 15:45:09   [DOTS]    org.openntf.news.playground.tasklets.SchedTest01.run5(SchedTest01.java:32) 
 14.02.2013 15:45:09   [DOTS]    sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
 14.02.2013 15:45:09   [DOTS]    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60) 
 14.02.2013 15:45:09   [DOTS]    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37) 
 14.02.2013 15:45:09   [DOTS]    java.lang.reflect.Method.invoke(Method.java:611) 
 14.02.2013 15:45:09   [DOTS]    com.ibm.dots.task.ServiceThread$ServiceTask.run(ServiceThread.java:171) 
 14.02.2013 15:45:09   [DOTS]    com.ibm.dots.task.ServiceThread.run(ServiceThread.java:286) 
 14.02.2013 15:46:09   [DOTS] Service Thread Scheduled Runs appears to be stuck on task stest01@run5. It has been more than 3 minute(s) 0 seconds since the last time a task ran. Throwing away new Task instances. Thread Stack trace to follow 
 14.02.2013 15:46:09   [DOTS] Thread Scheduled Runs appears to be deadlocked: 
 14.02.2013 15:46:09   [DOTS]    java.lang.Thread.sleep(Native Method) 
 14.02.2013 15:46:09   [DOTS]    java.lang.Thread.sleep(Thread.java:853) 
 14.02.2013 15:46:09   [DOTS]    org.openntf.news.playground.tasklets.SchedTest01.run5(SchedTest01.java:32) 
 14.02.2013 15:46:09   [DOTS]    sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
 14.02.2013 15:46:09   [DOTS]    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60) 
 14.02.2013 15:46:09   [DOTS]    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37) 
 14.02.2013 15:46:09   [DOTS]    java.lang.reflect.Method.invoke(Method.java:611) 
 14.02.2013 15:46:09   [DOTS]    com.ibm.dots.task.ServiceThread$ServiceTask.run(ServiceThread.java:171) 
 14.02.2013 15:46:09   [DOTS]    com.ibm.dots.task.ServiceThread.run(ServiceThread.java:286) 
 > tell dots cancel 
 14.02.2013 15:46:35   [DOTS] Canceling all running tasks... 
 14.02.2013 15:46:35   [DOTS] 1 task(s) cancelled 
 14.02.2013 15:46:36   [DOTS] (stest01) Cancel request taken... 
 14.02.2013 15:46:36   [DOTS] (stest01) END: 1 minutes 
 14.02.2013 15:46:36   [DOTS] (stest01) RUN: 15 seconds 
 14.02.2013 15:46:39   [DOTS] (stest01) RUN: 15 seconds

As you see, the second tasklet run and became deadlocked. If you leave it, it will shout forever but will never ever dispose the tasklet. As soon as we cancel this tasklet, the first one runs twice. "15:46:36" is because it has been queued. "15:46:39" is the normal schedule.

As a result, you have to be very careful about infinite loops. Occasional pings to monitoring desk might be a good idea for mission critical tasklets. Always catch the cancel signal. Because if you stop DOTS, it will eventually kill your tasklet but server will be unstable afterwards.

BTW, there are some situations that you don't have a control on catching a cancel signal. For instance, if your tasklet opens a URL connection and does not define a timeout value for the connection, it would wait forever in case remote server does not respond. Always set timeouts :)

Have fun...

DOTS Deep Dive 2: Cancel me or I will crash your server...

Serdar Basegmez — Wed, 13 Feb 2013 12:11:54 +0300

I just wanted to emphasize an important functionality within DOTS...

One of our slides in the recent DOTS session in IBM Connect 2013, we have talked about the "monitor" argument in tasklets. It has two important uses.

First of all, you might let DOTS container know about your progress. Second, it allows you to cancel your task in a less-disruptive manner.

Let's dive into code here.

Our tasklet is running every five seconds and wait 30 seconds each run:

@RunEvery( every=5, unit=RunUnit.second ) 
 public void run5(IProgressMonitor monitor) throws Exception { 
        logMessage("START: 5 Secs"); 
        
        monitor.beginTask("Sleeping well", 15); 
 
        for(int i=0; i<15; i++) { 
                monitor.worked(1); 
                Thread.sleep(2000); 
        } 
        
        logMessage("END: 5 secs"); 
 
 }

We have splitted 30 seconds wait into 15 parts of sleeping to make more sense. Using "beginTask" method at start, we have declared that "Our tasklet will do 15 units of job". Each unit, we inform progress monitor for "1 unit completed" message with "worked" method. We can check the status from the console:

11.02.2013 12:22:03   [DOTS] (stest01) START: 5 Secs 
 > tell dots taskstatus 
 11.02.2013 12:22:06   [DOTS] Display the list of OSGi tasklets currently in progress 
 11.02.2013 12:22:06   [DOTS] Scheduled Runs 
 11.02.2013 12:22:06   [DOTS] --- stest01 [Sleeping well] (13,333%) 
 11.02.2013 12:22:06   [DOTS] Triggered Runs 
 11.02.2013 12:22:06   [DOTS] --- No task running 
 11.02.2013 12:22:06   [DOTS] Manual Runs 
 11.02.2013 12:22:06   [DOTS] --- No task running

What if we cancel this task? Lets try...

11.02.2013 12:28:58   [DOTS] (stest01) START: 5 Secs 
 > tell dots q 
 11.02.2013 12:29:02   [DOTS] Canceling task stest01 
 11.02.2013 12:29:29   [DOTS] (stest01) END: 5 secs 
 11.02.2013 12:29:29   Domino OSGi Tasklet Container terminated ( profile DOTS )

When we stop DOTS task, it will try to cancel our task. However it's not a cancellation. Our tasklet finished its run (30 seconds) because it doesn't have an idea we have sent a cancel signal...

Now add a simple check inside our loop...

@RunEvery( every=5, unit=RunUnit.second ) 
 public void run5(IProgressMonitor monitor) throws Exception { 
        logMessage("START: 5 Secs"); 
        
        monitor.beginTask("Sleeping well", 15); 
        
        for(int i=0; i<15; i++) { 
                monitor.worked(1); 
                Thread.sleep(2000); 
                
                if(monitor.isCanceled()) { 
                        logMessage("Cancel request taken..."); 
                        break; 
                } 
                
        } 
        
        logMessage("END: 5 secs"); 
 
 }

When we cancel the tasklet, it will get the signal and break the execution...

11.02.2013 12:34:29   [DOTS] (stest01) START: 5 Secs 
 > tell dots q 
 11.02.2013 12:34:32   [DOTS] Canceling task stest01 
 11.02.2013 12:34:33   [DOTS] (stest01) Cancel request taken... 
 11.02.2013 12:34:33   [DOTS] (stest01) END: 5 secs 
 11.02.2013 12:34:34   Domino OSGi Tasklet Container terminated ( profile DOTS )

Receiving the cancel signal is extremely important. Suppose you are developing a very very long-running tasklet (it might be a manual tasklet as well). You should be able to break the execution at any point without waiting its normal run. At some point of cancellation, DOTS will decide it's crashed and fill your console with strange error messages...

One last point: Cancellation can be achived by "tell dots cancel (tasknumber)" command. I couldn't find out what the tasknumber is but if you don't provide a parameter, it will cancel all running tasks at once.

DOTS Deep Dive 1: Art of Scheduling Tasklets

Serdar Basegmez — Mon, 11 Feb 2013 12:05:00 +0300

After a successful IBM Connect session, I started a series of posts, based on feedbacks I received from other developers.

There was a little thing I didn't test before the session and this issue has been asked a couple of times: Possible conflicts between scheduled tasklets.

Unfortunately, current implementation within DOTS is based on single threaded approach for tasklets. There are three different threads responsible in DOTS tasklet container for scheduled, manual and triggerred tasklets. It means, there would be only one thread alive to run scheduled tasklets, therefore, only one scheduled tasklet can run at a time.

Let's dive into possible scenerios here.

Suppose we have a scheduled tasklet that will run every 5 seconds and wait for 7 seconds:

11.02.2013 11:10:19   [DOTS] (stest01) START: 5 Secs 
11.02.2013 11:10:26   [DOTS] (stest01) END: 5 secs 
11.02.2013 11:10:28   [DOTS] (stest01) START: 5 Secs 
11.02.2013 11:10:35   [DOTS] (stest01) END: 5 secs 
11.02.2013 11:10:38   [DOTS] (stest01) START: 5 Secs 
11.02.2013 11:10:45   [DOTS] (stest01) END: 5 secs

As you see, since it runs longer than its schedule, it will pass 5 secs schedule and runs every 10 seconds.

Suppose we add a second tasklet here, to run every 2 seconds but wait for 1 secs:

11.02.2013 11:11:15   [DOTS] (stest01) START: 2 Secs 
11.02.2013 11:11:16   [DOTS] (stest01) END: 2 secs 
11.02.2013 11:11:16   [DOTS] (stest01) START: 5 Secs 
11.02.2013 11:11:23   [DOTS] (stest01) END: 5 secs 
11.02.2013 11:11:23   [DOTS] (stest01) START: 2 Secs 
11.02.2013 11:11:24   [DOTS] (stest01) END: 2 secs 
11.02.2013 11:11:24   [DOTS] (stest01) START: 5 Secs 
11.02.2013 11:11:31   [DOTS] (stest01) END: 5 secs 
11.02.2013 11:11:31   [DOTS] (stest01) START: 2 Secs 
11.02.2013 11:11:32   [DOTS] (stest01) END: 2 secs 
11.02.2013 11:11:33   [DOTS] (stest01) START: 5 Secs 
11.02.2013 11:11:40   [DOTS] (stest01) END: 5 secs 
11.02.2013 11:11:40   [DOTS] (stest01) START: 2 Secs 
11.02.2013 11:11:41   [DOTS] (stest01) END: 2 secs 
11.02.2013 11:11:42   [DOTS] (stest01) START: 2 Secs 
11.02.2013 11:11:43   [DOTS] (stest01) END: 2 secs 
11.02.2013 11:11:43   [DOTS] (stest01) START: 5 Secs

This is a complicated scheduling right? They will never run at the sametime. If a tasklet should run in 15th, 20th, 25th and 30th seconds but there is another tasklet running at 20th and 30th seconds, it will just skip.

This is a big problem if you are going to use lots of long-running scheduled tasklets. A tasklet running longer than its schedule would not be a huge problem but multiple tasklets disrupting each other?

OK, there is a very nice solution for that:

Now we will have one tasklet running every 5 secs and waiting for 30 secs (very long job) and another one running every 3 secs waiting 2 secs. Here they are running friendly without intervening each other:

11.02.2013 11:49:06   [DOTS2] (stest01) START: 5 Secs 
11.02.2013 11:49:07   [DOTS] (stest01) END: 3 secs 
11.02.2013 11:49:08   [DOTS] (stest01) START: 3 Secs 
11.02.2013 11:49:10   [DOTS] (stest01) END: 3 secs 
11.02.2013 11:49:11   [DOTS] (stest01) START: 3 Secs 
11.02.2013 11:49:13   [DOTS] (stest01) END: 3 secs 
11.02.2013 11:49:14   [DOTS] (stest01) START: 3 Secs 
11.02.2013 11:49:16   [DOTS] (stest01) END: 3 secs 
11.02.2013 11:49:17   [DOTS] (stest01) START: 3 Secs 
11.02.2013 11:49:19   [DOTS] (stest01) END: 3 secs 
11.02.2013 11:49:20   [DOTS] (stest01) START: 3 Secs 
11.02.2013 11:49:22   [DOTS] (stest01) END: 3 secs 
11.02.2013 11:49:23   [DOTS] (stest01) START: 3 Secs 
11.02.2013 11:49:25   [DOTS] (stest01) END: 3 secs 
11.02.2013 11:49:26   [DOTS] (stest01) START: 3 Secs 
11.02.2013 11:49:28   [DOTS] (stest01) END: 3 secs 
11.02.2013 11:49:29   [DOTS] (stest01) START: 3 Secs 
11.02.2013 11:49:31   [DOTS] (stest01) END: 3 secs 
11.02.2013 11:49:32   [DOTS] (stest01) START: 3 Secs 
11.02.2013 11:49:34   [DOTS] (stest01) END: 3 secs 
11.02.2013 11:49:35   [DOTS] (stest01) START: 3 Secs 
11.02.2013 11:49:36   [DOTS2] (stest01) END: 5 secs 
11.02.2013 11:49:37   [DOTS] (stest01) END: 3 secs 
11.02.2013 11:49:38   [DOTS] (stest01) START: 3 Secs 
11.02.2013 11:49:40   [DOTS] (stest01) END: 3 secs 
11.02.2013 11:49:41   [DOTS2] (stest01) START: 5 Secs

Notice something different? They are running in different DOTS tasklet containers. 3-sec tasklet is running inside DOTS profile and the other one is running in DOTS2.

Multiple profiles are great if you run critical and complicated DOTS tasklets because it has its own Tasklet Container...

However, multiple profile helps you if you have seperate projects. What if you need multiple long-running tasklets within the same project?

For this, you have to enter into a red zone where every Domino-Java developers will eventually experience... In this jungle, you have to create your own thread and manage that. I will provide samples in the future...

A rookie speaker was here...

Serdar Basegmez — Wed, 6 Feb 2013 15:30:00 +0300

I was there, at IBM Connect 2013. My second Orlando visit was far more greater this time. A speaker badge combined to the IBM Champion ribbon, more experience and more fun!

This year, my schedule was packed. I have planned lots of sessions and attended most of them. After last year's disappointment in BDD, I attended master classes on sunday and they were great!

Two sessions were approaching development process in a different angle and they were especially exceptional. On sunday master class session, Nathan T. Freeman explained how to analyze any Notes application in a systematic way. Ryan J. Baxter and Mikkel F. Heisterberg had a best practices session when designing APIs for your applications.

I have been a good boy and got social again... I enjoyed meeting new people, talking to IBMers and other community people. The most valuable information you can get is hidden in these conversation (most of them should be consumed with good beer!).

IBM Champions at OGS...

The most important thing was my session with Bruce Elgort... I was not comfortable with the session because practices over Skype was too long and I was concerned about my accent. I have been speaking in front of crowd since I'm 19 and I have no problem about it. I am a passionated speaker and might talk several hours about a topic I really like. But talking in English? I was going to have an heart attack until 5 minutes before the session.

But I understood why all experienced speakers suggest newbies coupling with another experienced speaker. Bruce has managed me and the session with a great success. My audience might see minutes of silence and a shaking speaker if I was alone there :)

Our session was a great success! Right now looking the evaluation results and understanding we did a great job. One comment was especially came into my attention:

"This is the epitome of what a Lotusphere session should be. Veteran and rookie speaker teaching us how to get better value of the tools that we already own. Great stuff!!!!"

Thanks all for attending and filling out evaluation forms. Here is our session:

Full slides include step-by-step screen shots and explanations from Domino/Eclipse installation to deploying DOTS tasklets. Let me know if there is any problem...

As a bonus, here Justin interviews with me:

IBM Connect Session: BP207 - Meet the Java Application Server you already own...

Serdar Basegmez — Tue, 22 Jan 2013 12:41:03 +0300

Last year was my first Lotusphere. I had a great pleasure visiting labs and talking to developers from IBM.

I have met to David Taieb and we were discussing about an idea (or a suggestion) about XPages. We are lacking a (distributed) back-end processing system for XPages. When you have to run a really long process in your application which could be a reporting, calculation or so forth, you don't have anything but to keep users waiting with an hour glass.

David sketched a diagram showing how SmartCloud Social Business (a.k.a. LotusLive) handles background administration tasks with an OSGi-based service architecture. I was surprized to hear that they were using a technology that they have already put to OpenNTF: DOTS - Domino OSGi Tasklet Services...

When I return to my office, the first thing I did was downloading DOTS and playing with this new development method. It was fantastic! I was looking at the Next Generation Domino Agents. I have also learnt a lot about OSGi development because in contrast to XPages extensibility API, DOTS has a really simple OSGi Context with little dependencies, easy structure etc.

Later, we have developed a powerful FeedAggregator tasklet for CollaborationToday.info project and I have personally experienced how powerful and fast DOTS is (you may refer to this post about the performance of DOTS)

This year, in IBM Connect 2013, We have submitted DOTS as a session proposal with my dear friend Bruce Elgort. As many of you know, it has been accepted. BP207 session will be on January 30th at 10:00 am in SWAN 5-6.

We will talk about Java, OSGi, DOTS, preparing development environment, tips, tricks and some good practices. Our slidedeck includes every single detail to develop and deploy your first tasklet as well.

Java and in particular OSGi are now very important parts of the IBM Notes/Domino app dev model. In this session, you'll learn about OSGi and how easy it is to extend the capabilities of your Domino server and XPages applications with the Domino OSGi Tasklet Services (DOTS). Whether you want to replace your existing agents with DOTS tasks, or develop new services for your XPages apps utilizing the many Java libraries available - we'll show you how!

For attendees, there is no need to have any XPages experience but some level of Java knowledge would be needed to understand some concepts better.

See you there!

Still part of IBM Champions Program!

Serdar Basegmez — Wed, 12 Dec 2012 10:43:09 +0300

In 2011, I have been honored to be recognized as IBM Champion for IBM Collaboration Solutions (formerly, Lotus Software). Yesterday, 2013 class has been announced by our precious community manager Joyce Davis and my title has been renewed for 2013.

There are new champions on the list as well. But I am most delighted to see my dear friend, fellow founder of Turkish Lotus User Group and neighbour Ferhat Bulut (with the well-deserved nick name: bestcoder!).

Ferhat is working for Akbank, one of the largest banks in Turkey and he is an expert on XPages. We know him from last year, as the developer of silver prize contribution at the OpenNTF Contest. I'm frequently sending people to him suffering from XPages, web services or blackberry integration issues and he kindly helps many people in our local community in his free time.

Here is the list of people selected as IBM Champions.

6000 miles flight and I will speak at IBM Connect 2013...

Serdar Basegmez — Mon, 10 Dec 2012 20:14:00 +0300

Yes you heard right...

One of two sessions I have submitted has been accepted. It will be a great excitement for me to speak at IBM Connect for the first time, and also honored to do this with my dear friend Bruce Elgort (and I might be the first speaker from Turkey... Not sure...).

Meet the Java Application Server You Already Own - Domino

"Java and in particular OSGi are now very important parts of the Notes/Domino app dev model. In this session you will learn about OSGi and how easy it is to extend the capabilities of your Domino server and XPages applications with the Domino OSGi Tasklet Services (DOTS). Whether you want to replace your existing agents with DOTS tasks or develop new services for your XPages apps utilizing the many Java libraries available - we will show you how."

See you there...

Picture has been kindly stolen from Per Henrik

My first NotesIn9 show: Introduction to DOTS

Serdar Basegmez — Tue, 4 Dec 2012 23:47:27 +0300

Today, my first NotesIn9 show has been released :)

David Leedy is doing a great job in NotesIn9 video series. There are hours of videos, mostly on XPages.

Recently David announced an incredible project: "Drive to 99". The target is having 99 shows until the end of the year. Whole community responded well and many contributors created screencasts for Notes In 9. After weeks of procrastination, my video became ready as number 93 :)

So my video is about DOTS. DOTS is a great addition to Domino Server. Domino OSGi Tasklet Service provides a service architecture for tasklets that are running at the OSGi-level. They can be considered as the next generation server side agents for XPages paradigm. Read my previous post if you want to know why DOTS is so important...

This is an introduction for setting up runtime and development environments as well as building a simple hello world application. I hope you enjoy the video...

BTW, I really enjoyed screencasting. It is more difficult than blogging (in terms of preproduction effort) but I think you can deliver what you know much more easier. I will try to do it more in the future.

Second Article is on The View: Custom Controls - Deep Dive into Parameterization

Serdar Basegmez — Thu, 29 Nov 2012 21:23:10 +0300

Finally, I have finished my second article "Custom Controls, Part 2 - Deep Dive into Parameterization" and it has been published on The View magazine.

This is the second part of the article "Custom Controls — Basic Concepts and Techniques for Getting Started" published before.

Learn how to customize parameterization using property types and editors to provide a better development experience and increased readability for XPage developers. Find out how multi-value properties and grouping can handle some specific needs (e.g., list of values or logical groups) for parameterization. Also, get best practices on the programmatic use of custom property values.

As a procrastinator, I'd like to thank my editor Julia Wiseman... I would have fired myself if I was an editor :)

Experimenting DOTS task vs. Java Agent

Serdar Basegmez — Fri, 9 Nov 2012 19:45:00 +0300

DOTS (Domino OSGi Tasklet Service) project has been contributed into OpenNTF last year. Basically, it provides a way to create server tasks for Java developers.

This is huge. Basically we could run our Java code in a similar way with a couple of other techniques, but DOTS provide many advantages. I'll dive deep on this in a seperate blog post. For now, take it as a teaser :)

You can refer to Niklas's blog entry to setup DOTS in your environment.

For this experiment, I'm using an exploded version of FakeNames database by David Leedy. In the past, I have duplicated all contents for several times and I'm still using it to crash my server :)

First, I have created a 'terrible' Java agent which does nothing.

import java.util.Date; 
 
 import lotus.domino.AgentBase; 
 import lotus.domino.AgentContext; 
 import lotus.domino.Database; 
 import lotus.domino.Document; 
 import lotus.domino.NotesException; 
 import lotus.domino.Session; 
 import lotus.domino.View; 
 
 public class JavaAgent extends AgentBase { 
 
         public void NotesMain() { 
 
                 try { 
                         Session session = getSession(); 
                         AgentContext agentContext = session.getAgentContext(); 
 
                         Date startTime=new Date(); 
 
                         Database db=getSession().getDatabase("", "test/fakenames.nsf", false); 
 
                         longJob(db); 
 
                         long duration=((new Date().getTime()) - startTime.getTime())/1000;                 
                         System.out.println("Finished in " + String.valueOf(duration) + " secs... - "); 
 
                 } catch(Exception e) { 
                         e.printStackTrace(); 
                 } 
         } 
 
 
         public void longJob(Database db) throws NotesException { 
 
                 int count=0; 
 
                 View view=db.getView("byName"); 
 
                 System.out.println(view.getTopLevelEntryCount()); 
 
                 for(int i=0; i < view.getTopLevelEntryCount(); i++) { 
                         Document doc=view.getNthDocument(i); 
 
                         if(doc!=null) { 
                                 if(doc.getItemValueString("FirstName").startsWith("Sa")) { 
                                         count++; 
                                 } 
 
                                 doc.recycle(); 
                         } 
                 } 
 
                 view.recycle(); 
 
                 System.out.println(count); 
         } 
 
 }

As I said before, it does nothing but counting people that has a first name starts with "Sa". So it does something, consume memory and CPU. Who cares :)

Then I implemented the same code as a DOTS task:

import java.util.Date; 
 
 import lotus.domino.Database; 
 import lotus.domino.Document; 
 import lotus.domino.NotesException; 
 import lotus.domino.View; 
 
 import org.eclipse.core.runtime.IProgressMonitor; 
 
 import com.ibm.dots.annotation.RunOnStart; 
 import com.ibm.dots.task.AbstractServerTaskExt; 
 import com.ibm.dots.task.RunWhen; 
 
 public class AnnotatedTasklet extends AbstractServerTaskExt { 
 
         public AnnotatedTasklet() { 
         } 
 
         public void dispose() throws NotesException { 
         } 
 
         @Override 
         protected void doRun(RunWhen runWhen, IProgressMonitor monitor)        throws NotesException { 
         } 
         
         @RunOnStart 
         public void runOnStart( IProgressMonitor monitor ) throws NotesException { 
                 Date startTime=new Date(); 
 
                 Database db=getSession().getDatabase("", "test/fakenames.nsf", false); 
                 
                 longJob(db); 
                 
                 long duration=((new Date().getTime()) - startTime.getTime())/1000;                 
                 logMessage("Finished in " + String.valueOf(duration) + " secs..."); 
         } 
         
         public void longJob(Database db) throws NotesException { 
                 
                 int count=0; 
                 
                 View view=db.getView("byName"); 
 
                 System.out.println(view.getTopLevelEntryCount()); 
                 
                 for(int i=0; i < view.getTopLevelEntryCount(); i++) { 
                         Document doc=view.getNthDocument(i); 
                 
                         if(doc!=null) { 
                                 if(doc.getItemValueString("FirstName").startsWith("Sa")) { 
                                         count++; 
                                 } 
                                 
                                 doc.recycle(); 
                         } 
                 } 
                 
                 view.recycle(); 
                 
                 System.out.println(count); 
         } 
 }

Now, results:

> tell amgr run "test\XPagescrash.nsf" 'LongJobAgent' 
 09.11.2012 19:38:39   JVM: Java Virtual Machine initialized. 
 09.11.2012 19:38:39   AMgr: Start executing agent 'LongJobAgent' in 'test\XPagescrash.nsf' 
 09.11.2012 19:38:39   Agent Manager: Agent printing: 181349 
 09.11.2012 19:41:02   Agent Manager: Agent printing: 2227 
 09.11.2012 19:41:02   Agent Manager: Agent printing: Finished in 143 secs... - 
 09.11.2012 19:41:02   AMgr: Agent 'LongJobAgent' in 'test\XPagescrash.nsf' completed execution

> load dots 
 > Listening for transport dt_socket at address: 8001 
 > WARNING: Using pde configuration New_configuration located in C:\Lotus\Domino\Data\domino\workspace-dots\pde.launch.ini 
 09.11.2012 19:42:40   Domino OSGi Tasklet Container started ( profile DOTS ) 
 > 181349 
 > 2227 
 09.11.2012 19:43:22   [DOTS] (annotated) Finished in 41 secs...

You got the idea. The same Java routine runs in 143 seconds as a Java agent and 41 seconds as a DOTS task*.

(*) Clinical trials from Developi labs :)

Test platform for IBM Connections...

Serdar Basegmez — Fri, 7 Sep 2012 12:52:43 +0300

I've been in UKLUG lately. I heard lots of discussions about test platform for IBM Connections.

People who wants to implement new deployments, develop and test widgets or customize user interface, all need to install Connections.

So I just wanted to blog about my test platform. I'm currently doing some Connections business and my customers are expecting UI customization, error trapping, configuration changes, etc. Therefore, I have installed a full Connections image on my computer.

I am using Thinkpad T420, i5 2.4 Ghz CPU, 8 GB RAM, Windows 7 Professional installed, one SSD (for windows and apps) and another SATA disk for data and VM images.

There is a Windows 2008 64 bit image on VMWare Player. 2 processor assigned with 4704MB RAM in this image. I have found this memory level by trial and error. Less memory kills the VM and more slows down my computer. As LDAP resource, I'm using a Domino server on my computer which I'm also using for XPages development. this configuration is working smoothly. Start, stop and restart times for Connections are acceptable.

Configuring networks is a bit tricky. I am using a NAT to provide internet connection for the VM and host-only connection to provide connectivity between my computer and the VM.

The only problem is that I have used Windows 2008 for this setup. You can use it for evaluation purposes around 180 days (you can extend standard evaluation time for a couple of times). It has been expired last week. It's still working with a black screen, but sure it's not legal to use it anymore. Therefore I have to reinstall it on a Linux image now. Linux will probably consume much less memory, so I strongly suggest using Linux.

Of course, you have to consider other legal things like test period for IBM Connections or installing Domino on your local workstation etc.

Currently, I'm using the test server for pretty much everything and quite happy about its performance. Alternatively, Niklas is providing a Connection image to be used on OpenNTF projects. You can also use IBM cloud environment for a little payment, as a test platform.

Hope this helps.

Really don’t care...

Serdar Basegmez — Mon, 27 Aug 2012 11:38:01 +0300

IBM finally decided to announce that the last major yellow word in the community has changed... Lotusphere event has been renamed as "IBM Connect" and Lotusphere has become a sub-event or so now.

Shocked? No...

Expected? Yes...

IBM has been working on this transition for a couple of years. First, Lotus name has been removed from internal department hierarchy, then from some major products like Connections or Sametime. Last year Lotusphere 2012 was not yellow anymore. While, in spite of my expectations, Notes/Domino family is still 'Lotus' branded but sure, it will be renamed in a couple of years.

So what?

The only problem I'm having with the rebranding is that it's going so slow. I think that these kind of changes should be realised in one stroke. Of course it's not a clear way to rebrand a complete product tree, but Lotus is a special product because of its community. For more than a year, one of the the most discussed topics has been this rebranding issue. IMHO, this discussion is so unnecessary and real waste of time. We have all experienced some denial and some anger but the change was inevitable.

Now what?

The rebranding phase is almost over. We are all sleeping, waking up, talking and thinking inside a bubble. It is time to think out of this yellow bubble.But now, it should be an oppurtunity for all of us.

If you are old enough, you would remember how IBM survived near-bankruptcy-experience in 90s. Gerstner has made a very important move in IBM's point of view. It wasn't important to market names and products anymore, instead, IBM started to sell what the customer was looking for: Solution....

I am a business partner and I can only sell Domino or XPages or Sametime to a bunch of professionals who like yellow color as much as me. Companies need solutions that fit to their needs. They will need a web application development framework capable to integrate with numerous systems. None of CIOs will care if it's yellow or based on Java.

At the end of the day, I don't care the name...

Let's concentrate on the customer again.

Five weeks of Yammer experience...

Serdar Basegmez — Fri, 27 Jul 2012 22:30:00 +0300

I'm an author in an additional blog for more than a year. As a scientific skeptic blog, we are posting articles or short entries about hoaxes, legends, pseudo-science, alternative medicine, bad science news and similar stuff.

At the begining of this year, we have invited more authors to the group and we have become a micro-community. While we were using Google Groups, after a couple of weeks, it became a serious problem for group members to engage the discussion due to the e-mail over-loading.

Sometimes someone post an idea, news article or a research into a group to start the discussion. After replies and replies to replies (!), my friends (especially those are not monitoring messages every minutes like me!) are getting crazy about massive number of messages every day without any clue about which has been answered or not...

Two months ago we have decided to look for something else. We might create a private page on Facebook but we are all using FB in our private lives. While preventing e-mail over-loading, Facebook would cause an 'notification-overloading' for us. I have tested some other alternatives like Asana, Hall and Podio. Asana and Podio are too GTD-centric (getting things done) and Hall is just an alternative for Skype persistent chats (BTW, it's great, you should test it!).

I came up with the Yammer idea. While speculations about a possible Microsoft acquisition were everywhere around the web, I was really curious about what it is like. After a short testing period, we have decided to go with it. I have registered a network for my own company and created an external network (this is the only way if you want to invite people with different e-mail domains).

So what is it?
Many people are describing IBM Connections as a corporate Facebook. After seeing Yammer, this definition seems to be ridiculous to me. Because Yammer is a copy-cat of the Facebook. The visual and functional experience is almost the same as Facebook (But you can change its color!). I think a set of useful tools are coming with the paid version but I'd never pay for a private facebook :)

It really helped us. Especially desktop and mobile clients are more used than the web site. We can engage our discussion easily, share articles or files and work on new blog entries (even in real time).

About its shortcomings...
At the beginning, we have experienced a serious problem on connection speed. It was really slow (especially from Turkey). However, for a week or so, it seems that the speed problem has been resolved. Web site and mobile application work faster. Since the desktop application has been developed using Adobe Air; it is still slow...

The biggest problem is the notification. Sometimes it creates false signals about notifications and sometimes it skips. Desktop application notification is a silent small window appearing a couple of seconds on the screen and it cannot be customized, mobile notifications are not stable, etc.

It might not be preferred for big companies but in our case, we want to be notified about every conversation in the network which is not the case (or I couldn't find a related configuration). You can select people to be notified in conversations but there is no option to notify all users in the network. There is an announcement option but it's not a part of standard conversation stream, instead more useful for meeting alerts and so forth. I found interesting issues like when I create a page, it doesn't notify people unless I create a conversation. Clearly, Yammer tries to reduce information over-loading but it should be more customizable.

Another problem is that Yammer has no any task/todo mechanism. I cannot think of any social platform without some level of 'get things done' stuff. So we are currently unable to drop Asana.

Comparing to IBM Connections or SmartCloud solutions...
Yammer is definitely more smooth from an end-user perspective. Comparing to the current versions of social platforms in SmartCloud or Connections, the usability is better. Of course, the most important reasons are its simplicity and similarity to Facebook.

On the other hand, IBM solutions are far more capable than Yammer. Activities, Files and Blogs provide an important difference in functionality. I don't have paid version so I can't decide how much customization it allows for profiles, authentication, user directory, e-mail notifications and UI customization, but since it's a cloud-only solution, it would be similar to SmartCloud services.

Absense of desktop application in Connections portfolio is a problem. In addition, Connections mobile application is not conversation-centric, so it would be unfair to compare with Yammer's mobile app.

I should also underline that Yammer differs in style. It is a conversation-centric application. Information (files and contents) and categorization are not selected as a top priority.

At the bottom point, Yammer is still an ideal solution for smaller companies that need lots of micro-community interaction. We are using Yammer as a huge chat room, nothing more. Those who need more functionality like e-mail, files, forms etc; they will have to skip Yammer for now.

I am curious about what Microsoft will change in Yammer in the future to adapt it to big companies. The current version will probably be integrated into Office365 and Skype but I can't imagine how it will affect Microsoft's directions in the Sharepoint product.

Six long weeks...

Serdar Basegmez — Mon, 4 Jun 2012 11:35:09 +0300

Six weeks passed since my last post...

I have worked on XSnippets (OpenSearch support), xInvolve (Version 2.0 released) and a new OpenNTF project named VeryShort...

Not much time left for blogging. I'll make up for it :)

Stay tuned...

Completely unacceptable ’Theft’ by LotusLearns company...

Serdar Basegmez — Sat, 21 Apr 2012 11:29:07 +0300

I am blogging for more than two years. It's like a part-time job. I am doing this sacrificing my 'life' and 'time'. Many people attribute me and this is a great pleasure...

But sharing my ideas or knowledge on the WEB doesn't mean that people can steal them 'COMPLETELY' without my approval even with an attribution!

Yes, I have used the word "stealing" on purpose. Because this is what LotusLearns.com is doing! Kelley Smith, CEO of this company claims that she is doing this for Africans! Really!

For instance;

This has been stolen from my article and there is no attribution for that. Only a link above. Again no attribution on the bottom!

Even there was an attribution, RSS feeds don't provide a right to 'COPY' all contents. Many authors that have stolen articles in this site are not providing full articles on their RSS feeds. And Looking the structure of contents, we can easily see that these are not 'automatically' constructed. They have been edited one by one.

This is clear theft.

...and I am not alone;

Nathan T. Freeman: LotusLearns that stealing web content is shameful
Kathy Brown: How To Piss People Off – LotusLearns
Bill Malchisky: LotusLearns -- The Good, The Bad, and Yes, It’s Ugly (Underneath)
Chris Miller: LotusLearns.com might be using your content without permission - beware
Thomas Duff: If your blog content appears on PlanetLotus, LotusLearns.com is likely infringing on your copyright...
Jesse Gallagher: LotusLearns did a poor job copying my post
Tim Tripcony: LotusLearns wants you to think my content is theirs

OpenSearch functionality for XPages.info...

Serdar Basegmez — Mon, 16 Apr 2012 12:20:00 +0300

Today we have added a very simple feature to XPages.info site. It provides an integrated search functionality to be used with browsers.

Here how to use it:

For Chrome:

1. Open xpages.info site once.
2. Close your browser, reopen and at the address bar, start writing "xpages". Depending on how frequent you are using xpages.info, it will suggest the address in the bar. For me, "x" character is enough :)

3. Now you see a new instruction in the address bar: "Press Tab to search XPages". Do what it says :)

4. Press enter to see the search results related with your keyword:

For Firefox:

1. Open xpages.info site.
2. Open Search Engine alternatives on the right and click "Add XPages.info" option.

3. Now XPages.info has been added to Firefox registered search engines. To use it more practically, Click "Manage Search Engines" here.

4. Select our search engine, click "Edit Keywords" and enter a new keyword like "xp" here. Click OK to close this dialog.

5. Now go to a new window and enter "xp dynamic content" into the address bar... Voila!

I couldn't find a similar way to do this in Internet Explorer 9. If you know, please comment :)

How we did it?

It's pretty easy! You need to add two things to your database. First an XML file definition for opensearch:

 
XPages 
XPages.info Search 
UTF-8

You can also add some more properties here such as an icon to show in the search engine listing.

We put this xml files into File Resources section of our database. Then we added a resource definition into the HEAD section of our home page:

That's all!

Now I have two To-Do's for myself. I'll try to extend search functionality of XPages.info to provide single point of search for XPages-specific information.

The second To-Do is adding this to XSnippets. I noticed that searching snippets is not practical in the current situation.

OpenSearch provides much more possibilities than this. You can visit OpenSearch.org for more information...

"Dotless i", toLowerCase and toUpperCase functions: Use responsibly!

Serdar Basegmez — Thu, 5 Apr 2012 18:40:03 +0300

"Dotless i" problem is the most profitable area in my revenue stream :)

Since the beginning of my career, I found and isolated dozens of bugs simply related to 'dotless-i'.

I have already blogged about it couple of times, but let me sum up the problem.

There are Turkish characters in our language, as in many others. But two exceptional cases are "ğ" (curved g) and "ı/İ" (dotless-i/dotted-I) characters. The first one is a whole different story that some linguistic experts discuss its necessity. The second one is a serious headache for us.

Dotless-i, is a lowercase 'i' without dot. The uppercase of this character is the usual "I". There is another character, "I with dot". The lowercase of this character is the usual lowercase "i".

Have you noticed the problem? This unsymetrical conversion causes a serious problem in programming. We face this problem mostly in Java applications because of (IMHO) poor implementation of toLowerCase and toUpperCase functions.

In Java, String.toLowerCase() method converts characters to lowercase according to the default locale. This causes problems if your application works in Turkish locale and especially if you are using this function for a file name or a url that must obey a certain character set.

I have blogged about two serious examples before: The compile errors with Script libraries with "i" in their names and XSP Manager's fault if an XPage is in a database with "I" in its name.

There is a long history, as I said. For instance in some R7 version, router was unable to send a message to a recipient if his/her name starts with "I". Message reporting agents was not running in Turkish locale until R8. Anyone with Turkish locale could not install Lotus Notes 8.5.1 (it's real!). The list goes on...

There is almost no beta tester from Turkey and customers don't open PMR for these problems. So these problems are not going up to the first priority for development teams.

Even Java team has added a special warning to the latest documentation:

Note: This method is locale sensitive, and may produce unexpected results if used for strings that are intended to be interpreted locale independently. Examples are programming language identifiers, protocol keys, and HTML tags. For instance, "TITLE".toLowerCase() in a Turkish locale returns "t?tle", where '?' is the LATIN SMALL LETTER DOTLESS I character. To obtain correct results for locale insensitive strings, use toLowerCase(Locale.ENGLISH).

My point is to give a simple suggestion for all Java programmers.

If you are using toLowerCase and toUpperCase function for non-textual context, use them as suggested above!

Use responsibly :)